Meaningful Change Methods

Many methods are used to estimate the magnitude of change considered important by patients. For applications with individual patients, consider whether the change is due to chance or measurement error.

What is an important amount of change?

Defining the magnitude of change that patients find important or meaningful (versus statistically significant) is necessary for applications such as comparative effectiveness research. There are many terms for these levels of changes (e.g., clinically important change, minimally important difference, minimally important change, minimal clinically important difference) and many methods for estimating them. There is no single method for defining meaningful change that is accepted as the gold standard. No single method for defining meaningful change is adequate. Evaluations of what constitutes meaningful change should be based on multiple sources of evidence.

TIP: THE MAGNITUDE OF AN IMPORTANT SCORE DIFFERENCE IS AN ESTIMATE, AND THE ESTIMATE REFLECTS THE VALUES, CONCERNS, AND CONTEXT OF THE ESTIMATOR. THEREFORE, ANY ONE ESTIMATE SHOULD NOT BE TREATED AS THE “TRUE” IMPORTANT CHANGE.

AS MADELINE KING EXPLAINS:

“Specific estimates of minimally important differences (MIDs) should therefore not be over interpreted. For a given health-related quality-of-life scale, all available MID estimates (and their confidence intervals) should be considered, amalgamated into general guidelines and applied judiciously to any particular clinical or research context.”

· Finally, you should keep in mind that most MID estimates are an average across the range of scores. It may be that people require more or less change to consider it meaningful depending on where they started. [KFC1]

 [KFC1]Kozlowski AJ, Cella D, Nitsch KP, Heinemann AW. Evaluating Individual Change

With the Quality of Life in Neurological Disorders (Neuro-QoL) Short Forms. Arch

Phys Med Rehabil. 2016 Apr;97(4):650-4.e8. doi: 10.1016/j.apmr.2015.12.010.

PubMed PMID: 26740062; PubMed Central PMCID: PMC4994512.

Anchor- and Distribution-based Methods

Excellent reviews of methods for estimating meaningful change have been published. Typically such methods are divided into distribution-based and anchor based methods of estimation.

Crosby and colleagues describe the differences between these strategies.
A particularly helpful set of recommendations has been published by Revicki and colleagues.
Terwee and colleagues provide practical guidance for estimating minimal important change (MIC) using predictive modeling and, in some cases, the Receiver Operating Characteristic (ROC) curve methods.
Hays and Peipert examined appropriate methods for determining within-person significant change.
Other useful resources are written by Streiner and deVet.

Qualitative Methods

In 2017, a symposium on qualitative methods for establishing meaningful change thresholds was held at the International Society for Quality of Life Research (ISOQOL) annual conference. Advantages and disadvantages of semi-structured interviews, clinical trial exit interviews, focus groups, vignettes, and the Delphi panel method were explored. A summary was published in the Journal of Patient-Reported Outcomes in 2019 (see Staunton) with an update later that year. Learn more>>

Using PRO Scores to Define Responders

It has long been recognized that statistically significant differences are not equivalent to clinically meaningful differences. Further, there are different standards for what constitutes meaningful individual- and group-level differences. “Responder definitions” estimate the threshold of score change that can be judged, defensibly, as a meaningful change. A number of empirical methods can be used to establish responder thresholds (e.g., Coon 2017, Cappelleri 2014). The FDA recommends identifying an a priori responder threshold and reporting the number of responders in different clinical arms (see the FDA Guidance for Industry). This information is used to reveal the impact of interventions at the individual, not just the group-level.