Meaningful Change Methods

What is an important amount of change?

Defining the magnitude of change that corresponds to “important” change is necessary for applications such as comparative effectiveness research. There are many terms for these levels of changes (e.g., clinically important change, minimally important difference, minimally perceptible change) and many methods for estimating them.

TIP: THERE IS NO SUCH THING AS THE “TRUE” DIFFERENCE. THE MAGNITUDE OF AN IMPORTANT SCORE DIFFERENCE IS AN ESTIMATE, AND THE ESTIMATE REFLECTS THE VALUES, CONCERNS, AND CONTEXT OF THE ESTIMATOR. AS MADELINE KING EXPRESSES:

“Specific estimates of minimally important differences (MIDs) should therefore not be over interpreted. For a given health-related quality-of-life scale, all available MID estimates (and their confidence intervals) should be considered, amalgamated into general guidelines and applied judiciously to any particular clinical or research context.”

·         Finally, you should keep in mind that most MID estimates are an average across the range of scores. It may be that people require more or less change to consider it meaningful depending on where they started. [KFC1] 


 [KFC1]Kozlowski AJ, Cella D, Nitsch KP, Heinemann AW. Evaluating Individual Change

With the Quality of Life in Neurological Disorders (Neuro-QoL) Short Forms. Arch

Phys Med Rehabil. 2016 Apr;97(4):650-4.e8. doi: 10.1016/j.apmr.2015.12.010.

PubMed PMID: 26740062; PubMed Central PMCID: PMC4994512.

 

Guidance for Applying Meaningful Change Thresholds

Evidence continues to accumulate regarding reasonable estimates of meaningful change score thresholds. Studies of PROMIS® measures have calculated estimates of meaningful important differences.  There are several things you should keep in mind when using these estimates.

  • There are different methods for calculating (see below for more information) and they will yield different estimates of a meaningful difference. Also, analyses in different samples and contexts will lead to different estimates.
  • Picking an MID requires a judgment on your part. You should consider, for example, the use of the MID. Estimates on the lower end of the MID range might be appropriate for group comparisons. You might choose an estimate at the higher end to categorize changes in individuals.
  • If there is no empirical literature on which to base an MID estimate, you may want to use a half standard deviation (5 points on a T-score metric). This choice is not without its controversy, however.
  • Finally, you should keep in mind that most MID estimates are an average across the range of scores. It may be that people require more or less change to consider it meaningful depending on where they started.

Neuro-QoL™ Measures

Conditional minimal detectable change (cMDC) values were estimated for 14 Neuro-QoL measures. Learn more>> 

PROMIS Measures

Studies of PROMIS measures have estimated meaningful change. Learn more>>

 

Methods for Estimating Meaningful Change

Excellent reviews of methods for estimating meaningful change have been published. Typically such methods are divided into distribution-based and anchor based methods of estimation. Crosby and colleagues describe the differences between these strategies. A particularly helpful set of recommendations has been published by Revicki and colleagues.  Other useful resources are written by Streiner and deVet.  New methods for estimating important change are emerging (see Cook and Thissen).

In 2017, a symposium on qualitative methods for establishing meaningful change thresholds was held at the International Society for Quality of Life Research (ISOQOL) annual conference. Advantages and disadvantages of semi-structured interviews, clinical trial exit interviews, focus groups, vignettes, and the Delphi panel method were explored. A summary was published in the Journal of Patient-Reported Outcomes in 2019. Learn more>>

No single method for defining meaningful change is adequate. Evaluations of what constitutes meaningful change should be based on multiple sources of evidence. To the extent possible, estimates should be grounded in the context of use. It is prudent in setting threshold estimates to consider the consequences of those estimates (e.g., what is the cost-benefit ratio of a lower versus a higher threshold for a given context).

Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales: a practical guide to their development and use. Oxford University Press, USA

Using PRO Scores to Define Responders

It has long been recognized that statistically significant differences are not equivalent to clinically meaningful differences. Further, there are different standards for what constitutes meaningful individual- and group-level differences. “Responder definitions” estimate the threshold of score change that can be judged, defensibly, as a meaningful change. A number of empirical methods can be used to establish responder thresholds (e.g., Coon 2017, Cappelleri 2014). The FDA recommends identifying an a priori responder threshold and reporting the number of responders in different clinical arms (see the FDA Guidance for Industry). This information is used to reveal the impact of interventions at the individual, not just the group-level.