Meaningful Change

What is an important amount of change?

Defining the magnitude of change that corresponds to “important” change is necessary for applications such as comparative effectiveness research. There are many terms for these levels of changes (e.g., clinically important change, minimally important difference, minimally perceptible change) and many methods for estimating them.

TIP: THERE IS NO SUCH THING AS THE “TRUE” DIFFERENCE. THE MAGNITUDE OF AN IMPORTANT SCORE DIFFERENCE IS AN ESTIMATE, AND THE ESTIMATE REFLECTS THE VALUES, CONCERNS, AND CONTEXT OF THE ESTIMATOR. AS M.T. KING EXPRESSES:

“Specific estimates of minimally important differences (MIDs) should therefore not be over interpreted. For a given health-related quality-of-life scale, all available MID estimates (and their confidence intervals) should be considered, amalgamated into general guidelines and applied judiciously to any particular clinical or research context.”

·         Finally, you should keep in mind that most MID estimates are an average across the range of scores. It may be that people require more or less change to consider it meaningful depending on where they started. [KFC1] 


 [KFC1]Kozlowski AJ, Cella D, Nitsch KP, Heinemann AW. Evaluating Individual Change

With the Quality of Life in Neurological Disorders (Neuro-QoL) Short Forms. Arch

Phys Med Rehabil. 2016 Apr;97(4):650-4.e8. doi: 10.1016/j.apmr.2015.12.010.

PubMed PMID: 26740062; PubMed Central PMCID: PMC4994512.

 

Watch this 3-minute video about meaningful change in PROMIS scores.

 

Methods for Estimating Meaningful Change

Excellent reviews of methods for estimating meaningful change have been published. Typically such methods are divided into distribution-based and anchor based methods of estimation. Crosby and colleagues describe the differences between these strategies. A particularly helpful set of recommendations has been published by Revicki and colleagues.

No single method for defining meaningful change is adequate. Evaluations of what constitutes meaningful change should be based on multiple sources of evidence. To the extent possible, estimates should be grounded in the context of use. It is prudent in setting threshold estimates to consider the consequences of those estimates (e.g., what is the cost-benefit ratio of a lower versus a higher threshold for a given context).

Estimates of Meaningful Change Thresholds for HealthMeasures

Evidence continues to accumulate regarding reasonable estimates of meaningful change score thresholds. Several studies of PROMIS® measures have calculated estimates of meaningful important differences. These are summarized below. There are several things you should keep in mind when using these estimates.

  • There are different methods for calculating and they will yield different estimates of a meaningful difference. Also, analyses in different samples and contexts will lead to different estimates.
  • Picking an MID requires a judgment on your part. You should consider, for example, the use of the MID. Estimates on the lower end of the MID range might be appropriate for group comparisons. You might choose an estimate at the higher end to categorize changes in individuals.
  • The estimates below are only those published through 2016. Check PubMed to see if additional MID studies have been published.
  • If there is no empirical literature on which to base an MID estimate, you may want to use a half standard deviation (5 points on a T-score metric). This choice is not without its controversy, however.
  • Finally, you should keep in mind that most MID estimates are an average across the range of scores. It may be that people require more or less change to consider it meaningful depending on where they started.

PROMIS ADULT MID ESTIMATES

Fatigue
  • 17-item short form: 2.5-4.5 Yost
  • 7-item short form: 3.0-5.0 Yost

Based on Neuro-QoL short form, which shares metric with PROMIS Kozlowski:

  • Lower Quartile: 4.7-12.2
  • Middle Half: 4.7 – 5.1
  • Upper Quartile: 5.0-11.3
Pain Interference
  • Computer Adaptive Test: 3.5-5.5 Amtmann
  • 10-item short form: 4.0-6.0 Yost
Physical Functioning
  • 10-item short form: 4-6 Yost
  • 20-item short form: 2 Hays
Emotional Well-Being
  • Anxiety (9-item): 3.0-4.5 Yost
  • Depression (10-item): 3.0-4.5. Yost

PROMIS PEDIATRIC MID ESTIMATES

  • Depression: 2-3 Thissen
  • Pain Interference: 2-3 Thissen
  • Fatigue: 2-3 Thissen
  • Mobility: 2-3 Thissen
  • Morgan and colleagues found that estimates of MIDs varied by domain, the severity of symptom/dysfunction, and by who was making the judgement (pediatric patient, parent, or clinician).

Neuro-QoL Adult Measures

Conditional minimal detectable change (cMDC) values were estimated for 14 Neuro-QoL measures. Estimates vary based upon the patient’s symptom/dysfunction severity. Index tables and an interactive Excel workbook that calculates cMDCs are available. (Kozlowski)