Validation
Comparing ASCQ-Me® to PROMIS® to measure the functioning and wellbeing for adults with sickle cell disease
Sickle Cell Disease (SCD) causes profound suffering and decrements in daily functioning. Demand is growing for valid and reliable measures to systematically document these effects, particularly in adults. The Adult Sickle Cell Quality of Life Measurement System, ASCQ-Me℠, was developed for this purpose. To help users select the best of these measures for adults with SCD, we evaluated and compared PROMIS, which is designed to be “universally applicable” and ASCQ-Me℠ which was designed specifically for SCD.
Substantial qualitative and quantitative evidence supports the validity of ASCQ-Me measures.
Have ASCQ-Me measures been validated?
Though scales often are described as being “valid” or having been “validated,” validity resides in the use of the scores, not in the scale itself. Scores on a test of third grade vocabulary would be unlikely to be valid for discriminating the math skills of college-ready high school seniors. In the same way, self-report measures cannot be pronounced, once and for all purposes, “validated.” The role of validity analyses is to collect a body of evidence that reveals for what and with whom the scores on measures are valid.
What is the evidence for the validity of ASCQ-Me measures?
Though the ASCQ-Me measures are relatively new, substantial qualitative and quantitative evidence has been gathered that supports the validity of ASCQ-Me scores in discriminating levels of symptoms and outcomes important to individuals with sickle cell disease (SCD). Below are some highlights:
Content Validity
Content validity is evidence that measurements cover all the aspects of the target outcome, or symptoms that are important to affected patients. Attention to content validity is critical in the first steps of measurement development. As Anastasi stated it, “content validity is built into a test from the outset through the choice of appropriate items.”1 Assurance of the content validity of the ASCQ-Me scales began with structured interviews with both patients (N=120) and subject experts (N=15). Further, ASCQ-Me item content was grounded in life experiences discovered in participant interviews. Additional details related to the content validity of the ASCQ-ME measures have been published.2,3
Conceptual Model
Factor analytic results supported the essential unidimensionality of five of the six ASCQ-Me subscales: sleep, pain, stiffness, emotional, and social impacts of SCD.3 Unidimensionality of the cognitive impact scale was not supported and further development is needed to improve its ability to discriminate among levels of SCD disease severity.
Additional Validity Evidence
Researchers and clinicians interested in using the ASCQ-Me measures will find additional information about the properties of the measures in two important 2014 articles.2,3 A study of 490 adults with SCD supported the validity of ASCQ-Me measures.4 When evaluating this evidence, potential users should evaluate what has already been published and consider adding to the scientific body of evidence in their own samples.
Reliability
Inter-item consistency of the recommended subscales sleep, pain, stiffness, emotional, and social impacts of SCD was high (0.92 to 0.96).3 Trait-level dependent reliability (based on item response theory model) also has been evaluated.
Next Steps
Evaluating the validity and reliability of measurement scores is a continuing process. Future research will add to the body of knowledge regarding the ASCQ-Me measures. Below are sample future studies:
- Evaluate potential thresholds for ASCQ-Me scores that would improve interpretability and clinical usefulness.
- Conduct longitudinal studies of the function of ASCQ-Me scores to evaluate properties, such as responsiveness.
- Compare ASCQ-Me scores to scores on additional patient-relevant variables to enhance the evidence for convergent and divergent validity of scores.
- Expand the populations in which ASCQ-ME measures have been administered and evaluated.
Making the case for using ASCQ-Me measures
Suppose you have decided to use an ASCQ-Me measure in a clinic or in a study, but you must justify that decision to either administrators or a granting agency. Below are a few tips to make your case:
- Start by explaining why it is important to measure these constructs from the perspective of patients with SCD. Describe the impact of the disease on individuals who live with SCD, and how measures of the broader impact of SCD on patients’ lives may help develop treatment plans.
- Compare the population you are interested in to the population used in developing the ASCQ-Me measures.
- Summarize what is known about the reliability and validity of ASCQ-Me scores. Explain that validity is a continuing process that is never completed, and emphasize that evidence is still accumulating with regard to ASCQ-Me scores.
- Describe the alternatives to the ASCQ-Me measures and clearly state why you believe an ASCQ-Me measure is a good choice, particularly for your population and for your particular purpose. Remember, validity resides in the use of the scores.
1Anastasi A, 1988. Psychological Testing, New York, Macmillan Publishing Company, p. 122-127.
2Treadwell MJ, Hassell K, Levine R, Keller S. Adult sickle cell quality-of-life measurement information system (ASCQ-Me): conceptual model based on review of the literature and formative research. Clin J Pain. 2014 Oct;30(10):902-14.
3Keller SD, Yang M, Treadwell MJ, Werner EM, Hassell KL. Patient reports of health outcome for adults living with sickle cell disease: development and testing of the ASCQ-Me item banks. Health Qual Life Outcomes. 2014 Aug 22;12:125.
4Keller S, Yang M, Treadwell MJ, & Hassell KL. Sensitivity of alternative measures of functioning and wellbeing for adults with Sickle Cell Disease: Comparison of PROMIS® to ASCQ-Me℠. Health and Quality of Life Outcomes. 2017 Jun 2;15: 117.