Prepare Technology & Display Results

All administrative platforms require customization for healthcare systems and individual clinics.

Scores from HealthMeasures should be displayed to facilitate rapid and accurate interpretation by patients and clinicians when used in patient care.

Technology Requirements

Implementation of electronic collection of patient-reported outcome measures (e.g., integrated within the electronic health record) requires effort from informaticists. Necessary tasks include:

  • Technical build of administration platform including multiple options for completing assessments (e.g., via patient portal pre-visit and via tablet in waiting room).
  • Add selected patient-reported outcome measures.
  • Create assessment triggers, schedule, reminders, and opt-out options.
  • Generate list of patients with clinic visits today who have not yet completed their assessment.
  • Display patients’ responses and scores following best practices.
  • Implement notifications for scores as requested (e.g., automatic message to department’s social work pool for PROMIS Depression scores > 70).
  • Conduct end-to-end system test.
  • Configure tablets or kiosks for in-clinic assessment.
  • Support hardware and software when problems arise including creating a ticket support system if necessary.
  • Provide support for in-clinic connectivity issues.
  • Order data collection devices (e.g., tablets) and create an asset management model with an established product life cycle.
  • Create evaluation metrics (e.g., order rate, completion rate) and procedure for routine reporting about implementation.

BACKGROUND OF THE WEBSITE

Leverage Technology to Improve Efficiency and Use

The ePROs for Clinical Care Toolkit provides strategies for:

  • Utilizing best practices for the design of ePRO tools.
  • Identifying where technology can improve the efficiency of workflow.
  • Ensuring technology tools provide a seamless experience for patients and clinical teams.

Learn more>>

Recommendations for Displaying Results

  • Display scores over time (e.g., trends).
  • Graphs are often easier to interpret than tables of numbers.
  • Label which direction is better health and which direction is worse health.
    • Directionality should be consistent within a graph (e.g., up is always better health)
    • Sample figures are available in the ePROs for Clinical Care Toolkit (see below).
    • Sample figures used in research settings are available in Hanmer, J., Jensen, R.E. & Rothrock, N. A reporting checklist for HealthMeasures’ patient-reported outcomes: ASCQ-Me, Neuro-QoL, NIH Toolbox, and PROMIS. J Patient Rep Outcomes 4, 21 (2020). https://doi.org/10.1186/s41687-020-0176-4
  • Do not rely on color alone to convey information.
  • Use terminology that is understood by the viewer. This may be different for patients (e.g., “difficulty breathing”) and clinicians (e.g., “dyspnea”).
  • Provide interpretation aids when available. This can include score labels (e.g., “within normal limits,” “moderate”).
  • Engage care providers and patients before implementation to identify what scores should trigger specific clinical actions (e.g., further assessment, education on symptom management, referral).

Visualization Library, Guidelines, and Functional Requirements

The ePROs Toolkit provides detailed information on report design and use. This includes:

  • A visualization library containing longitudinal, comparative, and single score graphs and tables. Learn more>>
  • Guidelines for statistical presentation, longitudinal information, comparative information, and contextualizing data. Learn more>>
  • Guidelines for report function and interaction including automation, customization, drill up/down tools, filtering data, integrating clinical data, and accommodating multiple platforms. Learn more>>
  • Guidelines for presentation including enhancing key information, using simple graphs, and organizing visualizations. Learn more>>
  • An ePRO reporting functional requirements guide. Learn more>>
  • An ePRO clinic report design checklist. Learn more>>

Figure: ePRO Reporting Toolkit Domains and Guidelines

ePRO Report

Reprinted with permission. Source:  http://epros.becertain.org/reporting/overview/key-concepts-and-background

LeRouge C, Austin E, Lee J, Segal C, Sangameswaran S, Hartzler A, Lober B, Heim J, Lavallee DC. ePROs in Clinical Care: Guidelines and Tools for Health Systems. Seattle, WA: CERTAIN, University of Washington. May 2020.

Recent Research in Displaying Results

Best practices in displaying patient-reported outcome scores for use in clinical practice is an active area of research. Below are selected publications.

Arcia, A., Woollen, J., & Bakken, S. (2018). A systematic method for exploring data attributes in preparation for designing tailored infographics of patient reported outcomes. eGEMs (Generating Evidence & Methods to improve patient outcomes), 6(1), 2. doi: 10.5334/egems

Grossman, L. V., Feiner, S. K., Mitchell, E. G., & Creber, R. M. M. (2018). Leveraging patient-reported outcomes using data visualization. Applied clinical informatics, 9(03), 565-575. doi: 10.1055/s-0038-1667041

Hartzler A.L., Chaudhuri S., Fey B.C., Flum D.R., Lavallee D. (2015). Integrating patient-reported outcomes into spine surgical care through visual dashboards: Lessons learned from human-centered design. EGEMS (Wash DC), 3(2), 1133. doi: 10.13063/2327-9214.1133

Jensen, R.E., Bjorner, J.B. (2019). Applying PRO reference values to communicate clinically relevant information at the point-of-care. Medical Care, 57, S24-S30. doi: 10.1097/MLR.0000000000001113

LeRouge C., Hasselquist M.B., Kellogg L., Austin E., Fey B.C., Hartzler A.L., Flum D.R., and Lavallee D. (2017). Using heuristic evaluation to enhance the visual display of a provider dashboard for patient-reported outcomes. AcademyHealth eGEMs (Generating Evidence & Methods to improve patient outcomes): 5(2), article 6. doi: 10.13063/2327-9214.1283

Lor M., Koleck, T.A. Bakken S. (2019). Information visualizations of symptom information for patients and providers: a systematic review. Journal of the American Medical Informatics Association, 26(2), 162–171. doi: 10.1093/jamia/ocy152

Snyder, C., Smith, K., Holzner, B. et al. (2019). Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res 28, 345–356. doi: 10.1007/s11136-018-2020-3

    • A checklist from Snyder et al (2019) is also available from the PROTEUS network. Learn more>>