How Much Change Is Enough to Matter?

By Sandra D. Griffith, PhD; Deborah M. Miller, PhD; and Irene L. Katzan, MD

With more than 150 patient health status questionnaires in use across Cleveland Clinic and similar health systems, simply knowing the difference in scores between two time points does not provide sufficient information. Consider a five-point improvement on a scale between a patient’s initial and follow-up visits. Conclusions about the importance of this change will differ depending on whether the scale ranges from 1 to 20 or 1 to 100, as well as on the distribution of patient scores and the content of the questions.

How do we filter out the noise to find true changes in patient health rather than simply artifacts of the measurement tool? Even if a change is statistically meaningful, is it noticeable or important to the patient or provider? These questions underlie the concept of minimal clinically important difference (MCID), or the smallest change in a health status measure (HSM) required for a clinically meaningful difference.

As Cleveland Clinic’s Neurological Institute grapples with these questions, we increasingly draw on our Knowledge Program Care Path (KPCP) data collection system, a scalable platform developed by Cleveland Clinic for collection of data from both patient and provider. Patient-reported data obtained through the KPCP are immediately available within the electronic health record for use in clinical encounters. These data are also stored in a database and available for aggregate analyses. Now in its seventh year, the KPCP will soon reach the milestone of collecting HSM data from 1 million patient visits. Armed with an unprecedented number of measurements providing a comprehensive view of patient health, clinicians and researchers face the challenge of interpreting these data to describe patient populations, aid medical decision-making and evaluate care effectiveness. In addition to defining a patient’s health status at a single visit, the KPCP allows us to follow patients over time to provide crucial information on the progression of their health.

This article summarizes the essentials of MCID and how the KPCP is helping us evaluate, refine and apply various approaches to MCID assessment.

Distributional Approaches to MCID

Distributional approaches consider the statistical properties of a given instrument for measurement. Through the KPCP, we are using the large numbers of patient visits with HSM data collected during routine clinical care to better understand the measurement error inherent in HSM instruments. We are conducting studies to estimate the minimal detectable change (MDC) threshold of several HSMs, derived from the standard error of the measurements. If a difference in an HSM is above the MDC threshold, we can proceed with confidence that it is likely a true change and not simply noise from the measurement scale. However, we still need to determine whether the change is clinically important.

Anchor-Based Approaches to MCID

Clinical criteria for MCID rely on anchor-based methods, where an external criterion indicates clinically important changes. The external criterion must be valid for determining the change of interest and have good correlation with the target instrument. The KPCP collects global rating of change (GRC) questionnaires in tandem with some HSMs. These questionnaires ask patients or providers to rate the difference in the patient’s status with respect to a specific health domain (e.g., pain, depression, functional status) since the patient’s previous visit. External anchors, such as the GRC, provide us with metadata about the scales themselves, allowing detection of how much change is required for patients or providers to take notice. We are conducting analyses using KPCP data to link small, moderate or large changes, as described by GRC scales, to numeric differences on HSM scales.

Applying a Mixed-Methods Approach

We also are pursuing focused research studies to generate evidence that may not be attainable using only data collected through routine clinical care. With this goal in mind, we recently undertook a mixed-methods approach for establishing MCID by anchoring expert judgment of clinical severity to patients’ perceptions of the severity of their HSM scores. We implemented this approach in our multiple sclerosis clinic within the item response theory (IRT)-based HSM platform Neuro-QOL, using four of the assessment domains (upper extremity, mobility, sleep and fatigue).

The first phase of this implementation involved a qualitative process adapted from educational standard setting, which differentiates examinees from either passing or failing a given test. Two expert panels, one of patients and another of clinicians, were presented with multiple clinical vignettes consisting of sets of five items and the corresponding IRT-predicted responses for each item at successively more severe levels of the Neuro-QOL HSMs (0.5 standard deviations apart). Working individually and then as a group, panelists designated transition points between the vignettes where they thought the threshold was between “no problems,” “mild problems,” “moderate problems” and “severe problems” for each HSM.

During the next phase, which is quantitative, we will validate the expert anchors by administering the four Neuro-QOL HSMs through the KPCP to 1,000 patients with multiple sclerosis and asking them to rank the severity of their symptoms between “no problems,” “mild problems,” “moderate problems” and “severe problems.” Congruence between the expert anchors and patient self-report will indicate whether we have established meaningful transition points for each of these measures.

Implementing MCID in the Electronic Health Record

In our role as a learning healthcare system, we plan to implement MCID findings based on evidence generated at Cleveland Clinic and elsewhere. As a first step, we are developing strategies to communicate changes in HSM results to providers in the context of measure-specific MCIDs. This will provide clinicians at the point of care with visual information to help them determine the significance of changes in patient health over time. Figure 1 shows a potential display to be seen by providers in the electronic health record, giving a visual representation of the patient’s trajectory and signifying clinically important changes with an asterisk. Once implemented, we will evaluate its effectiveness and adjust strategies, as necessary, to ensure continual improvement.

FIGURE 1.Example of a potential provider display that indicates whether a clinically significant change in health status measure score has occurred since the patient’s previous visit.

Dr. Griffith is an assistant staff member in the Department of Quantitative Health Sciences and the Neurological Institute’s Center for Outcomes Research and Evaluation.

Dr. Miller is Associate Professor, Mellen Center for Multiple Sclerosis Treatment and Research, and the Neurological Institute’s Center for Outcomes Research and Evaluation.

Dr. Katzan is Director of the Neurological Institute’s Center for Outcomes Research and Evaluation and Director of the Knowledge Program Care Path.