Realizing Next-Generation Motor Assessment Through Massive Tech-Enabled Data Capture

By Jay Alberts, PhD, and Robert Bermel, MD

A few years ago, Cleveland Clinic’s Neurological Institute embarked on an initiative to develop, implement and test predictive care models by collecting objective, quantitative outcomes among patient populations with various neurologic conditions. The aim was to build large repositories of outcomes data through technology-enabled collection of common data elements across these conditions and more fully unleash the possibilities of data-guided neurologic care.

Those efforts have begun to bear fruit, most notably in the areas of multiple sclerosis (MS) and Parkinson disease (PD). This article shares updates on efforts in those areas and looks ahead to next steps.

Technology-enabled data capture and the waiting room of the future

Central to our efforts has been the development of technology-enabled tools for comprehensive and efficient quantitative assessment of neurological function. Such tools have been elusive in clinical practice, but we have made important progress with deployment of the Cleveland Clinic-developed Multiple Sclerosis Performance Test (MSPT),1 a battery of quantitative patient-reported outcome measures (PROMs) and neuroperformance tests (NPTs) administered using tablet (iPad)-based applications and supporting hardware.

As recently reported in a proof-of-principle analysis in Neurology: Clinical Practice,2 our use of the MSPT demonstrates that technology-enabled data capture can be integrated into routine clinical practice in a way that yields a wealth of comprehensive, standardized data for use in patient care and clinical research.

How it works. The analysis examined implementation of the MSPT into clinical care at Cleveland Clinic’s Mellen Center for Multiple Sclerosis Treatment and Research from September 2015 through May 2018.

Implementation involved reimagining the process for follow-up visits in our MS clinic. Under the new model, patients check in for their visit and undergo intake by a medical assistant, which includes an explanation of the purpose of the MSPT and how to complete it. Patients then complete the MSPT on an iPad by following visual and audio instructions, with help from the medical assistant if needed. A dedicated space was created for these assessments between the clinic’s waiting area and the corridor of exam rooms (Figure 1) so that up to eight patients can take the MSPT at once.

Figure 1. Dedicated space for MSPT completion in the “waiting room of the future.”

MSPT completion typically requires 30 to 40 minutes, so patients are instructed to arrive 50 minutes before their scheduled appointment with a clinician. After their first completion of the MSPT, patients are offered the option to take several of its PROM components at home up to three days before their next appointment. Development of an iPhone version of the NPT components of the MSPT is now underway to further expand options for remote completion.

As the MSPT is completed, raw data are automatically scored and uploaded to the patient’s electronic health record (EHR), to an internal database and, with patient consent, to a multicenter database for population-level research analysis. Real-time upload to the EHR makes the data available to the patient’s clinician during their visit that same day, allowing the data to be immediately reviewed with the patient and to inform clinical decision-making without delay.

The process was codesigned with patients, and implementation was gradual over the 33-month study period, with feedback invited from all stakeholders on an ongoing basis and continuous efforts to optimize efficiency. Implementation and monitoring were carried out by a multidisciplinary team of neurologists, research managers, nurse practitioners, medical assistants, biomedical engineers and statisticians.

What is captured. In addition to patient demographics and disease and treatment characteristics, the MSPT captures the measures listed in Table 1. The NPTs in the MSPT — which measure walking speed, manual dexterity, processing speed and visual contrast sensitivity — have been validated to perform as well as or better than the standard technician-administered counterpart tests from which they were derived.

What we found. Longitudinal data were prospectively collected and analyzed from 4,199 patients with MS across 8,022 follow-up visits during the study period. As detailed in our published analysis,2 we observed high completion rates for the MSPT modules overall. Among the primary findings:

• By the final five months of the study period, walking speed was tested in 89% of patients and manual dexterity in 95%, compared with 74% and 64%, respectively, before MSPT implementation.
• Data capture for processing speed and visual contrast sensitivity rose from 0% before implementation to 78% and 37%, respectively, by the study’s final five months.
• Four PROMs were completed by 41% to 98% of patients in the study’s final five months, whereas only one of the four PROMs was completed by any patients (at a rate of 71%) prior to implementation.

Completion rates and time required to take the NPT modules improved with successive visits. Younger age and lower levels of disability correlated with shorter completion times and higher completion rates, but there were no significant demographic differences between completers and noncompleters, suggesting good applicability of the MSPT to a diversity of patients.

What it means for care and practice. This approach to technology-enabled data capture represents a deliberate focus on the data points most relevant to the care of a given population — in this case, people with MS — and a deliberate effort to efficiently collect them for as many patients as possible.

Our proof-of-principle study shows this approach is feasible in a large clinic setting and yields a consistent and standardized range of data that promises more accurate phenotyping of individual patients and populations. The approach offers a number of additional benefits, including:

• Improved clinical workflow through standardized data acquisition and documentation
• More top-of-license practice time for clinicians, who are able to focus on interpreting and explaining PROM and NPT data and on integrating the data to inform management decisions
• Reduced documentation time through automatic integration of PROM and NPT data into visit notes, promising potential reductions in clinician burnout
• An efficient model for populating the EHR with structured longitudinal data, with potential benefits for enhancing disease monitoring over time at the patient level and for aggregation across centers to answer population-level research questions

These are benefits that reasonably can be expected in additional neurologic disease states, so an adaptation of the MSPT will soon be introduced to the Parkinson disease clinic in Cleveland Clinic’s Center for Neurological Restoration, as many of the same PROM and NPT assessments will apply. Indeed, this model holds promise for guiding management of any chronic neurologic illness where structured and standardized longitudinal motor and functional assessment proves valuable. We look forward to exploring more applications in the months and years ahead.

Future state: predictive analytics. As the precisely chosen data points from the MSPT accumulate, they are building an ever-growing database that will allow MS clinicians at Cleveland Clinic and beyond to identify the clinical state of their patients relative to others presenting with similar symptoms and disability. The clinician may then determine how the patient is likely to respond to various interventions (see Figure 2), the rate of disease progression and other fundamental aspects of management.

Work is also beginning on the development of artificial intelligence (AI) algorithms that will prompt clinicians, based on a given patient’s MSPT results in the context of giant AI-enhanced datasets, to consider evaluating for a specific finding or making a therapeutic change. The ultimate result will be a shift from an episodic care model to a patient-specific model in which clinical decisions are guided by outcomes from large datasets rather than clinical dogma or the treating clinician’s experience.

Figure 2. Plots depicting how care can be transformed through data-driven predictive analytics for individual patients. The top plot represents the traditional model of care for many patients, with variable outcomes (shown on the y-axis) as different treatments and doses are tried based on response to subjective clinical assessments. The result is time-consuming and often inefficient care that does not optimally serve the patient. The middle plot illustrates the initial value of collecting objective functional outcomes in combination with treatment protocols in a relatively large set of patients. Variability in outcomes over time has been reduced, but the overlap in confidence fields around the plots for Treatments A and B shows remaining uncertainty about which treatment is best for a patient at a given functional level. The bottom plot shows how clearly distinct outcomes between Treatments A and B might be predicted for a given patient early in management once predictive treatment models are well-populated through large-scale data collection of objective outcomes from tools such as the MSPT. As the size of the database grows, the predictive modeling becomes more patient-specific.

Use case in one aspect of MS neuroperformance

We are also employing one of the NPT components of the MSPT — an iPad adaptation of the Nine-Hole Peg Test that we call the Manual Dexterity Test (Figure 3) — to enable more-detailed assessment of upper extremity function when combined with advanced mathematical modeling. A preliminary report of how this assessment may better inform MS care was presented by Cleveland Clinic Mellen Center staff at the ECTRIMS 2019 Congress.3

Figure 3. The Manual Dexterity Test adapts the Nine-Hole Peg Test for administration on the iPad to enable more nuanced assessment and quantification of dexterity.

The conventional Nine-Hole Peg Test (NHPT) is limited to measuring a single outcome — trial duration, or total time to complete. To expand the utility of this test of upper extremity function, we developed an advanced mathematical model using data derived from use of the Manual Dexterity Test (MDT) across more than 6,500 patient visits along with closure coefficients derived from motion-capture analysis. The goal was to assess a number of novel biomechanical outcomes to evaluate specific mechanisms of fine and gross motor control, such as (among others):

• Insertion time
• Removal time
• Bulk movement time
• Average and maximum speed
• Number, type and duration of “fumbles,” or difficulties in removing and inserting pegs

We collected data (as part of MSPT data capture) from 4,471 patients with MS who took the MDT from September 2015 to February 2019. We used Spearman correlation coefficients to determine the relationship between conventional NHPT and novel MDT outcomes with two PROMs — Patient Determined Disease Steps (PDDS) and the upper extremity function measure of the Neuro-QoL instrument (Neuro-QoL UE).

Analysis showed that the conventional NHPT outcome of trial duration correlated moderately with the PDDS and Neuro-QoL UE. It also showed that the novel outcomes with the highest correlations with these PROMs were:

• Bulk movement average velocity
• Bulk distance moved
• Total good movement time
• Total movement time

These findings demonstrate that technology-enabled enhancement of the NHPT, as represented by the MDT, allows for more-detailed evaluation of upper extremity function in MS. Our next steps are to longitudinally examine the novel biomechanical measures with the highest correlations with PROMs to pinpoint those with the greatest sensitivity to change and correlation with specific aspects of function as measured by the Neuro-QoL UE.

This work is a good example of how generation of very well-characterized patient data promises more-personalized care and — when ultimately paired with large sets of consistently captured data — the potential for patient care to benefit from predictive analytics.

Quest for personalized exercise prescriptions for PD

Our Neurological Institute is pursuing similar work in PD with support from a five-year, $3 million National Institutes of Health grant. The project has two broad objectives:

• To determine the disease-modifying characteristics of a high-intensity exercise program in people with PD
• To identify exercise performance variables and patient characteristics that promote exercise-related improvement in PD symptoms

The project is premised on the hypothesis that high-intensity aerobic exercise will slow disease progression in PD, and it builds on findings from Cleveland Clinic’s recently completed CYCLE trial (Cyclical Lower-extremity Exercise for Parkinson’s).4 That 100-subject study showed that eight weeks of aerobic exercise on a stationary recumbent bicycle in a laboratory setting significantly improved global motor function and specific aspects of gait and cognitive function in individuals with PD.

For the new study, 250 individuals with PD are being randomized to 12 months of either high-intensity home exercise using the protocol from the CYCLE trial or usual and customary care. Participants will have their motor and nonmotor function systematically quantified via the PD Performance Test, a collection of validated Cleveland Clinic-developed iPad applications for biomechanical assessment similar to those in the MSPT.

The primary outcome will be the change in Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) III score from enrollment to evaluation after 12 months off medications. A secondary analysis will employ linear discriminant analysis to determine which exercise performance variables and patient-specific factors are associated with the greatest improvement in PD symptoms after 12 months of exercise. Classification analysis will be used to identify patient phenotypes in terms of MDS-UPDRS III score changes — i.e., responder, neutral or nonresponder. Figure 4 presents an example of a hypothesized result of the classification analysis based on preliminary analysis of CYCLE trial data.

Figure 4. Plot of potential phenotyping of PD patients’ response to exercise based on exercise performance and demographic variables. We hypothesize that responders (green dots and ellipse) will be patients with high heart rate compliance, high cadence and potentially shorter disease duration. Individuals who exercise at a slightly lower aerobic intensity or lower cadence may not demonstrate as substantial effects in terms of change in MDS-UPDRS III score and would be classified as having a neutral response (yellow dots and ellipse). Patients who exercise below the specified heart rate and at a low cadence are hypothesized to not respond to exercise and will be classified as nonresponders (red dots and ellipse).

This analysis will ultimately result in an equation consisting of a linear combination of demographic variables, disease variables, and exercise compliance and performance data that can accurately classify new patients into one of the three response groups based on patient-specific information. The analysis will be essential in developing a patient-specific “exercise prescription” — addressing exercise type, duration, frequency, etc. — based on an individual participant’s demographic and disease variables.

The ideal result will be an accurate and powerful predictive model that fundamentally alters how exercise is used and personalized in the treatment of PD, enabling a shift from general recommendations to patient-specific exercise prescriptions with the potential for reliable prediction of outcomes.

References

1. Rhodes JK, Schindler D, Rao SM, et al. Multiple Sclerosis Performance Test: technical development and usability. Adv Ther. 2019;36(7):1741-1755.
2. Macaron G, Moss BO, Li H, et al. Technology-enabled assessments to enhance multiple sclerosis clinical care and research. Neurol Clin Pract. 2019 Sept 6 [Epub ahead of print] doi:10.1212/CPJ.0000000000000710.
3. McGinley M, Qu J, Felix C, et al. Novel biomechanical outcomes derived from a technology-enabled adaptation of the 9-hole peg test. Abstract P792 at 35th Congress of the European Committee for Treatment and Research in Multiple Sclerosis (ECTRIMS 2019). Mult Scler J. 2019;25(S2):405.
4. The Cyclical Lower-extremity Exercise for Parkinson’s Trial (CYCLE). NCT01636297. Profile (with results) on ClinicalTrials.gov. Accessed Sept. 27, 2019.

Dr. Alberts is Vice Chair of Innovation in Cleveland Clinic’s Neurological Institute and staff in the Department of Biomedical Engineering.

Dr. Bermel is Director of the Mellen Center for Multiple Sclerosis Treatment and Research.