New Transplant Simulation Model Framework Addresses Limitations of Current Models

Organ transplant policymakers have relied heavily on simulations to predict the impact of potential policies on patient populations before implementation. The importance of accurate simulation has only increased as transplantation practices have become more complex, and policies are evolving quickly to adapt.

Simulation Allocation Models (SAMs) developed by the Scientific Registry of Transplant Recipients (SRTR) currently in use are built upon legacy data from previous models. But new technologies — ex vivo organ perfusion, in particular — as well as an evolving definition of what donor characteristics are considered “high-risk” have changed the equation. The historical data used in prior allocation simulation models doesn’t consider how these advancements affect the current and future landscapes. Relying on this historical data would limit policymaking as well as hamper researchers’ ability to explore and predict the potential effects of shifting donor or candidate populations realistically.

Recently published research in Journal of Heart and Lung Transplantation outlines the framework for a new organ allocation model simulation to address these limitations. The Computational Open-source Model for Evaluating Transplantation(COMET) model is an agent-based model simulating interactions of individual donors and candidates over time to project population outcomes.

“Organ transplantation can help improve a patient’s overall life expectancy and quality of life,” explains Maryam Valapour, MD, MPP, Director of Lung Transplant Outcomes at Cleveland Clinic’s Respiratory Institute and corresponding author of the paper. “But the logistics and prioritization involved with organ transplantation are complex. Allocation strategies have helped guide policy and clinical practice, but transplantation technology and practices have evolved rapidly over the past few years. The existing models cannot account for what kind of impact these changes in the supply of organs and risk trajectories for transplant candidates have had. We designed the COMET framework to examine system-level outcomes to better guide policy and clinical practice changes for organ transplant.”

The COMET framework

As an agent-based model, COMET simulates how individual entities that participate in the system, such as donors, candidates, and hospitals interact over time. Johnie Rose, MD, PhD, associate professor of medicine at Case Western Reserve University, first author and simulation scientist, notes that each agent has a set of fixed and/or time-varying properties. Fixed properties, such as candidate diagnosis or race, can be distributed across a population of agents according to real or hypothetical population characteristics. Time-varying properties consider how an agent’s status might change (e.g., accepting an organ, leaving the transplant list, or dying).

“The interactions between agents can produce complex system behaviors,” explains Dr. Rose. “The advantage of an agent-based model like COMET is that it’s able to model complexity with relative transparency and intuitiveness. In these types of models, researchers and policymakers can look at a complex situation and understand how and to what extent different factors are making an impact — and perhaps anticipate unintended consequences of a policy.”

COMET operates over fixed, 24-hour simulated time cycles, and its functionality is organized into interacting modules. Also, unlike prior systems, this team of researchers use hypothetical donors and candidates that are synthetically generated using data-driven probability models which are adaptable to account for ongoing or hypothetical donor/candidate population trends and evolving disease management. “Each day, the Donor Generation and Candidate Generation modules produce new synthetic donor and candidate populations, respectively,” says Dr. Valapour. “This is something that could not be done before and hampered our ability to accurately predict how policies would affect current or future patient populations.”

Validating the model

The first implementation of COMET was centered around lung transplantation. To validate COMET-Lung, the research group closely reproduced lung transplant outcomes for U.S. adults from 2018-2019 and also in the six months following the implementation of the Composite Allocation Score, a new allocation system implemented in the U.S. in early 2023.

The simulation was run 1,000 times, and the simulated (median [Interquartile Range, IQR]) versus observed outcomes for 2018-2019 were, respectively: 0.162 [0.157, 0.167] versus 0.170 waitlist deaths per waitlist year; 1.25 [1.23, 1.28] versus 1.26 transplants per waitlist year; 0.115 [0.112, 0.118] versus 0.113 post-transplant deaths per patient year; 202 [102, 377] versus 165 nautical miles travel distance.

The researchers also compared the TSAM-predicted and COMET-Lung-predicted distributions of lung transplant volume. This was done based on ABO blood type under both the Lung Allocation Score (LAS) and CAS systems to the observed distributions in the six months before and after the March 9, 2023, real-world transition from LAS to CAS. While TSAM predicted an increase in transplants for Type O candidates from 45.6% to 51.1% (+5.5%), COMET-Lung predicted a six-month decrease in transplants for type O candidates from 45.1% to 37.1% (-8.0%), similar to the actual observed decrease of 46.5% to 39.0% (-7.6%).

Advantages of COMET

In addition to the validity of the model, Dr. Rose says COMET can improve access to simulation research. “Prior models were all based on SRTR data, but obtaining access to sensitive SRTR data can be challenging,” he explains. “These barriers to access limit the policymaking and research communities’ abilities to explore the potential effects of shifting donor or candidate populations in a realistic clinical and policy future state. COMET is based on synthetic population data, so researchers without SRTR access can now conduct simulation research.”

Dr. Valapour believes that this is an important step in the field. “It’s also important to recognize that transplant centers differ in terms of their organ offer acceptance practices and to what extent they cooperate with other centers competing for the same organs. Since COMET associates transplant candidates with specific transplant centers, it can account for this. You can’t take a one-size-fits-all approach with organ transplantation, and the agent-based approach of COMET accounts for the discrepancies in this landscape.”

In the near term, Drs. Rose and Valapour and their team hope that COMET-Lung can be used to identify potential improvements and modifications to the new Composite Allocation Score-based system for lung allocation in the U.S. to help enhance equity and improve outcomes. The group believes that the framework can be applied to other transplant organs but doing so will require developing new organ-specific modules. “As the organ transplantation landscape continues to evolve, the importance of accurately forecasting the impacts of policy changes will only become more important,” says Dr. Valapour. “The use of synthetic populations, like what’s used in COMET, will hopefully lead to broader input on transplant policy and clinical practice changes that reflect a realistic future.”

This research was supported by an NIH grant (R01HL153175) where Dr. Valapour and Jarrod Dalton, PhD, Lerner Research Institute, serve as co-Principal Investigators.

Drs. Valapour and Rose recently sat down with The Journal of Heart and Lung Transplantation to further discuss their paper. Their podcast episode can be found here.