Congenital Heart Care: Performance Assessment Among Institutions and Surgeons

By Tara Karamlou, MD, MSc

One of the more vexing clinical questions facing surgical specialties today is performance assessment among institutions and surgeons. Many outcomes and quality measures in medical specialties and adult cardiac surgery have validity because they apply to large and fairly homogenous populations. In congenital heart disease, however, we tend to have much smaller numbers of patients, with huge heterogeneity in terms of both anatomy and surgical approach. Thus, when we try to synthesize the data that is available, it is easy to make missteps if we are not careful in how we compile and, more importantly, adjust the data for the different nuances we face in this field.

Tetralogy of Fallot (a defect which is fairly common), for example, is not just one anomaly – there are shades of tetralogy that include the entire anatomic spectrum. We have to carefully level the playing field when comparing outcomes among centers, surgeons and patients. In my career, I have spent a lot of time analyzing the data, seeking ways that we can better adjust for confounding factors and benchmark outcomes given all of this variability. There are several issues that add to the complexities of performance assessments among institutions and surgeons, including:

• Institution-specific and surgeon-specific factors.
• Institution-specific and surgeon-factors may be surrogates for one another and may interact with one another.
• Within-institution and within-patient factors across time are correlated and must be analyzed with mixed regression models that adjust for so-called random effects.
• An institutional or surgeon preference or bias for a given management algorithm may influence outcomes.

Volume always has a role

There is no question that centers, surgeons and teams that work together frequently have better outcomes. However, volume is an imperfect measure of performance. We have to understand that the team – its dynamics and the characteristics of team members – can influence how important (relatively) volume is in that equation. The overall feeling is that volume is always part of the equation, but with modifying variables.

Volume-related factors can influence outcomes in different ways. Consider, for example, two complex heart lesions. If you take a baby who has one pumping chamber and subject that patient to a complex operation, the center’s volume is more of a factor in that outcome — because it’s a multidisciplinary approach — than volume of procedures by the individual surgeon. In contrast, in a patient who has transposition of the great arteries who requires an arterial switch — a highly technical procedure — surgeon volume is much more important than the center volume. Volume matters in both cases, but it matters less in certain procedures.

Prognosticating risk: the devil is in the details

As clinicians, it is essential that we engage patients in a transparent and sound dialogue about the benefits and risks of procedures to empower them to give informed consent. Risk stratification allows us to tell patients what an expected outcome may be for a like-child with that diagnosis having surgery at that center. Stratifying risk gives us the benefit of being able to benchmark programs, surgeons and lesions according to a level playing field; it tells a patient broadly what may happen all things being equal.

However, stratifying does not necessarily give us information about that individual patient. It does not necessarily tell us for that particular patient, based on the patient’s own individual risk factors, what the outcome may be. For example, I can tell a family seeking treatment for tetralogy of Fallot that the risk of a particular complication is 10%. But when it comes to individual-level probability, it’s either 0% or 100%. This is part of the genesis and importance of precision medicine, which involves individual-level prediction of outcome.

At this point in time, risk stratification is the best we can do with the data we have as a fairly young specialty. Our ultimate goal is to compile the data needed in order to make good risk adjustment over the lifetime of patients, though we are likely decades away.

Although imperfect, risk stratification does allow you to compare programs that have similar case mix. If a small volume program performs low complexity surgeries, then it’s not accurate to tell a patient going to that small center that they would do just as well as they would if they had gone to a larger volume center that may be doing a highly complex spectrum of operations. In other words, you have to compare Cleveland Clinic Children’s to a center that’s like Cleveland Clinic Children’s. This is difficult as there are very few centers that like Cleveland Clinic Children’s in terms of the level of complexity we see. As a result, some of these comparisons become a little bit unfair. Additionally, they may give people incentives to game the system a bit by deflecting highly complicated patients away from their centers. However we measure risk, and discuss potential outcome scenarios with our patients, we must be sure to serve our patients best interests first and foremost.