AI-Generated R.E.N.A.L.+ Score Surpasses Traditional Methods for Prediction of Oncologic Outcomes

An artificial intelligence (AI)-generated R.E.N.A.L.+ score — developed by a team of researchers at Cleveland Clinic — proves superior to the traditional scoring system for the prediction of renal oncologic outcomes. These findings were recently presented during the American Urological Association Annual Meeting, held April 28 to May 1 in Chicago.

The R.E.N.A.L. nephrometry score, which was designed to standardize the assessment of renal tumors, is based on the following five features: radius (R), exophytic/endophytic (E), nearness to collecting system or sinus (N), anterior/posterior (A), and location relative to the polar lines (L).

While this scoring system has been validated as a predictive factor of oncologic outcomes, there has been limited adoption worldwide, notes study author Nour Abdallah, MD, a research fellow at the Glickman Urological and Kidney Institute, Cleveland Clinic. “We believe this is due in part to interobserver variability and also the substantial time and expertise it requires from the clinician in order to generate the score manually.”

Another inherent limitation of the traditional R.E.N.A.L. score is converting continuous variables into ordinal variables (e.g., a renal mass having a diameter of < 4 gets 1 point, 4-7 cm gets 2 points and > 7 gets three points). Although this was intended to simplify and standardize the score generation, according to Dr. Abdallah, valuable information might be lost when categorizing continuous variables, which could potentially reduce the score’s predictive accuracy.

To address these issues, Dr. Abdallah, alongside senior author Christopher J. Weight, MD, Center Director Urologic Oncology, Cleveland Clinic, and colleagues, initiated a study to determine if artificial intelligence could help by generating a fully automated model not bound by simplicity or lack of reproducibility.

The Cleveland Clinic team then tested a new AI-R.E.N.A.L.+ score (AI+ score) with continuous rather than ordinal components to see if it could surpass the predictive abilities of the traditional R.E.N.A.L. nephrometry score generated by human experts.

Study design

This retrospective study included 300 consecutive patients with preoperative CT scans that show suspected renal cancer from 2010-2018. Three clinicians tabulated traditional R.E.N.A.L. nephrometry scores for these patients.

Researchers used a deep neural network approach to automatically generate kidney segmentation masks of the renal parenchyma and tumor. Score components were then automatically estimated as ordinal and continuous variables via geometric algorithms based on the kidney segmentations.

The AI+ score was generated through multivariate logistic regression of continuous R.E.N.A.L. components, according to Dr. Abdallah. The research team compared the predictive utility between the AI+ score using continuous variables, the AI-generated R.E.N.A.L. score using ordinal variables, and the traditional human-generated R.E.N.A.L. score. The relative importance of AI+ score components was also assessed.

Key findings

The median age of study participants was 60 years old, and 40% were female. Study authors noted a median tumor size of 4.2 cm, and malignancy was found in 92 percent. Of those, 27%, 37% and 23% were high-stage, high-grade, and had necrosis, respectively.

Data showed that the AI+ score had superior predictive ability over the AI and traditional human-generated R.E.N.A.L. scores for all oncologic outcomes, including malignancy, high-stage, high-grade, and pathologic tumor necrosis, reports Dr. Abdallah. “The AI+ score also had superior performance in predicting partial versus radical nephrectomy when compared to the AI and the human-generated scores, both using ordinal variables.”

Another interesting aspect of their work, according to Dr. Abdallah, is the examination of the individual components—R.E.N.A.L.—of the AI+ score. “We found that the “R,” or maximal tumor diameter, was a statistically significant predictor of high-stage and high-grade disease as well as the presence of tumor necrosis.”

Significance, next steps

The R.E.N.A.L. nephrometry score has been proven to be a useful tool, but it isn’t being used to its full potential, notes Dr. Abdallah. “We hope that transitioning from human to AI scoring, and also from ordinal to continuous scoring, can help increase its use. Wider adoption of R.E.N.A.L. scoring is important at both the clinical and research levels. The AI+ score will reduce the time needed from the clinicians, making it much easier for them to adopt this score in their clinical practice.”

The AI+ score also has the potential to increase the efficiency, uniformity, and quality of kidney cancer research by eliminating interobserver variability in interpretation and measurement and potentially generating automated scores for large-volume data sets.

“The AI-R.E.N.A.L.+ score is a significant advance over existing clinically validated scoring systems. This AI+ model is reproducible, scalable, fully automated and demonstrated superior predictive accuracy to prevailing models.” Dr. Weight said. “This advance demonstrates how AI can help us improve upon renal outcome predictions, but we’re just scratching the surface.

“AI and computer vision has the capability to quantify kidneys and tumors on preoperative scans in a way that humans experts can’t,” he concludes. “With continued research you could envision a day when all CT scans could have an automated AI read to augment the radiologist read giving patients and their clinicians the most accurate information to make quality shared decisions.”