Using Artificial Intelligence to Diagnose Kidney Cancer

Incidence of kidney tumors is rising, and with it, the need for an efficient way to distinguish between benign and malignant masses on computed tomography (CT) scans. Manual nephrometry scores have been proposed as a way to assess the complexity and aggression of renal masses. However, they involve significant effort by experts and have shown limited power to predict which tumors are malignant.

Using artificial intelligence (AI), scientists at Cleveland Clinic have developed a reliable way of automating the characterization of kidneys and kidney tumors. The key is semantic segmentation of kidneys and kidney tumors — linking each pixel in a CT scan to a specific label — and training a computer to recognize the images, in a method known as deep learning.

“It’s similar to facial recognition software in that it’s taking pixels from images and making mathematical representations of them by computer in the background,” says Christopher Weight, MD, Director of the Center for Urologic Oncology at Cleveland Clinic Cancer Center. “We manually annotated about 40,000 slices from renal CT scans by drawing boundaries and indicating what was surrounding anatomy and what was tumor. After seeing enough examples, our computer model was able to look at slices it hadn’t seen before and make predictions.”

The winning AI model

In 2019, they posted a challenge to other institutions to develop reliable AI algorithms that could distinguish kidneys and kidney tumors. For the first Kidney and Kidney Tumor Segmentation Challenge (KiTS 19), they shared 210 of the CT images from the Cleveland Clinic set. More than 100 teams used data from them to develop their own automated systems to predict the true segmentation masks on a test set of another 90 CT images for which the corresponding ground truth segmentations were kept private.

“In 2019, the winning AI model for identifying a kidney had a Dice score of 97, which is roughly the same as for a human. For identifying a tumor, the human score is about 92 and the algorithm scored 87,” says Dr. Weight. “We used that algorithm, which was created by a German team, to create a fully automated, reproducible nephrectomy score to predict the complexity of a tumor.”

An expanded challenge to researchers — KiTS 21 — is underway; for more information about it, visit https://kits21.kits-challenge.org/.

Testing the algorithm

To test the performance of the scoring system, the Cleveland Clinic researchers have compared data generated by it versus assessments of the same CT scans by human experts. They used the Dice similarity coefficient, a common validation tool for image segmentation algorithms that measures the similarity between two sets of data.

Now, they are on the brink of operationalizing an algorithm that will result in a fully automated nephrometry score, which may be the first of its kind. The process from annotation of the CT scans to this point has taken nearly four years.

So far, the algorithm has only been applied clinically on a retrospective basis. A paper detailing results with the automated nephrectomy score in a prospective cohort has been submitted for publication. “The area under the curve for the model was almost identical to human expert scores for predictions in domains including whether a kidney would require total or partial removal, was high- or low-grade, or was cancer or not,” says Dr. Weight.

The team is also working on validating the algorithm in other cohorts to ensure that it performs well in a variety of populations.

Expanding availability

The next step is a website through which a patient can upload a CT scan of a renal mass and get an AI reading of it. “It could be a valuable option for a second opinion,” Dr. Weight believes, “with confirmation by an expert that the algorithm has performed correctly in that particular case.”

Because AI is easily scalable, he also sees it as a way to make high-quality cancer care more widely available — but not to the exclusion of human experts. “Despite what some computer experts might think, AI will never replace clinicians. It will probably extend their capabilities,” says Dr. Weight. “A human can only read so many scans, but there is no end to the number that a computer can handle. The results also are much more reproducible.”