HoloLens to Assess the Resectability of Ovarian Cancer

Just as the introduction of robotic surgery has revolutionized minimally invasive surgery (MIS), the next iteration of surgical advancement may be augmented reality (AR). This technology augments the surgeon’s ability to view, understand and manipulate data about the patient in incredible ways. Led by Karl West, MS, members of the Department of Biomedical Engineering (Jeff Yanof, PhD, Sara Al-Nimer, MS, and Aydan Hanlon, BS) have developed a non-invasive, real-time, AR platform that enables 3-D anatomical computer models to be viewed through Microsoft’s HoloLens AR device, allowing the surgeon to virtually see inside their patients.

“When we started experimenting with the technology, we realized we could take patient-specific images from a CT or MRI data set, do a 3D reconstruction of the images, and project them onto the headset’s lenses,” West says. “HoloLens allows you to not only see the real environment around you but also augments it with additional information projected onto the lenses.”

As part of this work, Robert DeBernarndo, MD, Director of the Peritoneal Malignancy Program in Cleveland Clinic’s Department of Gynecologic Oncology, and Ms. Al-Nimer are investigating the feasibility of AR in a cohort of patients with advanced ovarian cancer. Patients are currently being enrolled in the study and evaluated for surgical resectability. Dr. DeBernardo received a Caregiver Catalyst Award, sponsored by donations to the Cleveland Clinic Annual Fund, to conduct this research.

Building Virtual 3D Patient Models

In this application, bioengineers will use preoperative, 2-D patient imaging to create patient-specific, virtual 3-D models. “The images a surgeon interacts with through the headset are based on real patient data,” notes Mr. West. “Once a virtual model is constructed, we mark fixed points in the patient’s anatomy, such as the spine, and anchor the model to the patient. We can look down into the patient’s body and see views never before possible.”

“When viewed through the HoloLens,” Dr. DeBernardo states, “surgeons will gain an amazing perspective. We expect to be able to view and score a patient’s tumor(s) based on size, the presence of metastases, and location compared to other organs, thus determining if her cancer can be completely removed.”

This feasibility study is the next step of the development and testing process, and should enable the research team to determine how AR-assessment compares to the current model of MIS evaluation and the standard-of-care conventional surgery. “It is conceivable that AR can predict surgical resection better than either MIS or conventional surgery without having to bring patients into the operating room,” notes Dr. DeBernarndo.

Ovarian cancer, although uncommon, is one of the leading causes of cancer death in women. Typically it is discovered once the cancer has spread throughout the abdomen. Survival is directly correlated with the amount of residual cancer remaining following surgery. “It’s hard to believe,” Dr. DeBernardo says, “but in a small subset of patients, when women with advanced ovarian cancer have optimal surgery that completely removes all residual cancer, they can live more than 10 years.”

In contrast, those patients whose surgical outcomes are less than optimum may live 18-24 months. Predicting which patients can undergo optimal surgery—and when—is paramount in providing the best care for women with this disease.

Currently, preoperative evaluation of disease burden is limited. Surgeons can review static 2-D images, such as CT, MR or PET, and correlate these with exam findings. Only during open surgery can the true extent of the disease be appreciated and the determination be made as to whether or not the tumors can be completely removed. In an effort to minimize unnecessary surgery, our team has an ongoing, IRB-approved Cleveland Clinic study to investigate how MIS predicts disease burden compared to conventional open surgery. Regardless of approach, these patients still require assessment in the OR along with the inherent risks of surgery.

AR in the OR

“With this technology,” says Dr. DeBernardo, “we can take reality and make it even better. Wearing AR goggles, the surgeon is able to literally walk around the patient to visualize anatomy from any perspective and digitally subtract or enhance organs at will. Once perfected, this technology will revolutionize surgery. However, much work needs to be done to bring this technology into the operating theater for this application.

“We have the somewhat unique opportunity here at the Cleveland Clinic to have a talented team of biomedical engineers capable of developing and refining their platform for a broad range of medical applications, ongoing clinical trials with patients that will see real-time benefit, as well as experienced surgeons and radiologists with the passion and dedication to move this to clinical practice. If successful, this may serve as a spring board for further research, funding and development of AR surgery.”

Further refinement will be necessary. Currently, the process of importing imaging data and creating patient-specific 3-D digital models for this application is labor intensive. Throughout this study, the team will seek to better identify ways to generate these digital models and automate the process.

The information learned using AR in this patient cohort may serve as the foundation to expand the application of this exciting technology to other surgical disciplines. “In the near term,” Dr. DeBernardo notes, “the use of AR may improve the care of women with ovarian cancer, assuming the study shows that it is as good as or better than MIS or conventional surgery at predicting disease burden. More importantly though, by advancing the development of this platform here at Cleveland Clinic, we will be on the leading edge of this exciting technology.”