Researchers from Caltech and USC have developed the Surgical AI System (SAIS), an AI system that uses video data to assess the quality of surgical procedures and provide feedback to surgeons. SAIS assesses the performance of surgeons by evaluating individual discrete motions, like holding a needle or driving it through tissue. The researchers hope the system will help surgeons hone their skills and improve surgical outcomes. However, the system exhibited unintended bias in testing, and the research team is working to address this issue. The healthcare sector is showing a growing interest in using AI to improve surgical care.
Researchers from the California Institute of Technology (Caltech) and the University of Southern California (USC) Keck School of Medicine have created a new technology called the Surgical AI System (SAIS), which uses artificial intelligence (AI) to evaluate the quality of surgical procedures. The system is designed to provide surgeons with feedback on their work and help them improve their skills. SAIS uses videos of surgical procedures to identify the type of surgery being performed and the quality of the surgeon’s performance.
To train the AI system, the researchers used a huge collection of annotated operation films and related data. SAIS evaluates each distinct move a surgeon makes, such as grasping a needle, inserting it into a tissue, and taking it out of that tissue, to determine how well they perform as surgeons. After training, video footage from several hospitals and procedures was used to validate the technology. The researchers also designed SAIS to justify its skill assessment, similar to how an experienced surgeon might mentor a newer surgeon, but without some of the challenges that come with surgical mentorship.
The desire to help patients have better surgical results led to the creation of SAIS. The researchers wanted to explore how AI could help make human surgeons better and more effective. Anima Anandkumar, Ph.D., Bren Professor of Computing and Mathematical Sciences at Caltech and senior author of the studies, stated that “in high-stakes environments such as robotic surgery, it is not realistic for AI to replace human surgeons in the short term.” Instead, SAIS aims to provide surgeons with actionable feedback that can help them improve their skills.
“Human-derived surgical input is not currently objective or scalable,” according to Andrew Hung, MD, a urologist with Keck Medical of USC and associate professor of urology at Keck School of Medicine of USC. SAIS gives surgeons a significant opportunity to receive scalable and objective feedback thanks to AI-derived input. The research team shows that SAIS’ suggestions are consistent, accurate, and scalable by educating surgeons about their degree of skill and giving comments on their justification by referring to particular video clips.
The researchers also found that reliable AI-based explanations can pave the way for providing feedback when peer surgeons are not immediately available. If no seasoned surgeons are available to provide feedback, the system can be employed as a fallback.
However, SAIS exhibited unintended bias early on in testing. The tool would sometimes rate surgeons as more or less skilled than their experience indicated based only on an analysis of the surgeons’ overall movements. To address this, the researchers guided SAIS to narrow its focus to only the pertinent parts of the video. This reduced the AI’s bias but did not fully eliminate it. The research team is currently working to address the remaining bias in the tool.
The healthcare sector has shown a growing interest in using AI to improve surgical care. Earlier this month, the Advocate Aurora Research Institute announced that it will leverage surgical data analytics company KelaHealth’s Surgical Intelligence Platform to use AI and machine learning to determine the efficacy of robotic surgical techniques and enhance surgical care delivery. This trend reflects a growing recognition that AI has the potential to improve the quality of surgical care and help human surgeons become more effective.
