Researchers from Caltech and USC have developed the Surgical AI System (SAIS), an AI system that evaluates surgical performance. SAIS uses videos of surgical procedures to assess surgeon performance by evaluating individual, discrete motions. It can reveal the kind of surgery being performed as well as the caliber of the operation. Although the system has promise, the researchers found unexpected bias, which needs to be removed before clinicians can use it. The development of SAIS reflects a growing interest in using AI to improve surgical care.
The Surgical AI System (SAIS), developed by researchers from the California Institute of Technology (Caltech) and the University of Southern California (USC) Keck School of Medicine, is a new artificial intelligence (AI) system designed to provide surgeons with feedback on the quality of their work and which of their skills need improvement. The technology reviews surgical recordings to determine the process being carried out and the level of expertise the doctor used to carry it out. SAIS evaluates each discrete motion a surgeon makes, such as grasping a needle, inserting it into the tissue, and taking it out of that tissue, to determine how well they perform as surgeons.
According to Anima Anandkumar, Ph.D., Bren Professor of Computing and Mathematical Sciences at Caltech and senior author of the studies, “In high-stakes environments such as robotic surgery, it is not realistic for AI to replace human surgeons in the short term.” Instead, we asked how AI can safely improve surgical outcomes for patients; hence, our focus is on making human surgeons better and more effective through AI. To do this, the researchers used a significant amount of annotated surgery films and related data to train SAIS. Following training, the tool was validated using video data from multiple hospitals and procedure types. The researchers also designed SAIS to be able to justify its skill assessment, similar to how an experienced surgeon might if they were mentoring a newer surgeon, but without some of the challenges that come with surgical mentorship.
The SAIS system has a lot of potential for giving surgeons unbiased feedback on how they performed during surgery. AI-derived surgical feedback, such as that provided by the SAIS system, presents a significant opportunity to give surgeons actionable feedback because human-derived surgical feedback is neither objective nor scalable. The research team shows that SAIS’ suggestions are consistent, accurate, and scalable by educating surgeons about their degree of skill and giving comments on their justification by referring to particular video clips. When peer surgeons are not immediately available, trustworthy AI-based explanations can open the door for giving input.
However, the SAIS system also exhibits unintended bias early on in testing, in which the tool would sometimes rate surgeons as more or less skilled than their experience indicated based only on an analysis of the surgeons’ overall movements. The researchers addressed this by guiding SAIS to narrow its focus to only the pertinent parts of the video. This reduced the AI’s bias but did not fully eliminate it. The research team is currently working to address the remaining bias in the tool.
These initiatives are a reflection of a broader desire in the healthcare industry to use AI to enhance surgical treatment. For instance, the Advocate Aurora Research Institute revealed earlier this month that it would use the Surgical Intelligence Platform from KelaHealth, a startup that analyzes surgical data, to apply AI and machine learning (ML) to improve the delivery of surgical care.
The SAIS system has the potential to raise the standard of surgical care by giving doctors unbiased feedback on how well they perform during surgery. SAIS can assist surgeons in honing their abilities and improving the outcomes of their operations by detecting the type of surgery being performed and the caliber with which it is carried out. However, the system’s unintentional bias early in testing raises the possibility that SAIS still needs to be improved before clinicians can use it. The study team is currently striving to solve the tool’s remaining bias, and these efforts reflect a broader interest in employing AI to enhance surgical care across the healthcare sector.
