$1.2 million NSF funding for U-M research on surgical training

Researchers at the University of Michigan, including CSE professors Xu Wang and Anhong Guo as well as Prof. Vitaliy Popov of Michigan Medicine, have received a $1.2 million grant from the NSF to support their research leveraging advanced computational methods to evaluate and improve surgical training. The project also involves senior personnel Dr. Brian George and Dr. Gurjit Sandhu of Michigan Medicine. Through their project, titled “Multimodal Techniques to Enhance Intra- and Post-operative Learning and Coordination between Attending and Resident Surgeons,” they seek to deploy a multimodal modeling approach to assess and predict surgeon behaviors during operations with the aim of developing new educational tools for surgeons-in-training.

Surgeon training is an area of vital importance in the medical sector. Surgery requires a high degree of knowledge and precision, and the stakes of making a mistake are incredibly high; surgical errors can result in patient injury and even death. Despite these considerations, recent research has shown that many surgeons in the U.S. are not sufficiently trained when they complete their residencies, highlighting a need for more robust educational methods for surgeons.

To address this need, U-M researchers will adopt a computational approach to improve training experiences and outcomes for learning surgeons. Their work will build upon the team’s previous projects evaluating surgeons’ visual needs during surgery, which received a Best Paper Honorable Mention Award at CHI 2024, and designing a video-based surgical training tool called Surgment, which was spearheaded by CSE PhD student Jingying Wang and also presented at  CHI 2024. 

The first step will be to observe and model attending and resident surgeons’ behaviors in the operating room. This will involve performing a detailed assessment of approximately 100 laparoscopic gallbladder removal surgeries, with data collected on surgeons’ gaze during surgery, conversations that occur in the operating room, and a video stream of the surgery obtained by the laparoscopic camera. 

This examination will equip the team with robust data that will allow them to better understand and predict surgeon behavior and, by extension, the competence and independence of resident surgeons as well as the quality of instruction given by attending surgeons. Their predictive modeling approach will include a scene segmentation pipeline that automatically interprets and annotates surgery scenes (as demonstrated in Surgment), as well as a neural network architecture that will enable the researchers to simultaneously derive insight from multiple data streams—gaze, audio, and video.

Armed with this information, the team aims to develop an interactive dashboard that surgeons can use after an operation to debrief, review key moments from surgery, and deliver instructional feedback. They plan to test this postoperative dashboard in controlled experiments to determine its effectiveness in improving instruction and competence in the operating room. The team will also develop and evaluate augmented reality-based intraoperative visualizations to enhance coordination and instruction.

Through these innovative computational techniques and educational tools, the researchers aim to bridge existing gaps in surgical training, improving the readiness and independence of resident surgeons and, in the process, enhancing patient safety.

“We’re excited about the potential this research holds for transforming surgical training,” said Wang. “By leveraging advanced computational methods and partnerships with Michigan Medicine, we can gain unprecedented insights into surgeon behaviors and use this information to create more effective training tools, ultimately making surgeries safer for patients.”