This is how AI can assess the performance of surgeons

Researchers recruited fifty participants from four stages of neurosurgical training; neurosurgeons, fellows and senior residents, junior residents, and medical students. The participants performed 250 complex tumor resections using NeuroVR, a virtual reality surgical simulator. The National Research Council of Canada developed the system; CAE recorded all instrument movements in 20 millisecond intervals.

Using the raw data, the machine learning algorithm developed performance measures such as instrument position and force applied, as well as outcomes such as amount of tumor removed and blood loss, which could predict the level of expertise of each participant with 90% accuracy.

The top performing algorithm could classify participants using just six performance measures.

As reported in the Journal of the American Medical Association, the findings show that the fusion of artificial intelligence and virtual reality neurosurgical simulators can accurately and efficiently assess the performance of surgeon trainees.

This means that scientists can develop AI-assisted mentoring systems that focus on improving patient safety by guiding trainees through complex surgical procedures. These systems can determine areas that need improvement and how the trainee can develop these important skills before they operate on real patients.

“Physician educators are facing increased time pressure to balance their commitment to both patients and learners,” says lead author Rolando Del Maestro of McGill University.

“Our study proves that we can design systems that deliver on-demand surgical assessments at the convenience of the learner and with less input from instructors. It may also lead to better patient safety by reducing the chance for human error both while assessing surgeons and in the operating room.”

The Franco Di Giovanni Foundation, the Montreal English School Board, the McGill Department of Orthopedics, the Fonds de recherche du Québec-Santé (FRQS), and a Robert Maudsley Fellowship for Studies in Medical Education from the Royal College of Physicians and Surgeons of Canada supported the work.