Post written by Yichen Wang, MD, MSc, Vivek Kumbhari, MBChB, PhD, and Michael B. Wallace, MD, MPH, from the Division of Gastroenterology and Hepatology, Mayo Clinic, Jacksonville, Florida, USA.
Today’s endoscopic artificial intelligence (AI) can see, but it cannot act. Our perspective examines what may arrive next: embodied or physical, AI that can interpret the endoscopic environment, understand procedural intent, and assist with movement. We discuss vision-language-action models, robotic endoscopy, and humanoid manipulators as potential components of this transition, with particular attention to their application in flexible GI endoscopy.
Most current AI systems in endoscopy act as observers: they analyze images and provide information on a screen, while the endoscopist remains responsible for every physical action. At the same time, rapid advances in robotics and multimodal AI are making systems that connect perception, reasoning, and movement increasingly plausible. We felt it was important to introduce these developments to the endoscopy community, distinguish realistic near-term applications from more-speculative autonomy, and identify the clinical, technical, and safety questions that should shape the field before these technologies enter patient care.
Our analysis identifies shared autonomy as the most credible near-term pathway. Early systems may assist with constrained tasks such as stabilizing the endoscope, centering the lumen, maintaining a clear view, smoothing withdrawal, or holding a therapeutic target, while the physician retains control. We also propose that humanoid manipulators deserve consideration because they could operate conventional endoscopes within existing human-designed workflows and permit immediate takeover by an endoscopist.
A major translational barrier is the scarcity of synchronized vision-action data. Although endoscopy has generated extensive video archives, few data sets pair intraluminal images with hand movements, control-wheel actuation, forces, scope configuration, and procedural commands. Progress will require multimodal data sets, realistic simulation, staged benchtop and animal validation, clearly defined task boundaries, confidence monitoring, and reliable physician over-ride. Together, these priorities provide a practical research roadmap for the responsible development of embodied AI in endoscopy.
Embodied AI should be approached with cautious optimism. Its greatest near-term value may be reducing operator workload and variability rather than replacing the endoscopist. If developed and validated responsibly, it could eventually help disseminate expert technique, expand access to high-quality endoscopy, and allow physicians to focus more attention on clinical judgment and complex therapy.
Virtual and embodied artificial intelligence in flexible gastrointestinal endoscopy. Virtual artificial intelligence systems analyze endoscopic data and generate on-screen outputs to support clinical decision-making, but they do not directly interact with the physical environment. Embodied, or physical, artificial intelligence systems integrate multimodal sensing with vision–language–action (VLA) processing to generate robotic control policies and physical actions. This progression shifts artificial intelligence from passive decision support toward systems capable of perceiving, reasoning, and acting in the procedural environment.
Read the full article online.
The information presented in Endoscopedia reflects the opinions of the authors and does not represent the position of the American Society for Gastrointestinal Endoscopy (ASGE). ASGE expressly disclaims any warranties or guarantees, expressed or implied, and is not liable for damages of any kind in connection with the material, information, or procedures set forth.
