Post written by Michael A. Scaffidi, BSc(Hon), MEd, and Catharine M. Walsh, MD, MEd, PhD, from the Division of Gastroenterology, St. Michael’s Hospital and the Division of Gastroenterology, Hepatology and Nutrition, Hospital for Sick Children, University of Toronto, Toronto, Canada.

Assessment is a critical process to determine endoscopic competence. Real-time, or “live” assessment, involves an assessor who is present at the time of the colonoscopy and is the standard means by which most assessment occurs. Video-based assessment has been proposed as a complementary means by which to assess colonoscopy performance as it has several advantages over live assessment: raters can be blinded; it may capture more “authentic practice” in the absence of an assessor (ie, Hawthorne effect); recordings allow for evaluation by multiple reviewers; and videos can aid in feedback provision post-procedure.

We aimed to compare the reliability, validity, and feasibility of video-based assessments of competence in performing colonoscopy compared to live assessment. We video-recorded clinical colonoscopies performed by novice (<50 previous colonoscopies), intermediate (50-500), and experienced (>1000) endoscopists from 5 hospitals. Two views of each colonoscopy were videotaped: an endoscopic (intraluminal) view and a recording of the endoscopist’s hand movements. Recorded procedures were independently assessed by 2 blinded experts using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT), a validated procedure-specific assessment tool that comprises a global rating scale and checklist. The GiECAT assesses competency across 3 key domains of endoscopic competence: cognitive, technical, and integrative skills. The mean of the 2 blinded video assessors’ ratings were compared with live ratings conducted by a non-blinded expert endoscopist.

We found that blinded video-based assessments of colonoscopy performance are reliable, with excellent concordance between raters. Additionally, there was strong evidence of validity of video-based assessments, as assessors were able to accurately discriminate between novice, intermediate, and experienced endoscopists, and there was excellent agreement between video-based and live assessors. Agreement was good across all competency domains, but highest for technical skills. The high agreement between blinded video-based assessments and unblinded live assessments provides further evidence that assessor bias likely does not affect scores markedly within the clinical setting and that assessee behavior is not significantly altered by the act of being observed, as compared with being video-recorded. Video-based assessments were also considered feasible.

Our findings provide evidence in support of the reliability, validity, and feasibility of video-based assessments of competence in performing colonoscopy using the GiECAT.  While this study provides positive evidence for the use of video-based assessments, our findings also identified certain disadvantages of the approach. Video-based assessments may not adequately capture the entire procedure, as several GiECAT checklist items were unable to be adequately assessed based on video analysis, particularly those which reflect cognitive and integrative aspects of the procedure. Additionally, live assessments were perceived by assessors as being significantly easier than video-based assessments.

Figure 1. Assessments were based on video recordings, which displayed 2 views of each colonoscopy procedure: an endoscopic (intraluminal) view and a recording of the endoscopist’s hand movements.

A recent editorial on our study highlighted the importance of video-based assessments.  In addition to the advantages noted, the authors of the editorial discussed the potential role of videos in education and training. For example, the possibility of remote video-based instruction, especially for specialized procedures (eg, EUS). We encourage and challenge researchers, administrators, and educators to consider using videos in their endeavors to improve colonoscopy training and assessment.

Read the full article online.

The information presented in Endoscopedia reflects the opinions of the authors and does not represent the position of the American Society for Gastrointestinal Endoscopy (ASGE). ASGE expressly disclaims any warranties or guarantees, expressed or implied, and is not liable for damages of any kind in connection with the material, information, or procedures set forth.