Post written by Michael A. Scaffidi, BSc(Hon) MEd, and Catharine M. Walsh, MD, MEd, PhD, FRCPC, from the Division of Gastroenterology, St. Michael’s Hospital and the Division of Gastroenterology, Hepatology and Nutrition, Hospital for Sick Children, University of Toronto, Toronto, Canada.

Assessment of gastrointestinal endoscopy competence is integral to delivering high-quality patient care. Endoscopic competence involves 3 major skill domains: technical (psychomotor skills), cognitive (knowledge and recognition), and integrative (expertise and behavior). One approach used to assess these domains is self-assessment, wherein endoscopists appraise their own performances. Self-assessment has been acknowledged as an important skill as it allows individuals to identify their own strengths and weaknesses to better understand where additional practice is needed and to help inform self-directed learning activities. Moreover, the Joint Advisory Group (JAG) on Gastrointestinal Endoscopy recommends the use of a personal development plan with an element of self-assessment for trainees. It is unknown, however, whether endoscopists are accurate in assessing their own performance.

We aimed to determine the self-assessment accuracy of novice, intermediate, and experienced endoscopists. We presented data from 20 novice (performed <50 previous colonoscopies), 10 intermediate (50-500), and 10 experienced (>1000) endoscopists from 5 hospitals. The endoscopists each performed a clinical colonoscopy that was video-recorded. They subsequently self-assessed their own performances using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). The GiECAT is a task-specific direct observational assessment tool for colonoscopy that assesses competency in the 3 major skill domains: cognitive, technical, and integrative. Their video-recorded performances were also assessed independently by 2 blinded expert endoscopists using the GiECAT. Self-assessment accuracy between the externally assessed and self-assessed scores was evaluated.

This is the first study to examine self-assessment accuracy of clinical colonoscopy competence across levels of experience and competency domains. Overall, we found that endoscopists demonstrated moderate self-assessment accuracy. When analyzed by experience level, however, we found that experienced endoscopists were more accurate than novices. Novices tended to overrate their performance, whereas experienced endoscopists tended to underrate themselves. The positive association of endoscopic experience and self-assessment accuracy can be explained by the Dunning-Kruger effect—a cognitive bias whereby unskilled individuals have an illusory positive perception of their skills.

Figure 1. Bland-Altman analysis comparing externally assessed and self-assessed Gastrointestinal Endoscopy Competency Assessment Tool total scores. The mean of the externally assessed and self-assessed scores is on the horizontal axis, and the difference between externally assessed and self-assessed scores is on the vertical axis. The mean difference (central line) and 95% limits of agreement are shown. GiECAT, Gastrointestinal Endoscopy Competency Assessment Tool; SD, standard deviation.

Interestingly, we found that while experienced endoscopists were most accurate in self-assessing their technical and cognitive competencies, they were the least accurate in assessing their integrative competencies (expertise and behavior). These competencies, which include core skills like situational awareness and communication, allow individuals to integrate their knowledge and technical expertise to function effectively within a health care team, adapt to varied contexts, tolerate uncertainty, and are important for ensuring safe, high-quality endoscopic care. Upskilling courses that provide structured assessment and feedback for practicing endoscopists may help to promote ongoing awareness and learning regarding these important competencies.

Our findings have important implications for both novice and experienced endoscopists.  For experienced endoscopists, self-assessment can be a useful strategy to help self-identify learning needs (ie, knowledge and skills deficits). These same endoscopists, however, may need assistance in identifying deficits in their integrative competencies. Alternatively, we caution against the use of self-assessment among novices for assessment-related purposes. We suggest that endoscopic trainers should provide frequent, constructive feedback to learners to foster self-reflection and help them to develop greater self-awareness and a more accurate perception of their skill level and learning deficits.

Read the full article online.

The information presented in Endoscopedia reflects the opinions of the authors and does not represent the position of the American Society for Gastrointestinal Endoscopy (ASGE). ASGE expressly disclaims any warranties or guarantees, expressed or implied, and is not liable for damages of any kind in connection with the material, information, or procedures set forth.