Assessing learner engagement with virtual educational events: Development of the Virtual In-Class Engagement Measure (VIEM)

Background: The COVID-19 pandemic has necessitated virtual education, but effects on learner engagement are unknown. We developed a virtual in-class engagement measure (VIEM) to assess learner engagement in online surgical education events. Methods: Using the STROBE, an observer collected tool to document student engagement, as a template an ASE committee workgroup developed the VIEM. The VIEM had two parts: observer assessment and learner self-assessment of engagement. Trained observers collected engagement data from two institutions using the VIEM. Surgical attendings, fellows and residents were observed during virtual learning events. Educator attitudes towards online teaching were also assessed via survey. Results: 22 events with 839 learners were observed. VIEM distinguished between sessions with low and high engagement. 20% of learners pretended to participate. Half of instructors were comfortable with virtual teaching, but only 1/3 believed was as effective as in-person. 2/3 of teachers believed video learners were more engaged than audio learners. Conclusions: Virtual platforms do not automatically translate into increased engagement. Standard tools such as VIEM may help with assessment of engagement during virtual education.