Screen-Based Simulation for Training and Automated Assessment of Teamwork Skills: Comparing 2 Modes With Different Interactivity

Introduction The need for teamwork training is well documented; however, teaching these skills is challenging given the logistics of assembling individual team members together to train in person. We designed 2 modes of screen-based simulation for training teamwork skills to assess whether interactivity with nonplayer characters was necessary for in-game performance gains or for player satisfaction with the experience. Methods Mixed, randomized, repeated measures study with licensed healthcare providers block-stratified and randomized to evaluation - participant observes and evaluates the team player in 3 scenarios - and game play - participant is immersed as the leader in the same 3 scenarios. Teamwork construct scores (leadership, communication, situation monitoring, mutual support) from an ontology-based, Bayesian network assessment model were analyzed using mixed randomized repeated measures analyses of variance to compare performance, across scenarios and modes. Learning was measured by pretest and posttest quiz scores. User experience was evaluated using χ2 analyses. Results Among 166 recruited and randomized participants, 120 enrolled in the study and 109 had complete data for analysis. Mean composite teamwork Bayesian network scores improved for successive scenarios in both modes, with evaluation scores statistically higher than game play for every teamwork construct and scenario (r = 0.73, P = 0.000). Quiz scores improved from pretest to posttest (P = 0.004), but differences between modes were not significant. Conclusions For training teamwork skills using screen-based simulation, interactivity of the player with the nonplayer characters is not necessary for in-game performance gains or for player satisfaction with the experience.