Development of a new valid, reliable, and internationally applicable assessment tool of residents' competence in ophthalmic surgery

Purpose: To test the validity and reliability of a new tool for assessing residents' competence in ophthalmic surgery. Changing paradigms of ophthalmic education in the United States have influenced worldwide ophthalmic education and necessitated new methods of assessing resident competence. Accordingly, a new tool for assessing residents' competence in ophthalmic surgery (phacoemulsification) that could be applicable internationally was developed. We hypothesize that this instrument is valid and reliable. Methods: A panel of six international content experts adapted a previously published tool for assessing phacoemulsification. The tool (called the International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubric, or ICO-OSCAR:phaco) was reviewed by 12 international content experts for their constructive comments, which were incorporated to ensure content validity. Ten expert cataract surgery teachers then graded six recorded phacoemulsification surgeries with the ICO-OSCAR:phaco to investigate inter-rater reliability. Results: The coefficient alpha statistic (a measure of reliability/internal consistency) for the ICO-OSCAR:phaco as a whole was 0.92, and 17 of its 20 dimensions had alpha coefficients greater than 0.70. Conclusions: The ICO-OSCAR:phaco is a valid and reliable assessment tool that could be applied internationally to satisfy the global need of new instruments to comply with emerging trends in ophthalmic education. A toolbox of similar surgical competency assessment tools is being developed.