TY - JOUR
T1 - Results and Insights from Diagnostic Questions
T2 - 34th Demonstration and Competition Track at the 34th Annual Conference on Neural Information Processing Systems, NeurIPS 2020
AU - Wang, Zichao
AU - Lamb, Angus
AU - Saveliev, Evgeny
AU - Cameron, Pashmina
AU - Zaykov, Yordan
AU - Hernández-Lobato, José Miguel
AU - Turner, Richard E.
AU - Baraniuk, Richard G.
AU - Barton, Craig
AU - Jones, Simon Peyton
AU - Woodhead, Simon
AU - Zhang, Cheng
N1 - Funding Information:
We thank the Codalab team for their technical support throughout the competition and all competition participants who contributed. ZW and RGB are supported by NSF grants 1842378 and 1937134 and by ONR grant N0014-20-1-2534.
Publisher Copyright:
© 2021 Z. Wang et al.
PY - 2020
Y1 - 2020
N2 - This competition concerns educational diagnostic questions, which are pedagogically effective, multiple-choice questions (MCQs) whose distractors embody misconceptions. With a large and ever-increasing number of such questions, it becomes overwhelming for teachers to know which questions are the best ones to use for their students. We thus seek to answer the following question: how can we use data on hundreds of millions of answers to MCQs to drive automatic personalized learning in large-scale learning scenarios where manual personalization is infeasible? Success in using MCQ data at scale helps build more intelligent, personalized learning platforms that ultimately improve the quality of education en masse. To this end, we introduce a new, large-scale, real-world dataset and formulate 4 data mining tasks on MCQs that mimic real learning scenarios and target various aspects of the above question in a competition setting at NeurIPS 2020. We report on our NeurIPS competition in which nearly 400 teams submitted approximately 4000 submissions, with encouragingly diverse and effective approaches to each of our tasks.
AB - This competition concerns educational diagnostic questions, which are pedagogically effective, multiple-choice questions (MCQs) whose distractors embody misconceptions. With a large and ever-increasing number of such questions, it becomes overwhelming for teachers to know which questions are the best ones to use for their students. We thus seek to answer the following question: how can we use data on hundreds of millions of answers to MCQs to drive automatic personalized learning in large-scale learning scenarios where manual personalization is infeasible? Success in using MCQ data at scale helps build more intelligent, personalized learning platforms that ultimately improve the quality of education en masse. To this end, we introduce a new, large-scale, real-world dataset and formulate 4 data mining tasks on MCQs that mimic real learning scenarios and target various aspects of the above question in a competition setting at NeurIPS 2020. We report on our NeurIPS competition in which nearly 400 teams submitted approximately 4000 submissions, with encouragingly diverse and effective approaches to each of our tasks.
KW - Active learning
KW - Diagnostic questions
KW - Matrix completion
KW - Missing value prediction
KW - Personalized education
KW - Question analytics
KW - Unsupervised learning
UR - http://www.scopus.com/inward/record.url?scp=85162617842&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85162617842&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85162617842
SN - 2640-3498
VL - 133
SP - 191
EP - 205
JO - Proceedings of Machine Learning Research
JF - Proceedings of Machine Learning Research
Y2 - 6 December 2020 through 12 December 2020
ER -