Maximizing student benefit from instructor interactions with MOOC discussion forums

Abstract

We developed and tested algorithms to accurately identify MOOC discussion forum threads that would benefit from immediate instructor intervention. An automatic classifier was first used to generate sentiment, confusion, urgency, and question/answer/opinion status at the post level, and discussion forum threads were then reconstructed and ranked using algorithms that make use of these generated tags. We compared our four treatment algorithms (urgency, confusion, last_answers, and question_votes) with the three control algorithms currently used by edX to sort forum threads (timestamp, first_votes, and num_posts), using a corpus that includes all 1,182 threads from three iterations of an archived Statistics in Medicine MOOC hosted by Stanford on the Open edX platform. Nine domain experts in statistics rated the top three threads ranked by each algorithm for potential contributions to student learning and degree of urgency. Two of our treatment algorithms (confusion, question_votes) significantly outperformed the control algorithms at identifying urgent threads that would contribute to student learning.