QuanTyler : Apportioning Credit for Student Forum Participation

Abstract

We develop a random forest classifier that helps assign academic credit for student class forum participation. Our twelve predictors are quantities that are available from typical forum facilities, such as the number of endorsed contributions, and the number of answers and views. We add page rank and centrality to these base measures. The classification target are the four classes created by student rank quartiles. Course content experts provided ground truth by ranking a limited number of post pairs. We expand this labeled set via data augmentation. We compute the relative importance of the predictors, and compare performance in matching the human expert rankings as we vary the number of predictors used in training. We reach multiclass AUC measures between 0.64 and 0.94, depending on the number of deployed predictors. This performance is compared with the simple formulas that are currently used by some instructors for estimating the amount of credit to apportion for forum activity in their classes. To test generality and scalability we trained a classifier on the archive of the Web economics StackExchange reputation data. We used this classifier to predict the quartile assignments by human judges of forum posts from a university artificial intelligence course. Our first attempt reaches an average AUC of 0.69.