Wednesday, December 16, 2015 - 12:00pm to 1:00pm
Location:8102 Gates & Hillman Centers
Speaker:IOANNIS CARAGIANNIS, Associate Professor https://www.ceid.upatras.gr/webpages/faculty/caragian/
We investigate the potential of using ordinal peer grading for the evaluation of students in massive online open courses (MOOCs). According to such grading schemes, each student receives a few assignments (by other students) which she has to rank. Then, a global ranking (possibly translated into numerical scores) is produced by combining the individual ones. This is a novel application area for social choice concepts and methods, where the important problem to be solved is as follows: how should the assignments be distributed so that the collected individual rankings can be easily merged into a global one that is as close as possible to the ranking that represents the relative performance of the students in the assignment? Our main theoretical result suggests that using very simple ways to distribute the assignments so that each student has to rank only k of them, a Borda-like aggregation method can recover a 1-O(1/k) fraction of the true ranking when each student correctly ranks the assignments she receives. Experimental results strengthen our analysis further and also demonstrate that the same method is extremely robust even when students have imperfect capabilities as graders. Our results provide strong evidence that ordinal peer grading can be a highly effective and scalable solution for evaluation in MOOCs. — Ioannis Caragiannis is an associate professor at the Department of Computer Engineering and Informatics of the University of Patras, Greece, and a senior researcher at the Computer Technology Institute “Diophantus”. His research interests include design and analysis of algorithms, combinatorial optimization, algorithmic game theory, and computational aspects of social choice.