Doctoral Thesis Proposal - Alexander Koujianos Goldberg
— 3:30pm
Location:
In Person and VIrtual - ET
-
Traffic21 Classroom, Gates Hillman 6501 and Zoom
Speaker:
ALEXANDER KOUJIANOS GOLDBERG
,
Ph.D. Student, Computer Science Department, Carnegie Mellon University
https://akgoldberg.github.io/
Socially important decisions—from scientific funding, to college admissions and job hiring—rely on ratings or rankings supplied by multiple human evaluators. These judgments are prone to noise, bias, and strategic manipulation, and there is seldom an objective ground truth against which to determine their quality. The goal of this thesis is to understand and mitigate such errors in distributed human evaluation in order to make better decisions. Towards this end, we both conduct controlled experiments in review processes and develop principled algorithms with provable guarantees.
In particular, we conduct large-scale experiments at peer review conferences to expose sources of error in evaluation and identify opportunities for improvement. Then, we develop a method for selecting top candidates on the basis of uncertain evaluations, providing a principled instantiation of a "peer review lottery." Finally, we design privacy-preserving algorithms for releasing anonymized time-series and graph data, which can enable more transparency into review processes while preserving participant anonymity.
Thesis Committee
Giulia Fanti (Co-chair)
Nihar B. Shah (Co-chair)
Tom Mitchell
John Ioannidis (Stanford University)
Additional Information
In Person and Zoom Participation. See announcement.