Note: This content is accessible to all versions of every browser. However, this browser does not seem to support current Web standards, preventing the display of our site's design details.

  

Incentive Design in Peer Review: Rating and Repeated Endogenous Matching

Author(s):

Y. Xiao, F. Dörfler, M. van der Schaar
Conference/Journal:

Abstract:

Peer review (e.g., grading assignments in Massive Open Online Courses (MOOCs), academic paper review) is an effective and scalable method to evaluate the products (e.g., assignments, papers) of a large number of agents when the number of dedicated reviewing experts (e.g., teaching assistants, editors) is limited. Peer review poses two key challenges: 1) identifying the reviewers’ intrinsic capabilities (i.e., adverse selection) and 2) incentivizing the reviewers to exert high effort (i.e., moral hazard). Some works in mechanism design address pure adverse selection using one-shot matching rules, and pure moral hazard was addressed in repeated games with exogenously given and fixed matching rules. However, in peer review systems exhibiting both adverse selection and moral hazard, one-shot or exogenous matching rules do not link agents’ current behavior with future matches and future payoffs, and as we prove, will induce myopic behavior (i.e., exerting the lowest effort) resulting in the lowest review quality. In this paper, we propose for the first time a solution that simultaneously solves adverse selection and moral hazard. Our solution exploits the repeated interactions of agents, utilizes ratings to summarize agents’ past review quality, and designs matching rules that endogenously depend on agents’ ratings. Our proposed matching rules are easy to implement and require no knowledge about agents’ private information (e.g., their benefit and cost functions). Yet, they are effective in guiding the system to an equilibrium where the agents are incentivized to exert high effort and receive ratings that precisely reflect their review quality. Using several illustrative examples, we quantify the significant performance gains obtained by our proposed mechanism as compared to existing one-shot or exogenous matching rules.

Year:

2014
Type of Publication:

(15)Miscellaneous
Supervisor:



File Download:

Request a copy of this publication.
(Uses JavaScript)
% Autogenerated BibTeX entry
@Misc { XiaD_r:2014:IFA_5055,
    author={Y. Xiao and F. D{\"o}rfler and M. van der Schaar},
    title={{Incentive Design in Peer Review: Rating and Repeated
	  Endogenous Matching}},
    month=nov,
    year={2014},
    url={http://control.ee.ethz.ch/index.cgi?page=publications;action=details;id=5055}
}
Permanent link