text: We propose the PeerRank method for peer assessment. This constructs a grade for an agent based on the grades proposed by the agents evaluating the agent. Since the grade of an agent is a measure of their ability to grade correctly, the PeerRank method weights grades by the grades of the grading agent. The PeerRank method also provides an incentive for agents to grade correctly. As the grades of an agent depend on the grades of the grading agents, and as these grades themselves depend on the grades of other agents, we define the PeerRank method by a fixed point equation similar to the PageRank method for ranking web-pages. We identify some formal properties of the PeerRank method (for example, it satisfies axioms of unanimity, no dummy, no discrimination and symmetry), discuss some examples, compare with related work and evaluate the performance on some synthetic data. Our results show considerable promise, reducing the error in grade predictions by a factor of 2 or more in many cases over the natural baseline of averaging peer grades.
  @cite_2: A group of peers must choose one of them to receive a prize; everyone cares only about winning, not about who gets the prize if someone else. An award rule is impartial if one's message never influences whether or not one wins the prize. We explore the consequences of impartiality when each agent nominates a single (other) agent for the prize. @PARASPLIT On the positive side, we construct impartial nomination rules where both the influence of individual messages and the requirements to win the prize are not very different across agents. Partition the agents in two or more districts, each of size at least 3, and call an agent a local winner if he is nominated by a majority of members of his own district; the rule selects a local winner with the largest support from nonlocal winners, or a fixed default agent in case there is no local winner. @PARASPLIT On the negative side, impartiality implies that ballots cannot be processed anonymously as in plurality voting. Moreover, we cannot simultaneously guarantee that the winner always gets at least one nomination, and that an agent nominated by everyone else always wins.
summary: There are a number of closely related problems to ours in the social choice literature. For example, Holzman and Moulin have studied a related problem in which a set of agents wish to select one amongst them to receive a prize @cite_2 . A fundamental assumption of this work is that nominations are impartial: your message never influences whether you win the prize. In our setting, such an assumption is less appropriate. We want your evaluation of the work of another agent to influence your evaluation. There are several reasons behind this change. First, your ability to evaluate the work of other agents measures in part your command of the subject being examined. Second, you will be incentivized to grade accurately by a better final grade. If your message cannot influence your evaluation, then you have no incentive to provide good evaluations. For this reason, even if we extend the sort of methods proposed by Holzman and Moulin to the task of ranking, the starting assumptions are very different.