text: We consider crowdsourcing problems where the users are asked to provide evaluations for items; the user evaluations are then used directly, or aggregated into a consensus value. Lacking an incentive scheme, users have no motive in making effort in completing the evaluations, providing inaccurate answers instead. We propose incentive schemes that are truthful and cheap: truthful as the optimal user behavior consists in providing accurate evaluations, and cheap because the truthfulness is achieved with little overhead cost. We consider both discrete evaluation tasks, where an evaluation can be done either correctly, or incorrectly, with no degrees of approximation in between, and quantitative evaluation tasks, where evaluations are real numbers, and the error is measured as distance from the correct value. For both types of tasks, we propose hierarchical incentive schemes that can be effected with a small amount of additional evaluations, and that scale to arbitrarily large crowd sizes: they have the property that the strength of the incentive does not weaken with increasing hierarchy depth. Interestingly, we show that for these schemes to work, the only requisite is that workers know their place in the hierarchy in advance.
  @cite_16: The success of a human computation system depends critically on the humans in the system actually behaving, or acting, as necessary for the system to function effectively. Since users have their own costs and benefits from participation, they will undertake desirable actions only if properly incentivized to do so: Indeed, while there are a vast number of human computation systems on the Web, the extent of participation and quality of contribution varies widely across systems. How can a game-theoretic approach help understand why, and provide guidance on designing systems that incentivize high participation and effort from contributors?
summary: Providing incentives to human agents to return truthful responses is one of the central challenges for crowdsourcing algorithms and applications @cite_16 .