  How do author perceptions match up to the outcomes of the peer-review process
and perceptions of others? In a top-tier computer science conference (NeurIPS
2021) with more than 23,000 submitting authors and 9,000 submitted papers, we
survey the authors on three questions: (i) their predicted probability of
acceptance for each of their papers, (ii) their perceived ranking of their own
papers based on scientific contribution, and (iii) the change in their
perception about their own papers after seeing the reviews. The salient results
are: (1) Authors have roughly a three-fold overestimate of the acceptance
probability of their papers: The median prediction is 70% for an approximately
25% acceptance rate. (2) Female authors exhibit a marginally higher
(statistically significant) miscalibration than male authors; predictions of
authors invited to serve as meta-reviewers or reviewers are similarly
calibrated, but better than authors who were not invited to review. (3)
Authors' relative ranking of scientific contribution of two submissions they
made generally agree (93%) with their predicted acceptance probabilities, but
there is a notable 7% responses where authors think their better paper will
face a worse outcome. (4) The author-provided rankings disagreed with the
peer-review decisions about a third of the time; when co-authors ranked their
jointly authored papers, co-authors disagreed at a similar rate -- about a
third of the time. (5) At least 30% of respondents of both accepted and
rejected papers said that their perception of their own paper improved after
the review process. The stakeholders in peer review should take these findings
into account in setting their expectations from peer review.
