  In 2023, the International Conference on Machine Learning (ICML) required
authors with multiple submissions to rank their submissions based on perceived
quality. In this paper, we aim to employ these author-specified rankings to
enhance peer review in machine learning and artificial intelligence conferences
by extending the Isotonic Mechanism to exponential family distributions. This
mechanism generates adjusted scores that closely align with the original scores
while adhering to author-specified rankings. Despite its applicability to a
broad spectrum of exponential family distributions, implementing this mechanism
does not require knowledge of the specific distribution form. We demonstrate
that an author is incentivized to provide accurate rankings when her utility
takes the form of a convex additive function of the adjusted review scores. For
a certain subclass of exponential family distributions, we prove that the
author reports truthfully only if the question involves only pairwise
comparisons between her submissions, thus indicating the optimality of ranking
in truthful information elicitation. Moreover, we show that the adjusted scores
improve dramatically the estimation accuracy compared to the original scores
and achieve nearly minimax optimality when the ground-truth scores have bounded
total variation. We conclude the paper by presenting experiments conducted on
the ICML 2023 ranking data, which show significant estimation gain using the
Isotonic Mechanism.
