  Despite an increasing reliance on fully-automated algorithmic decision-making
in our day-to-day lives, human beings still make highly consequential
decisions. As frequently seen in business, healthcare, and public policy,
recommendations produced by algorithms are provided to human decision-makers to
guide their decisions. While there exists a fast-growing literature evaluating
the bias and fairness of such algorithmic recommendations, an overlooked
question is whether they help humans make better decisions. We develop a
statistical methodology for experimentally evaluating the causal impacts of
algorithmic recommendations on human decisions. We also show how to examine
whether algorithmic recommendations improve the fairness of human decisions and
derive the optimal decision rules under various settings. We apply the proposed
methodology to preliminary data from the first-ever randomized controlled trial
that evaluates the pretrial Public Safety Assessment (PSA) in the criminal
justice system. A goal of the PSA is to help judges decide which arrested
individuals should be released. On the basis of the preliminary data available,
we find that providing the PSA to the judge has little overall impact on the
judge's decisions and subsequent arrestee behavior. However, our analysis
yields some potentially suggestive evidence that the PSA may help avoid
unnecessarily harsh decisions for female arrestees regardless of their risk
levels while it encourages the judge to make stricter decisions for male
arrestees who are deemed to be risky. In terms of fairness, the PSA appears to
increase the gender bias against males while having little effect on any
existing racial differences in judges' decision. Finally, we find that the
PSA's recommendations might be unnecessarily severe unless the cost of a new
crime is sufficiently high.
