  AI recommender systems are sought for decision support by providing
suggestions to operators responsible for making final decisions. However, these
systems are typically considered black boxes, and are often presented without
any context or insight into the underlying algorithm. As a result, recommender
systems can lead to miscalibrated user reliance and decreased situation
awareness. Recent work has focused on improving the transparency of recommender
systems in various ways such as improving the recommender's analysis and
visualization of the figures of merit, providing explanations for the
recommender's decision, as well as improving user training or calibrating user
trust. In this paper, we introduce an alternative transparency technique of
structuring the order in which contextual information and the recommender's
decision are shown to the human operator. This technique is designed to improve
the operator's situation awareness and therefore the shared situation awareness
between the operator and the recommender system. This paper presents the
results of a two-phase between-subjects study in which participants and a
recommender system jointly make a high-stakes decision. We varied the amount of
contextual information the participant had, the assessment technique of the
figures of merit, and the reliability of the recommender system. We found that
providing contextual information upfront improves the team's shared situation
awareness by improving the human decision maker's initial and final judgment,
as well as their ability to discern the recommender's error boundary.
Additionally, this technique accurately calibrated the human operator's trust
in the recommender. This work proposes and validates a way to provide
model-agnostic transparency into AI systems that can support the human decision
maker and lead to improved team performance.
