  While demands for change and accountability for harmful AI consequences
mount, foreseeing the downstream effects of deploying AI systems remains a
challenging task. We developed AHA! (Anticipating Harms of AI), a generative
framework to assist AI practitioners and decision-makers in anticipating
potential harms and unintended consequences of AI systems prior to development
or deployment. Given an AI deployment scenario, AHA! generates descriptions of
possible harms for different stakeholders. To do so, AHA! systematically
considers the interplay between common problematic AI behaviors as well as
their potential impacts on different stakeholders, and narrates these
conditions through vignettes. These vignettes are then filled in with
descriptions of possible harms by prompting crowd workers and large language
models. By examining 4113 harms surfaced by AHA! for five different AI
deployment scenarios, we found that AHA! generates meaningful examples of
harms, with different problematic AI behaviors resulting in different types of
harms. Prompting both crowds and a large language model with the vignettes
resulted in more diverse examples of harms than those generated by either the
crowd or the model alone. To gauge AHA!'s potential practical utility, we also
conducted semi-structured interviews with responsible AI professionals (N=9).
Participants found AHA!'s systematic approach to surfacing harms important for
ethical reflection and discovered meaningful stakeholders and harms they
believed they would not have thought of otherwise. Participants, however,
differed in their opinions about whether AHA! should be used upfront or as a
secondary-check and noted that AHA! may shift harm anticipation from an
ideation problem to a potentially demanding review problem. Drawing on our
results, we discuss design implications of building tools to help practitioners
envision possible harms.
