  Conversational AI systems exhibit a level of human-like behavior that
promises to have profound impacts on many aspects of daily life -- how people
access information, create content, and seek social support. Yet these models
have also shown a propensity for biases, offensive language, and conveying
false information. Consequently, understanding and moderating safety risks in
these models is a critical technical and social challenge. Perception of safety
is intrinsically subjective, where many factors -- often intersecting -- could
determine why one person may consider a conversation with a chatbot safe and
another person could consider the same conversation unsafe. In this work, we
focus on demographic factors that could influence such diverse perceptions. To
this end, we contribute an analysis using Bayesian multilevel modeling to
explore the connection between rater demographics and how raters report safety
of conversational AI systems. We study a sample of 252 human raters stratified
by gender, age group, race/ethnicity group, and locale. This rater pool
provided safety labels for 1,340 human-chatbot conversations. Our results show
that intersectional effects involving demographic characteristics such as
race/ethnicity, gender, and age, as well as content characteristics, such as
degree of harm, all play significant roles in determining the safety of
conversational AI systems. For example, race/ethnicity and gender show strong
intersectional effects, particularly among South Asian and East Asian women. We
also find that conversational degree of harm impacts raters of all
race/ethnicity groups, but that Indigenous and South Asian raters are
particularly sensitive to this harm. Finally, we observe the effect of
education is uniquely intersectional for Indigenous raters, highlighting the
utility of multilevel frameworks for uncovering underrepresented social
perspectives.
