  Large language models (LLMs) with chat-based capabilities, such as ChatGPT,
are widely used in various workflows. However, due to a limited understanding
of these large-scale models, users struggle to use this technology and
experience different kinds of dissatisfaction. Researchers have introduced
several methods such as prompt engineering to improve model responses. However,
they focus on crafting one prompt, and little has been investigated on how to
deal with the dissatisfaction the user encountered during the conversation.
Therefore, with ChatGPT as the case study, we examine end users'
dissatisfaction along with their strategies to address the dissatisfaction.
After organizing users' dissatisfaction with LLM into seven categories based on
a literature review, we collected 511 instances of dissatisfactory ChatGPT
responses from 107 users and their detailed recollections of dissatisfied
experiences, which we release as a publicly accessible dataset. Our analysis
reveals that users most frequently experience dissatisfaction when ChatGPT
fails to grasp their intentions, while they rate the severity of
dissatisfaction the highest with dissatisfaction related to accuracy. We also
identified four tactics users employ to address their dissatisfaction and their
effectiveness. We found that users often do not use any tactics to address
their dissatisfaction, and even when using tactics, 72% of dissatisfaction
remained unresolved. Moreover, we found that users with low knowledge regarding
LLMs tend to face more dissatisfaction on accuracy while they often put minimal
effort in addressing dissatisfaction. Based on these findings, we propose
design implications for minimizing user dissatisfaction and enhancing the
usability of chat-based LLM services.
