  Online peer-to-peer support platforms enable conversations between millions
of people who seek and provide mental health support. If successful, web-based
mental health conversations could improve access to treatment and reduce the
global disease burden. Psychologists have repeatedly demonstrated that empathy,
the ability to understand and feel the emotions and experiences of others, is a
key component leading to positive outcomes in supportive conversations.
However, recent studies have shown that highly empathic conversations are rare
in online mental health platforms.
  In this paper, we work towards improving empathy in online mental health
support conversations. We introduce a new task of empathic rewriting which aims
to transform low-empathy conversational posts to higher empathy. Learning such
transformations is challenging and requires a deep understanding of empathy
while maintaining conversation quality through text fluency and specificity to
the conversational context. Here we propose PARTNER, a deep reinforcement
learning agent that learns to make sentence-level edits to posts in order to
increase the expressed level of empathy while maintaining conversation quality.
Our RL agent leverages a policy network, based on a transformer language model
adapted from GPT-2, which performs the dual task of generating candidate
empathic sentences and adding those sentences at appropriate positions. During
training, we reward transformations that increase empathy in posts while
maintaining text fluency, context specificity and diversity. Through a
combination of automatic and human evaluation, we demonstrate that PARTNER
successfully generates more empathic, specific, and diverse responses and
outperforms NLP methods from related tasks like style transfer and empathic
dialogue generation. Our work has direct implications for facilitating empathic
conversations on web-based platforms.
