  Open-ended questions in surveys are valuable because they do not constrain
the respondent's answer, thereby avoiding biases. However, answers to
open-ended questions are text data which are harder to analyze. Traditionally,
answers were manually classified as specified in the coding manual. Most of the
effort to automate coding has gone into the easier problem of single label
prediction, where answers are classified into a single code. However, open-ends
that require multi-label classification, i.e., that are assigned multiple
codes, occur frequently. This paper focuses on multi-label classification of
text answers to open-ended survey questions in social science surveys. We
evaluate the performance of the transformer-based architecture BERT for the
German language in comparison to traditional multi-label algorithms (Binary
Relevance, Label Powerset, ECC) in a German social science survey, the GLES
Panel (N=17,584, 55 labels). We find that classification with BERT (forcing at
least one label) has the smallest 0/1 loss (13.1%) among methods considered
(18.9%-21.6%). As expected, it is much easier to correctly predict answer texts
that correspond to a single label (7.1% loss) than those that correspond to
multiple labels ($\sim$50% loss). Because BERT predicts zero labels for only
1.5% of the answers, forcing at least one label, while recommended, ultimately
does not lower the 0/1 loss by much. Our work has important implications for
social scientists: 1) We have shown multi-label classification with BERT works
in the German language for open-ends. 2) For mildly multi-label classification
tasks, the loss now appears small enough to allow for fully automatic
classification (as compared to semi-automatic approaches). 3) Multi-label
classification with BERT requires only a single model. The leading competitor,
ECC, iterates through individual single label predictions.
