Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Membership Inference Attack Susceptibility of Clinical Language Models   Deep Neural Network (DNN) models have been shown to have high empirical
privacy leakages. Clinical language models (CLMs) trained on clinical data have
been used to improve performance in biomedical natural language processing
tasks. In this work, we investigate the risks of training-data leakage through
white-box or black-box access to CLMs. We design and employ membership
inference attacks to estimate the empirical privacy leaks for model
architectures like BERT and GPT2. We show that membership inference attacks on
CLMs lead to non-trivial privacy leakages of up to 7%. Our results show that
smaller models have lower empirical privacy leakages than larger ones, and
masked LMs have lower leakages than auto-regressive LMs. We further show that
differentially private CLMs can have improved model utility on clinical domain
while ensuring low empirical privacy leakage. Lastly, we also study the effects
of group-level membership inference and disease rarity on CLM privacy leakages.

8
The categories of this paper are: hep-ex
28
The categories of this paper are: alg-geom
16
The categories of this paper are: stat
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
7
The categories of this paper are: hep-th
21
The categories of this paper are: adap-org
10
The categories of this paper are: q-bio
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
2
The categories of this paper are: math
31
The categories of this paper are: ao-sci
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
12
The categories of this paper are: nucl-th
13
The categories of this paper are: quant-ph