Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Model Compression for Domain Adaptation through Causal Effect Estimation   Recent improvements in the predictive quality of natural language processing
systems are often dependent on a substantial increase in the number of model
parameters. This has led to various attempts of compressing such models, but
existing methods have not considered the differences in the predictive power of
various model components or in the generalizability of the compressed models.
To understand the connection between model compression and out-of-distribution
generalization, we define the task of compressing language representation
models such that they perform best in a domain adaptation setting. We choose to
address this problem from a causal perspective, attempting to estimate the
average treatment effect (ATE) of a model component, such as a single layer, on
the model's predictions. Our proposed ATE-guided Model Compression scheme
(AMoC), generates many model candidates, differing by the model components that
were removed. Then, we select the best candidate through a stepwise regression
model that utilizes the ATE to predict the expected performance on the target
domain. AMoC outperforms strong baselines on dozens of domain pairs across
three text classification and sequence tagging tasks.

33
The categories of this paper are: plasm-ph
29
The categories of this paper are: funct-an
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
16
The categories of this paper are: stat
14
The categories of this paper are: math-ph
13
The categories of this paper are: quant-ph
24
The categories of this paper are: dg-ga
26
The categories of this paper are: bayes-an
9
The categories of this paper are: nlin
2
The categories of this paper are: math
10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
30
The categories of this paper are: q-alg
34
The categories of this paper are: chem-ph
17
The categories of this paper are: q-fin