Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Prompt Waywardness: The Curious Case of Discretized Interpretation of
  Continuous Prompts   Fine-tuning continuous prompts for target tasks has recently emerged as a
compact alternative to full model fine-tuning. Motivated by these promising
results, we investigate the feasibility of extracting a discrete (textual)
interpretation of continuous prompts that is faithful to the problem they
solve. In practice, we observe a "wayward" behavior between the task solved by
continuous prompts and their nearest neighbor discrete projections: We can find
continuous prompts that solve a task while being projected to an arbitrary text
(e.g., definition of a different or even a contradictory task), while being
within a very small (2%) margin of the best continuous prompt of the same size
for the task. We provide intuitions behind this odd and surprising behavior, as
well as extensive empirical analyses quantifying the effect of various
parameters. For instance, for larger model sizes we observe higher waywardness,
i.e, we can find prompts that more closely map to any arbitrary text with a
smaller drop in accuracy. These findings have important implications relating
to the difficulty of faithfully interpreting continuous prompts and their
generalization across models and tasks, providing guidance for future progress
in prompting language models.

26
The categories of this paper are: bayes-an
21
The categories of this paper are: adap-org
25
The categories of this paper are: patt-sol
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
11
The categories of this paper are: hep-lat
20
The categories of this paper are: acc-phys
8
The categories of this paper are: hep-ex
36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
7
The categories of this paper are: hep-th
19
The categories of this paper are: eess
16
The categories of this paper are: stat
5
The categories of this paper are: gr-qc
27
The categories of this paper are: comp-gas
14
The categories of this paper are: math-ph