Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Vokenization: Improving Language Understanding with Contextualized,
  Visual-Grounded Supervision   Humans learn language by listening, speaking, writing, reading, and also, via
interaction with the multimodal real world. Existing language pre-training
frameworks show the effectiveness of text-only self-supervision while we
explore the idea of a visually-supervised language model in this paper. We find
that the main reason hindering this exploration is the large divergence in
magnitude and distributions between the visually-grounded language datasets and
pure-language corpora. Therefore, we develop a technique named "vokenization"
that extrapolates multimodal alignments to language-only data by contextually
mapping language tokens to their related images (which we call "vokens"). The
"vokenizer" is trained on relatively small image captioning datasets and we
then apply it to generate vokens for large language corpora. Trained with these
contextually generated vokens, our visually-supervised language models show
consistent improvements over self-supervised alternatives on multiple
pure-language tasks such as GLUE, SQuAD, and SWAG. Code and pre-trained models
publicly available at https://github.com/airsplay/vokenization

11
The categories of this paper are: hep-lat
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
29
The categories of this paper are: funct-an
6
The categories of this paper are: astro-ph
8
The categories of this paper are: hep-ex
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph
5
The categories of this paper are: gr-qc
16
The categories of this paper are: stat
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
4
The categories of this paper are: cond-mat