Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: On the Dimensionality of Word Embedding   In this paper, we provide a theoretical understanding of word embedding and
its dimensionality. Motivated by the unitary-invariance of word embedding, we
propose the Pairwise Inner Product (PIP) loss, a novel metric on the
dissimilarity between word embeddings. Using techniques from matrix
perturbation theory, we reveal a fundamental bias-variance trade-off in
dimensionality selection for word embeddings. This bias-variance trade-off
sheds light on many empirical observations which were previously unexplained,
for example the existence of an optimal dimensionality. Moreover, new insights
and discoveries, like when and how word embeddings are robust to over-fitting,
are revealed. By optimizing over the bias-variance trade-off of the PIP loss,
we can explicitly answer the open question of dimensionality selection for word
embedding.

27
The categories of this paper are: comp-gas
36
The categories of this paper are: cmp-lg
8
The categories of this paper are: hep-ex
23
The categories of this paper are: solv-int
3
The categories of this paper are: physics
26
The categories of this paper are: bayes-an
31
The categories of this paper are: ao-sci
30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex
35
The categories of this paper are: mtrl-th
34
The categories of this paper are: chem-ph
21
The categories of this paper are: adap-org
0
The categories of this paper are: hep-ph
7
The categories of this paper are: hep-th
32
The categories of this paper are: atom-ph
28
The categories of this paper are: alg-geom