  Learning embedding table plays a fundamental role in Click-through rate(CTR)
prediction from the view of the model performance and memory usage. The
embedding table is a two-dimensional tensor, with its axes indicating the
number of feature values and the embedding dimension, respectively. To learn an
efficient and effective embedding table, recent works either assign various
embedding dimensions for feature fields and reduce the number of embeddings
respectively or mask the embedding table parameters. However, all these
existing works cannot get an optimal embedding table. On the one hand, various
embedding dimensions still require a large amount of memory due to the vast
number of features in the dataset. On the other hand, decreasing the number of
embeddings usually suffers from performance degradation, which is intolerable
in CTR prediction. Finally, pruning embedding parameters will lead to a sparse
embedding table, which is hard to be deployed. To this end, we propose an
optimal embedding table learning framework OptEmbed, which provides a practical
and general method to find an optimal embedding table for various base CTR
models. Specifically, we propose pruning the redundant embeddings regarding
corresponding features' importance by learnable pruning thresholds.
Furthermore, we consider assigning various embedding dimensions as one single
candidate architecture. To efficiently search the optimal embedding dimensions,
we design a uniform embedding dimension sampling scheme to equally train all
candidate architectures, meaning architecture-related parameters and learnable
thresholds are trained simultaneously in one supernet. We then propose an
evolution search method based on the supernet to find the optimal embedding
dimensions for each field. Experiments on public datasets show that OptEmbed
can learn a compact embedding table which can further improve the model
performance.
