  Click-through rate (CTR) prediction plays as a core function module in
various personalized online services. The traditional ID-based models for CTR
prediction take as inputs the one-hot encoded ID features of tabular modality,
which capture the collaborative signals via feature interaction modeling. But
the one-hot encoding discards the semantic information conceived in the
original feature texts. Recently, the emergence of Pretrained Language Models
(PLMs) has given rise to another paradigm, which takes as inputs the sentences
of textual modality obtained by hard prompt templates and adopts PLMs to
extract the semantic knowledge. However, PLMs generally tokenize the input text
data into subword tokens and ignore field-wise collaborative signals.
Therefore, these two lines of research focus on different characteristics of
the same input data (i.e., textual and tabular modalities), forming a distinct
complementary relationship with each other. In this paper, we propose to
conduct Fine-grained feature-level ALignment between ID-based Models and
Pretrained Language Models (FLIP) for CTR prediction. We design a novel joint
reconstruction pretraining task for both masked language and tabular modeling.
Specifically, the masked data of one modality (i.e., tokens or features) has to
be recovered with the help of the other modality, which establishes the
feature-level interaction and alignment via sufficient mutual information
extraction between dual modalities. Moreover, we propose to jointly finetune
the ID-based model and PLM for downstream CTR prediction tasks, thus achieving
superior performance by combining the advantages of both models. Extensive
experiments on three real-world datasets demonstrate that FLIP outperforms SOTA
baselines, and is highly compatible for various ID-based models and PLMs.
