  Recently, there is a growing interest in developing next-generation
recommender systems (RSs) based on pretrained large language models (LLMs),
fully utilizing their encoded knowledge and reasoning ability. However, the
semantic gap between natural language and recommendation tasks is still not
well addressed, leading to multiple issues such as spuriously-correlated
user/item descriptors, ineffective language modeling on user/item contents, and
inefficient recommendations via auto-regression, etc. In this paper, we propose
CLLM4Rec, the first generative RS that tightly integrates the LLM paradigm and
ID paradigm of RS, aiming to address the above challenges simultaneously. We
first extend the vocabulary of pretrained LLMs with user/item ID tokens to
faithfully model the user/item collaborative and content semantics.
Accordingly, in the pretraining stage, a novel soft+hard prompting strategy is
proposed to effectively learn user/item collaborative/content token embeddings
via language modeling on RS-specific corpora established from user-item
interactions and user/item features, where each document is split into a prompt
consisting of heterogeneous soft (user/item) tokens and hard (vocab) tokens and
a main text consisting of homogeneous item tokens or vocab tokens that
facilitates stable and effective language modeling. In addition, a novel mutual
regularization strategy is introduced to encourage the CLLM4Rec to capture
recommendation-oriented information from user/item contents. Finally, we
propose a novel recommendation-oriented finetuning strategy for CLLM4Rec, where
an item prediction head with multinomial likelihood is added to the pretrained
CLLM4Rec backbone to predict hold-out items based on the soft+hard prompts
established from masked user-item interaction history, where recommendations of
multiple items can be generated efficiently.
