  User modeling, which aims to capture users' characteristics or interests,
heavily relies on task-specific labeled data and suffers from the data sparsity
issue. Several recent studies tackled this problem by pre-training the user
model on massive user behavior sequences with a contrastive learning task.
Generally, these methods assume different views of the same behavior sequence
constructed via data augmentation are semantically consistent, i.e., reflecting
similar characteristics or interests of the user, and thus maximizing their
agreement in the feature space. However, due to the diverse interests and heavy
noise in user behaviors, existing augmentation methods tend to lose certain
characteristics of the user or introduce noisy behaviors. Thus, forcing the
user model to directly maximize the similarity between the augmented views may
result in a negative transfer. To this end, we propose to replace the
contrastive learning task with a new pretext task: Augmentation-Adaptive
SelfSupervised Ranking (AdaptSSR), which alleviates the requirement of semantic
consistency between the augmented views while pre-training a discriminative
user model. Specifically, we adopt a multiple pairwise ranking loss which
trains the user model to capture the similarity orders between the implicitly
augmented view, the explicitly augmented view, and views from other users. We
further employ an in-batch hard negative sampling strategy to facilitate model
training. Moreover, considering the distinct impacts of data augmentation on
different behavior sequences, we design an augmentation-adaptive fusion
mechanism to automatically adjust the similarity order constraint applied to
each sample based on the estimated similarity between the augmented views.
Extensive experiments on both public and industrial datasets with six
downstream tasks verify the effectiveness of AdaptSSR.
