  Sequential recommendation predicts users' next behaviors with their
historical interactions. Recommending with longer sequences improves
recommendation accuracy and increases the degree of personalization. As
sequences get longer, existing works have not yet addressed the following two
main challenges. Firstly, modeling long-range intra-sequence dependency is
difficult with increasing sequence lengths. Secondly, it requires efficient
memory and computational speeds. In this paper, we propose a Sparse Attentive
Memory (SAM) network for long sequential user behavior modeling. SAM supports
efficient training and real-time inference for user behavior sequences with
lengths on the scale of thousands. In SAM, we model the target item as the
query and the long sequence as the knowledge database, where the former
continuously elicits relevant information from the latter. SAM simultaneously
models target-sequence dependencies and long-range intra-sequence dependencies
with O(L) complexity and O(1) number of sequential updates, which can only be
achieved by the self-attention mechanism with O(L^2) complexity. Extensive
empirical results demonstrate that our proposed solution is effective not only
in long user behavior modeling but also on short sequences modeling.
Implemented on sequences of length 1000, SAM is successfully deployed on one of
the largest international E-commerce platforms. This inference time is within
30ms, with a substantial 7.30% click-through rate improvement for the online
A/B test. To the best of our knowledge, it is the first end-to-end long user
sequence modeling framework that models intra-sequence and target-sequence
dependencies with the aforementioned degree of efficiency and successfully
deployed on a large-scale real-time industrial recommender system.
