  Graph Neural Networks (GNNs) have emerged as promising solutions for
collaborative filtering (CF) through the modeling of user-item interaction
graphs. The nucleus of existing GNN-based recommender systems involves
recursive message passing along user-item interaction edges to refine encoded
embeddings. Despite their demonstrated effectiveness, current GNN-based methods
encounter challenges of limited receptive fields and the presence of noisy
``interest-irrelevant'' connections. In contrast, Transformer-based methods
excel in aggregating information adaptively and globally. Nevertheless, their
application to large-scale interaction graphs is hindered by inherent
complexities and challenges in capturing intricate, entangled structural
information. In this paper, we propose TransGNN, a novel model that integrates
Transformer and GNN layers in an alternating fashion to mutually enhance their
capabilities. Specifically, TransGNN leverages Transformer layers to broaden
the receptive field and disentangle information aggregation from edges, which
aggregates information from more relevant nodes, thereby enhancing the message
passing of GNNs. Additionally, to capture graph structure information
effectively, positional encoding is meticulously designed and integrated into
GNN layers to encode such structural knowledge into node attributes, thus
enhancing the Transformer's performance on graphs. Efficiency considerations
are also alleviated by proposing the sampling of the most relevant nodes for
the Transformer, along with two efficient sample update strategies to reduce
complexity. Furthermore, theoretical analysis demonstrates that TransGNN offers
increased expressiveness compared to GNNs, with only a marginal increase in
linear complexity. Extensive experiments on five public datasets validate the
effectiveness and efficiency of TransGNN.
