  Subgraph-based graph representation learning (SGRL) has recently emerged as a
powerful tool in many prediction tasks on graphs due to its advantages in model
expressiveness and generalization ability. Most previous SGRL models face
computational challenges associated with the high cost of subgraph extraction
for each training or test query. Recently, SUREL was proposed to accelerate
SGRL, which samples random walks offline and joins these walks online as a
proxy of subgraph for representation learning. Thanks to the reusability of
sampled walks across different queries, SUREL achieves state-of-the-art
performance in terms of scalability and prediction accuracy. However, SUREL
still suffers from high computational overhead caused by node duplication in
sampled walks. In this work, we propose a novel framework SUREL+ that upgrades
SUREL by using node sets instead of walks to represent subgraphs. This
set-based representation eliminates repeated nodes by definition but can also
be irregular in size. To address this issue, we design a customized sparse data
structure to efficiently store and access node sets and provide a specialized
operator to join them in parallel batches. SUREL+ is modularized to support
multiple types of set samplers, structural features, and neural encoders to
complement the structural information loss after the reduction from walks to
sets. Extensive experiments have been performed to validate SUREL+ in the
prediction tasks of links, relation types, and higher-order patterns. SUREL+
achieves 3-11$\times$ speedups of SUREL while maintaining comparable or even
better prediction performance; compared to other SGRL baselines, SUREL+
achieves $\sim$20$\times$ speedups and significantly improves the prediction
accuracy.
