  Attention-based Neural Networks (NN) have demonstrated their effectiveness in
accurate memory access prediction, an essential step in data prefetching.
However, the substantial computational overheads associated with these models
result in high inference latency, limiting their feasibility as practical
prefetchers. To close the gap, we propose a new approach based on
tabularization that significantly reduces model complexity and inference
latency without sacrificing prediction accuracy. Our novel tabularization
methodology takes as input a distilled, yet highly accurate attention-based
model for memory access prediction and efficiently converts its expensive
matrix multiplications into a hierarchy of fast table lookups. As an exemplar
of the above approach, we develop DART, a prefetcher comprised of a simple
hierarchy of tables. With a modest 0.09 drop in F1-score, DART reduces 99.99%
of arithmetic operations from the large attention-based model and 91.83% from
the distilled model. DART accelerates the large model inference by 170x and the
distilled model by 9.4x. DART has comparable latency and storage costs as
state-of-the-art rule-based prefetcher BO but surpasses it by 6.1% in IPC
improvement. DART outperforms state-of-the-art NN-based prefetchers TransFetch
by 33.1% and Voyager by 37.2% in terms of IPC improvement, primarily due to its
low prefetching latency.
