ursuit of reduced modeling complexity. Building on these improvements,
we reduce the computational complexity of attention calculation from O(N^2) to
O(Nk), enabling training with just eight NVIDIA RTX 4090 GPUs and eliminating
the need for KV cache during inference. Extensive experiments on ImageNet
demonstrate that MVAR achieves comparable or superior performance with both
small model trained from scratch and large fine-tuned models, while reducing
the average GPU memory footprint by 3.0x.</p>
           