  Developing a generative model of realistic order flow in financial markets is
a challenging open problem, with numerous applications for market participants.
Addressing this, we propose the first end-to-end autoregressive generative
model that generates tokenized limit order book (LOB) messages. These messages
are interpreted by a Jax-LOB simulator, which updates the LOB state. To handle
long sequences efficiently, the model employs simplified structured state-space
layers to process sequences of order book states and tokenized messages. Using
LOBSTER data of NASDAQ equity LOBs, we develop a custom tokenizer for message
data, converting groups of successive digits to tokens, similar to tokenization
in large language models. Out-of-sample results show promising performance in
approximating the data distribution, as evidenced by low model perplexity.
Furthermore, the mid-price returns calculated from the generated order flow
exhibit a significant correlation with the data, indicating impressive
conditional forecast performance. Due to the granularity of generated data, and
the accuracy of the model, it offers new application areas for future work
beyond forecasting, e.g. acting as a world model in high-frequency financial
reinforcement learning applications. Overall, our results invite the use and
extension of the model in the direction of autoregressive large financial
models for the generation of high-frequency financial data and we commit to
open-sourcing our code to facilitate future research.
