  Lakehouse systems have reached in the past few years unprecedented size and
heterogeneity and have been embraced by many industry players. However, they
are often difficult to use as they lack the declarative language and
optimization possibilities of relational engines. This paper introduces
RumbleML, a high-level, declarative library integrated into the RumbleDB engine
and with the JSONiq language. RumbleML allows using a single platform for data
cleaning, data preparation, training, and inference, as well as management of
models and results. It does it using a purely declarative language (JSONiq) for
all these tasks and without any performance loss over existing platforms (e.g.
Spark). The key insights of the design of RumbleML are that training sets,
evaluation sets, and test sets can be represented as homogeneous sequences of
flat objects; that models can be seamlessly embodied in function items mapping
input test sets into prediction-augmented result sets; and that estimators can
be seamlessly embodied in function items mapping input training sets to models.
We argue that this makes JSONiq a viable and seamless programming language for
data lakehouses across all their features, whether database-related or
machine-learning-related. While lakehouses bring Machine Learning and Data
Wrangling on the same platform, RumbleML also brings them to the same language,
JSONiq. In the paper, we present the first prototype and compare its
performance to Spark showing the benefit of a huge functionality and
productivity gain for cleaning up, normalizing, validating data, feeding it
into Machine Learning pipelines, and analyzing the output, all within the same
system and language and at scale.
