  Data processing engines increasingly leverage distributed file systems for
scalable, cost-effective storage. While the Apache Parquet columnar format has
become a popular choice for data storage and retrieval, the immutability of
Parquet files renders it impractical to meet the demands of frequent updates in
contemporary analytical workloads. Log-Structured Tables (LSTs), such as Delta
Lake, Apache Iceberg, and Apache Hudi, offer an alternative for scenarios
requiring data mutability, providing a balance between efficient updates and
the benefits of columnar storage. They provide features like transactions,
time-travel, and schema evolution, enhancing usability and enabling access from
multiple engines. Moreover, engines like Apache Spark and Trino can be
configured to leverage the optimizations and controls offered by LSTs to meet
specific business needs. Conventional benchmarks and tools are inadequate for
evaluating the transformative changes in the storage layer resulting from these
advancements, as they do not allow us to measure the impact of design and
optimization choices in this new setting.
  In this paper, we propose a novel benchmarking approach and metrics that
build upon existing benchmarks, aiming to systematically assess LSTs. We
develop a framework, LST-Bench, which facilitates effective exploration and
evaluation of the collaborative functioning of LSTs and data processing engines
through tailored benchmark packages. A package is a mix of use patterns
reflecting a target workload; LST-Bench makes it easy to define a wide range of
use patterns and combine them into a package, and we include a baseline package
for completeness. Our assessment demonstrates the effectiveness of our
framework and benchmark packages in extracting valuable insights across diverse
environments. The code for LST-Bench is open-sourced and is available at
https://github.com/microsoft/lst-bench/ .
