  As real-time analysis of the new data become increasingly compelling, more
organizations deploy Hybrid Transactional/Analytical Processing (HTAP) systems
to support real-time queries on data recently generated by online transaction
processing. This paper argues that real-time queries, semantically consistent
schema, and domain-specific workloads are essential in benchmarking, designing,
and implementing HTAP systems. However, most state-of-the-art and
state-of-the-practice benchmarks ignore those critical factors. Hence, they are
incommensurable and, at worst, misleading in benchmarking, designing, and
implementing HTAP systems. This paper presents OLxPBench, a composite HTAP
benchmark suite. OLxPBench proposes: (1) the abstraction of a hybrid
transaction, performing a real-time query in-between an online transaction, to
model widely-observed behavior pattern -- making a quick decision while
consulting real-time analysis; (2) a semantically consistent schema to express
the relationships between OLTP and OLAP schema; (3) the combination of
domain-specific and general benchmarks to characterize diverse application
scenarios with varying resource demands. Our evaluations justify the three
design decisions of OLxPBench and pinpoint the bottlenecks of two mainstream
distributed HTAP DBMSs. International Open Benchmark Council (BenchCouncil)
sets up the OLxPBench homepage at https://www.benchcouncil.org/olxpbench/. Its
source code is available from https://github.com/BenchCouncil/olxpbench.git.
