  Benchmarking is crucial for testing and validating any system, even more so
in real-time systems. Typical real-time applications adhere to well-understood
abstractions: they exhibit a periodic behavior, operate on a well-defined
working set, and strive for stable response time avoiding non-predicable
factors such as page faults. Unfortunately, available benchmark suites fail to
reflect key characteristics of real-time applications. Practitioners and
researchers must resort to either benchmark heavily approximated real-time
environments, or to re-engineer available benchmarks to add -- if possible --
the sought-after features. Additionally, the measuring and logging capabilities
provided by most benchmark suites are not tailored "out-of-the-box" to
real-time environments, and changing basic parameters such as the scheduling
policy often becomes a tiring and error-prone exercise.
  In this paper, we present RT-bench, an open-source framework adding standard
real-time features to virtually any existing benchmark. Furthermore, RT-bench
provides an easy-to-use, unified command line interface to customize key
aspects of the real-time execution of a set of benchmarks. Our framework is
guided by four main criteria: 1) cohesive interface, 2) support for periodic
application behavior and deadline semantics, 3) controllable memory footprint,
and 4) extensibility and portability. We have integrated within the framework
applications from the widely used SD-VBS and IsolBench suites. We showcase a
set of use-cases that are representative of typical real-time system evaluation
scenarios and that can be easily conducted via RT-Bench.
