Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Tarema: Adaptive Resource Allocation for Scalable Scientific Workflows
  in Heterogeneous Clusters   Scientific workflow management systems like Nextflow support large-scale data
analysis by abstracting away the details of scientific workflows. In these
systems, workflows consist of several abstract tasks, of which instances are
run in parallel and transform input partitions into output partitions. Resource
managers like Kubernetes execute such workflow tasks on cluster
infrastructures. However, these resource managers only consider the number of
CPUs and the amount of available memory when assigning tasks to resources; they
do not consider hardware differences beyond these numbers, while computational
speed and memory access rates can differ significantly.
  We propose Tarema, a system for allocating task instances to heterogeneous
cluster resources during the execution of scalable scientific workflows. First,
Tarema profiles the available infrastructure with a set of benchmark programs
and groups cluster nodes with similar performance. Second, Tarema uses online
monitoring data of tasks, assigning labels to tasks depending on their resource
usage. Third, Tarema uses the node groups and task labels to dynamically assign
task instances evenly to resources based on resource demand. Our evaluation of
a prototype implementation for Kubernetes, using five real-world Nextflow
workflows from the popular nf-core framework and two 15-node clusters
consisting of different virtual machines, shows a mean reduction of isolated
job runtimes by 19.8% compared to popular schedulers in widely-used resource
managers and 4.54% compared to the heuristic SJFN, while providing a better
cluster usage. Moreover, executing two long-running workflows in parallel and
on restricted resources shows that Tarema is able to reduce the runtimes even
more while providing a fair cluster usage.

24
The categories of this paper are: dg-ga
5
The categories of this paper are: gr-qc
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
36
The categories of this paper are: cmp-lg
18
The categories of this paper are: econ
6
The categories of this paper are: astro-ph
26
The categories of this paper are: bayes-an
13
The categories of this paper are: quant-ph
16
The categories of this paper are: stat
20
The categories of this paper are: acc-phys
17
The categories of this paper are: q-fin
0
The categories of this paper are: hep-ph
11
The categories of this paper are: hep-lat
2
The categories of this paper are: math