Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: OS4M: Achieving Global Load Balance of MapReduce Workload by Scheduling
  at the Operation Level   The efficiency of MapReduce is closely related to its load balance. Existing
works on MapReduce load balance focus on coarse-grained scheduling. This study
concerns fine-grained scheduling on MapReduce operations, with each operation
representing one invocation of the Map or Reduce function. By default,
MapReduce adopts the hash-based method to schedule Reduce operations, which
often leads to poor load balance. In addition, the copy phase of Reduce tasks
overlaps with Map tasks, which significantly hinders the progress of Map tasks
due to I/O contention. Moreover, the three phases of Reduce tasks run in
sequence, while consuming different resources, thereby under-utilizing
resources. To overcome these problems, we introduce a set of mechanisms named
OS4M (Operation Scheduling for MapReduce) to improve MapReduce's performance.
OS4M achieves load balance by collecting statistics of all Map operations, and
calculates a globally optimal schedule to distribute Reduce operations. With
OS4M, the copy phase of Reduce tasks no longer overlaps with Map tasks, and the
three phases of Reduce tasks are pipelined based on their operation loads. OS4M
has been transparently incorporated into MapReduce. Evaluations on standard
benchmarks show that OS4M's job duration can be shortened by up to 42%,
compared with a baseline of Hadoop.

22
The categories of this paper are: chao-dyn
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
30
The categories of this paper are: q-alg
37
The categories of this paper are: supr-con
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
29
The categories of this paper are: funct-an
19
The categories of this paper are: eess
32
The categories of this paper are: atom-ph
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
12
The categories of this paper are: nucl-th
36
The categories of this paper are: cmp-lg