  Key-based workload partitioning is a common strategy used in parallel stream
processing engines, enabling effective key-value tuple distribution over worker
threads in a logical operator. While randomized hashing on the keys is capable
of balancing the workload for key-based partitioning when the keys generally
follow a static distribution, it is likely to generate poor balancing
performance when workload variance occurs on the incoming data stream. This
paper presents a new key-based workload partitioning framework, with practical
algorithms to support dynamic workload assignment for stateful operators. The
framework combines hash-based and explicit key-based routing strategies for
workload distribution, which specifies the destination worker threads for a
handful of keys and assigns the other keys with the hashing function. When
short-term distribution fluctuations occur to the incoming data stream, the
system adaptively updates the routing table containing the chosen keys, in
order to rebalance the workload with minimal migration overhead within the
stateful operator. We formulate the rebalance operation as an optimization
problem, with multiple objectives on minimizing state migration costs,
controlling the size of the routing table and breaking workload imbalance among
worker threads. Despite of the NP-hardness nature behind the optimization
formulation, we carefully investigate and justify the heuristics behind key
(re)routing and state migration, to facilitate fast response to workload
variance with ignorable cost to the normal processing in the distributed
system. Empirical studies on synthetic data and real-world stream applications
validate the usefulness of our proposals and prove the huge advantage of our
approaches over state-of-the-art solutions in the literature.
