  Byzantine State Machine Replication (SMR) is a long studied topic that
received increasing attention recently with the advent of blockchains as
companies are trying to scale them to hundreds of nodes. Byzantine SMRs try to
increase throughput by either reducing the latency of consensus instances that
they run sequentially or by reducing the number of replicas that send messages
to others in order to reduce the network usage. Unfortunately, the former
approach makes use of resources in burst whereas the latter requires
CPU-intensive authentication mechanisms.
  In this paper, we propose a new Byzantine SMR called Dispel (Distributed
Pipeline) that allows any node to distributively start new consensus instances
whenever they detect sufficient resources locally. We evaluate the performance
of Dispel within a single datacenter and across up to 380 machines over 3
continents by comparing it against four other SMRs. On 128 nodes, Dispel speeds
up HotStuff, the Byzantine fault tolerant SMR being integrated within
Facebook's blockchain, by more than 12 times. In addition, we also test Dispel
under isolated and correlated failures and show that the Dispel distributed
design is more robust than HotStuff. Finally, we evaluate Dispel in a
cryptocurrency application with Bitcoin transactions and show that this SMR is
not the bottleneck.
