  The rapidly growing parameter volume of deep neural networks (DNNs) hinders
the artificial intelligence applications on resource constrained devices, such
as mobile and wearable devices. Neural network pruning, as one of the
mainstream model compression techniques, is under extensive study to reduce the
number of parameters and computations. In contrast to irregular pruning that
incurs high index storage and decoding overhead, structured pruning techniques
have been proposed as the promising solutions. However, prior studies on
structured pruning tackle the problem mainly from the perspective of
facilitating hardware implementation, without analyzing the characteristics of
sparse neural networks. The neglect on the study of sparse neural networks
causes inefficient trade-off between regularity and pruning ratio.
Consequently, the potential of structurally pruning neural networks is not
sufficiently mined.
  In this work, we examine the structural characteristics of the irregularly
pruned weight matrices, such as the diverse redundancy of different rows, the
sensitivity of different rows to pruning, and the positional characteristics of
retained weights. By leveraging the gained insights as a guidance, we first
propose the novel block-max weight masking (BMWM) method, which can effectively
retain the salient weights while imposing high regularity to the weight matrix.
As a further optimization, we propose a density-adaptive regular-block (DARB)
pruning that outperforms prior structured pruning work with high pruning ratio
and decoding efficiency. Our experimental results show that DARB can achieve
13$\times$ to 25$\times$ pruning ratio, which are 2.8$\times$ to 4.3$\times$
improvements than the state-of-the-art counterparts on multiple neural network
models and tasks. Moreover, DARB can achieve 14.3$\times$ decoding efficiency
than block pruning with higher pruning ratio.
