  Deep learning methods have shown outstanding performance in many
applications, including single-image super-resolution (SISR). With residual
connection architecture, deeply stacked convolutional neural networks provide a
substantial performance boost for SISR, but their huge parameters and
computational loads are impractical for real-world applications. Thus,
designing lightweight models with acceptable performance is one of the major
tasks in current SISR research. The objective of lightweight network design is
to balance a computational load and reconstruction performance. Most of the
previous methods have manually designed complex and predefined fixed
structures, which generally required a large number of experiments and lacked
flexibility in the diversity of input image statistics. In this paper, we
propose a dynamic residual self-attention network (DRSAN) for lightweight SISR,
while focusing on the automated design of residual connections between building
blocks. The proposed DRSAN has dynamic residual connections based on dynamic
residual attention (DRA), which adaptively changes its structure according to
input statistics. Specifically, we propose a dynamic residual module that
explicitly models the DRA by finding the interrelation between residual paths
and input image statistics, as well as assigning proper weights to each
residual path. We also propose a residual self-attention (RSA) module to
further boost the performance, which produces 3-dimensional attention maps
without additional parameters by cooperating with residual structures. The
proposed dynamic scheme, exploiting the combination of DRA and RSA, shows an
efficient trade-off between computational complexity and network performance.
Experimental results show that the DRSAN performs better than or comparable to
existing state-of-the-art lightweight models for SISR.
