  Recent works indicate that convolutional neural networks (CNN) need large
receptive fields (RF) to compete with visual transformers and their attention
mechanism. In CNNs, RFs can simply be enlarged by increasing the convolution
kernel sizes. Yet the number of trainable parameters, which scales
quadratically with the kernel's size in the 2D case, rapidly becomes
prohibitive, and the training is notoriously difficult. This paper presents a
new method to increase the RF size without increasing the number of parameters.
The dilated convolution (DC) has already been proposed for the same purpose. DC
can be seen as a convolution with a kernel that contains only a few non-zero
elements placed on a regular grid. Here we present a new version of the DC in
which the spacings between the non-zero elements, or equivalently their
positions, are no longer fixed but learnable via backpropagation thanks to an
interpolation technique. We call this method "Dilated Convolution with
Learnable Spacings" (DCLS) and generalize it to the n-dimensional convolution
case. However, our main focus here will be on the 2D case. We first tried our
approach on ResNet50: we drop-in replaced the standard convolutions with DCLS
ones, which increased the accuracy of ImageNet1k classification at
iso-parameters, but at the expense of the throughput. Next, we used the recent
ConvNeXt state-of-the-art convolutional architecture and drop-in replaced the
depthwise convolutions with DCLS ones. This not only increased the accuracy of
ImageNet1k classification but also of typical downstream and robustness tasks,
again at iso-parameters but this time with negligible cost on throughput, as
ConvNeXt uses separable convolutions. Conversely, classic DC led to poor
performance with both ResNet50 and ConvNeXt. The code of the method is
available at:
https://github.com/K-H-Ismail/Dilated-Convolution-with-Learnable-Spacings-PyTorch.
