Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks   In many robotics and VR/AR applications, 3D-videos are readily-available
sources of input (a continuous sequence of depth images, or LIDAR scans).
However, those 3D-videos are processed frame-by-frame either through 2D
convnets or 3D perception algorithms. In this work, we propose 4-dimensional
convolutional neural networks for spatio-temporal perception that can directly
process such 3D-videos using high-dimensional convolutions. For this, we adopt
sparse tensors and propose the generalized sparse convolution that encompasses
all discrete convolutions. To implement the generalized sparse convolution, we
create an open-source auto-differentiation library for sparse tensors that
provides extensive functions for high-dimensional convolutional neural
networks. We create 4D spatio-temporal convolutional neural networks using the
library and validate them on various 3D semantic segmentation benchmarks and
proposed 4D datasets for 3D-video perception. To overcome challenges in the 4D
space, we propose the hybrid kernel, a special case of the generalized sparse
convolution, and the trilateral-stationary conditional random field that
enforces spatio-temporal consistency in the 7D space-time-chroma space.
Experimentally, we show that convolutional neural networks with only
generalized 3D sparse convolutions can outperform 2D or 2D-3D hybrid methods by
a large margin. Also, we show that on 3D-videos, 4D spatio-temporal
convolutional neural networks are robust to noise, outperform 3D convolutional
neural networks and are faster than the 3D counterpart in some cases.

22
The categories of this paper are: chao-dyn
13
The categories of this paper are: quant-ph
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
6
The categories of this paper are: astro-ph
15
The categories of this paper are: nucl-ex
29
The categories of this paper are: funct-an
7
The categories of this paper are: hep-th
0
The categories of this paper are: hep-ph
2
The categories of this paper are: math
30
The categories of this paper are: q-alg
10
The categories of this paper are: q-bio
17
The categories of this paper are: q-fin
16
The categories of this paper are: stat
27
The categories of this paper are: comp-gas
19
The categories of this paper are: eess