Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: LambdaNetworks: Modeling Long-Range Interactions Without Attention   We present lambda layers -- an alternative framework to self-attention -- for
capturing long-range interactions between an input and structured contextual
information (e.g. a pixel surrounded by other pixels). Lambda layers capture
such interactions by transforming available contexts into linear functions,
termed lambdas, and applying these linear functions to each input separately.
Similar to linear attention, lambda layers bypass expensive attention maps, but
in contrast, they model both content and position-based interactions which
enables their application to large structured inputs such as images. The
resulting neural network architectures, LambdaNetworks, significantly
outperform their convolutional and attentional counterparts on ImageNet
classification, COCO object detection and COCO instance segmentation, while
being more computationally efficient. Additionally, we design LambdaResNets, a
family of hybrid architectures across different scales, that considerably
improves the speed-accuracy tradeoff of image classification models.
LambdaResNets reach excellent accuracies on ImageNet while being 3.2 - 4.4x
faster than the popular EfficientNets on modern machine learning accelerators.
When training with an additional 130M pseudo-labeled images, LambdaResNets
achieve up to a 9.5x speed-up over the corresponding EfficientNet checkpoints.

35
The categories of this paper are: mtrl-th
25
The categories of this paper are: patt-sol
30
The categories of this paper are: q-alg
6
The categories of this paper are: astro-ph
24
The categories of this paper are: dg-ga
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
3
The categories of this paper are: physics
34
The categories of this paper are: chem-ph
26
The categories of this paper are: bayes-an
15
The categories of this paper are: nucl-ex
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci
33
The categories of this paper are: plasm-ph
14
The categories of this paper are: math-ph