Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Why and when should you pool? Analyzing Pooling in Recurrent
  Architectures   Pooling-based recurrent neural architectures consistently outperform their
counterparts without pooling. However, the reasons for their enhanced
performance are largely unexamined. In this work, we examine three commonly
used pooling techniques (mean-pooling, max-pooling, and attention), and propose
max-attention, a novel variant that effectively captures interactions among
predictive tokens in a sentence. We find that pooling-based architectures
substantially differ from their non-pooling equivalents in their learning
ability and positional biases--which elucidate their performance benefits. By
analyzing the gradient propagation, we discover that pooling facilitates better
gradient flow compared to BiLSTMs. Further, we expose how BiLSTMs are
positionally biased towards tokens in the beginning and the end of a sequence.
Pooling alleviates such biases. Consequently, we identify settings where
pooling offers large benefits: (i) in low resource scenarios, and (ii) when
important words lie towards the middle of the sentence. Among the pooling
techniques studied, max-attention is the most effective, resulting in
significant performance gains on several text classification tasks.

18
The categories of this paper are: econ
28
The categories of this paper are: alg-geom
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
13
The categories of this paper are: quant-ph
10
The categories of this paper are: q-bio
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
36
The categories of this paper are: cmp-lg
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
14
The categories of this paper are: math-ph
30
The categories of this paper are: q-alg
22
The categories of this paper are: chao-dyn
25
The categories of this paper are: patt-sol