Instruct: Identify the main category of Biorxiv papers based on the titles and abstracts
Query: The Flip-flop neuron - A memory efficient alternative for solving challenging sequence processing and decision making problems Sequential decision making tasks that require information integration over extended durations of time are challenging for several reasons including the problem of vanishing gradients, long training times and significant memory requirements. To this end we propose a neuron model fashioned after the JK flip-flops in digital systems. A flip-flop is a sequential device that can store state information of the previous history. We incorporate the JK flip-flop neuron into several deep network architectures and apply the networks to difficult sequence processing problems. The proposed architectures include flip-flop neural networks (FFNNs), bidirectional flip-flop neural networks (BiFFNNs), convolutional flip-flop neural networks (ConvFFNNs), and bidirectional convolutional flip-flop neural networks (BiConvFFNNs). Learning rules of proposed architectures have also been derived. We have considered the most popular benchmark sequential tasks like signal generation, sentiment analysis, handwriting generation, text generation, video frame prediction, lung volume prediction, and action recognition to evaluate the proposed networks. Finally, we compare the results of our networks with the results from analogous networks with Long Short-Term Memory (LSTM) neurons on the same sequential tasks. Our results show that the JK flip-flop networks outperform the LSTM networks significantly or marginally on all the tasks, with only half of the trainable parameters.
1
The categories of this paper are: bioinformatics
11
The categories of this paper are: cognition
5
The categories of this paper are: cell
10
The categories of this paper are: and
7
The categories of this paper are: ecology
20
The categories of this paper are: pharmacology
18
The categories of this paper are: systems
6
The categories of this paper are: plant
22
The categories of this paper are: physiology
0
The categories of this paper are: genetics
26
The categories of this paper are: biochemistry
29
The categories of this paper are: education
21
The categories of this paper are: toxicology
28
The categories of this paper are: communication
25
The categories of this paper are: bioengineering
31
The categories of this paper are: paleontology