  We investigate the role of attention and memory in complex reasoning tasks.
We analyze Transformer-based self-attention as a model and extend it with
memory. By studying a synthetic visual reasoning test, we refine the taxonomy
of reasoning tasks. Incorporating self-attention with ResNet50, we enhance
feature maps using feature-based and spatial attention, achieving efficient
solving of challenging visual reasoning tasks. Our findings contribute to
understanding the attentional needs of SVRT tasks. Additionally, we propose
GAMR, a cognitive architecture combining attention and memory, inspired by
active vision theory. GAMR outperforms other architectures in sample
efficiency, robustness, and compositionality, and shows zero-shot
generalization on new reasoning tasks.
