  Human ability to recognize complex visual patterns arises through
transformations performed by successive areas in the ventral visual cortex.
Deep neural networks trained end-to-end for object recognition approach human
capabilities, and offer the best descriptions to date of neural responses in
the late stages of the hierarchy. But these networks provide a poor account of
the early stages, compared to traditional hand-engineered models, or models
optimized for coding efficiency or prediction. Moreover, the gradient
backpropagation used in end-to-end learning is generally considered to be
biologically implausible. Here, we overcome both of these limitations by
developing a bottom-up self-supervised training methodology that operates
independently on successive layers. Specifically, we maximize feature
similarity between pairs of locally-deformed natural image patches, while
decorrelating features across patches sampled from other images. Crucially, the
deformation amplitudes are adjusted proportionally to receptive field sizes in
each layer, thus matching the task complexity to the capacity at each stage of
processing. In comparison with architecture-matched versions of previous
models, we demonstrate that our layerwise complexity-matched learning (LCL)
formulation produces a two-stage model (LCL-V2) that is better aligned with
selectivity properties and neural activity in primate area V2. We demonstrate
that the complexity-matched learning paradigm is critical for the emergence of
the improved biological alignment. Finally, when the two-stage model is used as
a fixed front-end for a deep network trained to perform object recognition, the
resultant model (LCL-V2Net) is significantly better than standard end-to-end
self-supervised, supervised, and adversarially-trained models in terms of
generalization to out-of-distribution tasks and alignment with human behavior.
