  Pruning coupled with learning aims to optimize the neural network (NN)
structure for solving specific problems. This optimization can be used for
various purposes: to prevent overfitting, to save resources for implementation
and training, to provide explainability of the trained NN, and many others. The
minimal structure that cannot be pruned further is not unique. Ensemble of
minimal structures can be used as a committee of intellectual agents that
solves problems by voting. Each minimal NN presents an "empirical knowledge"
about the problem and can be verbalized. The non-uniqueness of such knowledge
extracted from data is an important property of data-driven Artificial
Intelligence (AI). In this work, we review an approach to pruning based on the
principle: What controls training should control pruning. This principle is
expected to work both for artificial NN and for selection and modification of
important synaptic contacts in brain. In back-propagation artificial NN
learning is controlled by the gradient of loss functions. Therefore, the first
order sensitivity indicators are used for pruning and the algorithms based on
these indicators are reviewed. The notion of logically transparent NN was
introduced. The approach was illustrated on the problem of political
forecasting: predicting the results of the US presidential election. Eight
minimal NN were produced that give different forecasting algorithms. The
non-uniqueness of solution can be utilised by creation of expert panels
(committee). Another use of NN pluralism is to identify areas of input signals
where further data collection is most useful. In Conclusion, we discuss the
possible future of widely advertised XAI program.
