  In this paper we introduce a new method which employs the concept of
"Orientation Vectors" to train a feed forward neural network and suitable for
problems where large dimensions are involved and the clusters are
characteristically sparse. The new method is not NP hard as the problem size
increases. We `derive' the method by starting from Kolmogrov's method and then
relax some of the stringent conditions. We show for most classification
problems three layers are sufficient and the network size depends on the number
of clusters. We prove as the number of clusters increase from N to N+dN the
number of processing elements in the first layer only increases by d(logN), and
are proportional to the number of classes, and the method is not NP hard.
  Many examples are solved to demonstrate that the method of Orientation
Vectors requires much less computational effort than Radial Basis Function
methods and other techniques wherein distance computations are required, in
fact the present method increases logarithmically with problem size compared to
the Radial Basis Function method and the other methods which depend on distance
computations e.g statistical methods where probabilistic distances are
calculated. A practical method of applying the concept of Occum's razor to
choose between two architectures which solve the same classification problem
has been described. The ramifications of the above findings on the field of
Deep Learning have also been briefly investigated and we have found that it
directly leads to the existence of certain types of NN architectures which can
be used as a "mapping engine", which has the property of "invertibility", thus
improving the prospect of their deployment for solving problems involving Deep
Learning and hierarchical classification. The latter possibility has a lot of
future scope in the areas of machine learning and cloud computing.
