  Statistical mechanics is used to study unrealizable generalization in two
large feed-forward neural networks with binary weights and output, a perceptron
and a tree committee machine. The student is trained by a teacher being larger,
i.e. having more units than the student. It is shown that this is the same as
using training data corrupted by Gaussian noise. Each machine is considered in
the high temperature limit and in the replica symmetric approximation as well
as for one step of replica symmetry breaking. For the perceptron a phase
transition is found for low noise. However the transition is not to optimal
learning. If the noise is increased the transition disappears. In both cases
$\epsilon _{g}$ will approach optimal performance with a $(\ln\alpha
/\alpha)^k$ decay for large $\alpha$. For the tree committee machine noise in
the input layer is studied, as well as noise in the hidden layer. If there is
no noise in the input layer there is, in the case of one step of repl! ica
symmetry breaking, a phase tra nsition to optimal learning at some finite
$\alpha$ for all levels of noise in the hidden layer. When noise is added to
the input layer the generalization behavior is similar to that of the
perceptron. For one step of replica symmetry breaking, in the realizable limit,
the values of the spinodal points found in this paper disagree with previously
reported estimates \cite{seung1},\cite{schwarze1}. Here the value $\alpha _{sp}
= 2.79$ is found for the tree committee machine and $\alpha _{sp} = 1.67$ for
the perceptron.
