input layer represents the independent variables while the hidden and output layers represent the latent and dependent variables, respectively (Zeng, 1999). Feed-forward neural networks provide a framework to represent a nonlinear functional mapping of a set of d **input variables** 
xi*,, i = 1, ...,d* into a set of c output variables yj *, j = 1, ..., c* **( Bishop, 1995).** 
The relationship between the input and output units of the neural network is represented by the following function (Bishop, 1995; Marwala and Lagazio, 2004)]: 

$$y_{k}=f_{\mbox{\tiny{\it source}}}\biggl{(}\sum\nolimits_{j=1}^{M}W_{kj}^{(2)}f_{\mbox{\tiny{\it inner}}}\biggl{(}\sum\nolimits_{i=1}^{d}W_{ji}^{(1)}x_{i}+W_{ji}^{(1)}\biggr{)}+W_{k0}^{(2)}\biggr{)}\tag{1}$$

where )1( Wji and )2( Wkj are the first and second layer weights going from input i **to hidden unit** j and hidden unit j to output unit k, respectively. M is number of the hidden units, d **is number of** 
input units, while )1( Wj0 and )2( Wk 0**represent biases of the hidden and output units, respectively.** 
The output activation function is given by f*oute*r while for the hidden unit it is f*inner.*

![0_image_0.png](0_image_0.png)

