(IJCSIS) International Journal of Computer Science and Information Security, Vol. 7, No. 3, March 2010 weights on the connections between the input and the hidden units.

 The behavior of the output units depends on the activity of the hidden units and the weights between the hidden and output units.
This simple type of network is interesting because the hidden units are free to construct their own representations of the input. The weights between the input and hidden units determine when each hidden unit is active, and so by modifying these weights, a hidden unit can choose what it represents.

We also distinguish single-layer and multi-layer architectures. The single-layer organization, in which all units are connected to one another, constitutes the most general case and is of more potential computational power than hierarchically structured multi-layer organizations. 

## 7.4 Feedback Networks

Feedback networks can have signals traveling in both directions by introducing loops in the network. Feedback networks are very powerful and can get extremely complicated. Feedback networks are dynamic; their 'state' is changing continuously until they reach an equilibrium point. 

## 7.5 Back-Propagation Algorithm - A Mathematical Approach

Units are connected to one another. 

Connections correspond to the edges of the underlying directed graph. There is a real number associated with each connection, which is called the weight of the connection. We denote by Wij the weight of the connection from unit ui to unit uj. It is then convenient to represent the pattern of connectivity in the network by a weight matrix W whose elements are the weights Wij. Two types of connection are usually distinguished: excitatory and inhibitory. A positive weight represents an excitatory connection whereas a negative weight 

![0_image_0.png](0_image_0.png)

Fig 11: Back Propagation Neural Network A unit in the output layer determines its activity by following a two step procedure. 

 First, it computes the total weighted input xj, 

$$X_{j}=2$$
$\neg\neg\neg\neg$
using the formula: 
Where yi is the activity level of the jth unit in the previous layer and Wij is the weight of the connection between the ith and the jth unit.

 Next, the unit calculates the activity yj using some function of the total weighted input. 

$$\gamma_{j}={\frac{1}{1+e^{-x_{j}}}}$$

Typically we use the sigmoid function:
 Once the activities of all output units have been determined, the network computes the error E, 

* [16] A. A. K.  
which is defined by the expression:
Where yj is the activity level of the jth unit in the top layer and dj is the desired output of the jth unit.

## 7.5.1 The Back-Propagation Algorithm Consists Of Four Steps:

1. Compute how fast the error changes as the activity of an output unit is changed. This error derivative (EA) is the difference between the actual 

* [16] A. A. K.  
and the desired activity.