and support vectors. Support vectors consist of the data points that sit on the boundaries of the acceptable region defined byε **and are extracted from the training dataset [5, 13]. This** 
constrained optimisation problem can be solved using quadratic programming with the training data and, as a result, is guaranteed to find a global optimum [5, 12]. 

$$\frac{1}{2}\|w\|^{2}+C\sum_{i=1}^{N}\Bigl{(}\xi_{i}+\xi_{i}^{*}\Bigr{)}\tag{22}$$
Minimise :
$$\begin{array}{l}{{\delta_{i}\phi(x_{i})-b\leq\varepsilon+\xi_{i}}}\\ {{\ \ \ \ w\phi(x_{i})+b-t_{i}\leq\varepsilon+\xi_{i}^{*}}}\\ {{\ \ \ \xi_{i},\xi_{i}^{*}\geq0}}\end{array}$$
Subject *to :*
_where $w$ - weight vector_  $C$ - _penalty parameter_  $N$ - _number of data points_  $\epsilon$ - _deviation from function_  $\epsilon$ - _Slack Variables_
The constraints above deal with a linear ε **-insensitive loss** function used to penalise the data points in the training dataset that are outside the specified deviation ε **. A loss** function is used to determine which function (f(x)) best describes the dependency observed in the training dataset [4]. The purpose of the loss function is to determine the cost of the difference between the actual and predicted outputs for a given set of inputs. The ε **-insensitive loss function is defined** 
by Equation 23 [4, 5]. As seen in Figure 3, in regression problems a ε **-tube is formed around the function, and any** 
data points outside this ε **-tube have an associated cost given** by Equation 23. Most data points should lay within the ε - tube, however, the slack variable allow some data points to lie outside the ε **-tube [5]. There are two slack variables to** account for the upper and lower bounds of the ε **-tube. Both** 
ε **and C are user-defined parameters. The parameter C is a** 
regularisation parameter that controls the trade-off between the complexity of the machine and the number of data points that lie outside the ε -tube [5]. The deviation ε **, determines** 
the approximation accuracy enforced on the training data points [13]. For regression, the parameters C and ε **should be** 
tuned simultaneously [5]. 

$$\ell(f(x),t)=\begin{cases}\left|f(x)-t\right|-\varepsilon&\text{if}\left|f(x)-t\right|>\varepsilon\\ 0&\text{otherwise}\end{cases}$$

$\eqref{eq:walpha}$
There are other ε **-insensitive loss functions that can be used** such as a quadratic ε **-insensitive loss function. Also, a least** 
squares cost function can be used. This results in a Least Squares Support Vector Machine (LS-SVM) that has a few different properties to the original Vapnik's SVM presented above. 

In a least squares SVM, the ε **-insensitive loss function is** 
replaced by a least squares cost function which corresponds to a form of ridge regression [14]. The inequality constraints that Equation 22 is subject to are replaced by equality. As a consequence, the training process of a LS-SVM involves solving a set of linear equations instead of a quadratic programming problem. The set of linear equations that result are of the dimension N+1, where N is the number of training samples [15]. In the case of a standard SVM, the quadratic programming (QP) problem to be solved is roughly exponential to the size of the training dataset [14]. Therefore, the number of training samples used to train an SVM should be considered carefully. However, the set of linear equations is still not as time and computationally consuming to solve as the QP problem. 

In a LS-SVM, the weight vector that results from minimizing the summed squared approximation error over all training samples is searched for, where the approximation error is the difference between the SVM's output and the desired target output [15]. The equation for a LS-SVM is shown below in Equation 24 [15]. 

$$\frac{1}{2}\left\|w\right\|^{2}+C\sum_{i=0}^{N}e_{i}^{2}\tag{24}$$
Minimise :
$$t_{i}=w\phi(x_{i})+b+e_{i}$$
Subject *to :*
where ei = ti- f(xi) 
The main difference between Neural networks and Support Vector Machines is that support vector machines minimise an upper bound of the generalisation errors instead of minimising the error on the training dataset [13]. Support vector machines utilise risk minimisation, measured using a loss function. Normally, support vector machines have a slower execution time as there is little control over the number of support vectors defined [5]. An SVM has less parameters to tune than a neural network, and the optimisation procedure can be performed efficiently. In the case of the LS-SVM, the parameters that need to be tuned are the penalty or regularisation constant and the deviation of the Gaussian function, if an RBF kernel is used. While for a standard SVM the ε accuracy for the ε **-insensitive loss** 
function needs to determined additionally. 

## V. Fuzzy Logic And Neuro-Fuzzy **Systems**

Neuro-Fuzzy Systems are based on Fuzzy logic which was formulated in the 1960s by Zadeh. These systems combine Fuzzy Logic and certain principles of Neural Networks in 