From, the results in Tables 4 - 6, it can be seen that the Bayesian MLP gave a better accuracy than the single MLP implemented using standard approaches. However, it took a substantial amount more time to train and execute compared to the single MLP. 

The Bayesian techniques using Hybrid Monte Carlo were attempted with an RBF, however, difficulties were experienced and no definite results were obtained. 

## F. **Support Vector Machines**

The LS-SVMlab Toolbox for Matlab was used to simulate the SVM for the given dataset. This toolbox implements Least Squares Support Vector Machines for both classification and Regression problems [24]. Another toolbox that implemented the ε **-insensitivity loss function SVM was found, however,** 
to run a simulation was extremely time consuming even when the number of samples used to train the SVM were substantially decreased. 

Since an SVM determines an unknown dependency between a set of inputs and an output, the toolbox handles the case of multiple outputs by treating each of the outputs separately. Therefore, there are effectively 4 SVMs modelling the dataset. This is different to the neural networks where one network was trained to model all 4 outputs. As a result, 4 SVMs were simultaneously implemented and trained, one for each output of the dataset. 

The implementation of the LS-SVM required the selection of two free parameters since a Radial Basis function was used for the kernel function. Therefore, the optimum values of the two free parameters needed to be determined: the width or bandwidth of basis function (σ 2**), and the regularisation or** 
penalty parameter (C). An empirical approach was taken in determining the 2 free parameters, and is similar to the approach taken in [13]. Since there are 4 outputs, the parameters for each corresponding SVM had to be determined. The procedure used is discussed below with respect to the determination of the parameters for modelling Output 1. The same procedure was used for the determination of the parameters for the other outputs in the dataset. 

First, the regularisation constant was set at a value of 10 while varying the bandwidth of the basis function for training data corresponding to Output 1. The basis function's width was varied for values from 0.3 to 1000. For a small σ 2**, the** 
training error was at its minimum; however, the validation error was very large. This gives an indication that the LSSVM is over-trained for small σ 2**. At around** σ 2 **= 1, the** 
training and validation errors crossed and remained constant for a while. Then from about σ 2 **= 10, both the validation and** 
training error started to increase which indicates that the SVM was not even able to model the training data well for large values of σ 2**, and is under-trained. An appropriate** 
choice for the bandwidth of the basis function was decided to be 1, from the above experiments carried out. 

Secondly, the bandwidth of the basis function was kept constant at 1 while the value of the regularisation constant was varied. The regularisation constant was varied between 1 and 1000 while observing the training and validation errors.

As the regularisation constant (C) was increased both the validation and training error decreased together, until a certain point where the validation error started to increase while the testing error continued to decrease. Thus, for a small value of C it appears to under-fit the training data, and for large values of C the SVM appears to over-fit the training dataset. Therefore, the most appropriate value for the regularisation constant was 10, as beyond this value the validation error starts to increase. The optimum parameter values chosen to model Output 1 were C=10 and σ 2**= 1. The** 
optimum parameters for each of the SVM corresponding to the 4 outputs can be seen in Table 7. Figures 14-17 show the Actual vs. the Predicted values of the first 60 samples of the each output for test dataset applied to the LS-SVMs. 

| Table 7: Showing the results obtained for the implementations of the LS-SVM  SVM Output1 Output2 Output3 Output4  MSE for Test Dataset 0.023300 0.023600 0.01125 0.015400  Training Time (s) 90 60 50 120  Execution Time (s) 2.7 2s 2.7 2.6  σ 2 1 1 10 0.1  C 10 1 10 10   |
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

From Table 7, it can be seen that the LS-SVM took longer to train and execute than the neural networks produced using the standard approach. Even tough, the neural network was modelling 4 relationships it was much faster than the LSSVM which is only modelling one relationship at a time. The results obtained from LS-LVM were easily reproducible as opposed to neural networks where one can easily obtain different and less accurate results when the simulation is rerun, due to the optimisation techniques used. If the error of each of the SVMs are added together as if they 

![0_image_0.png](0_image_0.png) are working in a committee to predicted each output of the Steam Generator separately, the effective MSE would be approximately 0.07355. 