In order to try to discriminate between the modeling error and the method error, a fifth model is also run, on the same test cases and with similar experimental conditions than the four developmental approaches described above:
the layout is exactly the same (a 2D grid of cells), the same NEAT parameters are used (to evolve a feedforward neural network), and selection proceeds using the same fitness.

However, there is no chemical nor any exchange of information between neighboring cells, and on the other hand, all cells receive as inputs their (x,y) coordinates on the grid.

Hence the flag approximation problem is reduced to a simple regression problem. In the following, the results of this model will be considered as reference results, as it is not expected that any developmental approach can ever beat a totally informed model using the same NEAT optimization tool. This experiment is termed "f(x, y) = z**" from now on.**

## 3.3 Experimental Setup

All 5 models described in previous section have been run on the 4 flags showed on Figure 2. All results presented in the following are statistics over 16 independent runs.

As already said, the evolutionary neural network optimizer is NEAT, with the settings that are described in Table 1. It is worth noticing that during all runs, no bloat was ever observed for the NEAT genotypes. The mean size of the networks (measured by the total number of edges between neurons) gently grew from its starting value (between 5 and 10 depending on the model) to some final value below 40
- the largest experiment reaching 45. This first result confirms the robustness of this optimization tool, but also, to some extent, demonstrates the well-posedness of the problems NEAT was solving (bloating for Neural Networks can be a sign of overfitting ill-conditioned data).

As argued in section 2.3, the halting of the growth process is based on the stabilization of the energy of the organism, checked over some time window. The width of this time window has been set to 8 time-steps in all experiments. However, because not all networks will stabilize, a maximum number of iterations has to be imposed. This maximum number was set to 1024, and if no stabilization has occurred at that time, the fitness is set to value 0: as 0 is the worst possible value for the fitness, this amounts to using some death penalty for the stabilization constraint.

With such settings, a typical run lasts about one day on a 3.4GHz Pentium IV. Though this might seem a huge computational cost, we believe that it is not a critical issue when designing real-world structures: On the one hand, designing mechanical parts is already a time-consuming process, involving highly trained engineers - and human time nowadays costs much more than CPU time. On the other hand, when the structure that is being designed is bound to be built by thousands or millions, a few days represent a very small overhead indeed.

## 3.4 Results

3.4.1 Comparing fitnesses The statistics for the off-line results are displayed as the usual box-plots1**on Figures 3, 4 and 5 respectively for the**
1**as generated by the R statistical package, see**
http://en.wikipedia.org/wiki/Box plot **for a precise** description.

![0_image_0.png](0_image_0.png) ![0_image_1.png](0_image_1.png)

3-bands, disc and half-discs problems of Figure 2, and online results (the average over the 16 runs of the fitness of the best-of-generation individuals as evolution progresses)
are shown on Figure 6 for the 3-bands problem.

The results for the 2-bands problem are almost identical for the 5 models, and are not presented here: same average fitness of 0.999, with a slightly larger variance for the developmental approaches (and variance 0 for the regression model). For each setting of the embryogenic approach, though, some runs were able to find a marginally better solution than that of the regression model - but without any statistical significance. For the slightly more difficult target three-bands, the reference model is still able to find an ex-

![0_image_2.png](0_image_2.png)

