Alternatives for the overall architecture will also be looked for. One crucial issue in autonomous robotics is the adaptivity of the controller. Several architectures have been proposed in that direction (see [13] and references herein) and will be tried, like for instance the idea of auto-teaching networks.

Finally, in the longer run, the library approach helps to keep tracks of the behavior of the robot at a level of generality that can be later exploited by some data mining technique. Gathering the Frequent Item Sets in the best evolved controllers can help deriving some brand new macro-actions. The issue will then be to check how useful such macro-actions can be if added to the library.

## References

1. P. J. Bentley, editor. *Evolutionary Design by Computers***. Morgan Kaufman Publishers Inc., 1999.**
2. R. A. Brooks. A robust layered control system for a mobile robot. *Journal of* Robotics and Automation**, 2(1):14–23, 1986.**
3. R. A. Brooks. How to build complete creatures rather than isolated cognitive simulators. In Kurt VanLehn, editor, Architectures for Intelligence: The 22nd Carnegie Mellon Symp. on Cognition**, pages 225–239. Lawrence Erlbaum Associates, 1991.**
4. R. Dawkins. *The blind watchmaker***. Norton, W. W. and Company, 1988.**
5. D. Floreano and F. Mondada. Evolution of homing navigation in a real mobile robot. *IEEE Transactions on Systems, Man, and Cybernetics***, 26:396–407, 1994.**
6. I. Harvey. *The Artificial Evolution of Adaptive Behaviour***. PhD thesis, University**
of Sussex, 1993.

7. M. Humphrys. *Action Selection Methods Using Reinforcement Learning***. PhD**
thesis, University of Cambridge, 1997.

8. Leslie Pack Kaelbling, Michael L. Littman, and Andrew P. Moore. Reinforcement learning: A survey. *Journal of Artificial Intelligence Research***, 4:237–285, 1996.**
9. M. Keijzer, J. J. Merelo, G. Romero, and M. Schoenauer. Evolving objects: a general purpose evolutionary computation library. In P. Collet et al., editors, Artificial Evolution'01**, pages 229–241. Springer Verlag, LNCS 2310, 2002.**
10. P. Maes. The dynamics of action selection. In Proceedings of the 11th International Joint Conference on Artificial Intelligence**, 1989.**
11. J.R. Millan, D. Posenato, and E. Dedieu. Continuous-action Q-learning. Machine Learning Journal**, 49(23):247–265, 2002.**
12. S. Nolfi and D. Floreano. How co-evolution can enhance the **adaptive power of**
artificial evolution: implications for evolutionary robotics. In P. Husbands and J.A.

Meyer, editors, *Proceedings of EvoRobot98***, pages 22–38. Springer Verlag, 1998.**
13. S. Nolfi and D. Floreano. *Evolutionary Robotics***. MIT Press, 2000.**
14. H.-P. Schwefel. *Numerical Optimization of Computer Models***. John Wiley & Sons,**
New-York, 1981. 1995 - 2nd **edition.**
15. R.S. Sutton and A. G. Barto. *Reinforcement learning***. MIT Press, 1998.**
16. E. Tuci, I. Harvey, and M. Quinn. Evolving integrated controllers for autonomous learning robots using dynamic neural networks. In B. Hallam **et al., editor,** *Proc.*
of SAB'02**. MIT Press, 2002.**
17. V. N. Vapnik. *Statistical Learning Theory***. Wiley, 1998.**
18. B.M. Yamauchi and R.D. Beer. Integrating reactive, sequential, and learning behavior using dynamical neural network. In D. Cliff et al., editor, *Proc. SAB'94*.

MIT Press, 1994.