# Brain-Like Stochastic Search: A Research Challenge And Funding Opportunity

## Paul J. Werbos

Room 675, NSF*
Arlington VA 22230 pwerbos@nsf.gov Abstract - Brain-Like Stochastic Search (BLiSS) refers to this task: given a family of utility functions U(u,), where is a vector of parameters or task descriptors, maximize or minimize U with respect to u, using networks (Option Nets) 
which input  and learn to generate good options u stochastically. This paper discusses why this is crucial to brain-like intelligence (an area funded by NSF) and to many applications, and discusses various possibilities for network design and training.

## 1 Introduction

Evolutionary computing is probably the lead technology today for finding global minima or maxima to a function U(u). Of course, there are many forms of evolutionary computing. There are also classical methods, like Gibbs search and the sophisticated trust region approaches recently developed by Barhen et al and used on the Desert Storm tank routing problem. There are a few neural net designs (like Kohonen nets, but not Hopfield nets) which have had competitive performance on some specialized large-scale optimization problems.

On the other had, it is hard to believe that the human brain uses these kinds of algorithms directly in making complex, novel decisions. As a result, many people doing basic research in neural networks have essentially ignored the need for this kind of systematic stochastic search. Some kinds of stochastic exploration methods have been developed (e.g., see Thrun in White and Sofge, 1992), 
but nothing with the kind of complexity and richness one finds in the evolutionary computation literature. The neural network field, in turn, still pays a lead role in developing the kind of highly functional network models which, in my view, are the only models which have a serious chance of eventually explaining the functional power of biological neural networks. (See Werbos in Pribram 1998.)
More recent work on neural networks suggests that this is a major omission - that greater attention to stochastic search will be a necessary part of replicating or understanding the higher-order intelligence that we find even in the lowest of mammal brains. Because this goal is a major goal for research sponsored at NSF, we are changing our priorities in the Control, Network and Computational Intelligence (CNCI) program to encourage research in this topic, and to encourage several related topics. Section 2 of this paper will discuss those other, related topics, and will discuss my general reasoning here. 

From a practical viewpoint, BLiSS offers a number of obvious advantage. In essence, the training of Option Nets over a variety of problems in a given domain allows you (your network) to build up domain-specific knowledge about how to solve optimization problems in a given domain. For example, if you define  to be the set of coordinates of 100 cities in the 100-city Traveling Salesman Problem (TSP), an Option net could learn how to set up good initial guesses for the optimum. Perhaps a recurrent Option Net (taking the previous best guess as an input) could make good guesses about how to improve the initial guesses. It would be interesting to see how training could improve the strength of such a search mechanism, and compare with other less domain-specific search methods. as another example, there would be great value in training a system specifically to solve problems in VLSI design, or to solve problems in aircraft design, and so on.

In a brain-like context, the system somehow needs to search over millions of choices, not just the dozen or two which are most comon in engineering today. In order to handle such very large problems, there is a need for the system to learn which variables to focus on in the search. 

Generic search techniques, which do not incorporate the learning of where to focus, cannot ultimately handle such large problems.

Of course, there has been a little work in the evolutionary computing area on ways to tweak parameters of an evolutionary search, analogous to tweaking the learning rates in neural networks. But the goal here is to achieve a more all-encompassing sort of learning, which, once again, has some serious hope of explaining how the human brain handles these kinds of problems.

The next section will say more about the concept of brain-like stochastic search, neural networks, and the broader goals of the CNCI program. Those who really do not care about that background may want to jump ahead to sections 3 and 4 which discuss the training and structure of Option Nets and Bliss Systems. **Please note that the** suggestion in sections 3 and 4 are very tentative; they are only my attempt to indicate that something like BLiSS is possible, and to suggest a couple of possible ways to get started in such work. NSF strongly encourages any innovative approach whatsoever that seriously addresses the underlying research challenge here; there is absolutely no 