

![0_image_0.png](0_image_0.png)

achieved. This is akin to Game Theory's "static equilibrium" 

## F. Bluffing

A bluff is an action, usually in the context of a card game that misrepresents one's cards with the intent of causing one's opponents to drop theirs. There are two opposing schools of thought regarding bluffing. One school claims that bluffing is purely psychological, while the other maintains that a bluff is a purely statistical act, and therefore no less sensible than any other strategy. Astoundingly enough, the intelligent agents do in fact learn to bluff! A classic example is illustrated in Fig. 11, which depicts a hand in which bluffing was evidenced 

![0_image_1.png](0_image_1.png)

In the above hand, Randy is the first caller, and diamonds have been declared trumps. Randy's hand is not particularly impressive, having only one low trump, and two low supporting cards. Still, he has the lead, and a trump could become a trick, although his risks are high for minimal reward. Nonetheless, Randy chooses to play this hand. Ronald, having nothing to speak of, unsurprisingly folds. Roderick, on the other hand, has a very good hand. One high trump, and an outside ace. However, with one still to call, and Randy already representing a strong hand by playing, Roderick chooses to fold. AIden, whose hand is very strong with two high trumps and an outside jack, plays the hand. When the hand is played repeatedly, Randy eventually chooses not to play, since he loses all three to AIden. 

Instantly, Roderick chooses to play the hand, indicating that the bluff was successful, that it chased a player out of the hand! Depending on which of the schools of thought regarding bluffing one follows this astonishing result leads us to one of two possible conclusions. If, like the author, one hold that bluffing is simply playing the odds, making the odds for one's opponent unfavourable by representing a strong hand, then this result shows that the agents learn each other's patterns well enough to factor their opponent's strategies into the game evaluation, something game theory does a very poor job of. Should one follow the theory that bluffing is purely psychological, then the only conclusion that can be reached from this result is that the agents have in fact developed their own 'psyches', their own personalities which can then be exploited. Regardless of which option the reader holds to, the fact remains that agents have been shown to learn, on their own and without external prompting, to bluff! 

## V. Conclusions

While the exact nature of bluffing is still unknown, it has been shown that a system involving agents capable of learning adaptively not only from the game being played, but also from their opponents, is in fact able to learn to predict its opponent's reactions. This knowledge in turn changes the statistical nature of a game being played, allowing agents to learn to bluff, based purely on rational reasoning, lending strong support to the theory that bluffing is simply *playing the* odds, and not an illogical, psychologically based action. The use of the Reinforcement learning paradigm [1, 6], along with the TD(λ) algorithm [5] for adaptively training neural networks, ahs been shown to meet all of the requirements to produce such agents. Lastly, the design of the agent "view", has been seen to be the most important facet of creating bluffing agents, since their view of the game as inclusive of the other players allows for the incorporation of those players into it's estimation of the game's outcome. With all of these steps adhered to, artificially intelligent agents can learn to bluff! 

## References

[1] R.S. Sutton, and A.G. Barto, *Reinforcement learning: An Introduction*, 
MIT press, 1998. 

[2] R.S. Sutton, "Learning to predict by the methods of temporal differences". *Machine Learning*, vol 3, pp. 9-44, 1988. 

[3] E. Hurwitz, and T. Marwala, Optimising Reinforcement Learning for Neural Networks, *In Proceedings of the 6th Annual European on* Intelligent Games and Simulation, Leicester, UK, pp. 13-18, 2005,.ISBN: 90-77381-23-6. 

[4] A.H. Morehead, G. Mott-Smith, and P.D. Morehead, *Hoyle's Rules of* Games, Third revised and updated edition. Signet book. 2001. 

[5] R.S. Sutton, *Implementation details of the TD(λ) procedure for the case* of vector predictions and Backpropogation. GTE laboratories technical note TN87-509.1, 1989. 

[6] R.S. Sutton, Frequenlty *asked questions about reinforcement learning*, 
http://www.cs.ualberta.ca/~sutton/RL-FAQ.html: last viewed 24/08/2005, 
[7] O. Smart, *Line Search and Descents Methods.*
http://www.biochemistry.bham.ac.uk/osmart/msc/lect/lect2a.html 02/02/1996. Last Viewed 20/04/2006. 

[8] J. Wesley Hines, *Matlab supplement to Fuzzy and Neural Approaches* in Engineering. John Wiley & Sons Inc. New York. 1997. 

[9] Miller, Sutton and Werbos. *Neural Networks for Control.* MIT Press. 

Cambridge, Massecheussettes. 1990. 