For the first test, we evaluated the strength of GIB's cardplay using Bridge Master (BM),
a commercial program developed by Canadian internationalist Fred Gitelman. B M contains 10 deals at 5 levels of difficulty. Each of the 36 deals on each level is a problem in declarer play.  If you misplay the hand, B M moves the defenders' cards around if necessary to ensure your defeat.

BM was used for the test instead of randomly dealt deals because the signal to noise ratio is far higher; good plays are generally rewarded and bad ones punished. Every deal also contains a lesson of some kind; there are no completely uninteresting deals where the line of play is irrelevant or obvious. There are drawbacks to testing GIB's performance on nonrandomly dealt deals, of course, since the B M deals may in some way not be representative of the problems a bridge player would actually encounter at the table.

The test was run under Microsoft Windows on a 200 MHz Pentium Pro. As a benchmark, Bridge Baron (BB) version 6 was also tested on the same deals using the same hardware. 6 B was given 10 seconds to select each play, and GIB was given 90 seconds to play the entire deal with a maximum Monte Carlo sample size of 50. 7 New deals were generated each time a play decision needed to be made.

These numbers approximately equalized the computational resources used by the two programs; BB could in theory take 260 seconds per deal (ten seconds on each of 26 plays), but in practice took substantially less. GIB was given the auctions as well; there was no facility for doing this in BB. This information was critical on a small number of deals.

Here is how the two systems performed:

| Level   | B    | GIB   |
|---------|------|-------|
| 16      | 31   |       |
| 2       | 8    | 23    |
|         | 2    | 12    |
| 4       | 1    | 21    |
| 5       | 4    | 13    |
| Total   | 3    | 100   |
| 18.3%   | 5.6% |       |

Each entry is the number of deals that were played successfully by the program in question.

GIB's mistakes are illuminating. While some of them involve failing to gather information, most are problems in combining multiple ances (as in case D above). As BM's deals get more difficult, they more often involve combining a variety of possibly winning options and that is why GIB's performance falls off at levels 2 and 3.

At still higher levels, however, BM typically involves the successful development of complex end positions, and GIB's performance rebounds. This appeared to happen to BB as well, although to a much lesser extent. It was gratifying to see GIB discover for itself the complex end positions around which the BM deals are designed, and more gratifying still to witness GIB's discovery of a maneuver that had hitherto not been identified in the bridge literature, as described in Appendix B.