Wisdom is knowing how

   little we know -- Socrates

About Company

Started in 1980, retired in 2004 REBEL was baptized into ProDeo, latin for gratis according to Dutch tradition.

Other Information

    Quick downloads

 

 

    Quick links

 

 

    Technical

 

 

    Misc

 

 

    Personal links

 

 

 

MATCH.EXE (version 1.3a)

engine - engine utility running in the background

 

MATCH.EXE is a command line utility running in the background when playing engine - engine matches. It now also can be used as an stand alone PGN analysis tool.

 

The utility is especially handy when running multiple matches at the same time via separate threads (cores) because it calculates the progress of all running matches.

 

For instance when running a match under Arena using 4 threads every 10 seconds the screen is updated. Example from the ALL.PGN file included in the download.

 

 

 2879-2452-2668 (7999)  match score 4105.0 - 3894.0 (51.3%)

 

 Won-loss 2879-2668 = 211 (7999 games) draws 30.7%

 

 LOS = 99.8%  Elo Error Margin +6 -6

 

 Engine MICKEY (elo 2500) vs Engine MOUSE (elo 2500) estimated TPR 2508 (+8)


 

HOW IT WORKS

 

Both Arena and cutechess-cli have an option to collect every game it plays in a separate PGN file defined by the user. For Arena do: Options -> Appearance -> PGN -> Save all games and choose (for instance) ALL.PGN which is included in the download.

 

MATCH.EXE will poll ALL.PGN every 10 seconds and display the progress of the match(es).

 


 

SETUP example match ProDeo vs Fruit 2.1

using 4 cores (threads).

 

Copy the ProDeo folder to 4 different folders and do the same for Fruit 2.1. Using Arena install the 4 ProDeo engines as MICKEY-1, MICKEY-2, MICKEY-3, MICKEY-4 and install Fruit 2.1 as MOUSE-1, MOUSE-2, MOUSE-3, MOUSE-4. Then start the 4 matches, run MATCH.EXE and enjoy the progress of the match.

 


 

New features

 

 1.  ORDO elo calculation  Type O to activate (deactivate)
 2.  Crosstable support  Type C to activate (deactivate)
 3.  HTML support  Type H
 4.  Flexible duration

 Type 0-9 to change the number of seconds MATCH to poll the PGN again.

 A value of 0 means the PGN polling is deactivated.

 

An example of a typical 10 rounds robin match (with Arena) is given here. In case you want to change the outlook simply modify the EXAMPLE.HTM template with an HTML editor.

 


 

PARAMETERS

 

The program is controlled by 10 parameters in the corresponding configuration file MATCH.CFG

 

 1.  all.pgn  default setting  PGN location
 2.  MICKEY  default setting  The nickname you have given ProDeo in Arena or cutechess-cli
 3.

 2500

 default setting  ELO of MICKEY (ProDeo)
 4.

 MOUSE

 default setting  The nickname you have given Fruit in Arena or cutechess-cli
 5.

 2500

 default setting  ELO of MOUSE (Fruit)
 6.

 10

 default setting  Number of seconds to wait for next PGN polling
 7.

 100

 default setting  Create a data point after each 100 games for an Excel histogram
 8.

 output.txt

 default setting  Write match results to file output.txt
 9.

 history.txt

 default setting

 Version control - keep track of your testing results 

10.

 description

 

 20 lines are available to describe in detail what you are testing and this

 is stored in the file as defined in "9" (history.txt). See example below.

 

The most easiest way to run MATCH.EXE is to install the program in the same folder where the games are stored.

 


 

VERSION CONTROL

example tuning the Bishop Pair (100 vs 150)

then 100 vs 125

MATCH.CFG

 1.  all.pgn  PGN location
 2.  MICKEY

 The nickname you have given ProDeo 1.81x

 in Arena or cutechess-cli

 3.

 2500

 ELO of MICKEY (ProDeo 1.81x)
 4.

 MOUSE

 The nickname you have given ProDeo 1.81

 in Arena or cutechess-cli

 5.

 2500

 ELO of MOUSE (ProDeo 1.81)
 6.

 10

 Number of seconds to wait for next PGN

 polling.

 7.

 100

 Create a data point after each 100 games

 for an Excel histogram.

 8.

 output.txt

 Write match results to file output.
 9.

 history.txt

 Version control - keep track of your testing

 results in history.txt

10.

 ProDeo 1.81x vs ProDeo 1.81 tuning Bishop Pair

 

 [Bishop_Pair = 150]     * ProDeo 1.81 has 100

 Match description

 

Version control (history.txt) will look like:

 

 ProDeo 1.81x vs ProDeo 1.81 tuning Bishop Pair

 

 [Bishop_Pair = 150]     * ProDeo 1.81 has 100

 

 Engine MICKEY (elo 2500) vs Engine MOUSE (elo 2500) estimated TPR 2500 (+0)
 1276-1366-1280 (3922) match score 1959.0 - 1963.0 (49.9%)
 Won-loss 1276-1280 = -4 (3922 games) draws 34.8%
 LOS = 46.8%  Elo Error Margin +8 -8

 ProDeo 1.81x vs ProDeo 1.81 tuning Bishop Pair

 

 [Bishop_Pair = 125]     * ProDeo 1.81 has 100

 

 Engine MICKEY (elo 2500) vs Engine MOUSE (elo 2500) estimated TPR 2506 (+6)
 1347-1388-1265 (4000) match score 2041.0 - 1959.0 (51.0%)
 Won-loss 1347-1265 = 82 (4000 games) draws 34.7%
 LOS = 94.6%  Elo Error Margin +8 -8

 

As it looks the Bishop Pair value of 150 is too high and the 125 setting very likely is an improvement.

 


 

EXCEL HISTOGRAMS

 

In testing almost 2 identical versions it happens too often one version takes an incredible lead at the start because of the random nature of the testing procedure and the other version is unable to catch up. At times this can be misleading and the end score does not represent the true score.

 

In doubt Histograms can be useful here. MATCH after each 100 games (configurable see option 7) will create a data point for the use of EXCEL (or Open Office CALC). For EXCEL do as follows:

 

Open the file EXCEL_EU.TXT (or EXCEL_US.TXT for USA format) and click FINISH. Chose INSERT + LINE and select the wished histogram type. You might need Daniel's XL Toolbox for that depending on the EXCEL version you are using. Open Office CALC works in a similar way.

 

 

In this case the histogram shows a quite normal pattern, the usual fluctuations at the start of a match and then relaxing after 1000 games.

 


 

DOWNLOAD and credits

 

Download MATCH 1.3a with source code.

 

Special thanks to:

 

Rémi Coulom for the use of the WhoIsBest algorithm that calculates the LOS (Likelihood of superiority) percentage.

 

Kai Laskos for supplying a formula for the calculation of the elo error margin.

 

Miguel A. Ballicora for the use of ORDO.

 

Peter Mckenzie for the use of PGNRES.

 

Norman Pollock for the use of CLEANUP.

 

The (estimated) calculated elo rating (listed as TPR) as used in MATCH is taken from the Mephisto Polgar (1988) plus some modifcations to handle large PGN collections. The used formula might be not totally accurate when elo ratings between 2 engines differ a lot.

 

Comments and suggestions at Talkchess.

 

 

Copyright ® 2012  Ed Schröder