Quality and performance test framework creation

[kryczkal]

Performance todos:

• minimal remote testing framework (1500, 1900/2000, 2200, 2400)
• mini local comparing tests

[kryczkal]

Foundation for quality testing laid out in #4 and #10

[kryczkal]

Idea for testing framework:

1. Run Checkmate-Chariot through a gauntlet tournament against many (eq. 10) engines of known and varying strength (ideally 1 engine for each 100elo above 1500, but then the tests would take too long).

• UCI compliant engines will be taken from: CCRL - Index
• the tournament will be run using: c-chess-cli
• the output will be saved as .pgn
• we will approve some sensible number for the number of games played against each engine

2. the .pgn will be fed to Ordo | Bayeselo to retrieve an estimation of the elo

[kryczkal]

@Jlisowskyy

[kryczkal]

with #17 Uci implemented fully, we can now run c-chess-cli and get the elo of the engine

[Jlisowskyy]

We should decide about the time for each game, I belive it should be at least longer than bullet or blitz games. Do we would guarantee quality of the tests, there could be some shorter blitz like option for dynamic and fast tests also

[Jlisowskyy]

Also we will be running tests on release build so if the bug happens it would be nice to have some kind of script to reproduce the failure based on the command input sequence on the debug build

[kryczkal]

the Engine is currently below 1900 elo and i need to find worthy oponents

[Jlisowskyy]

Testing framework should additionally return timestamps inside the logs to simplify analysis. I also think that there should be ready testing script for evaluation case.