[RFC] Considerations for "Skill Level"

[Sopel97]

There was some discussion recently about possible improvements to the implementation of "Skill Level" in Stockfish on discord. @vondele suggested to try eval randomization, in particular interpolation of the NNUE eval with N(0, RookValueEg) using some parameter. I've implemented it and tested briefly, but it requires more work to assess the quality of the games played and calibrate the Elo rating. The initial results suggest that it might be a good direction.

The experiment I performed was to use pure NNUE evaluation at fixed nodes and vary the interpolation parameter. 46 configurations played a round-robin tournament with 50 games in each pair. The following c-chess-cli command was used:

#!/bin/bash
 
c-chess-cli \
    -concurrency 16 \
    -rounds 1 \
    -games 50 \
    -openings file=/home/sopel/nnue/c-chess-cli/noob_3moves.epd order=random -repeat -resign 3 700 -draw 8 10 \
    -pgn tournament.pgn 2 \
    -each tc=1000+1 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_100k option.RandomEvalPerturb=0 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_90k option.RandomEvalPerturb=0 nodes=90000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_80k option.RandomEvalPerturb=0 nodes=80000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_70k option.RandomEvalPerturb=0 nodes=70000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_60k option.RandomEvalPerturb=0 nodes=60000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_50k option.RandomEvalPerturb=0 nodes=50000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_40k option.RandomEvalPerturb=0 nodes=40000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_30k option.RandomEvalPerturb=0 nodes=30000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_20k option.RandomEvalPerturb=0 nodes=20000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_10k option.RandomEvalPerturb=0 nodes=10000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_5k option.RandomEvalPerturb=0 nodes=5000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_4k option.RandomEvalPerturb=0 nodes=4000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_3k option.RandomEvalPerturb=0 nodes=3000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_2k option.RandomEvalPerturb=0 nodes=2000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_0_1k option.RandomEvalPerturb=0 nodes=1000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_1_100k option.RandomEvalPerturb=1 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_2_100k option.RandomEvalPerturb=2 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_3_100k option.RandomEvalPerturb=3 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_4_100k option.RandomEvalPerturb=4 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_5_100k option.RandomEvalPerturb=5 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_6_100k option.RandomEvalPerturb=6 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_7_100k option.RandomEvalPerturb=7 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_8_100k option.RandomEvalPerturb=8 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_9_100k option.RandomEvalPerturb=9 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_10_100k option.RandomEvalPerturb=10 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_12_100k option.RandomEvalPerturb=12 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_14_100k option.RandomEvalPerturb=14 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_16_100k option.RandomEvalPerturb=16 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_18_100k option.RandomEvalPerturb=18 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_20_100k option.RandomEvalPerturb=20 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_25_100k option.RandomEvalPerturb=25 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_30_100k option.RandomEvalPerturb=30 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_35_100k option.RandomEvalPerturb=35 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_40_100k option.RandomEvalPerturb=40 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_45_100k option.RandomEvalPerturb=45 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_50_100k option.RandomEvalPerturb=50 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_55_100k option.RandomEvalPerturb=55 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_60_100k option.RandomEvalPerturb=60 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_65_100k option.RandomEvalPerturb=65 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_70_100k option.RandomEvalPerturb=70 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_75_100k option.RandomEvalPerturb=75 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_80_100k option.RandomEvalPerturb=80 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_85_100k option.RandomEvalPerturb=85 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_90_100k option.RandomEvalPerturb=90 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_95_100k option.RandomEvalPerturb=95 nodes=100000 \
    -engine cmd=./engines/stockfish/stockfish name=stockfish_pure_100_100k option.RandomEvalPerturb=100 nodes=100000

Naming: stockfish_pure_{RandomEvalPerturb}_{nodes}
Code: Sopel97@56a8a4f
Experiment results: https://drive.google.com/drive/folders/14SZEV6TICedYtNZ2Ym2sFCQynoTBMeQ0?usp=sharing (includes a .pgn with moves saved).
Ordo:

   # PLAYER                     :   RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)
   1 stockfish_pure_1_100k      :      9.8   19.7  1826.0    2250    81      84
   2 stockfish_pure_0_100k      :      0.0   ----  1810.5    2250    80      79
   3 stockfish_pure_2_100k      :     -8.1   19.6  1797.5    2250    80      51
   4 stockfish_pure_4_100k      :     -8.4   20.0  1797.0    2250    80      68
   5 stockfish_pure_0_90k       :    -13.1   19.7  1789.5    2250    80      65
   6 stockfish_pure_3_100k      :    -16.8   19.0  1783.5    2250    79      75
   7 stockfish_pure_5_100k      :    -23.2   19.7  1773.0    2250    79      93
   8 stockfish_pure_6_100k      :    -37.8   19.1  1749.0    2250    78      72
   9 stockfish_pure_0_80k       :    -43.2   19.0  1740.0    2250    77      56
  10 stockfish_pure_8_100k      :    -44.7   19.2  1737.5    2250    77      86
  11 stockfish_pure_7_100k      :    -54.8   19.0  1720.5    2250    76      85
  12 stockfish_pure_0_70k       :    -65.2   19.4  1703.0    2250    76      67
  13 stockfish_pure_9_100k      :    -69.6   19.0  1695.5    2250    75      55
  14 stockfish_pure_10_100k     :    -70.8   20.1  1693.5    2250    75      98
  15 stockfish_pure_0_60k       :    -89.6   19.2  1661.5    2250    74      63
  16 stockfish_pure_12_100k     :    -92.8   19.9  1656.0    2250    74     100
  17 stockfish_pure_0_50k       :   -126.1   19.1  1599.0    2250    71      71
  18 stockfish_pure_14_100k     :   -131.6   19.2  1589.5    2250    71     100
  19 stockfish_pure_16_100k     :   -174.4   19.8  1517.0    2250    67      59
  20 stockfish_pure_0_40k       :   -176.5   19.4  1513.5    2250    67     100
  21 stockfish_pure_18_100k     :   -210.8   19.9  1457.0    2250    65     100
  22 stockfish_pure_0_30k       :   -246.6   19.5  1400.0    2250    62      64
  23 stockfish_pure_20_100k     :   -250.2   20.2  1394.5    2250    62     100
  24 stockfish_pure_0_20k       :   -352.1   21.2  1248.5    2250    55      87
  25 stockfish_pure_25_100k     :   -365.2   20.6  1231.5    2250    55     100
  26 stockfish_pure_30_100k     :   -513.1   24.1  1067.0    2250    47     100
  27 stockfish_pure_0_10k       :   -587.6   25.5   999.5    2250    44     100
  28 stockfish_pure_35_100k     :   -669.2   27.1   933.5    2250    41     100
  29 stockfish_pure_40_100k     :   -831.1   30.0   816.5    2250    36      63
  30 stockfish_pure_0_5k        :   -836.2   29.1   813.0    2250    36     100
  31 stockfish_pure_0_4k        :   -927.1   31.5   752.0    2250    33      99
  32 stockfish_pure_45_100k     :   -966.2   31.4   726.5    2250    32     100
  33 stockfish_pure_0_3k        :  -1031.4   33.1   685.0    2250    30     100
  34 stockfish_pure_50_100k     :  -1161.1   34.6   607.0    2250    27     100
  35 stockfish_pure_0_2k        :  -1209.6   36.0   579.5    2250    26     100
  36 stockfish_pure_55_100k     :  -1333.4   38.0   514.0    2250    23     100
  37 stockfish_pure_0_1k        :  -1426.8   41.5   469.5    2250    21      97
  38 stockfish_pure_60_100k     :  -1464.0   42.1   453.0    2250    20     100
  39 stockfish_pure_65_100k     :  -1690.6   49.2   368.5    2250    16     100
  40 stockfish_pure_70_100k     :  -1936.4   62.2   298.5    2250    13     100
  41 stockfish_pure_75_100k     :  -2076.9   68.5   263.5    2250    12     100
  42 stockfish_pure_80_100k     :  -2360.9   84.0   201.0    2250     9     100
  43 stockfish_pure_85_100k     :  -2581.0   93.3   158.5    2250     7     100
  44 stockfish_pure_90_100k     :  -2913.0  115.3   106.0    2250     5     100
  45 stockfish_pure_95_100k     :  -3417.6  175.1    44.0    2250     2     100
  46 stockfish_pure_100_100k    :  -3677.9  186.7    10.0    2250     0     ---
 
White advantage = 8.06 +/- 1.85
Draw rate (equal opponents) = 55.79 % +/- 0.45

Plots:

[vondele]

I've played a few games against it for RandomEvalPerturb in the upper quarter, and could at least win.

Probably instead of percent I would use per mill so it can be a bit finer tuned.

If we see that as an alternative for Skill Level (which has clear deficiencies), would be interesting to know if other places rely on Skill Level (e.g. Lichess?).

It would be nice if some people try to play a few games against it to see if that would be an interesting opponent for a suitable value of RandomEvalPerturb.

[scchess]

Thanks! I'm going to test it a few games and write back here. Also, we need to establish an anchor chess engine to measure the ELO. What about measure it against one of the Maia ELO calibrated versions?

[scchess]

I played a few games. I could easily win the level 100, but suffered losses to level 50 after some close games. It looked like the level 50 is somewhere around 2000 to 2200 level. Overall it looks cool.

[Sopel97]

100 has effectively completely random evaluation, so it's about as expected. Did you play the games at some time control or fixed nodes like in my tests? Anything particularily visible about the "style"?

[scchess]

I tried it with fixed 5 minutes games, not nodes. As a human vs AI, I could only do only a few games. It was clearly still "computer style", meaning there was strong consistency throughout the games. Not sure there is anything you can do. However, the ability to assess the position also became a little worse, which is welcome. Next, we should calibrate against an engine with a well defined human rating like Maia.

[tillchess]

If we see that as an alternative for Skill Level (which has clear deficiencies), would be interesting to know if other places rely on Skill Level (e.g. Lichess?).

Lichess playing vs computer levels 1-8 relies on search depth, UCI_Elo and move time values.
Therefore since UCI_ELo and UCI_LimitStrength is used Lichess relies on Skill Level

[lucasart]

My experience is as follows:

• blitz rating 1793 on lichess.
• I score about 50% against SF level 5 on lichess. That's playing in real blitz conditions 5'+3" no take backs, no restarting games when blundering (just counter attack until SF counter blunders, same as I would against a human).

Most of the game, computer plays ok. Weird moves, but nothing terrible. I win playing the way humans should play against a computer, by stiffling the opponent like a boa constrictor.

Eventually, when I break through, and the materialistic computer can't find a way out without losing material, it stops resisting, and just throws away its pieces.

That is lame. A key skill in chess is learning to win in a winning position, in the most irrefutable way, leaving no counter play to the opponent (ie. so that even full strength SF will eventually be forced to bend the knee). And you don't learn that if the opponent goes hara-kiri on you.

[Sopel97]

@lucasart have you tried playing the version described in this issue to see whether there's a difference in style?

[lucasart]

@Sopel97 I will try when my computer is fixed. One thing I notice is that random eval perturbation + depth (or nodes?) limit should suffice. Running a multi PV search behind the scene complicates for no reason. I can't see the purpose of this once random eval is introduced. This could be an important simplification.

*nodes might be better than depth limits, because of endgame, where branching factors reduce and even humans can out-calculate depth limits easily.

[vondele]

There is a more up-to-date branch which gets rid of multipv and has a calibrated scheme to set user Elo here :
https://github.com/vondele/Stockfish/commits/rep
It is calibrated at 1500 Elo against the maia engine at lichess.

[lucasart]

@vondele I tried it at 1500 elo: 5+3 blitz, no takebacks, FRC (to reduce opening knowledge factor and focus purely on gameplay, SF is bookless here). As expected, SF played slightly odd moves (positionally), but destroyed me tactically. I think we still need limits on depth or nodes in combination to eval pollution.

Another problem is aspiration search. Because the search returns polluted scores going all over the place, there is a lot of search inconsistency and re-searches going on. When using skill level, it probably makes sense to switch off aspiration, and just search +/-INF window directly. The user doesn't see the logs, so you might say it doesn't matter. But it changes time management, and that's the part visible to the user (ie. lots of searhc instability make SF use time more aggressively).

[Date "2021.09.27"]
[White "Human"]
[Black "vondele_rep"]
[Result "0-1"]
[FEN "bbqnnrkr/pppppppp/8/8/8/8/PPPPPPPP/BBQNNRKR w KQkq - 0 1"]
[GameDuration "00:11:41"]
[GameEndTime "2021-09-27T20:51:29.068 HKT"]
[GameStartTime "2021-09-27T20:39:47.861 HKT"]
[PlyCount "30"]
[SetUp "1"]
[Termination "time forfeit"]
[TimeControl "300+3"]

1. e4 Ne6 {-0.35/21 31s} 2. Nf3 {17s} Nf4 {0.00/21 28s} 3. Re1 {20s}
f5 {+0.74/20 26s} 4. exf5 {19s} e6 {+1.09/20 32s} 5. fxe6 {8.3s}
dxe6 {+1.30/19 25s} 6. Ne3 {15s} b5 {+1.10/20 17s} 7. Qd1 {35s} h5 {0.00/19 39s}
8. d4 {73s} Nd6 {+1.31/20 37s} 9. b3 {6.4s} Rh6 {+0.77/19 35s} 10. O-O {65s}
Ne4 {+3.19/20 23s} 11. c4 {35s} c5 {+0.96/20 29s} 12. dxc5 {2.8s}
Nh3+ {+1.57/17 8.3s} 13. gxh3 {20s} Rg6+ {+4.78/14 0.45s} 14. Ng2 {8.9s}
Rxg2+ {+1.94/19 6.1s} 15. Kxg2 {3.4s} Ng5 {+2.44/15 0.43s, White loses on time}
0-1

Strong chess players are probably laughing at this game ;-)

For the sake of comparison, here is the last game I played against maia5 on lichess in blitz 5+3: https://lichess.org/9pNVarMAimZ6. Basically a draw, and the engine blundered the king+pawn endgame.