Wrong evaluation for wrong color bishop.

[AliceRoselia]

Describe the issue

Stockfish gives a very wrong evaluation on wrong color bishop positions. (1 pawn or more). This does not occur with other drawn endgames I've tested such as two knights endgame. Whether or not this affects playing strength has yet to be tested.

Expected behavior

The evaluation should be drawn or close to draw.

Steps to reproduce

Try Stockfish on wrong color bishop position such as 4k3/8/p3b3/8/8/8/8/K7 w - - 0 2

Tablebase off.

Anything else?

No response

Operating system

All

Stockfish version

Stockfish dev-20251203-c109a88e

[aryandadwal2006]

Hey I fixed this. Lemme make a pull request now, simple thing.

[vondele]

The evaluation I see with this fen is correct:

Stockfish dev-20251223-c475024b by the Stockfish developers (see AUTHORS file)
position fen 4k3/8/p3b3/8/8/8/8/K7 w - - 0 2
go depth 1
info string Available processors: 0-31
info string Using 1 thread
info string NNUE evaluation using nn-c288c895ea92.nnue (125MiB, (102384, 1024, 15, 32, 1))
info string NNUE evaluation using nn-37f18f62d772.nnue (6MiB, (22528, 128, 15, 32, 1))
info string Network replica 1: Shared memory.
info depth 1 seldepth 2 multipv 1 score cp -3 nodes 2 nps 2000 hashfull 0 tbhits 0 time 1 pv a1b2
bestmove a1b2

And so is static:

 NNUE network contributions (White to move)
+------------+------------+------------+------------+
|   Bucket   |  Material  | Positional |   Total    |
|            |   (PSQT)   |  (Layers)  |            |
+------------+------------+------------+------------+
|  0         |  -  2.10   |  +  2.09   |  -  0.00   | <-- this bucket is used
|  1         |  -  2.64   |  +  2.74   |  +  0.10   |
|  2         |  -  2.91   |  +  2.16   |  -  0.75   |
|  3         |  -  3.09   |  +  1.34   |  -  1.76   |
|  4         |  -  3.16   |  +  2.09   |  -  1.06   |
|  5         |  -  3.16   |  +  0.01   |  -  3.15   |
|  6         |  -  3.08   |  +  0.46   |  -  2.62   |
|  7         |  -  2.97   |  +  0.68   |  -  2.29   |
+------------+------------+------------+------------+

NNUE evaluation        +0.00 (white side)
Final evaluation       +0.09 (white side) [with scaled NNUE, ...]

Generally, fixing these evals should be done via training better .nnue nets, either with additional data or otherwise.

[vondele]

Issue is visible after a deeper search though, so keeping this open:

info depth 30 seldepth 76 multipv 1 score cp -108 upperbound nodes 98890851 nps 3641583 hashfull 121 tbhits 0 time 27156 pv a1b2 e8d7
info depth 30 currmove a1b2 currmovenumber 1

[vondele]

It might be interesting to make a kind of map to see if which static evals are right and which are wrong?

[vondele]

This is a set of draw fens derived from the original, for current master, at depth 1 the engine score is very wrong compared to the syzygy score.

8/8/8/8/p7/k1K5/b7/8 w - - 0 7  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -556, 'mate': None}]
8/8/8/8/p7/k1K5/b7/8 w - - 2 7  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -556, 'mate': None}]
8/8/8/8/p7/k2K4/b7/8 w - - 0 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -556, 'mate': None}]
8/8/8/8/p7/k7/b2K4/8 w - - 2 7  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -548, 'mate': None}]
8/8/8/8/p7/k7/b7/2K5 w - - 2 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -548, 'mate': None}]
8/8/8/8/p7/k7/b2K4/8 w - - 0 7  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -548, 'mate': None}]
8/8/8/8/p7/k7/b2K4/8 w - - 2 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -547, 'mate': None}]
8/8/8/8/p7/k2K4/b7/8 w - - 4 7  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -547, 'mate': None}]
8/8/p7/1k1b4/8/3K4/8/8 b - - 11 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 353, 'mate': None}]
8/8/p7/2kb4/8/3K4/8/8 b - - 11 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 339, 'mate': None}]
6b1/8/8/p1k5/8/3K4/8/8 b - - 5 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 316, 'mate': None}]
8/8/p7/k2b4/8/3K4/8/8 b - - 11 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 297, 'mate': None}]
8/5b2/p7/1k6/8/3K4/8/8 b - - 11 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 279, 'mate': None}]
8/8/8/p7/1kb5/8/2K5/8 b - - 5 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 245, 'mate': None}]
8/8/8/p7/1kb5/8/2K5/8 b - - 1 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 245, 'mate': None}]
8/8/8/p7/1kb5/8/2K5/8 b - - 3 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 245, 'mate': None}]
8/8/8/p7/1kb5/3K4/8/8 w - - 4 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': -245, 'mate': None}]
6b1/8/8/pk6/8/3K4/8/8 b - - 5 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 238, 'mate': None}]
6b1/8/8/pk6/8/3K4/8/8 b - - 3 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 238, 'mate': None}]
8/5b2/p7/2k5/8/3K4/8/8 b - - 11 6  c0  [{'type': 'syzygy', 'wdl': 0, 'dtz': 0}, {'type': 'engine', 'score': 222, 'mate': None}]

The static eval is very wrong for a few of those only:

NNUE evaluation        -5.74 (white side)
NNUE evaluation        -5.74 (white side)
NNUE evaluation        -5.04 (white side)
NNUE evaluation        -5.33 (white side)
NNUE evaluation        -1.69 (white side)
NNUE evaluation        -5.33 (white side)
NNUE evaluation        -5.33 (white side)
NNUE evaluation        -5.04 (white side)
NNUE evaluation        +0.04 (white side)
NNUE evaluation        +0.06 (white side)
NNUE evaluation        -0.07 (white side)
NNUE evaluation        +0.02 (white side)
NNUE evaluation        -0.01 (white side)
NNUE evaluation        -0.01 (white side)
NNUE evaluation        -0.01 (white side)
NNUE evaluation        -0.01 (white side)
NNUE evaluation        +0.04 (white side)
NNUE evaluation        +0.04 (white side)
NNUE evaluation        +0.01 (white side)

[vondele]

A larger set of fens with wrong static evals:

8/8/p7/8/8/8/k1b5/2K5 w - -
8/8/8/p7/8/8/k1b5/2K5 w - -
8/8/8/8/p7/k1K5/b7/8 w - -
8/8/p7/8/2bK4/8/k7/8 w - -
8/8/8/8/p7/k7/b1K5/8 b - -
8/8/8/8/p7/k1K5/2b5/8 w - -
8/8/8/8/p7/k2K4/b7/8 w - -
8/8/8/8/p7/k7/b2K4/8 w - -
8/8/p7/8/8/8/2b5/k1K5 w - -
8/8/8/p7/8/k1K5/2b5/8 w - -
8/8/8/8/p7/k7/b7/3K4 w - -
8/8/p7/8/1Kb5/8/k7/8 w - -
8/7b/8/p7/8/2K5/8/k7 w - -
8/8/8/p7/8/8/k1bK4/8 w - -
8/8/p7/8/8/k1K5/2b5/8 w - -
8/8/8/p7/2b5/k1K5/8/8 w - -
8/8/p7/8/2b5/k1K5/8/8 w - -
8/8/p7/8/8/8/k1bK4/8 w - -
8/8/8/p7/k1bK4/8/8/8 w - -
2b5/8/8/p7/2K5/8/k7/8 w - -
8/8/p7/8/1Kb5/8/1k6/8 w - -
8/8/p7/8/2bK4/k7/8/8 w - -
8/8/p7/8/2b5/2K5/k7/8 w - -
8/8/8/p7/k2K4/8/2b5/8 w - -
8/8/8/p7/2b5/2K5/8/k7 w - -
8/8/8/p7/8/1bK5/8/k7 w - -
8/8/p7/8/3K4/3b4/8/k7 w - -
8/8/p7/8/1Kb5/8/8/k7 w - -
8/8/8/p7/8/k7/2bK4/8 w - -
8/8/p7/8/k1bK4/8/8/8 w - -
8/8/8/p7/2bK4/k7/8/8 w - -
8/8/8/p7/k7/8/8/3bK3 w - -
8/8/8/p7/2K5/7b/1k6/8 w - -
8/8/8/p7/1k6/8/8/3bK3 w - -
8/8/p7/8/2bK4/8/8/k7 w - -
8/8/p7/8/8/1bK5/8/k7 w - -
8/8/p7/3b4/3K4/8/8/k7 w - -
8/8/8/p4b2/2K5/8/k7/8 w - -
8/8/p7/3b4/8/3k4/8/3K4 w - -
8/8/p7/8/8/k7/2bK4/8 w - -
8/8/8/8/p7/k7/2bK4/8 w - -
8/8/p7/8/2bK4/8/8/1k6 w - -
8/8/p7/8/1Kb5/8/8/1k6 w - -
8/8/p7/8/1k6/8/8/3bK3 w - -
8/8/8/p7/2K5/7b/k7/8 w - -
8/8/p7/8/k7/8/8/3bK3 w - -
8/8/8/p7/2bK4/8/k7/8 w - -
8/8/8/p7/2b5/2K5/k7/8 w - -
b7/8/8/p7/2K5/8/1k6/8 w - -
8/8/p7/8/bK6/8/k7/8 w - -
6b1/8/8/8/pk6/3K4/8/8 w - -
8/8/8/p7/k1K5/8/2b5/8 w - -
8/8/p7/8/8/3b4/3K4/1k6 w - -
8/8/p7/8/2b5/2K5/8/k7 w - -
8/8/8/p7/2bK4/8/8/k7 w - -
6b1/8/8/p7/k7/3K4/8/8 w - -
8/8/8/p7/2K5/8/k7/1b6 w - -
8/5b2/p7/2k5/4K3/8/8/8 w - -
8/8/p7/8/1K6/1b6/8/k7 w - -
8/8/p7/8/2k5/1b6/3K4/8 b - -
8/8/8/3b4/pk6/3K4/8/8 w - -
8/8/p7/1k6/8/5b2/8/4K3 w - -
6b1/8/8/p1k5/4K3/8/8/8 w - -
8/8/8/8/p7/k7/2b5/2K5 w - -
8/8/8/p7/2K5/5b2/k7/8 w - -
8/8/p7/8/2k5/1b6/4K3/8 w - -
8/8/p7/8/k2K4/8/2b5/8 w - -
8/8/8/p7/2K3b1/8/k7/8 w - -
8/8/p7/8/2bK4/8/1k6/8 w - -
8/8/8/p7/2K5/8/k7/7b w - -
8/8/p7/8/8/k7/8/1bK5 w - -
8/8/p7/1k6/b7/8/8/4K3 w - -
4b3/8/8/p7/1k6/3K4/8/8 w - -
8/8/p7/k7/3K4/8/8/3b4 w - -
8/8/p7/1k6/3K4/8/8/3b4 w - -
8/8/8/p7/8/2K5/k1b5/8 w - -
8/8/p7/1k6/2bK4/8/8/8 w - -
8/5b2/8/8/pk6/3K4/8/8 w - -
8/8/p7/k7/3K4/8/b7/8 w - -
8/8/8/p7/8/k7/2b5/2K5 w - -
8/8/4b3/p7/1k6/3K4/8/8 w - -
8/8/p7/8/k1K5/8/2b5/8 w - -
8/8/8/8/p7/k7/b7/2K5 w - -
6b1/8/8/p7/8/3k4/8/3K4 w - -
b7/8/8/p7/2K5/8/k7/8 w - -
8/8/p7/2k5/8/5b2/8/4K3 w - -
8/8/p7/1k1b4/8/4K3/8/8 w - -
8/8/p3b3/2k5/4K3/8/8/8 w - -
6b1/8/8/pk6/4K3/8/8/8 w - -
8/7b/p7/k7/3K4/8/8/8 w - -
8/8/p7/8/2b5/2K5/8/1k6 w - -
8/8/8/p6b/1k6/3K4/8/8 w - -
8/8/2b5/p7/k7/3K4/8/8 w - -
8/8/8/p6b/2K5/8/k7/8 w - -
8/8/p7/k4b2/3K4/8/8/8 w - -
8/8/8/p7/8/1k6/8/1bK5 w - -
8/8/p7/8/2k5/8/b7/3K4 w - -
8/8/p7/8/8/2Kb4/8/1k6 w - -
8/8/p7/2k5/8/8/6b1/4K3 w - -
8/7b/8/p7/2K5/8/8/k7 w - -

[rn5f107s2]

I think this is caused by correction history, yesterday I wrote a patch trying to track the leaf correction and prevent correction history to be updated based on itself.

When printing the leaf correction at root it looks like the correction makes out the entire score (I think normalization is roughly / 3, so 300 internal units should roughly correspond to the 100 cp displayed)

info depth 46 seldepth 115 multipv 1 score cp -109 nodes 193213143 nps 3628483 hashfull 229 tbhits 0 time 53249 pv a1b1 e8d7 b1a1 d7c6 a1b2 c6b5 b2c2 b5a4 c2b2 e6d5 b2c2 a4a3 c2b1 d5f7 b1a1 f7h5 a1b1 a3b3 b1a1 a6a5 a1b1 b3a3 b1a1 a5a4 a1b1 a3b3 b1a1 h5f7 a1b1 a4a3 b1a1 b3b4 a1b1 b4c3 b1a1 f7e6 a1b1 c3b4 b1a1 b4b3 a1b1 b3c3 b1a1 e6g8 a1b1 c3d3 b1a1 g8f7 a1b1 d3d4 b1a1 f7c4 a1b1 c4g8 b1a1 d4d3 a1b1 g8c4 b1a1 d3c3 a1b1 c3b4 b1a1 c4g8 a1b1 g8f7 b1a1 b4c3
info string leaf correction -376

When disabling correction history entirely the draw is also easily found

info depth 14 seldepth 17 multipv 1 score cp 1 nodes 26301 nps 1878642 hashfull 3 tbhits 0 time 14 pv a1b2 e8d7 b2c1 e6c4 c1b1 d7c7 b1b2 c7c6 b2c3 c6b5 c3c2 b5c5 c2b1

with the score not leaving the [2, -2] intervall upto atleast depth 62

I think the core issue that I also tried to tackle in this test: https://tests.stockfishchess.org/tests/view/694ab695572093c1986d6944
is that if one correction entry gets the value raised, e.g white messes up and black queens while the bishop is on f3, then the search can through reaching a leaf position with the bishop on f3 update all corrhist entries leading up to the position. The issue is, that those corrhist entries will then never go back down again, since the corrhist at the leafnode makes it seem like the correction is correct and thus never decreases these values. However though searching lines where white blunders the corrhist values can be increased and then continue in the same cycle.

I noticed that the test I submitted has (atleast) 2 major flaws, the leafCorrection is not reset to 0 on draws and I forgot to flip the sign, so I will fix this and submit another test. There is another issue with correction for ttScores being inaccessible, but I dont think this can be fixed.

[rn5f107s2]

The fixed test is here: https://tests.stockfishchess.org/tests/view/694bf005572093c1986d6c56

running it on the position still results in the score slowly trending upwards but slower than master

This test:

info depth 57 seldepth 90 multipv 1 score cp -50 nodes 1500812491 nps 3319808 hashfull 315 tbhits 0 time 452078 pv a1b2 e8d7 b2b1 d7d6 b1a1 d6c6 a1b2 c6b5 b2b1 b5a4 b1a1 a4b3 a1b1 e6d5 b1a1 b3c3 a1b1 a6a5 b1a1 a5a4 a1b1 d5e6 b1a1 e6c4 a1b1 c3b3 b1a1 a4a3 a1b1 c4d5 b1a1 d5e6 a1b1 b3a4 b1a1 a3a2 a1b2 a4b5 b2a1 e6d5 a1b2 d5c4 b2a1 c4e6 a1b2 e6d5 b2a1 b5a4 a1b2 d5c4 b2a1 a4b4 a1b2 c4e6 b2a1 e6d5 a1b2 b4a4 b2a1 d5c4 a1b2 c4e6 b2a1 e6g8 a1b2 a4b4 b2a1 g8e6 a1b2 e6c4 b2a1 c4g8 a1b2 b4b5 b2a1 b5c6 a1b2 g8e6 b2a1 e6d5

Master:

info depth 34 seldepth 79 multipv 1 score cp -98 nodes 36541499 nps 3235192 hashfull 122 tbhits 0 time 11295 pv a1b2 e8d7 b2a1 d7c6 a1b1 c6b5 b1c1 e6d5 c1b1 b5b4 b1c2 b4a3 c2b1 a3b3 b1a1 d5e6 a1b1 a6a5 b1a1 a5a4 a1b1 a4a3 b1a1 e6c4 a1b1 a3a2 b1a1 b3b4 a1b2 b4b5 b2a1 c4e6 a1b2 b5c5 b2a1 c5d6 a1b2 e6d5 b2a1 d6e5 a1b2 d5g8 b2a1 e5d4 a1b2 g8d5 b2a1 d5e6 a1b2 e6g8 b2a1 d4e4 a1b2 g8d5 b2a1 e4d4 a1b2 d4c5 b2a1 c5b4 a1b2 d5g8 b2a1 b4b5 a1b2
info depth 35 currmove a1b2 currmovenumber 1
info depth 35 currmove a1b1 currmovenumber 2
info depth 35 seldepth 82 multipv 1 score cp -100 upperbound nodes 36640592 nps 3234230 hashfull 123 tbhits 0 time 11329 pv a1b2 e8d7
info depth 35 currmove a1b2 currmovenumber 1
info depth 35 currmove a1b1 currmovenumber 2
info depth 35 seldepth 86 multipv 1 score cp -105 upperbound nodes 36663062 nps 3234500 hashfull 123 tbhits 0 time 11335 pv a1b2 e8d7
info depth 35 currmove a1b2 currmovenumber 1

[vondele]

Interesting, it is indeed somewhat consistent with this branch:
https://github.com/vondele/Stockfish/tree/staticEvalPV

It prints the static eval of the positions along the PV line.
For

position fen 8/8/8/3b4/pk6/3K4/8/8 w - - 0 1
go depth 20
ucinewgame
quit

It shows:

...
info depth 20 seldepth 44 multipv 1 score cp -33 nodes 788005 nps 2717258 hashfull 36 tbhits 0 time 290 pv -261 d3c2 -14 b4a3 -44 c2b1 -8 a3b3 2 b1a1 -7 d5e6 6 a1b1 -5 e6f7 0 b1a1 -10 f7g8 3 a1b1 -3 b3c3 4 b1a1 -3 g8e6 4 a1b1 -8 e6f7 4 b1a1 -4 f7b3 4 a1b1 -5 a4a3 8 b1a1 -4 c3c4 8 a1b1 -6 c4b5 3 b1a1 -7 b5b6 1
bestmove d3c2 ponder b4a3

so a fairly large score, even though most positions along the PV are evaluated statically near 0.

[AliceRoselia]

Maybe try adding these positions to the NNUE training data to see if it helps? Maybe it could even gain.