Endgame Ranking or Value Bug

[nwalton125]

Running "endgame -multiple-vars 10 -endgame-plies 10" on https://www.cross-tables.com/annotated.php?u=57496#26# returned the following result:

MA is ranked highest despite showing a value lower than the second ranked play, AMA, shows. Other plays are also misranked. This happened on two runs of the command, after I had run "endgame -multiple-vars 10" (default plies) before and gotten a similar misranking.

[nwalton125]

Hm, I just ran some other stuff. First I ran "endgame", which showed a pretty different result than when I had run "endgame -multiple-vars 10" before. Then I ran "endgame -plies 10" and then "endgame -multiple-vars 10 -endgame-plies 10" again. This time, the plays were correctly ranked. Then I ran "endgame". Then I ran "endgame -multiple-vars 10" and "endgame -multiple-vars 10 -endgame-plies 10" again and got a misranking again both times. So for some reason the misranking seems to only happen sometimes.

[domino14]

What version of Macondo is this btw?

[domino14]

@nwalton125 I know what's causing this. The problem has to do with the endgame algorithm and the limited search depth.

This endgame seems like it requires more than 10 plies to determine the correct solution as it only shows up when you look that deep, even though the actual solution is only 4 plies long. I wrote a blog post about this here:

https://blog.woogles.io/articles/2024-09-07-the-omgwords-endgame-why-more-plies-sometimes-reveal-the-true-best-move/

To get a consistent answer look a few plies deeper. Use this command:

endgame -plies 14 -multiple-vars 10

This should give you the consistent (correct) answer every time. If you always want to get the correct answer you should technically look like 25 plies deep -- 25 plies is the maximum plies an endgame could ever be. But for a number of endgames this would take prohibitively long. 14 plies is the most for the vast majority of endgames I would bet, many that require a higher ply count would likely be contrived.

Also note that -endgameplies or -endgame-plies are not options for the endgame command, it's just -plies. If you download the latest Macondo you can use TAB to auto-complete the endgame and other commands.

[nwalton125]

What you said doesn’t seem to address the issue in my screenshot, which is that plays that are shown with a worse value are displayed as ranked higher than those which show a better value.

[nwalton125]

I guess it could be that running for 25 plies fixes that issue too…but I don’t see why it would, and it seems like it should at least be possible in principle to have a program which always returns a correct ranking of the plays within whatever depth search it is using. I may also not be using a fully up-to-date version of macondo.

[domino14]

so the plays being ranked incorrectly is an artifact of the lower search depth -- at low depth the endgame algorithm thinks it's solved the endgame, so then when the variations search runs, it discards that "winner" and sometimes by doing so it finds the true winner, which has a higher value. it's the interplay with the transposition table and the lower search depth that causes this confusing result. increasing the search depth will make it so the true answer is found the first time, and thus all subsequent answers will necessarily have values that monotonically decrease.

but you're right that the answer looks confusing and should be fixed for lower search depths. if i sort the output by value right before displaying it then it should rank correctly no matter what the search depth is. i'll reopen this bug report. thank you!