Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lc0 missing mate in #1 in tight endgame #627

Closed
MelleKoning opened this issue Dec 29, 2018 · 10 comments
Closed

Lc0 missing mate in #1 in tight endgame #627

MelleKoning opened this issue Dec 29, 2018 · 10 comments
Labels
bug Something isn't working release blocker Bugs which block releases

Comments

@MelleKoning
Copy link
Contributor

I've posted this bug in the forum:
https://groups.google.com/forum/#!topic/lczero/O0OWPd3ujE4

The position is visible on lichess: https://lichess.org/8j8a0wQy
The blunder move is 47. Qa8-a7, while it should be 47. Bf1-g2

How to reproduce:

  1. Start lc0 with a recent weightsfile, example:

lc0 --weights=nets\32300

  1. set the position right at the position the mate can still be prevented with this command:

position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 f8e7 f2f3 c8e6 d1d2 h7h5 c3d5 f6d5 e4d5 e6f5 b3a5 b7b6 a5c4 b8d7 a2a4 e8g8 f1e2 a8c8 e1g1 h5h4 h2h3 f5g6 b2b4 d8c7 f1c1 c7b7 c2c3 f8e8 e2f1 c8b8 e3f2 f7f5 d2e1 b7c7 f2h4 e7h4 e1h4 f5f4 a4a5 b6b5 c4d2 d7f6 c3c4 b5c4 d2c4 b8b4 c4b6 c7e7 f1a6 e5e4 h4f4 f6h5 f4e3 e7e5 f3f4 h5f4 a6f1 f4d3 c1d1 e5b2 b6d7 b4b3 a5a6 d3e5 e3g5 b3h3 g5d2 b2b3 d7e5 b3g3 e5g4 g3g4 d2e1 h3h8 a6a7 e4e3 a7a8Q g4f4 g2g3 f4h6

  1. now let leela think for 10.000 nodes and you will see this:
go nodes 10000
Loading weights file from: nets\32300
Creating backend [cudnn]...
info depth 1 seldepth 2 time 123 nodes 5 score cp 270 hashfull 0 nps 40 tbhits 0 pv a8e8 g6e8
info depth 2 seldepth 3 time 196 nodes 6 score cp 406 hashfull 0 nps 30 tbhits 0 pv a8e8 g8h7 g3g4
info depth 2 seldepth 3 time 252 nodes 11 score cp 95 hashfull 0 nps 43 tbhits 0 pv f1g2 g8f7 g3g4
info depth 3 seldepth 4 time 306 nodes 21 score cp 446 hashfull 0 nps 68 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 3 seldepth 5 time 455 nodes 24 score cp 382 hashfull 0 nps 52 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 3 seldepth 6 time 515 nodes 25 score cp 311 hashfull 0 nps 48 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 3 seldepth 7 time 585 nodes 26 score cp 238 hashfull 0 nps 44 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 3 seldepth 8 time 641 nodes 28 score cp 185 hashfull 0 nps 43 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 4 seldepth 9 time 736 nodes 36 score cp 109 hashfull 1 nps 48 tbhits 0 pv a8e8 g8h7 e8h8 h7h8
info depth 4 seldepth 9 time 841 nodes 71 score cp 196 hashfull 1 nps 84 tbhits 0 pv f1g2 h8h7 a8e8 g6e8
info depth 4 seldepth 10 time 1739 nodes 281 score cp 129 hashfull 3 nps 161 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1e2 h8f8 g2f3 h2g3
info depth 4 seldepth 11 time 2944 nodes 692 score cp 152 hashfull 6 nps 235 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2
info depth 4 seldepth 12 time 3189 nodes 770 score cp 146 hashfull 6 nps 241 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2
info depth 5 seldepth 12 time 3411 nodes 858 score cp 133 hashfull 7 nps 251 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2
info depth 4 seldepth 12 time 3465 nodes 1037 score cp 133 hashfull 7 nps 299 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2
info depth 4 seldepth 13 time 3659 nodes 1132 score cp 119 hashfull 8 nps 309 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2 f1e2
info depth 5 seldepth 13 time 3958 nodes 1250 score cp 125 hashfull 8 nps 315 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1c3 e3e2 f1f2 h7h6 d1h1 h2h1
info depth 3 seldepth 13 time 4285 nodes 11336 score cp -9167 hashfull 9 nps 2645 tbhits 0 pv d1d2 h6h1
bestmove d1d2 ponder h6h1

What you see here, is that lc0 can see the right move Bf1-g2 at seldepth 13, but at the end goes for another blunder Rd1-d2?? which enables black to mate again with Qh1 #1.

There is clearly something wrong in evaluation of this position.
Characteristics: both sides are close to checkmate, so there is a very thin line to operate.

  1. Only when extending the search to 30.000 nodes, lc0 finds its way out of this mess:
go nodes 30000
(cut for brevity)
info depth 14 seldepth 26 time 14176 nodes 20525 score cp -6532 hashfull 48 nps 631 tbhits 0 pv d1d2 h6h1 <-- still wrong

info depth 15 seldepth 26 time 15605 nodes 21998 score cp -30 hashfull 53 nps 668 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 e4f5 g6f5 d4a4 b2d2 a4a7 f5g6 h4h5 <-- finally found the right move again

Expected behaviour: once the right move is found (only 2 plies deep!) it should keep with that move and not jump to a blunder move missing the mate in 1.
@mooskagh mooskagh added bug Something isn't working release blocker Bugs which block releases labels Dec 29, 2018
@jhellis3
Copy link

jhellis3 commented Dec 29, 2018 •
edited
Loading

I have not kept up with all of the changes and not sure if this is even part of the issue here, but I wonder if this might be avoided in a rather naive way.

I believe moves are scored based upon the number of visits they receive; only counting visits to our moves (check node ply is even or odd depending on if root is 0 or 1) for selection purposes might be worth a try.

@oscardssmith
Copy link
Contributor

CP should fix this, right?

@Tilps
Copy link
Contributor

Tilps commented Dec 29, 2018

To be clear I have no debugged this scenario, I am purely hypothesizing... but...
I think this is a side effect of conflating multi-collision with multi-terminal hit.
We will run as many terminal hits as it takes to move a parent from one choice to a different choice, under the assumption that the Q value does not shift since they are collisions. With terminal hits this is clearly not true as it is known that q will shift. Thus we can vastly overshoot how many visits are required to shift a parent choice.

@Tilps Tilps mentioned this issue Dec 29, 2018
Closed
@Tilps
Copy link
Contributor

Tilps commented Dec 30, 2018

PR629 fixes this as far as I can tell. Still testing whether its an elo gain or loss though...

@MelleKoning MelleKoning mentioned this issue Dec 30, 2018
Closed
@Videodr0me
Copy link
Collaborator

@oscardssmith yes CP fixes this, but it should not happen even without CP. More general the multi terminal hits, while inflating nps (these are not real NN nps) never showed any elo gain in my testing, also they ceased to be a mere optimization since we included out-of-order eval. Multi collisions, I did not test much, but these should actually help (also they do not get OOEd). I fully agree with Tilps that both concepts are kind of conflated in the current codebase and probably shouldn't.

@MelleKoning MelleKoning mentioned this issue Jan 13, 2019
Closed
@MelleKoning
Copy link
Contributor Author

Does the suggestion to have different MakeTerminal values for training and playing make sense?
See: #677

@MelleKoning
Copy link
Contributor Author

@mooskagh Tested pull request #683 and that version quickly finds the right move, and does not get lost anymore.

Debug:

C:\dev\lc0>lc0pr683 --verbose-move-stats -t 1 --minibatch-size=256 --weights=.\nets\11258
       _
|   _ | |
|_ |_ |_| v0.21.0-dev built Jan 20 2019
position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 f8e7 f2f3 c8e6 d1d2 h7h5 c3d5 f6d5 e4d5 e6f5 b3a5 b7b6 a5c4 b8d7 a2a4 e8g8 f1e2 a8c8 e1g1 h5h4 h2h3 f5g6 b2b4 d8c7 f1c1 c7b7 c2c3 f8e8 e2f1 c8b8 e3f2 f7f5 d2e1 b7c7 f2h4 e7h4 e1h4 f5f4 a4a5 b6b5 c4d2 d7f6 c3c4 b5c4 d2c4 b8b4 c4b6 c7e7 f1a6 e5e4 h4f4 f6h5 f4e3 e7e5 f3f4 h5f4 a6f1 f4d3 c1d1 e5b2 b6d7 b4b3 a5a6 d3e5 e3g5 b3h3 g5d2 b2b3 d7e5 b3g3 e5g4 g3g4 d2e1 h3h8 a6a7 e4e3 a7a8Q g4f4 g2g3 f4h6
go nodes 25000
Loading weights file from: .\nets\11258
Creating backend [cudnn]...
info depth 1 seldepth 2 time 119 nodes 4 score cp 277 hashfull 0 nps 33 tbhits 0 pv a8e8 g6e8
info depth 2 seldepth 3 time 187 nodes 5 score cp 115 hashfull 0 nps 26 tbhits 0 pv a8e8 g6e8 f1g2
info depth 2 seldepth 4 time 256 nodes 6 score cp 27 hashfull 0 nps 23 tbhits 0 pv a8e8 g6e8 f1g2 h6h2
info depth 2 seldepth 5 time 318 nodes 8 score cp 98 hashfull 0 nps 25 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1
info depth 3 seldepth 6 time 367 nodes 10 score cp 80 hashfull 0 nps 27 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5
info depth 3 seldepth 7 time 430 nodes 11 score cp 21 hashfull 0 nps 25 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5 e1e2
info depth 4 seldepth 8 time 487 nodes 13 score cp -66 hashfull 0 nps 26 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5 e1e2 b5e2
info depth 4 seldepth 9 time 551 nodes 18 score cp -131 hashfull 0 nps 32 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5 e1e2 b5e2 f1e2
info depth 5 seldepth 10 time 618 nodes 23 score cp -183 hashfull 0 nps 37 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5 e1e2 b5e2 f1e2 h2g2
info depth 5 seldepth 10 time 741 nodes 33 score cp 56 hashfull 1 nps 44 tbhits 0 pv f1g2 h6h2 g1f1 e3e2 e1e2 g8h7 a8e8
info depth 6 seldepth 10 time 856 nodes 42 score cp 51 hashfull 1 nps 49 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4
info depth 7 seldepth 11 time 977 nodes 76 score cp 98 hashfull 1 nps 77 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 g6h5 g2e4 e8e4 a2h2
info depth 7 seldepth 12 time 1248 nodes 120 score cp 121 hashfull 2 nps 96 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 g6h5 g2e4 e8e4 a2h2
info depth 7 seldepth 13 time 1312 nodes 144 score cp 56 hashfull 2 nps 109 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 e8f8 g2f3 f8f3 e1f2 f3f2
info depth 8 seldepth 14 time 1377 nodes 175 score cp -13 hashfull 2 nps 127 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 e8f8 a2f2 e3f2 e1f2 f8f2
info depth 8 seldepth 15 time 1442 nodes 208 score cp -55 hashfull 2 nps 144 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 e8f8 a2f2 e3f2 e1f2 f8f2
info depth 8 seldepth 16 time 1539 nodes 254 score cp -98 hashfull 3 nps 165 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 e8f8 a2f2 e3f2 e1f2 f8f2 f1f2
info depth 9 seldepth 17 time 1797 nodes 384 score cp -130 hashfull 3 nps 213 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a2 e8f8 a2f2 e3f2 e1f2 f8f2 f1f2
info depth 9 seldepth 18 time 1891 nodes 424 score cp -122 hashfull 3 nps 224 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 g2f3 b2b4 f1f2
info depth 10 seldepth 19 time 1999 nodes 482 score cp -118 hashfull 4 nps 241 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1
info depth 9 seldepth 19 time 2070 nodes 506 score cp -105 hashfull 4 nps 244 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1
info depth 10 seldepth 19 time 2117 nodes 509 score cp -103 hashfull 4 nps 240 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1
info depth 10 seldepth 20 time 2271 nodes 608 score cp -95 hashfull 4 nps 267 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 g6f5 e1e8 h7g6 g2f3
info depth 11 seldepth 20 time 2336 nodes 626 score cp -96 hashfull 4 nps 267 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 g6f5 e1e8 h7g6 g2f3 b2a2
info depth 11 seldepth 21 time 2514 nodes 762 score cp -98 hashfull 5 nps 303 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1 f1f2 h7g6
info depth 11 seldepth 22 time 2708 nodes 905 score cp -105 hashfull 5 nps 334 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1 f1f2 h7g6
info depth 12 seldepth 22 time 2957 nodes 1080 score cp -102 hashfull 6 nps 365 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1 f1f2 h7g6 f2e3 g6h5
info depth 12 seldepth 23 time 3245 nodes 1316 score cp -102 hashfull 7 nps 405 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 e1b1 g6b1 f1e2 h7g6 e2f3 g6f5
info depth 13 seldepth 25 time 3580 nodes 1577 score cp -108 hashfull 8 nps 440 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1e1 b2b1 g2f3 b1e1 f1e1 g6f7 e1f2 h7g6
info depth 13 seldepth 26 time 3758 nodes 1713 score cp -100 hashfull 8 nps 455 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 e4f5 g6f5 d4a4 b2d2
info depth 14 seldepth 26 time 4420 nodes 2287 score cp -98 hashfull 11 nps 517 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 d4a4 b2d2 f1e1 d2h2
info depth 14 seldepth 27 time 5359 nodes 3097 score cp -97 hashfull 14 nps 577 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 e4f5 g6f5 d4a4 b2d2 a4a7 f5f6
info depth 15 seldepth 27 time 5719 nodes 3444 score cp -96 hashfull 15 nps 602 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 e4f5 g6f5 d4a4 b2d2 a4a7 f5f6
info depth 15 seldepth 28 time 6335 nodes 3977 score cp -95 hashfull 17 nps 627 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 e4f5 g6f5 d4a4 b2d2 a4a7 f5f6 a7a5
info depth 16 seldepth 28 time 7832 nodes 5358 score cp -93 hashfull 22 nps 684 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 d4a4 b2d2 f1e1 d2h2 e4f3
info depth 16 seldepth 29 time 8172 nodes 5639 score cp -92 hashfull 23 nps 690 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 g2e4 h7g6 d4a4 b2d2 f1e1 d2h2 e4f3
info depth 16 seldepth 30 time 8560 nodes 5981 score cp -92 hashfull 24 nps 698 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 d4f4 g7g6 h4h5 h7g7 h5g6 g7g6 g2e4 f5e4 f4e4 g6f5 e4e6 b2b6
info depth 16 seldepth 31 time 10056 nodes 7443 score cp -93 hashfull 29 nps 740 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 d4f4 g7g6 h4h5 h7g7 h5g6 g7g6 g2e4 f5e4 f4e4 g6f5 e4e6 b2b6
info depth 17 seldepth 32 time 10816 nodes 8182 score cp -92 hashfull 31 nps 756 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 d4f4 g7g6 h4h5 h7g7 h5g6 g7g6 g2e4 f5e4 f4e4 g6f5 e4e6 b2b6
info depth 17 seldepth 32 time 15838 nodes 13090 score cp -88 hashfull 48 nps 826 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 a1a4 e3e2 e1e2 e8e2 a4h4 h2h4 g3h4 e2b2 d1d4 g6f5 d4f4 g7g6 g2f3 h7g7 h4h5 g7f6 h5g6 f6g6 f3e4 f5e4 f4e4 g6f5 e4e6 b2b6 f1e2 b6b5
info string f1a6  (143 ) N:       3 (+ 0) (P:  0.59%) (Q: -0.99301) (U: 0.68336) (Q+U: -0.30965) (V: -0.9790)
info string f1b5  (141 ) N:       4 (+ 0) (P:  0.79%) (Q: -0.98608) (U: 0.73044) (Q+U: -0.25565) (V: -0.9443)
info string e1a5  (117 ) N:       5 (+ 0) (P:  1.02%) (Q: -0.99266) (U: 0.78307) (Q+U: -0.20959) (V: -0.9633)
info string a1a7  (19  ) N:       6 (+ 0) (P:  1.11%) (Q: -0.99507) (U: 0.72873) (Q+U: -0.26634) (V: -0.9704)
info string a1c1  (1   ) N:       6 (+ 0) (P:  1.13%) (Q: -0.99770) (U: 0.74682) (Q+U: -0.25089) (V: -0.9862)
info string a1a6  (17  ) N:       6 (+ 0) (P:  1.14%) (Q: -0.99528) (U: 0.74807) (Q+U: -0.24721) (V: -0.9717)
info string a8c6  (1610) N:       6 (+ 0) (P:  1.15%) (Q: -0.99696) (U: 0.75661) (Q+U: -0.24035) (V: -0.9818)
info string a1a3  (10  ) N:       6 (+ 0) (P:  1.16%) (Q: -0.99483) (U: 0.76264) (Q+U: -0.23219) (V: -0.9690)
info string a1a4  (13  ) N:       6 (+ 0) (P:  1.18%) (Q: -0.99594) (U: 0.77545) (Q+U: -0.22049) (V: -0.9756)
info string e1b4  (114 ) N:       6 (+ 0) (P:  1.19%) (Q: -0.99577) (U: 0.78525) (Q+U: -0.21052) (V: -0.9746)
info string a1a5  (15  ) N:       6 (+ 0) (P:  1.22%) (Q: -0.99522) (U: 0.80560) (Q+U: -0.18962) (V: -0.9713)
info string a8a3  (1602) N:       6 (+ 0) (P:  1.24%) (Q: -0.99683) (U: 0.81539) (Q+U: -0.18144) (V: -0.9810)
info string a8b7  (1612) N:       7 (+ 0) (P:  1.27%) (Q: -0.99672) (U: 0.72908) (Q+U: -0.26765) (V: -0.9771)
info string d1c1  (74  ) N:       7 (+ 0) (P:  1.28%) (Q: -0.99669) (U: 0.73677) (Q+U: -0.25992) (V: -0.9768)
info string a1b1  (0   ) N:       7 (+ 0) (P:  1.32%) (Q: -0.99819) (U: 0.76117) (Q+U: -0.23702) (V: -0.9873)
info string e1c3  (109 ) N:       7 (+ 0) (P:  1.33%) (Q: -0.99644) (U: 0.76820) (Q+U: -0.22824) (V: -0.9751)
info string f1c4  (139 ) N:       7 (+ 0) (P:  1.30%) (Q: -0.97137) (U: 0.75084) (Q+U: -0.22054) (V: -0.9705)
info string a8a4  (1604) N:       7 (+ 0) (P:  1.39%) (Q: -0.99724) (U: 0.80293) (Q+U: -0.19431) (V: -0.9807)
info string d1b1  (73  ) N:       7 (+ 0) (P:  1.40%) (Q: -0.99677) (U: 0.80623) (Q+U: -0.19055) (V: -0.9774)
info string d1d2  (81  ) N:       7 (+ 0) (P:  1.40%) (Q: -0.99203) (U: 0.80447) (Q+U: -0.18757) (V: -0.9442)
info string a8a2  (1600) N:       7 (+ 0) (P:  1.40%) (Q: -0.99067) (U: 0.80425) (Q+U: -0.18643) (V: -0.9347)
info string d1d3  (86  ) N:       7 (+ 0) (P:  1.41%) (Q: -0.99631) (U: 0.81062) (Q+U: -0.18569) (V: -0.9742)
info string a8b8  (1614) N:       7 (+ 0) (P:  1.41%) (Q: -0.99739) (U: 0.81194) (Q+U: -0.18545) (V: -0.9817)
info string a8a5  (1606) N:       8 (+ 0) (P:  1.48%) (Q: -0.99764) (U: 0.75826) (Q+U: -0.23938) (V: -0.9811)
info string a1a2  (7   ) N:       8 (+ 0) (P:  1.55%) (Q: -0.99081) (U: 0.79382) (Q+U: -0.19699) (V: -0.9265)
info string g3g4  (615 ) N:       8 (+ 0) (P:  1.58%) (Q: -0.99675) (U: 0.80847) (Q+U: -0.18828) (V: -0.9740)
info string a8a6  (1608) N:       8 (+ 0) (P:  1.59%) (Q: -0.99746) (U: 0.81316) (Q+U: -0.18430) (V: -0.9797)
info string f1d3  (134 ) N:       8 (+ 0) (P:  1.48%) (Q: -0.93854) (U: 0.75650) (Q+U: -0.18204) (V: -0.9719)
info string d1d4  (90  ) N:       8 (+ 0) (P:  1.59%) (Q: -0.99609) (U: 0.81511) (Q+U: -0.18097) (V: -0.9687)
info string a8a7  (1611) N:       8 (+ 0) (P:  1.60%) (Q: -0.99703) (U: 0.82098) (Q+U: -0.17606) (V: -0.9763)
info string e1e3  (111 ) N:       8 (+ 0) (P:  1.60%) (Q: -0.99153) (U: 0.82176) (Q+U: -0.16977) (V: -0.9636)
info string a8c8  (1615) N:       9 (+ 0) (P:  1.64%) (Q: -0.99733) (U: 0.75611) (Q+U: -0.24122) (V: -0.9759)
info string a8d8  (1616) N:       9 (+ 0) (P:  1.66%) (Q: -0.99796) (U: 0.76420) (Q+U: -0.23376) (V: -0.9816)
info string g1g2  (156 ) N:       9 (+ 0) (P:  1.67%) (Q: -0.98362) (U: 0.76983) (Q+U: -0.21379) (V: -0.9301)
info string e1f2  (107 ) N:       9 (+ 0) (P:  1.75%) (Q: -0.99576) (U: 0.80464) (Q+U: -0.19112) (V: -0.9619)
info string f1e2  (130 ) N:       9 (+ 0) (P:  1.74%) (Q: -0.97655) (U: 0.80359) (Q+U: -0.17296) (V: -0.9735)
info string e1d2  (105 ) N:      10 (+ 0) (P:  1.97%) (Q: -0.99565) (U: 0.82613) (Q+U: -0.16952) (V: -0.9565)
info string e1e2  (106 ) N:      15 (+ 0) (P:  2.79%) (Q: -0.98910) (U: 0.80337) (Q+U: -0.18573) (V: -0.8365)
info string f1h3  (138 ) N:      75 (+ 0) (P: 10.69%) (Q: -0.82589) (U: 0.64839) (Q+U: -0.17750) (V: -0.6030)
info string a8e8  (1617) N:     174 (+ 1) (P: 20.32%) (Q: -0.71494) (U: 0.53215) (Q+U: -0.18278) (V:  0.4921)
info string f1g2  (132 ) N:   12563 (+259) (P: 15.48%) (Q: -0.18999) (U: 0.00556) (Q+U: -0.18442) (V: -0.7062)
bestmove f1g2 ponder h6h2

@MelleKoning
Copy link
Contributor Author

@Videodr0me tested this position with your PR #700 and that fixes this for same net and same search amount of nodes. Nice work!

C:\dev\lc0>lc0cp --verbose-move-stats -t 1 --minibatch-size=256 --weights=.\nets\32300
       _
|   _ | |
|_ |_ |_| v0.21.0-dev built Jan 27 2019
position startpos moves e2e4 c7c5 g1f3 d7d6 d2d4 c5d4 f3d4 g8f6 b1c3 a7a6 c1e3 e7e5 d4b3 f8e7 f2f3 c8e6 d1d2 h7h5 c3d5 f6d5 e4d5 e6f5 b3a5 b7b6 a5c4 b8d7 a2a4 e8g8 f1e2 a8c8 e1g1 h5h4 h2h3 f5g6 b2b4 d8c7 f1c1 c7b7 c2c3 f8e8 e2f1 c8b8 e3f2 f7f5 d2e1 b7c7 f2h4 e7h4 e1h4 f5f4 a4a5 b6b5 c4d2 d7f6 c3c4 b5c4 d2c4 b8b4 c4b6 c7e7 f1a6 e5e4 h4f4 f6h5 f4e3 e7e5 f3f4 h5f4 a6f1 f4d3 c1d1 e5b2 b6d7 b4b3 a5a6 d3e5 e3g5 b3h3 g5d2 b2b3 d7e5 b3g3 e5g4 g3g4 d2e1 h3h8 a6a7 e4e3 a7a8Q g4f4 g2g3 f4h6
go nodes 5000
Loading weights file from: .\nets\32300
Creating backend [cudnn]...
info depth 1 seldepth 2 time 120 nodes 5 score cp 270 hashfull 0 nps 41 tbhits 0 pv a8e8 g6e8
info depth 2 seldepth 3 time 187 nodes 11 score cp 95 hashfull 0 nps 58 tbhits 0 pv f1g2 g8f7 g3g4
info depth 2 seldepth 4 time 254 nodes 15 score cp 272 hashfull 0 nps 59 tbhits 0 pv a8e8 g6e8 f1g2 h6h2
info depth 3 seldepth 5 time 317 nodes 19 score cp 349 hashfull 0 nps 59 tbhits 0 pv a8e8 g8h7 g3g4 h8g8 f1g2
info depth 3 seldepth 6 time 368 nodes 21 score cp 281 hashfull 0 nps 57 tbhits 0 pv a8e8 g8h7 g3g4 h8g8 f1g2 g8h8
info depth 3 seldepth 7 time 429 nodes 22 score cp 197 hashfull 0 nps 51 tbhits 0 pv a8e8 g8h7 g3g4 h8g8 f1g2 g8h8
info depth 4 seldepth 8 time 479 nodes 30 score cp 97 hashfull 0 nps 62 tbhits 0 pv a8e8 g6e8 f1g2 h6h2 g1f1 e8b5 d1d3 b5d3
info depth 4 seldepth 9 time 576 nodes 73 score cp 229 hashfull 1 nps 126 tbhits 0 pv f1g2 g8f7 a8e8 h8e8
info depth 4 seldepth 10 time 1446 nodes 409 score cp 124 hashfull 3 nps 282 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e3e2 f1f2
info depth 4 seldepth 11 time 2090 nodes 739 score cp 90 hashfull 4 nps 353 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2
info depth 4 seldepth 12 time 2154 nodes 752 score cp 88 hashfull 4 nps 349 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2
info depth 4 seldepth 13 time 2220 nodes 783 score cp 67 hashfull 5 nps 352 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2 f1e2
info depth 5 seldepth 13 time 2339 nodes 834 score cp 68 hashfull 5 nps 356 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 a8e8 h8e8 d1d4 e8f8 d4f4 g6d3 e1e2 d3e2 f1e2
info depth 6 seldepth 13 time 3931 nodes 1629 score cp 87 hashfull 9 nps 414 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1c3 h2g3 f1g1 g3f2 g1h1 e8e7 a8h8 h7h8
info depth 6 seldepth 14 time 5137 nodes 2253 score cp 42 hashfull 12 nps 438 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h2g3 d1e1 h7h6 b4a3 g3f2
info depth 6 seldepth 15 time 5620 nodes 2403 score cp 30 hashfull 14 nps 427 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h2g3 d1e1 h7h6 b4a3 g3f2
info depth 6 seldepth 16 time 5857 nodes 2656 score cp -2 hashfull 14 nps 453 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h8f8 f1e1 h2g1 g2f1 g1f1
info depth 7 seldepth 16 time 6241 nodes 2919 score cp -28 hashfull 15 nps 467 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h8f8 f1e1 h2g1 g2f1 g1f1
info depth 7 seldepth 17 time 6571 nodes 3148 score cp -52 hashfull 16 nps 479 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h8f8 f1e1 h2g1 g2f1 g1f1
info depth 7 seldepth 17 time 6881 nodes 3450 score cp -88 hashfull 17 nps 501 tbhits 0 pv f1g2 h6h2 g1f1 g8h7 e1b4 h8f8 f1e1 h2g1 g2f1 g1f1
info string d1d4  (90  ) N:       3 (+ 0) (P:  0.82%) (Q: -0.74813) (U: 0.39939) (Q+U: -0.34874) (V: -0.2444)  C:00000000
info string f1a6  (143 ) N:       4 (+ 0) (P:  1.71%) (Q: -0.93241) (U: 0.66485) (Q+U: -0.26756) (V: -0.7296)  C:00000000
info string a8d8  (1616) N:       4 (+ 0) (P:  1.88%) (Q: -0.89588) (U: 0.72985) (Q+U: -0.16603) (V: -0.5835)  C:00000000
info string a8a3  (1602) N:       5 (+ 0) (P:  2.02%) (Q: -0.88859) (U: 0.65397) (Q+U: -0.23463) (V: -0.4430)  C:00000000
info string a1a5  (15  ) N:       5 (+ 0) (P:  2.07%) (Q: -0.89546) (U: 0.67128) (Q+U: -0.22418) (V: -0.4773)  C:00000000
info string a1a6  (17  ) N:       5 (+ 0) (P:  2.32%) (Q: -0.90234) (U: 0.75068) (Q+U: -0.15167) (V: -0.5117)  C:00000000
info string a1a4  (13  ) N:       6 (+ 0) (P:  2.28%) (Q: -0.88359) (U: 0.63453) (Q+U: -0.24906) (V: -0.3016)  C:00000000
info string a1a7  (19  ) N:       6 (+ 0) (P:  2.63%) (Q: -0.92559) (U: 0.73163) (Q+U: -0.19396) (V: -0.5535)  C:00000000
info string a1c1  (1   ) N:       7 (+ 0) (P:  2.11%) (Q: -0.75623) (U: 0.51329) (Q+U: -0.24294) (V: -0.4722)  C:00000000
info string f1e2  (130 ) N:       9 (+ 0) (P:  2.99%) (Q: -0.80006) (U: 0.58248) (Q+U: -0.21758) (V: -0.4647)  C:00000000
info string a8a5  (1606) N:       9 (+ 0) (P:  1.84%) (Q: -0.55078) (U: 0.35825) (Q+U: -0.19253) (V: -0.2792)  C:00000000
info string e1f2  (107 ) N:       9 (+ 0) (P:  3.52%) (Q: -0.81663) (U: 0.68503) (Q+U: -0.13159) (V: -0.5040)  C:00000000
info string e1e3  (111 ) N:      10 (+ 0) (P:  1.58%) (Q: -0.48804) (U: 0.27927) (Q+U: -0.20877) (V:  0.2219)  C:00000000
info string a8c6  (1610) N:      12 (+ 0) (P:  1.99%) (Q: -0.47664) (U: 0.29795) (Q+U: -0.17869) (V: -0.1585)  C:00000000
info string a8a4  (1604) N:      12 (+ 0) (P:  1.51%) (Q: -0.39561) (U: 0.22609) (Q+U: -0.16952) (V:  0.1335)  C:00000000
info string e1d2  (105 ) N:      17 (+ 0) (P:  2.60%) (Q: -0.42785) (U: 0.28147) (Q+U: -0.14638) (V: -0.4225)  C:00000000
info string a8c8  (1615) N:      18 (+ 0) (P:  1.93%) (Q: -0.35243) (U: 0.19769) (Q+U: -0.15474) (V: -0.3441)  C:00000000
info string a8a2  (1600) N:      19 (+ 0) (P:  2.04%) (Q: -0.36114) (U: 0.19871) (Q+U: -0.16243) (V: -0.2011)  C:00000000
info string f1b5  (141 ) N:      27 (+ 0) (P:  1.17%) (Q: -0.24289) (U: 0.08162) (Q+U: -0.16127) (V: -0.4083)  C:00000000
info string a8b8  (1614) N:      28 (+ 0) (P:  2.63%) (Q: -0.33195) (U: 0.17614) (Q+U: -0.15581) (V: -0.4417)  C:00000000
info string e1a5  (117 ) N:      29 (+ 0) (P:  1.12%) (Q: -0.23683) (U: 0.07264) (Q+U: -0.16419) (V: -0.1233)  C:00000000
info string d1d3  (86  ) N:      30 (+ 0) (P:  1.14%) (Q: -0.22687) (U: 0.07178) (Q+U: -0.15508) (V: -0.2625)  C:00000000
info string f1c4  (139 ) N:      31 (+ 0) (P:  1.08%) (Q: -0.21286) (U: 0.06572) (Q+U: -0.14715) (V: -0.4358)  C:00000000
info string f1d3  (134 ) N:      32 (+ 0) (P:  1.65%) (Q: -0.26439) (U: 0.09700) (Q+U: -0.16739) (V: -0.3755)  C:00000000
info string a8b7  (1612) N:      32 (+ 0) (P:  1.69%) (Q: -0.25962) (U: 0.09966) (Q+U: -0.15996) (V: -0.1713)  C:00000000
info string a1b1  (0   ) N:      34 (+ 0) (P:  2.71%) (Q: -0.30700) (U: 0.15048) (Q+U: -0.15652) (V: -0.3363)  C:00000000
info string e1e2  (106 ) N:      36 (+ 0) (P:  3.68%) (Q: -0.33497) (U: 0.19373) (Q+U: -0.14124) (V: -0.0508)  C:00000000
info string a1a3  (10  ) N:      40 (+ 0) (P:  2.32%) (Q: -0.26528) (U: 0.10993) (Q+U: -0.15535) (V: -0.1194)  C:00000000
info string e1c3  (109 ) N:      40 (+ 0) (P:  1.26%) (Q: -0.21141) (U: 0.05994) (Q+U: -0.15147) (V: -0.0946)  C:00000000
info string a8e8  (1617) N:      44 (+ 0) (P:  6.33%) (Q: -0.42410) (U: 0.27359) (Q+U: -0.15051) (V:  0.4840)  C:00000000
info string e1b4  (114 ) N:      44 (+ 0) (P:  1.10%) (Q: -0.19668) (U: 0.04764) (Q+U: -0.14904) (V: -0.0900)  C:00000000
info string g1g2  (156 ) N:      49 (+ 0) (P:  3.15%) (Q: -0.26821) (U: 0.12246) (Q+U: -0.14575) (V: -0.1497)  C:00000000
info string d1b1  (73  ) N:      51 (+ 0) (P:  1.14%) (Q: -0.19476) (U: 0.04257) (Q+U: -0.15220) (V: -0.0335)  C:00000000
info string a1a2  (7   ) N:      51 (+ 0) (P:  3.64%) (Q: -0.27782) (U: 0.13607) (Q+U: -0.14175) (V: -0.0840)  C:00000000
info string d1c1  (74  ) N:      67 (+ 0) (P:  1.74%) (Q: -0.20273) (U: 0.04991) (Q+U: -0.15282) (V: -0.1800)  C:00000000
info string g3g4  (615 ) N:      73 (+ 0) (P:  5.12%) (Q: -0.27885) (U: 0.13465) (Q+U: -0.14420) (V: -0.2887)  C:00000000
info string a8a7  (1611) N:      77 (+ 0) (P:  2.33%) (Q: -0.20042) (U: 0.05814) (Q+U: -0.14227) (V:  0.2101)  C:00000000
info string d1d2  (81  ) N:     101 (+ 0) (P:  1.37%) (Q: -0.17036) (U: 0.02607) (Q+U: -0.14429) (V:  0.0355)  C:00000000
info string a8a6  (1608) N:     103 (+ 0) (P:  2.32%) (Q: -0.18573) (U: 0.04331) (Q+U: -0.14242) (V:  0.2275)  C:00000000
info string f1h3  (138 ) N:     110 (+ 0) (P:  4.71%) (Q: -0.23528) (U: 0.08249) (Q+U: -0.15279) (V:  0.3139)  C:00000000
info string f1g2  (132 ) N:    2160 (+97) (P:  8.74%) (Q: -0.19139) (U: 0.00753) (Q+U: -0.18386) (V:  0.4014)  C:00000000
bestmove f1g2 ponder h6h2

@oscardssmith
Copy link
Contributor

Close this then?

@MelleKoning
Copy link
Contributor Author

Closing now that #683 is merged and also #700 has that merged in already

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working release blocker Bugs which block releases
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

To ensure MakeTerminal value overwrites any MCTS calculated value MelleKoning/lc0
6 participants
@MelleKoning @jhellis3 @mooskagh @oscardssmith @Tilps @Videodr0me