v0.31.0-rc1

[borg323]

In this version:

• The blas, cuda, eigen, metal and onnx backends now have support for multihead network architecture and can run BT3/BT4 nets.
• Updated the internal Elo model to better align with regular Elo for human players.
• There is a new XLA backend that uses OpenXLA compiler to produce code to execute the neural network. See https://github.com/LeelaChessZero/lc0/wiki/XLA-backend for details. Related are new leela2onnx options to output the HLO format that XLA understands.
• There is a vastly simplified lc0 interface available by renaming the executable to lc0simple.
• The backends can now suggest a minibatch size to the search, this is enabled by --minibatch-size=0 (the new default).
• If the cudnn backend detected an unsupported network architecture it will switch to the cuda backend.
• Two new selfplay options enable value and policy tournaments. A policy tournament is using a single node policy to select the move to play, while a value tournament searches all possible moves at depth 1 to select the one with the best q.
• While it is easy to get a single node policy evaluation (go nodes 1 using uci), there was no simple way to get the effect of a value only evaluation, so the --value-only option was added.
• Button uci options were implemented and a button to clear the tree was added (as hidden option).
• Support for the uci go mate option was added.
• The rescorer can now be built from the lc0 code base instead of a separate branch.
• A dicrete onnx layernorm implementation was added to get around a onnxruntime bug with directml - this has some overhead so it is only enabled for onnx-dml and can be switched off with the alt_layernorm=false backend option.
• The --onnx2pytoch option was added to leela2onnx to generate pytorch compatible models.
• There is a cuda min_batch backend option to reduce non-determinism with small batches.
• New options were added to onnx2leela to fix tf exported onnx models.
• The onnx backend can now be built for amd's rocm.
• Fixed a bug where the Contempt effect on eval was too low for nets with natively higher draw rates.
• Made the WDL Rescale sharpness limit configurable via the --wdl-max-s hidden option.
• The search task workers can be set automatically, to either 0 for cpu backends or up to 4 depending on the number of cpu cores. This is enabled by --task-workers=-1 (the new default).
• Several assorted fixes and code cleanups.

---

This discussion was created from the release v0.31.0-rc1.

[Massa7ca]

USE BT4-1024x15x32h-swa-6147500 model networks

[mooskagh]

Could you record a log file (preferably a leela log, there is an engine option LogFile for that; but cutechess UCI log is also fine) and attach it?

Also it may be easier to debug this in our Discord chat at http://lc0.org/chat.

[borg323]

Can you tell us the GPU used?

[Massa7ca]

rtx 4090 with 551.86 driver. I add log file
log.log

[borg323]

Can you try the v0.31.0-rc2 release?

[fsmosca]

What is wrong?

uciok
isready
readyok
ucinewgame
Found pb network file: F:\Chess\Engines\Lc0\lc0-v0.31.0-rc1-windows-gpu-nvidia-cudnn/791556.pb.gz
Creating backend [cudnn-auto]...
Switching to [cudnn-fp16]...
CUDA Runtime version: 10.0.0
Cudnn version: 7.4.2
Latest version of CUDA supported by the driver: 12.2.0
GPU: NVIDIA GeForce GTX 1650 SUPER
GPU memory: 3.99957 GiB
GPU clock frequency: 1770 MHz
GPU compute capability: 7.5
isready
readyok
ucinewgame
isready
readyok
position startpos
go movetime 60000
Unhandled exception in worker thread: CUDA error: an illegal memory access was encountered (c:\projects\lc0\src\neural\cuda\network_cudnn.cc:865)

[fsmosca]

It worked with v0.30.0

uciok
isready
readyok
ucinewgame
Found pb network file: F:\Chess\Engines\Lc0\lc0-v0.30.0-windows-gpu-nvidia-cudnn/791556.pb.gz
Creating backend [cudnn-auto]...
Switching to [cudnn-fp16]...
CUDA Runtime version: 10.0.0
Cudnn version: 7.4.2
Latest version of CUDA supported by the driver: 12.2.0
GPU: NVIDIA GeForce GTX 1650 SUPER
GPU memory: 3.99957 GiB
GPU clock frequency: 1770 MHz
GPU compute capability: 7.5
isready
readyok
position startpos
go movetime 10000
info depth 1 seldepth 2 time 6094 nodes 2 score cp 27 nps 23 tbhits 0 pv e2e4 e7e5
info depth 2 seldepth 3 time 6211 nodes 6 score cp 27 nps 29 tbhits 0 pv e2e4 e7e5 g1f3
info depth 2 seldepth 4 time 6223 nodes 11 score cp 25 nps 51 tbhits 0 pv e2e4 e7e5 g1f3 g8f6

[mooskagh]

Converted to a bug here: #2007

[borg323]

Can you also test with https://ci.appveyor.com/api/buildjobs/xpg6y3c063prploq/artifacts/build%2Flc0.exe? This is the latest cuda/cudnn code merged in v0.30.0.

[fsmosca]

That one failed too.

PS F:\Chess\Engines\Lc0\lc0-v0.31.0-rc1-windows-gpu-nvidia-cudnn> ./lc0-1
       _
|   _ | |
|_ |_ |_| v0.30.0+git.b4dc0ebb built Mar 29 2024
uci
id name Lc0 v0.30.0+git.b4dc0ebb
id author The LCZero Authors.

...

uciok
isready
readyok
ucinewgame
Found pb network file: F:\Chess\Engines\Lc0\lc0-v0.31.0-rc1-windows-gpu-nvidia-cudnn/791556.pb.gz
Creating backend [cudnn-auto]...
Switching to [cudnn-fp16]...
CUDA Runtime version: 10.0.0
Cudnn version: 7.4.2
Latest version of CUDA supported by the driver: 12.2.0
GPU: NVIDIA GeForce GTX 1650 SUPER
GPU memory: 3.99957 GiB
GPU clock frequency: 1770 MHz
GPU compute capability: 7.5
isready
readyok
position startpos
go movetime 30000
Unhandled exception in worker thread: CUDA error: an illegal memory access was encountered (c:\projects\lc0\src\neural\cuda\network_cudnn.cc:865)

[borg323]

I added two more tests to do in issue #2007