Skip to content

Commit 84f3e86

Browse files
nodchipvondele
authored andcommitted
Add NNUE evaluation
This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish. Both the NNUE and the classical evaluations are available, and can be used to assign a value to a position that is later used in alpha-beta (PVS) search to find the best move. The classical evaluation computes this value as a function of various chess concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation computes this value with a neural network based on basic inputs. The network is optimized and trained on the evalutions of millions of positions at moderate search depth. The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward. It can be evaluated efficiently on CPUs, and exploits the fact that only parts of the neural network need to be updated after a typical chess move. [The nodchip repository](https://github.com/nodchip/Stockfish) provides additional tools to train and develop the NNUE networks. This patch is the result of contributions of various authors, from various communities, including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather, rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler, dorzechowski, and vondele. This new evaluation needed various changes to fishtest and the corresponding infrastructure, for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged. The first networks have been provided by gekkehenker and sergiovieri, with the latter net (nn-97f742aaefcd.nnue) being the current default. The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option, provided the `EvalFile` option points the the network file (depending on the GUI, with full path). The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest: 60000 @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c ELO: 92.77 +-2.1 (95%) LOS: 100.0% Total: 60000 W: 24193 L: 8543 D: 27264 Ptnml(0-2): 609, 3850, 9708, 10948, 4885 40000 @ 20+0.2 th 8 https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58 ELO: 89.47 +-2.0 (95%) LOS: 100.0% Total: 40000 W: 12756 L: 2677 D: 24567 Ptnml(0-2): 74, 1583, 8550, 7776, 2017 At the same time, the impact on the classical evaluation remains minimal, causing no significant regression: sprt @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f2906a2a5abc164f05e4c5b LLR: 2.94 (-2.94,2.94) {-6.00,-4.00} Total: 34936 W: 6502 L: 6825 D: 21609 Ptnml(0-2): 571, 4082, 8434, 3861, 520 sprt @ 60+0.6 th 1 https://tests.stockfishchess.org/tests/view/5f2906cfa5abc164f05e4c5d LLR: 2.93 (-2.94,2.94) {-6.00,-4.00} Total: 10088 W: 1232 L: 1265 D: 7591 Ptnml(0-2): 49, 914, 3170, 843, 68 The needed networks can be found at https://tests.stockfishchess.org/nns It is recommended to use the default one as indicated by the `EvalFile` UCI option. Guidelines for testing new nets can be found at https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests Integration has been discussed in various issues: #2823 #2728 The integration branch will be closed after the merge: #2825 https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip closes #2912 This will be an exciting time for computer chess, looking forward to seeing the evolution of this approach. Bench: 4746616
1 parent 9587eee commit 84f3e86

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+2470
-241
lines changed

.travis.yml

Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,12 @@
11
language: cpp
2-
dist: xenial
2+
dist: bionic
33

44
matrix:
55
include:
66
- os: linux
77
compiler: gcc
88
addons:
99
apt:
10-
sources: ['ubuntu-toolchain-r-test']
1110
packages: ['g++-8', 'g++-8-multilib', 'g++-multilib', 'valgrind', 'expect', 'curl']
1211
env:
1312
- COMPILER=g++-8
@@ -17,23 +16,23 @@ matrix:
1716
compiler: clang
1817
addons:
1918
apt:
20-
sources: ['ubuntu-toolchain-r-test', 'llvm-toolchain-xenial-6.0']
21-
packages: ['clang-6.0', 'llvm-6.0-dev', 'g++-multilib', 'valgrind', 'expect', 'curl']
19+
packages: ['clang-10', 'llvm-10-dev', 'g++-multilib', 'valgrind', 'expect', 'curl']
2220
env:
23-
- COMPILER=clang++-6.0
21+
- COMPILER=clang++-10
2422
- COMP=clang
25-
- LDFLAGS=-fuse-ld=lld
2623

2724
- os: osx
25+
osx_image: xcode12
2826
compiler: gcc
2927
env:
3028
- COMPILER=g++
3129
- COMP=gcc
3230

3331
- os: osx
32+
osx_image: xcode12
3433
compiler: clang
3534
env:
36-
- COMPILER=clang++ V='Apple LLVM 9.4.1' # Apple LLVM version 9.1.0 (clang-902.0.39.2)
35+
- COMPILER=clang++
3736
- COMP=clang
3837

3938
branches:
@@ -48,26 +47,34 @@ script:
4847
- git log HEAD | grep "\b[Bb]ench[ :]\+[0-9]\{7\}" | head -n 1 | sed "s/[^0-9]*\([0-9]*\).*/\1/g" > git_sig
4948
- export benchref=$(cat git_sig)
5049
- echo "Reference bench:" $benchref
50+
51+
#
52+
# Compiler version string
53+
- $COMPILER -v
54+
5155
#
5256
# Verify bench number against various builds
5357
- export CXXFLAGS="-Werror -D_GLIBCXX_DEBUG"
5458
- make clean && make -j2 ARCH=x86-64 optimize=no debug=yes build && ../tests/signature.sh $benchref
55-
- make clean && make -j2 ARCH=x86-32 optimize=no debug=yes build && ../tests/signature.sh $benchref
56-
- make clean && make -j2 ARCH=x86-32 build && ../tests/signature.sh $benchref
59+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then make clean && make -j2 ARCH=x86-32 optimize=no debug=yes build && ../tests/signature.sh $benchref; fi
60+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then make clean && make -j2 ARCH=x86-32 build && ../tests/signature.sh $benchref; fi
5761

5862
#
5963
# Check perft and reproducible search
64+
- export CXXFLAGS="-Werror"
65+
- make clean && make -j2 ARCH=x86-64 build
6066
- ../tests/perft.sh
6167
- ../tests/reprosearch.sh
68+
6269
#
6370
# Valgrind
6471
#
6572
- export CXXFLAGS="-O1 -fno-inline"
6673
- if [ -x "$(command -v valgrind )" ]; then make clean && make -j2 ARCH=x86-64 debug=yes optimize=no build > /dev/null && ../tests/instrumented.sh --valgrind; fi
6774
- if [ -x "$(command -v valgrind )" ]; then ../tests/instrumented.sh --valgrind-thread; fi
75+
6876
#
6977
# Sanitizer
7078
#
71-
# Use g++-8 as a proxy for having sanitizers, might need revision as they become available for more recent versions of clang/gcc
72-
- if [[ "$COMPILER" == "g++-8" ]]; then make clean && make -j2 ARCH=x86-64 sanitize=undefined optimize=no debug=yes build > /dev/null && ../tests/instrumented.sh --sanitizer-undefined; fi
73-
- if [[ "$COMPILER" == "g++-8" ]]; then make clean && make -j2 ARCH=x86-64 sanitize=thread optimize=no debug=yes build > /dev/null && ../tests/instrumented.sh --sanitizer-thread; fi
79+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then make clean && make -j2 ARCH=x86-64 sanitize=undefined optimize=no debug=yes build > /dev/null && ../tests/instrumented.sh --sanitizer-undefined; fi
80+
- if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then make clean && make -j2 ARCH=x86-64 sanitize=thread optimize=no debug=yes build > /dev/null && ../tests/instrumented.sh --sanitizer-thread; fi

AUTHORS

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,17 @@
1-
# List of authors for Stockfish, as of March 30, 2020
1+
# List of authors for Stockfish, as of August 4, 2020
22

3+
# Founders of the Stockfish project and fishtest infrastructure
34
Tord Romstad (romstad)
45
Marco Costalba (mcostalba)
56
Joona Kiiski (zamar)
67
Gary Linscott (glinscott)
78

9+
# Authors and inventors of NNUE, training, NNUE port
10+
Yu Nasu (ynasu87)
11+
Motohiro Isozaki (yaneurao)
12+
Hisayori Noda (nodchip)
13+
14+
# all other authors of the code in alphabetical order
815
Aditya (absimaldata)
916
Adrian Petrescu (apetresc)
1017
Ajith Chandy Jose (ajithcj)
@@ -36,6 +43,7 @@ Dariusz Orzechowski
3643
David Zar
3744
Daylen Yang (daylen)
3845
DiscanX
46+
Dominik Schlösser (domschl)
3947
double-beep
4048
Eduardo Cáceres (eduherminio)
4149
Eelco de Groot (KingDefender)
@@ -115,7 +123,8 @@ Nick Pelling (nickpelling)
115123
Nicklas Persson (NicklasPersson)
116124
Niklas Fiekas (niklasf)
117125
Nikolay Kostov (NikolayIT)
118-
Nguyen Pham
126+
Nguyen Pham (nguyenpham)
127+
Norman Schmidt (FireFather)
119128
Ondrej Mosnáček (WOnder93)
120129
Oskar Werkelin Ahlin
121130
Pablo Vazquez
@@ -135,6 +144,7 @@ Richard Lloyd
135144
Rodrigo Exterckötter Tjäder
136145
Ron Britvich (Britvich)
137146
Ronald de Man (syzygy1, syzygy)
147+
rqs
138148
Ryan Schmitt
139149
Ryan Takker
140150
Sami Kiminki (skiminki)
@@ -143,6 +153,7 @@ Sergei Antonov (saproj)
143153
Sergei Ivanov (svivanov72)
144154
sf-x
145155
Shane Booth (shane31)
156+
Shawn Varghese (xXH4CKST3RXx)
146157
Stefan Geschwentner (locutus2)
147158
Stefano Cardanobile (Stefano80)
148159
Steinar Gunderson (sesse)
@@ -155,9 +166,11 @@ Tom Vijlbrief (tomtor)
155166
Tomasz Sobczyk (Sopel97)
156167
Torsten Franz (torfranz, tfranzer)
157168
Tracey Emery (basepr1me)
169+
tttak
158170
Unai Corzo (unaiic)
159171
Uri Blass (uriblass)
160172
Vince Negri (cuddlestmonkey)
173+
zz4032
161174

162175

163176
# Additionally, we acknowledge the authors and maintainers of fishtest,

Readme.md renamed to README.md

Lines changed: 86 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,13 @@
44
[![Build Status](https://ci.appveyor.com/api/projects/status/github/official-stockfish/Stockfish?branch=master&svg=true)](https://ci.appveyor.com/project/mcostalba/stockfish/branch/master)
55

66
[Stockfish](https://stockfishchess.org) is a free, powerful UCI chess engine
7-
derived from Glaurung 2.1. It is not a complete chess program and requires a
7+
derived from Glaurung 2.1. It features two evaluation functions, the classical
8+
evaluation based on handcrafted terms, and the NNUE evaluation based on
9+
efficiently updateable neural networks. The classical evaluation runs efficiently
10+
on most 64bit CPU architectures, while the NNUE evaluation benefits strongly from the
11+
vector intrinsics available on modern CPUs (avx2 or similar).
12+
13+
Stockfish is not a complete chess program and requires a
814
UCI-compatible GUI (e.g. XBoard with PolyGlot, Scid, Cute Chess, eboard, Arena,
915
Sigma Chess, Shredder, Chess Partner or Fritz) in order to be used comfortably.
1016
Read the documentation for your GUI of choice for information about how to use
@@ -22,20 +28,19 @@ This distribution of Stockfish consists of the following files:
2228
* src, a subdirectory containing the full source code, including a Makefile
2329
that can be used to compile Stockfish on Unix-like systems.
2430

31+
To use the NNUE evaluation an additional data file with neural network parameters
32+
needs to be downloaded. The filename for the default set can be found as the default
33+
value of the `EvalFile` UCI option, with the format
34+
`nn-[SHA256 first 12 digits].nnue` (e.g. nn-c157e0a5755b.nnue). This file can be downloaded from
35+
```
36+
https://tests.stockfishchess.org/api/nn/[filename]
37+
```
38+
replacing `[filename]` as needed.
2539

26-
## UCI parameters
27-
28-
Currently, Stockfish has the following UCI options:
29-
30-
* #### Debug Log File
31-
Write all communication to and from the engine into a text file.
3240

33-
* #### Contempt
34-
A positive value for contempt favors middle game positions and avoids draws.
41+
## UCI options
3542

36-
* #### Analysis Contempt
37-
By default, contempt is set to prefer the side to move. Set this option to "White"
38-
or "Black" to analyse with contempt for that side, or "Off" to disable contempt.
43+
Currently, Stockfish has the following UCI options:
3944

4045
* #### Threads
4146
The number of CPU threads used for searching a position. For best performance, set
@@ -44,50 +49,51 @@ Currently, Stockfish has the following UCI options:
4449
* #### Hash
4550
The size of the hash table in MB. It is recommended to set Hash after setting Threads.
4651

47-
* #### Clear Hash
48-
Clear the hash table.
49-
5052
* #### Ponder
5153
Let Stockfish ponder its next move while the opponent is thinking.
5254

5355
* #### MultiPV
5456
Output the N best lines (principal variations, PVs) when searching.
5557
Leave at 1 for best performance.
5658

57-
* #### Skill Level
58-
Lower the Skill Level in order to make Stockfish play weaker (see also UCI_LimitStrength).
59-
Internally, MultiPV is enabled, and with a certain probability depending on the Skill Level a
60-
weaker move will be played.
59+
* #### Use NNUE
60+
Toggle between the NNUE and classical evaluation functions. If set to "true",
61+
the network parameters must be availabe to load from file (see also EvalFile).
6162

62-
* #### UCI_LimitStrength
63-
Enable weaker play aiming for an Elo rating as set by UCI_Elo. This option overrides Skill Level.
63+
* #### EvalFile
64+
The name of the file of the NNUE evaluation parameters. Depending on the GUI the
65+
filename should include the full path to the folder/directory that contains the file.
6466

65-
* #### UCI_Elo
66-
If enabled by UCI_LimitStrength, aim for an engine strength of the given Elo.
67-
This Elo rating has been calibrated at a time control of 60s+0.6s and anchored to CCRL 40/4.
67+
* #### Contempt
68+
A positive value for contempt favors middle game positions and avoids draws,
69+
effective for the classical evaluation only.
70+
71+
* #### Analysis Contempt
72+
By default, contempt is set to prefer the side to move. Set this option to "White"
73+
or "Black" to analyse with contempt for that side, or "Off" to disable contempt.
74+
75+
* #### UCI_AnalyseMode
76+
An option handled by your GUI.
77+
78+
* #### UCI_Chess960
79+
An option handled by your GUI. If true, Stockfish will play Chess960.
6880

6981
* #### UCI_ShowWDL
7082
If enabled, show approximate WDL statistics as part of the engine output.
7183
These WDL numbers model expected game outcomes for a given evaluation and
7284
game ply for engine self-play at fishtest LTC conditions (60+0.6s per game).
7385

74-
* #### Move Overhead
75-
Assume a time delay of x ms due to network and GUI overheads. This is useful to
76-
avoid losses on time in those cases.
77-
78-
* #### Slow Mover
79-
Lower values will make Stockfish take less time in games, higher values will
80-
make it think longer.
81-
82-
* #### nodestime
83-
Tells the engine to use nodes searched instead of wall time to account for
84-
elapsed time. Useful for engine testing.
86+
* #### UCI_LimitStrength
87+
Enable weaker play aiming for an Elo rating as set by UCI_Elo. This option overrides Skill Level.
8588

86-
* #### UCI_Chess960
87-
An option handled by your GUI. If true, Stockfish will play Chess960.
89+
* #### UCI_Elo
90+
If enabled by UCI_LimitStrength, aim for an engine strength of the given Elo.
91+
This Elo rating has been calibrated at a time control of 60s+0.6s and anchored to CCRL 40/4.
8892

89-
* #### UCI_AnalyseMode
90-
An option handled by your GUI.
93+
* #### Skill Level
94+
Lower the Skill Level in order to make Stockfish play weaker (see also UCI_LimitStrength).
95+
Internally, MultiPV is enabled, and with a certain probability depending on the Skill Level a
96+
weaker move will be played.
9197

9298
* #### SyzygyPath
9399
Path to the folders/directories storing the Syzygy tablebase files. Multiple
@@ -114,6 +120,47 @@ Currently, Stockfish has the following UCI options:
114120
Limit Syzygy tablebase probing to positions with at most this many pieces left
115121
(including kings and pawns).
116122

123+
* #### Move Overhead
124+
Assume a time delay of x ms due to network and GUI overheads. This is useful to
125+
avoid losses on time in those cases.
126+
127+
* #### Slow Mover
128+
Lower values will make Stockfish take less time in games, higher values will
129+
make it think longer.
130+
131+
* #### nodestime
132+
Tells the engine to use nodes searched instead of wall time to account for
133+
elapsed time. Useful for engine testing.
134+
135+
* #### Clear Hash
136+
Clear the hash table.
137+
138+
* #### Debug Log File
139+
Write all communication to and from the engine into a text file.
140+
141+
## classical and NNUE evaluation
142+
143+
Both approaches assign a value to a position that is used in alpha-beta (PVS) search
144+
to find the best move. The classical evaluation computes this value as a function
145+
of various chess concepts, handcrafted by experts, tested and tuned using fishtest.
146+
The NNUE evaluation computes this value with a neural network based on basic
147+
inputs (e.g. piece positions only). The network is optimized and trained
148+
on the evalutions of millions of positions at moderate search depth.
149+
150+
The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward.
151+
It can be evaluated efficiently on CPUs, and exploits the fact that only parts
152+
of the neural network need to be updated after a typical chess move.
153+
[The nodchip repository](https://github.com/nodchip/Stockfish) provides additional
154+
tools to train and develop the NNUE networks.
155+
156+
On CPUs supporting modern vector instructions (avx2 and similar), the NNUE evaluation
157+
results in stronger playing strength, even if the nodes per second computed by the engine
158+
is somewhat lower (roughly 60% of nps is typical).
159+
160+
Note that the NNUE evaluation depends on the Stockfish binary and the network parameter
161+
file (see EvalFile). Not every parameter file is compatible with a given Stockfish binary.
162+
The default value of the EvalFile UCI option is the name of a network that is guaranteed
163+
to be compatible with that binary.
117164

118165
## What to expect from Syzygybases?
119166

appveyor.yml

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,9 @@ clone_depth: 50
44
branches:
55
only:
66
- master
7-
- appveyor
87

98
# Operating system (build VM template)
10-
os: Visual Studio 2017
9+
os: Visual Studio 2019
1110

1211
# Build platform, i.e. x86, x64, AnyCPU. This setting is optional.
1312
platform:
@@ -36,8 +35,11 @@ before_build:
3635
$src = $src.Replace("\", "/")
3736
3837
# Build CMakeLists.txt
39-
$t = 'cmake_minimum_required(VERSION 3.8)',
38+
$t = 'cmake_minimum_required(VERSION 3.17)',
4039
'project(Stockfish)',
40+
'set(CMAKE_CXX_STANDARD 17)',
41+
'set(CMAKE_CXX_STANDARD_REQUIRED ON)',
42+
'set (CMAKE_CXX_EXTENSIONS OFF)',
4143
'set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/src)',
4244
'set(source_files', $src, ')',
4345
'add_executable(stockfish ${source_files})'
@@ -51,10 +53,11 @@ before_build:
5153
$b = git log HEAD | sls "\b[Bb]ench[ :]+[0-9]{7}" | select -first 1
5254
$bench = $b -match '\D+(\d+)' | % { $matches[1] }
5355
Write-Host "Reference bench:" $bench
54-
$g = "Visual Studio 15 2017"
55-
If (${env:PLATFORM} -eq 'x64') { $g = $g + ' Win64' }
56-
cmake -G "${g}" .
57-
Write-Host "Generated files for: " $g
56+
$g = "Visual Studio 16 2019"
57+
If (${env:PLATFORM} -eq 'x64') { $a = "x64" }
58+
If (${env:PLATFORM} -eq 'x86') { $a = "Win32" }
59+
cmake -G "${g}" -A ${a} .
60+
Write-Host "Generated files for: " $g $a
5861
5962
build_script:
6063
- cmake --build . --config %CONFIGURATION% -- /verbosity:minimal

0 commit comments

Comments
 (0)