Skip to content

Commit 737c02e

Browse files
SjorsSjors
committed
doc: explain Stratum v2 design, testing and usage
1 parent 44787d1 commit 737c02e

File tree

2 files changed

+380
-1
lines changed

2 files changed

+380
-1
lines changed

‎doc/stratum-v2.md

Lines changed: 379 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,379 @@
1+
# Stratum v2
2+
3+
## Design
4+
5+
The Stratum v2 protocol specification can be found here: https://github.com/stratum-mining/sv2-spec
6+
7+
Bitcoin Core performs the Template Provider role, and for that it implements the
8+
Template Distribution Protocol. When launched with `-sv2` we listen for connections
9+
from Job Declarator clients.
10+
11+
A Job Declarator client might run on the same machine, e.g. for a single ASIC
12+
hobby miner. In a more advanced setup it might run on another machine, on the same
13+
local network or remote. A third possible use case is where a miner relies on a
14+
node run by someone else to provide the templates. Trust may not go both ways in
15+
that scenario, see the section on DoS.
16+
17+
We send them a new block template whenenver out tip is updated, or when mempool
18+
fees have increased sufficiently. If the pool finds a block, we attempt to
19+
broadcast it based on a cached template.
20+
21+
Communication with other roles uses the Noise Protocol, which has been implemented
22+
to the extend necessary. Its cryptographic primitives were chosen so that they
23+
were already present in the Bitcoin Core project at the time of writing the spec.
24+
25+
### Advantage over getblocktemplate RPC
26+
27+
Although under the hood the Template Provider uses `CreateNewBlock()` just like
28+
the `getblocktemplate` RPC, there's a number of advantages in running a
29+
server with a stateful connection, and avoiding JSON RPC in general.
30+
31+
1. Stateful, so we can have back-and-forth, e.g. requesting transaction data,
32+
processing a block solution.
33+
2. Less (de)serializing and data sent over the wire, compared to plain text JSON
34+
3. Encrypted, safer (for now: less unsafe) to expose on the public internet
35+
4. Push based: new template is sent immediately when a new block is found rather
36+
than at the next poll interval. Combined with Cluster Mempool this can
37+
hopefully be done for higher fee templates too.
38+
5. Low friction deployment with other Stratum v2 software / devices
39+
40+
### Message flow(s)
41+
42+
See the [Message Types](https://github.com/stratum-mining/sv2-spec/blob/main/08-Message-Types.md)
43+
and [Protocol Overview](https://github.com/stratum-mining/sv2-spec/blob/main/03-Protocol-Overview.md)
44+
section of the spec for all messages and their details.
45+
46+
When a Job Declarator client connects to us, it first sends a `SetupConnection`
47+
message. We reply with `SetupConnection.Success` unless something went wrong,
48+
e.g. version mismatch, in which case we reply with `SetupConnection.Error`.
49+
50+
Next the client sends us their `CoinbaseOutputDataSize`. If this is invalid we
51+
disconnect. Otherwise we start the cycle below that repeats with every block.
52+
53+
We send a `NewTemplate` message with `future_template` set `true`, immedidately
54+
followed by `SetNewPrevHash`. We _don't_ send any transaction information
55+
at this point. The Job Declarator client uses this to announce upstream that
56+
it wants to declare a new template.
57+
58+
In the simplest setup with SRI the Job Declarator client doubles as a proxy and
59+
sends these two messages to all connected mining devices. They will keep
60+
working on their previous job until the `SetNewPrevHash` message arrives.
61+
Future implementations could provide an empty or speculative template before
62+
a new block is found.
63+
64+
Meanwhile the pool will request, via the Job Declarator client, the transaction
65+
lists belonging to the template: `RequestTransactionData`. In case of a problem
66+
we reply with `RequestTransactionData.Error`. Otherwise we reply with the full[0]
67+
transaction data in `RequestTransactionData.Success`.
68+
69+
When we find a template with higher fees, we send a `NewTemplate` message
70+
with `future_template` set to `false`. This is _not_ followed by `SetNewPrevHash`.
71+
72+
Finally, if we find an actual block, the client sends us `SubmitSolution`.
73+
We then lookup the template (may not be the most recent one), reconstruct
74+
the block and broadcast it. The pool will do the same.
75+
76+
`[0]`: When the Job Declarator client communicates with the Job Declarator
77+
server there is an intermediate message which sends short transaction ids
78+
first, followed by a `ProvideMissingTransactions` message. The spec could be
79+
modified to introduce a similar message here. This is especially useful when
80+
the Template Provider runs on a different machine than the Job Declarator
81+
client. Erlay might be useful here too, in a later stage.
82+
83+
### Noise Protocol
84+
85+
As detailed in the [Protocol Security](https://github.com/stratum-mining/sv2-spec/blob/main/04-Protocol-Security.md)
86+
section of the spec, Stratum v2 roles use the Noise Protocol to communicate.
87+
88+
We only implement the parts needed for inbound connections, although not much
89+
code would be needed to support outbound connections as well if this is required later.
90+
91+
The spec was written before BIP 324 peer-to-peer encryption was introduced. It
92+
has much in common with Noise, but for the purposes of Stratum v2 it currently
93+
lacks authentication. Perhaps a future version of Stratum will use this. Since
94+
we only communicate with the Job Declarator role, a transition to BIP 324 would
95+
not require waiting for the entire mining ecosystem to adopt it.
96+
97+
An alternative to implementing the Noise Protocol in Bitcoin Core is to use a
98+
unix socket instead and rely on the user to install a separate tool to convert
99+
to this protocol. Such a tool could be provided by developers of the Job
100+
Declarator client.
101+
102+
ZMQ may be slightly more convenient than a unix socket. Since the Stratum v2
103+
protocol is stateful we would need to use the [request-reply](https://zguide.zeromq.org/docs/chapter3/)
104+
mode. Currently we only use the unidirectional `ZMQ_PUB` mode, see
105+
[zmq_socket](http://api.zeromq.org/4-2:zmq-socket). But then Stratum v2 messages
106+
can be sent and received without dealing with low level sockets / buffers / bytes.
107+
This could be implemented as a ZmqTransport subclass of Transport. Whether this
108+
involves less new code than the Noise Protocol remains to be seen.
109+
110+
### Mempool monitoring
111+
112+
The current design calls `CreateNewBlock()` internally every `-sv2interval` seconds.
113+
We then broadcast the resulting block template if fees have increased enough to make
114+
it worth the overhead (`-sv2feedelta`). A pool may have additional rate limiting in
115+
place.
116+
117+
This is better than the Stratum v1 model of a polling call to the `getblocktemplate` RPC.
118+
It avoids (de)serializing JSON, uses an encrypted connection and only sends data
119+
over the wire if fees increased.
120+
121+
But it's still a poll based model, as opposed to the push based approach
122+
whenever a new block arrives. It would be better if a new template is generated
123+
as soon as a potentially revenue-increasing transaction is added to the mempool.
124+
The Cluster Mempool project might enable that.
125+
126+
### DoS and privacy
127+
128+
The current Template Provider should not be run on the public internet with
129+
unlimited access. It is not harneded against DoS attacks, nor against mempool probing.
130+
131+
There's currently no limit to the number of Job Declarator clients that can connect,
132+
which could exhaust memory. There's also no limit to the amount of raw transaction
133+
data that can be requested.
134+
135+
Templates reveal what is in the mempool without any delay or randomization.
136+
137+
This is why the use of `-sv2allowip` is required when `-sv2bind` is set to
138+
anything other than localhost on mainnet.
139+
140+
Future improvements should aim to reduce or eliminate the above concerns such
141+
that any node can run a Template Provider as a public service.
142+
143+
## Usage
144+
145+
Using this in a production environment is not yet recommended, but see the testing guide below.
146+
147+
### Build
148+
149+
Follow the instructions in [build-unix.md](build-unix.md), [build-osx.md](build-osx.md),
150+
etc, but add `-DWITH_SV2=ON` to `cmake -B build`.
151+
152+
### Parameters
153+
154+
See also `bitcoind --help`.
155+
156+
Start Bitcoin Core with `-sv2` to start a Template Provider server with default settings.
157+
The listening port can be changed with `-sv2port`.
158+
159+
By default it only accepts connections from localhost. This can be changed
160+
using `-sv2bind`, which requires the use of `-sv2allowip`. See DoS and Privacy below.
161+
162+
Use `-debug=sv2` to see Stratum v2 related log messages. Set `-loglevel=sv2:trace`
163+
to see which messages are exchanged with the Job Declarator client.
164+
165+
The frequency at which new templates are generated can be controlled with
166+
`-sv2interval`. The new templates are only submitted to connected clients if
167+
they are for a new block, or if fees have increased by at least `-sv2feedelta`.
168+
169+
You may increase `-sv2interval`` to something your node can handle, and then
170+
adjust `-sv2feedelta` to limit back and forth with the pool.
171+
172+
You can use `-debug=bench` to see how long block generation typically takes on
173+
your machine, look for `CreateNewBlock() ... (total ...ms)`. Another factor to
174+
consider is upstream rate limiting, see the [Job Declaration Protocol](https://github.com/stratum-mining/sv2-spec/blob/main/06-Job-Declaration-Protocol.md).
175+
Mining hardware may also incur a performance dip when it receives a new job.
176+
177+
## Testing Guide
178+
179+
Unfortunately testing still requires quite a few moving parts, and each setup has
180+
its own merits and issues.
181+
182+
To get help with the stratum side of things, this Discord may be useful: https://discord.gg/fsEW23wFYs
183+
184+
The Stratum Reference Implementation (SRI) provides example implementations of
185+
the various (other) Stratum v2 roles: https://github.com/stratum-mining/stratum
186+
187+
You can set up an entire pool on your own machine. You can also connect to an
188+
existing pool and only run a limited set of roles on your machine, e.g. the
189+
Job Declarator client and Translator (v1 to v2).
190+
191+
SRI includes a v1 and v2 CPU miner, but at the time of writing neither seems to work.
192+
Another CPU miner that does work, when used with the Translator: https://github.com/pooler/cpuminer
193+
194+
### Regtest
195+
196+
TODO
197+
198+
This is also needed for functional test suite coverage. It's also the only test
199+
network doesn't need a standalone CPU miner or ASIC.
200+
201+
Perhaps a mock Job Declator client can be added. We also need a way mine a given
202+
block template, akin to `generate`.
203+
204+
To make testing easier it should be possible to use a connection without Noise Protocol.
205+
206+
### Testnet
207+
208+
The difficulty on testnet4 varies wildly, but typically much too high for CPU mining.
209+
Even when using a relatively cheap second hand miner, e.g. an S9, it could take
210+
days to find a block.
211+
212+
The above means it's difficult to test the `SubmitSolution` message.
213+
214+
#### Bring your own ASIC, use external testnet pool
215+
216+
This uses an existing testnet pool. There's no need to create an account anywhere.
217+
The pool does not pay out the testnet coins it generates. It also currently
218+
doesn't censor anything, so you can't test the (solo mining) fallback behavior.
219+
220+
First start the node:
221+
222+
```
223+
build/src/bitcoind -testnet4 -sv2 -debug=sv2
224+
```
225+
226+
Build and run a Job Declator client: [stratum-mining/stratum/tree/main/roles/jd-client](https://github.com/stratum-mining/stratum/tree/main/roles/jd-client
227+
228+
This client connects to your node to receive new block templates and then "declares"
229+
them to a Job Declarator server. Additionally it connects to the pool itself.
230+
231+
Copy [jdc-config-hosted-example.toml](https://github.com/stratum-mining/stratum/blob/main/roles/jd-client/config-examples/jdc-config-hosted-example.toml)
232+
to e.g. `~/.stratum/testnet4-jdc.toml`, change `tp_address` to `127.0.0.1:48336` and comment out `tp_authority_public_key`.
233+
234+
The `coinbase_outputs` is used for fallback to solo mining. Generate an address
235+
of any type and then use the `getaddressinfo` RPC to find its public key.
236+
237+
Finally you most likely need to use the v1 to v2 translator: [stratum-mining/stratum/tree/main/roles/translator](https://github.com/stratum-mining/stratum/tree/main/roles/translator),
238+
even when you have a stratum v2 capable miner (see notes on ASIC's and Firmware below).
239+
240+
You need to point the translator to your job declator client, which in turn takes
241+
care of connecting to the pool. Try [tproxy-config-local-jdc-example.toml](https://github.com/stratum-mining/stratum/blob/main/roles/translator/tproxy-config-local-jdc-example.toml).
242+
243+
As soon as you turn on the translator, the Bitcoin Core log should show a `SetupConnection` [message](https://github.com/stratum-mining/sv2-spec/blob/main/08-Message-Types.md).
244+
245+
Now point your ASIC to the translator. At this point you should be seeing
246+
`NewTemplate`, `SetNewPrevHash` and `SetNewPrevHash` messages.
247+
248+
If the pool is down, notify someone on the above mentioned Discord.
249+
250+
### Custom Signet
251+
252+
Unlike testnet4, signet(s) use the regular difficulty adjustment mechanism.
253+
Although the default signet has very low difficulty, you can't mine on it,
254+
because to do so requires signing blocks using a private key that only two people have.
255+
256+
It's possible to create a signet that does not require signatures. There's no
257+
such public network, because it would risk being "attacked" by very powerful
258+
ASIC's. They could massively increase the difficulty and then disappear, making
259+
it impossible for a CPU miner to append new blocks.
260+
261+
Instead, you can create your own custom unsigned signet. Unlike regtest this
262+
network does have difficulty (adjustment). This allows you to test if e.g. pool
263+
software correctly sets and adjusts the share difficulty for each participant.
264+
Although for the Template Provider role this is not relevant.
265+
266+
#### Creating the signet
267+
268+
See also [signet/README.md](../contrib/signet/README.md)
269+
270+
If you use the default signet for anything else, create a fresh data directory.
271+
272+
Add the following to `bitcoin.conf`:
273+
274+
```ini
275+
[signet]
276+
# OP_TRUE
277+
signetchallenge=51
278+
```
279+
280+
This challenge represents "the special case where an empty solution is valid
281+
(i.e. scriptSig and scriptWitness are both empty)", see [BIP 325](https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki). For mining software things will look just like testnet.
282+
283+
The new chain needs to have at least 16 blocks, or the SRI software will panick.
284+
So we'll mine those using `bitcoin-util grind`:
285+
286+
```sh
287+
CLI="build/src/bitcoin-cli"
288+
MINER="contrib/signet/miner"
289+
GRIND="build/src/bitcoin-util grind"
290+
ADDR=...
291+
NBITS=1d00ffff
292+
$MINER --cli="$CLI" generate --grind-cmd="$GRIND" --address="$ADDR" --nbits=$NBITS
293+
```
294+
295+
#### Mining
296+
297+
The cleanest setup involves two connected nodes, each with their own data
298+
directory: one for the pool and one for the miner. By selectively breaking the
299+
connection you can inspect how unknown transactions in the template are requested
300+
by the pool, and how a newly found block is submitted is submitted both by the
301+
pool and the miner.
302+
303+
However things should work fine with just one node.
304+
305+
Start the miner node first, with a GUI for convenience:
306+
307+
```sh
308+
src/qt/bitcoin-qt -datadir=$HOME/.stratum/bitcoin -signet
309+
```
310+
311+
Suggested config for the pool node:
312+
313+
```ini
314+
[signet]
315+
# OP_TRUE
316+
signetchallenge=51
317+
server=0
318+
listen=0
319+
connect=127.0.0.1
320+
```
321+
322+
The above disables its RPC server and p2p listening to avoid a port conflict.
323+
324+
Start the pool node:
325+
326+
```sh
327+
build/src/bitcoind -datadir=$HOME/.stratum/bitcoin-pool -signet
328+
```
329+
330+
Configure an SRI pool:
331+
332+
```
333+
cd roles/pool
334+
mkdir -p ~/.stratum
335+
cp
336+
```
337+
338+
Start the SRI pool:
339+
340+
```sh
341+
cargo run -p pool_sv2 -- -c ~/.stratum/signet-pool.toml
342+
```
343+
344+
For the Job Declarator _client_ and Translator, see Testnet above.
345+
346+
Now use the [CPU miner](https://github.com/pooler/cpuminer) and point it to the translator:
347+
348+
```
349+
./minerd -a sha256d -o stratum+tcp://localhost:34255 -q -D -P
350+
```
351+
352+
353+
#### Mining after being away
354+
355+
The Template Provider will not start until the node is caught up.
356+
Use `bitcoin-util grind` as explained above for it to catch up.
357+
358+
### Mainnet
359+
360+
See testnet for how to use an external pool. See signet for how to configure your own pool.
361+
362+
Pools that support Stratum v2 on mainnet:
363+
364+
* Braiins: unclear if they are currently compatible with latest spec. URL's are
365+
listed [here](https://academy.braiins.com/en/braiins-pool/stratum-v2-manual/#servers-and-ports). There's no Job Declarator server.
366+
* DEMAND : No account needed for solo mining. Both the pool and Job Declarator
367+
server are at `dmnd.work:2000`. Requires a custom SRI branch, see [instructions](https://dmnd.work/#solo-mine).
368+
369+
### Notes on ASIC's and Firmware:
370+
371+
#### BraiinsOS
372+
373+
* v22.08.1 uses an (incompatible) older version of Stratum v2
374+
* v23.12 is untested (and not available on S9)
375+
* v22.08.1 when used in Stratum v1 mode, does not work with the SRI Translator
376+
377+
#### Antminer stock OS
378+
379+
This should work with the Translator, but has not been tested.

‎src/init.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -669,7 +669,7 @@ void SetupServerArgs(ArgsManager& argsman, bool can_listen_ipc)
669669
argsman.AddArg("-blockmintxfee=<amt>", strprintf("Set lowest fee rate (in %s/kvB) for transactions to be included in block creation. (default: %s)", CURRENCY_UNIT, FormatMoney(DEFAULT_BLOCK_MIN_TX_FEE)), ArgsManager::ALLOW_ANY, OptionsCategory::BLOCK_CREATION);
670670
argsman.AddArg("-blockversion=<n>", "Override block version to test forking scenarios", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::BLOCK_CREATION);
671671
#ifdef WITH_SV2
672-
argsman.AddArg("-sv2", "Bitcoind will act as a Stratum v2 Template Provider (default: false)", ArgsManager::ALLOW_ANY, OptionsCategory::BLOCK_CREATION);
672+
argsman.AddArg("-sv2", "Bitcoind will act as a Stratum v2 Template Provider, see doc/stratum-v2.md (default: false)", ArgsManager::ALLOW_ANY, OptionsCategory::BLOCK_CREATION);
673673
argsman.AddArg("-sv2interval", strprintf("Template Provider block template update interval (default: %d seconds)", Sv2TemplateProviderOptions().fee_check_interval.count()), ArgsManager::ALLOW_ANY, OptionsCategory::BLOCK_CREATION);
674674
argsman.AddArg("-sv2feedelta", strprintf("Minimum fee delta for Template Provider to send update upstream (default: %d sat)", uint64_t(Sv2TemplateProviderOptions().fee_delta)), ArgsManager::ALLOW_ANY, OptionsCategory::BLOCK_CREATION);
675675
#endif

0 commit comments

Comments
 (0)