Add leios-late-ib-inclusion to the Haskell simulator

[nfrisby]

Closes #410.

[nfrisby]

The Haskell simulator is configurable to use a single stage for voting (Vote) instead of two stages (VoteSend and VoteRecv).

The Short-to-Full spec only considers the two stage variant, when it specifies to include 3 iterations' IBs in each EB.

Would the Short-to-Full spec only include 2 iterations' IBs if Vote were a single stage? Should the Haskell simulator do that? @pagio @SupernaviX

[SupernaviX]

@nfrisby I don't think we care about supporting the behavior when leios-vote-send-recv-stages is false, we've been assuming the uniform leios variant for a long time.

[nfrisby]

FYI, I did some force pushes that.

• fixed a bug
• added tests (very similar to implementation, but that's largely because this first implementation is unoptimized)
• ran fourmolu locally

[nfrisby]

I'm now expecting CI to build green.

[ch1bo]

Looks good and I love that you added a test for it!

[nfrisby]

I converted this to a Draft PR because my study of the whole simulator has turned up at least three reasons this current diff is insufficient (see #413 (comment)).

[bwbush]

I don't have a global enough understanding of the implementation's inclusion logic to fully assess this, but changes seem to have the basics correct.

I'd very much like it if we could cite the specs as comments in the Haskell and Rust code. That direct link would make reviews much easier. Our use of google docs is not aligned very well with this style of commenting/citation.

This diff is now much bigger than it was when Sebastian approved it.

[nfrisby]

Here's a partial record of my manual validation of this feature.

Record which commit I'm using.

$ git rev-parse HEAD
801b110e806e4ca16b29e138f5b2e15df128f819

Generate the config-on.log and config-off.log files.

$ cabal run ols -- sim short-leios --topology-file simulation/docs/topology-\[5-cores\].yaml --leios-config-file my-config.yaml --output-file catch-on --output-seconds 300
$ vi my-config.yaml   # disable leios-late-ib-inclusion
$ cabal run ols -- sim short-leios --topology-file simulation/docs/topology-\[5-cores\].yaml --leios-config-file my-config.yaml --output-file catch-off --output-seconds 300

(my-config.yaml generates many fewer IBs per stage and has a reduced stage length compared to the default file. Nothing crucial---just to make my runs faster and easier to parse.)

Demonstrate the EBs' byte sizes are roughly three times bigger.

$ cat catch-off.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 2 -d']' | tr ',:' ' ' | awk '{print $1, $2}' | head
"size_bytes" 496
"size_bytes" 496
"size_bytes" 496
"size_bytes" 752
"size_bytes" 752
"size_bytes" 752
"size_bytes" 464
"size_bytes" 464
"size_bytes" 464
"size_bytes" 464
$ for i in off on; do echo $i; cat catch-$i.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 2 -d']' | tr ',:' ' ' | awk '{x = x + $2} END {print x}'; done
off
26000
on
56240

Demonstrate the number of IBs per EB is roughly three times bigger.

$ cat catch-off.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 1 -d']' | cut -c 50- | head -n5 
":"EB","id":"7-2","slot":50,"input_blocks":["16-0","49-0","58-1","58-2","63-1","67-0","68-1","77-0"
":"EB","id":"50-2","slot":50,"input_blocks":["16-0","49-0","58-1","58-2","63-1","67-0","68-1","77-0"
":"EB","id":"81-3","slot":50,"input_blocks":["16-0","49-0","58-1","58-2","63-1","67-0","68-1","77-0"
":"EB","id":"5-3","slot":60,"input_blocks":["11-0","13-0","17-0","26-0","27-0","29-1","41-0","49-1","51-0","54-0","66-0","68-2","68-3","88-0","90-0","93-0"
":"EB","id":"30-3","slot":60,"input_blocks":["11-0","13-0","17-0","26-0","27-0","29-1","41-0","49-1","51-0","54-0","66-0","68-2","68-3","88-0","90-0","93-0"
$ for i in off on; do echo $i; cat catch-$i.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 2 -d'[' | cut -f 1 -d']' | tr -d '0123456789",' | head; done
off
--------
--------
--------
----------------
----------------
----------------
-------
-------
-------
-------
on
-----------------------------
-----------------------------
-----------------------------
-----------------------------------
-----------------------------------
-----------------------------------
-------------------------------
-------------------------------
-------------------------------
------------------------------

I also manually checked the slots of the IBs of an arbitrary EB with the feature on and they spanned more than one stage length.

[nfrisby]

@bwbush suggested to double-check it works with Short-to-Full, since the default config file just uses short.

Both validations look right:

$ # used 'z' delimiter since initial EBs drop the endorser_blocks field since it's empty
$ for j in short full; do for i in off on; do echo $i-$j; cat catch-$i-$j.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 2 -dz | tr ',:' ' ' | awk '{x = x + $2} END {print x}'; done; done
off-short
26000
on-short
56240
off-full
33872
on-full
63536

for i in off on; do echo $i; cat catch-$i-full.log | grep -e '"tag":"generated"' | grep -e '"kind":"EB"' | cut -f 2 -d'[' | cut -f 1 -d']' | tr -d '0123456789",' | head; done
off
--------
--------
--------
----------------
----------------
----------------
-------
-------
-------
-------
on
-----------------------------
-----------------------------
-----------------------------
-----------------------------------
-----------------------------------
-----------------------------------
-------------------------------
-------------------------------
-------------------------------
------------------------------

[nfrisby]

We discussed the PR process in our working call today.

• I realized I haven't worked on a non-deployed codebase for a while :D
• So PRs in this repo don't need as much direct/immediate scrutiny as my instincts were telling me.
• Basic idea: do what I've already done here. Some tests. Some manual validation. Ask Brian/Yves/Simon for high-level input and give them a heads up. Then merge.
• Problems will (hopefully) become apparent as me/Brian/Yves do future work that directly or indirectly scrutinizes the merged PR.

My situation (urgent technical backfill on a prototype/research/brainstorming project) demanded some intensive onboarding. This created an opportunity for me to write down "stuff that wasn't immediately obvious".

I think such a document is valuable in its own right, but it also could become the basis of an architectural decision record (ADR). So in addition to the above list, Non-trivial PRs should also amend that ADR.

---

I already had drafted a summary of my onboarding, so I'll add that doc in a separate PR and then rebase this one on that doc, add an amendment, and then merge it.

[nfrisby]

I rebased so that the SimulatorModel.md is in-place and also amended it according to the changes in this PR.

I'm Merging now. Beyond the code review and my ad-hoc manual tests, the idea is that the Agda trace-verifier and/or confusing disagreements with the Rust simulator will eventually any catch bugs in this PR.