Skip to content

feat(dag-builder): Add a tool to generate vertices from a DAG description #1181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 14, 2024

Conversation

msbrogli
Copy link
Member

@msbrogli msbrogli commented Nov 11, 2024

Motivation

Manually generating DAGs for tests is overly verbose and lacks readability. Testers need to focus on the higher-level test objectives without getting bogged down in transaction creation details. So this PR introduces a DSL to describe DAGs and a tool to generate a list of vertices that fulfill the DAG description.

Acceptance Criteria

  1. Add a cli.py that outputs a file format compatible with load_from_logs cli command.
  2. Implement a filler that gets an incomplete DAG and fill all gaps.
  3. Implement a tokenizer that gets the text and return tokens.
  4. Implement a vertex exporter that gets a complete DAG and generate the vertices.
  5. Implement a DAGBuilder class that integrates these components to generate the vertices from a DAG description.
  6. Add tests.unittest.TestCase.get_dag_builder() that creates a DAG builder from a HathorManager object.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@msbrogli msbrogli requested a review from jansegre as a code owner November 11, 2024 15:52
@msbrogli msbrogli self-assigned this Nov 11, 2024
@msbrogli msbrogli requested a review from glevco November 11, 2024 15:52
@msbrogli msbrogli mentioned this pull request Nov 11, 2024
1 task
Copy link

github-actions bot commented Nov 11, 2024

🐰 Bencher Report

Branchfeat/dag-builder
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Lower Boundary
nanoseconds (ns)
(Limit %)
Upper Boundary
nanoseconds (ns)
(Limit %)
sync-v2 (up to 20000 blocks)📈 view plot
🚷 view threshold
103,989,761,030.36
(+1.63%)
92,089,692,645.99
(88.56%)
112,554,068,789.54
(92.39%)
🐰 View full continuous benchmarking report in Bencher

@msbrogli msbrogli force-pushed the feat/dag-builder branch 2 times, most recently from 7b52412 to 5b0be7f Compare November 11, 2024 17:11
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

Attention: Patch coverage is 88.52201% with 73 lines in your changes missing coverage. Please review.

Project coverage is 84.89%. Comparing base (759c1d7) to head (3a38194).
Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
hathor/dag_builder/cli.py 7.69% 24 Missing ⚠️
hathor/dag_builder/tokenizer.py 75.00% 11 Missing and 8 partials ⚠️
hathor/dag_builder/default_filler.py 89.53% 7 Missing and 11 partials ⚠️
hathor/dag_builder/builder.py 92.85% 5 Missing and 4 partials ⚠️
hathor/dag_builder/vertex_exporter.py 98.35% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1181      +/-   ##
==========================================
+ Coverage   84.86%   84.89%   +0.03%     
==========================================
  Files         314      322       +8     
  Lines       24335    24970     +635     
  Branches     3708     3833     +125     
==========================================
+ Hits        20652    21199     +547     
- Misses       2974     3034      +60     
- Partials      709      737      +28     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@msbrogli msbrogli force-pushed the feat/dag-builder branch 5 times, most recently from 321c91c to d8ad4d5 Compare November 11, 2024 20:22
@msbrogli msbrogli force-pushed the feat/dag-builder branch 3 times, most recently from 3385aaf to df9d0e6 Compare November 11, 2024 20:45
@msbrogli msbrogli force-pushed the feat/dag-builder branch 5 times, most recently from c4b256d to c7512e8 Compare November 12, 2024 20:15
@msbrogli msbrogli changed the title feat(simulator): DAG builder feat(dag-builder): DAG builder Nov 13, 2024
@msbrogli msbrogli changed the title feat(dag-builder): DAG builder feat(dag-builder): Add a tool to generate vertices from a DAG description Nov 13, 2024
@msbrogli msbrogli force-pushed the feat/dag-builder branch 3 times, most recently from c90b105 to f24ba29 Compare November 13, 2024 18:13
@msbrogli msbrogli force-pushed the feat/dag-builder branch 4 times, most recently from 33fdc20 to f5b1de5 Compare November 13, 2024 18:34
glevco
glevco previously approved these changes Nov 13, 2024
jansegre
jansegre previously approved these changes Nov 14, 2024
@msbrogli msbrogli force-pushed the feat/dag-builder branch 3 times, most recently from f58e81f to fda29c6 Compare November 14, 2024 16:53
jansegre
jansegre previously approved these changes Nov 14, 2024
glevco
glevco previously approved these changes Nov 14, 2024
@msbrogli msbrogli dismissed stale reviews from glevco and jansegre via d76d190 November 14, 2024 19:07
@msbrogli msbrogli merged commit 3a38194 into master Nov 14, 2024
9 checks passed
@msbrogli msbrogli deleted the feat/dag-builder branch November 14, 2024 23:18
@jansegre jansegre mentioned this pull request Nov 21, 2024
2 tasks
@jansegre jansegre mentioned this pull request Dec 11, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants