Skip to content

Load generator: add a new mode to pre-generate load offline #4671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 27, 2025

Conversation

marta-lokhova
Copy link
Contributor

@marta-lokhova marta-lokhova commented Mar 22, 2025

Resolves stellar/stellar-core-internal#345
Resolves https://github.com/stellar/stellar-core-internal/issues/339

Offline load generation has a few nice properties:

  • It eliminates account creation phase in our simulations completely. We can now pre-fill the DB by configuring GENESIS_TEST_ACCOUNT_COUNT. On my machine, creating 1,000,000 accounts takes ~20 seconds. This should speed up our iteration speed when working with simulations.
  • It eliminates the overhead from creating transactions in real-time, and allows for a more realistic performance measurement. Transactions are pre-generated into LOADGEN_PREGENERATED_TRANSACTIONS_FILE via command line.
  • It removes mempool race conditions that are making the simulation fail incorrectly (because we can create arbitrarily many accounts very quickly now, we can wait a long time before re-using the same account)

In addition, the new loadgen skips certain expensive validity checks, and adds pre-generated transactions directly into the tx queue, this further reduces the overhead of a load generation step.

@marta-lokhova marta-lokhova force-pushed the offlineLoadGeneration branch from 040d6de to 4b1a608 Compare March 22, 2025 01:26
@marta-lokhova marta-lokhova force-pushed the offlineLoadGeneration branch 2 times, most recently from c9bf22c to f02632e Compare March 26, 2025 00:07
@marta-lokhova marta-lokhova force-pushed the offlineLoadGeneration branch from f02632e to eea1358 Compare March 26, 2025 16:34
SirTyson
SirTyson previously approved these changes Mar 26, 2025
Copy link
Contributor

@SirTyson SirTyson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall a very helpful change! A few comments from me, mostly minor stuff.

SirTyson
SirTyson previously approved these changes Mar 27, 2025
Copy link
Contributor

@SirTyson SirTyson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marta-lokhova marta-lokhova added this pull request to the merge queue Mar 27, 2025
Merged via the queue into stellar:master with commit 8a4a5ba Mar 27, 2025
13 checks passed
@marta-lokhova marta-lokhova deleted the offlineLoadGeneration branch March 27, 2025 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants