[sui-custom-indexer] How to enable checkpoint stream for use with custom indexer on local node?

[vladilen11]

Description

I'm currently running a local Sui node and working on a custom indexer that needs to consume checkpoint stream data in real-time.

Question

So I'd like to ask:

How can I enable the checkpoint stream from the node?
Are there specific configuration options or flags I need to set when starting the node?

Are there any example projects or docs you could point me to for integrating with the checkpoint stream?

[github-actions]

Thank you for opening this issue, a team member will review it shortly. Until then, please do not interact with any users that claim to be from Sui support and do not click on any links!

[hyd628]

Hey @vladilen11 , are you using the Mysten customer indexer? https://docs.sui.io/guides/developer/advanced/custom-indexer

[vladilen11]

Yes, I’ve found a working setup for enabling checkpoint data output on a local node.

Here’s what worked for me:

sui start --with-faucet --data-ingestion-dir=./chk --force-regenesis

• This starts a local node with faucet enabled.

• The --data-ingestion-dir=./chk flag ensures that checkpoint data is continuously written to the ./chk directory.

• A custom indexer can then directly read and process checkpoint files from that directory.

However, there's one caveat:

The --force-regenesis flag does not automatically clear the previous contents of the --data-ingestion-dir.
To avoid potential data inconsistency, developers need to manually delete the contents of the ./chk directory before starting the node again with --force-regenesis.

Hope this helps others looking to use checkpoint data in their custom indexing pipelines.

Anyway thanks @hyd628 ❤️