Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find out why dataset height rolls back #149

Open
kalabukdima opened this issue Mar 19, 2025 · 1 comment
Open

Find out why dataset height rolls back #149

kalabukdima opened this issue Mar 19, 2025 · 1 comment

Comments

@kalabukdima
Copy link
Collaborator

Currently, the last dataset block known (from metadata written by scheduler) by the portal sometimes decreases.
It can't be the case that parquet chuks are removed from S3. We should find out whether it's a bug on the scheduler side or in the portal.
Image

@kalabukdima
Copy link
Collaborator Author

It gets written like that by the scheduler indeed. Probably because it restarts and loses state sometimes.
It will be fixed when we have a new scheduler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant