Skip to content

Improve and mitigate warnings around dataloss when flushing #9846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 2, 2025

Conversation

grtlr
Copy link
Contributor

@grtlr grtlr commented Apr 30, 2025

Related

What

Important

This PR also changes the way RecordingStream is free'd in the C/C++ API. Before we called stream.disconnect, which unnecessarily replaced the current sink with a buffered sink that would be immediately dropped afterwards. Not only did this cause spam in the log outputs, it also lead to race conditions upon (log) application shutdown.

This PR makes it more explicit why we drop data during flushing, by bumping the log messages to warn!. It also improves the message by pointing the users to flush_timeout.

We also bump the default timeout from two seconds to now 3 seconds.

It's worth taking note that explicitly calling flush_blocking from our SDKs should be able to opt-out of this timeout, to ensure all data is sent. This will be tracked here:

@grtlr grtlr changed the title Grtlr/dataloss flush warnings Improve and mitigate warnings around dataloss when flushing Apr 30, 2025
@grtlr grtlr added include in changelog 🪵 Log & send APIs Affects the user-facing API for all languages labels Apr 30, 2025
Copy link

github-actions bot commented Apr 30, 2025

Web viewer built successfully. If applicable, you should also test it:

  • I have tested the web viewer
Result Commit Link Manifest
10aa0ac https://rerun.io/viewer/pr/9846 +nightly +main

Note: This comment is updated whenever you push a commit.

@grtlr grtlr marked this pull request as ready for review May 2, 2025 09:57
@grtlr grtlr added this to the 0.23.2 milestone May 2, 2025
Copy link
Member

@abey79 abey79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, thanks.

@grtlr grtlr merged commit b2e1fea into main May 2, 2025
36 checks passed
@grtlr grtlr deleted the grtlr/dataloss-flush-warnings branch May 2, 2025 10:47
abey79 pushed a commit that referenced this pull request May 2, 2025
### Related

* Closes #9818.

### What

> [!IMPORTANT]
> This PR also changes the way `RecordingStream` is free'd in the C/C++
API. Before we called `stream.disconnect`, which unnecessarily replaced
the current sink with a _buffered_ sink that would be immediately
dropped afterwards. Not only did this cause spam in the log outputs, it
also lead to race conditions upon (log) application shutdown.

This PR makes it more explicit why we drop data during flushing, by
bumping the log messages to `warn!`. It also improves the message by
pointing the users to `flush_timeout`.

We also bump the default timeout from two seconds to now 3 seconds.

It's worth taking note that explicitly calling `flush_blocking` from our
SDKs should be able to opt-out of this timeout, to ensure all data is
sent. This will be tracked here:

* #9845.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
include in changelog 🪵 Log & send APIs Affects the user-facing API for all languages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[C++ repro] Logging lots of image looses data
2 participants