Write bigger chunks to ram_file #136
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`ram_file` is implemented as a port driver, and has quiet a significant overhead if every single series is emitted separately. What happens here is that bigger chunk is collected before writing to ram_file. Chunk is collected using binary append, in such a way that is can benefit from binary append optimization in ERTS. One nice side-effect of collecting binaries is that a lot of iolists in the hottest code path are no longer created, putting significantly less pressure on the garbage collector.
binarin
added a commit
to binarin/prometheus.erl
that referenced
this pull request
Oct 18, 2021
Builds on top of prometheus-erl#136 This is useful when e.g. escaping is not needed, as it is in case with RabbitMQ, where no escaping is needed because strict rules already apply to names. This change provides not only direct speedup, but also significantly reduced garbage collection overhead, as for every output line the proplist with labels and the list with converted `#'LabelPair'{}`'s are replaced with just a single binary. Using RabbitMQ as an example, time to produce ~630k metrics went from ~6 to ~3 seconds. Downside is that it's only compatible with text format, and spec for `#'LabelPair'{}' is not respected.
|
@binarin please look at the pipeleines - some minor code style things |
binarin
added a commit
to binarin/prometheus.erl
that referenced
this pull request
Nov 3, 2021
Builds on top of prometheus-erl#136 This is useful when e.g. escaping is not needed, as it is in case with RabbitMQ, where no escaping is needed because strict rules already apply to names. This change provides not only direct speedup, but also significantly reduced garbage collection overhead, as for every output line the proplist with labels and the list with converted `#'LabelPair'{}`'s are replaced with just a single binary. Using RabbitMQ as an example, time to produce ~630k metrics went from ~6 to ~3 seconds. Downside is that it's only compatible with text format, and spec for `#'LabelPair'{}' is not respected.
binarin
added a commit
to binarin/prometheus.erl
that referenced
this pull request
Nov 5, 2021
Builds on top of prometheus-erl#136 This is useful when e.g. escaping is not needed, as it is in case with RabbitMQ, where no escaping is needed because strict rules already apply to names. This change provides not only direct speedup, but also significantly reduced garbage collection overhead, as for every output line the proplist with labels and the list with converted `#'LabelPair'{}`'s are replaced with just a single binary. Using RabbitMQ as an example, time to produce ~630k metrics went from ~6 to ~3 seconds. Downside is that it's only compatible with text format, and spec for `#'LabelPair'{}' is not respected.
deadtrickster
pushed a commit
that referenced
this pull request
Nov 5, 2021
Builds on top of #136 This is useful when e.g. escaping is not needed, as it is in case with RabbitMQ, where no escaping is needed because strict rules already apply to names. This change provides not only direct speedup, but also significantly reduced garbage collection overhead, as for every output line the proplist with labels and the list with converted `#'LabelPair'{}`'s are replaced with just a single binary. Using RabbitMQ as an example, time to produce ~630k metrics went from ~6 to ~3 seconds. Downside is that it's only compatible with text format, and spec for `#'LabelPair'{}' is not respected. Co-authored-by: Alexey Lebedeff <[email protected]>
12 tasks
michaeldjeffrey
added a commit
to helium/packet-purchaser
that referenced
this pull request
Dec 7, 2021
current reporting takes about 500ms prometheus-erl/prometheus.erl#136 This might be able to speed us up quite a bit
michaeldjeffrey
added a commit
to helium/packet-purchaser
that referenced
this pull request
Dec 8, 2021
current reporting takes about 500ms prometheus-erl/prometheus.erl#136 This might be able to speed us up quite a bit
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ram_fileis implemented as a port driver, and has quiet a significant overhead if every single series is emitted separately.What happens here is that bigger chunk is collected before writing to ram_file. Chunk is collected using binary append, in such a way that is can benefit from binary append optimization in ERTS.
One nice side-effect of collecting binaries is that a lot of iolists in the hottest code path are no longer created, putting significantly
less pressure on the garbage collector.
Observed performance gains can be as high as 20 times (700k metrics emitted by RabbitMQ went from 2 minutes to 5 seconds). Such big difference can be mostly attributed to garbage collection overhead (for some reason it's non-linear in the number of metrics), but profiling shows that 20% improvement is there even without accounting for gc.