Skip to content

Commit 51412b3

Browse files
authored
Remove queue from the Prometheus remote write (#2951)
Prometheus remote write (PWR) exporter is supporting queued retry out of the box, but due to the strict requirements of PWR, the queue causes "out of order sample" errors. The queue won't be suitable for Prometheus unless it has a way to shard the data by timeseries. This is not going to break the existing users because if they have this feature enabled, they already can't use the PRW exporter correctly. Fixes #2949.
1 parent 0627fa8 commit 51412b3

File tree

2 files changed

+7
-3
lines changed

2 files changed

+7
-3
lines changed

exporter/prometheusremotewriteexporter/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -51,4 +51,4 @@ Several helper files are leveraged to provide additional capabilities automatica
5151
5252
- [HTTP settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/confighttp/README.md)
5353
- [TLS and mTLS settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)
54-
- [Queuing, retry and timeout settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md)
54+
- [Retry and timeout settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md), note that the exporter doesn't support `sending_queue`.

exporter/prometheusremotewriteexporter/factory.go

+6-2
Original file line numberDiff line numberDiff line change
@@ -55,12 +55,17 @@ func createMetricsExporter(_ context.Context, params component.ExporterCreatePar
5555
return nil, err
5656
}
5757

58+
// Don't support the queue.
59+
// See https://github.com/open-telemetry/opentelemetry-collector/issues/2949.
60+
// Prometheus remote write samples needs to be in chronological
61+
// order for each timeseries. If we shard the incoming metrics
62+
// without considering this limitation, we experience
63+
// "out of order samples" errors.
5864
prwexp, err := exporterhelper.NewMetricsExporter(
5965
cfg,
6066
params.Logger,
6167
prwe.PushMetrics,
6268
exporterhelper.WithTimeout(prwCfg.TimeoutSettings),
63-
exporterhelper.WithQueue(prwCfg.QueueSettings),
6469
exporterhelper.WithRetry(prwCfg.RetrySettings),
6570
exporterhelper.WithShutdown(prwe.Shutdown),
6671
)
@@ -75,7 +80,6 @@ func createDefaultConfig() config.Exporter {
7580
ExternalLabels: map[string]string{},
7681
TimeoutSettings: exporterhelper.DefaultTimeoutSettings(),
7782
RetrySettings: exporterhelper.DefaultRetrySettings(),
78-
QueueSettings: exporterhelper.DefaultQueueSettings(),
7983
HTTPClientSettings: confighttp.HTTPClientSettings{
8084
Endpoint: "http://some.url:9411/api/prom/push",
8185
// We almost read 0 bytes, so no need to tune ReadBufferSize.

0 commit comments

Comments
 (0)