Skip to content

Commit 72562bf

Browse files
WorksButNotTestedYour Name
authored and
rmalmain
committed
Librasan (#3023)
* Fixes to main * Add librasan * Party like it's 2024 * Fix snapshot module to work with guest asan * Fix guest_asan module * Fixes to runner * Fix linking issues using a REL * Fix qemu_launcher * Change modify_mapping to a method * Fix gasan_test * Remove debug from Justfile * Optimize release build of librasan * Set ulimit for qasan and gasan tests * Tidy up symbol renaming * Add missing symbols for PPC * Change to support rustix 1.0.0 * Canonicalize the CUSTOM_ASAN_PATH * Review changes * Restructure backends * release_max_level_info * More review changes * Clippy fixes * Changes to reduce the burden on the CI * Fix macos clippy --------- Co-authored-by: Your Name <[email protected]>
1 parent c265f3f commit 72562bf

File tree

249 files changed

+15876
-222
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

249 files changed

+15876
-222
lines changed

‎.github/workflows/build_and_test.yml

+39-2
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,10 @@ jobs:
6262
if: runner.os == 'MacOS'
6363
run: cd docs && mdbook test -L ../target/debug/deps $(python3-config --ldflags | cut -d ' ' -f1)
6464
- name: Run tests (Windows)
65-
if: runner.os == 'Windows'
65+
if: runner.os == 'Windows'
6666
run: cargo test -- --test-threads 1
6767
- name: Run tests (Linux)
68-
if: runner.os != 'Windows'
68+
if: runner.os != 'Windows'
6969
run: cargo test -- --test-threads 1
7070
- name: Test libafl no_std
7171
run: cd libafl && cargo test --no-default-features
@@ -404,6 +404,43 @@ jobs:
404404
shell: bash
405405
run: ARCH=${{ matrix.arch }} RUN_ON_CI=1 LLVM_CONFIG=llvm-config-${{env.MAIN_LLVM_VERSION}} ./scripts/test_fuzzer.sh ${{ matrix.fuzzer }}
406406

407+
librasan-build:
408+
runs-on: ubuntu-24.04
409+
needs:
410+
- changes
411+
if: ${{ needs.changes.outputs.qemu == 'true' }}
412+
steps:
413+
- uses: actions/checkout@v4
414+
- uses: ./.github/workflows/librasan-prepare
415+
- name: Build
416+
if: runner.os == 'Linux'
417+
shell: bash
418+
run: |
419+
RUN_ON_CI=1 \
420+
LLVM_CONFIG=llvm-config-${{env.MAIN_LLVM_VERSION}} \
421+
just \
422+
-f ./libafl_qemu/librasan/Justfile \
423+
build_everything_dev \
424+
build_x86_64_release
425+
426+
librasan-test:
427+
runs-on: ubuntu-24.04
428+
needs:
429+
- changes
430+
if: ${{ needs.changes.outputs.qemu == 'true' }}
431+
steps:
432+
- uses: actions/checkout@v4
433+
- uses: ./.github/workflows/librasan-prepare
434+
- name: Build
435+
if: runner.os == 'Linux'
436+
shell: bash
437+
run: |
438+
RUN_ON_CI=1 \
439+
LLVM_CONFIG=llvm-config-${{env.MAIN_LLVM_VERSION}} \
440+
just \
441+
-f ./libafl_qemu/librasan/Justfile \
442+
test_everything
443+
407444
fuzzers-qemu-system:
408445
needs:
409446
- changes

‎.github/workflows/librasan-prepare/action.yml

+73
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Setup QEMU librasan environment
2+
description: Sets up the QEMU librasan environment
3+
runs:
4+
using: composite
5+
steps:
6+
- name: Enable i386
7+
shell: bash
8+
run: sudo dpkg --add-architecture i386
9+
- name: Install QEMU deps
10+
shell: bash
11+
run: |
12+
sudo apt-get update && \
13+
DEBIAN_FRONTEND=noninteractive \
14+
sudo apt-get install -y \
15+
build-essential \
16+
clang-18 \
17+
clang++-18 \
18+
cmake \
19+
curl \
20+
g++-aarch64-linux-gnu \
21+
g++-arm-linux-gnueabi \
22+
g++-i686-linux-gnu \
23+
g++-mipsel-linux-gnu \
24+
g++-powerpc-linux-gnu \
25+
gcc-aarch64-linux-gnu \
26+
gcc-arm-linux-gnueabi \
27+
gcc-i686-linux-gnu \
28+
gcc-mipsel-linux-gnu \
29+
gcc-powerpc-linux-gnu \
30+
gdb \
31+
gdb-multiarch \
32+
git \
33+
gnupg \
34+
libc6-dev:i386 \
35+
libclang-dev \
36+
libgcc-13-dev:i386 \
37+
libglib2.0-dev \
38+
lsb-release \
39+
ninja-build \
40+
python3 \
41+
python3-pip \
42+
python3-venv \
43+
qemu-user \
44+
software-properties-common \
45+
wget
46+
- uses: dtolnay/rust-toolchain@nightly
47+
- name: install just
48+
uses: extractions/setup-just@v2
49+
with:
50+
just-version: 1.39.0
51+
- name: Install cargo-binstall
52+
shell: bash
53+
run: |
54+
curl -L --proto '=https' --tlsv1.2 -sSf \
55+
https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | \
56+
bash
57+
- name: Install nextest
58+
shell: bash
59+
run: |
60+
cargo binstall --no-confirm cargo-nextest
61+
- name: Install Rust Targets
62+
shell: bash
63+
run: |
64+
rustup target add armv7-unknown-linux-gnueabi && \
65+
rustup target add aarch64-unknown-linux-gnu && \
66+
rustup target add i686-unknown-linux-gnu && \
67+
rustup target add powerpc-unknown-linux-gnu
68+
- uses: actions/checkout@v4
69+
with:
70+
submodules: true
71+
fetch-depth: 0
72+
- uses: Swatinem/rust-cache@v2
73+
with: { shared-key: "${{ runner.os }}-shared-fuzzer-cache" }

‎.github/workflows/ubuntu-prepare/action.yml

+4
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ runs:
77
shell: bash
88
run: sudo apt-get update && sudo apt-get install -y curl lsb-release wget software-properties-common gnupg ninja-build shellcheck pax-utils nasm libsqlite3-dev libc6-dev libgtk-3-dev gcc g++ gcc-arm-none-eabi gcc-arm-linux-gnueabi g++-arm-linux-gnueabi libslirp-dev libz3-dev build-essential cmake
99
- uses: dtolnay/rust-toolchain@stable
10+
- name: install just
11+
uses: extractions/setup-just@v2
12+
with:
13+
just-version: 1.39.0
1014
- name: Add stable clippy
1115
shell: bash
1216
run: rustup toolchain install stable --component clippy --allow-downgrade

‎Cargo.toml

+1
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ cmake = "0.1.51"
9696
document-features = "0.2.10"
9797
fastbloom = { version = "0.9.0", default-features = false }
9898
hashbrown = { version = "0.14.5", default-features = false } # A faster hashmap, nostd compatible
99+
just = "1.40.0"
99100
libc = "0.2.159" # For (*nix) libc
100101
libipt = "0.3.0"
101102
log = "0.4.22"

‎fuzzers/binary_only/qemu_launcher/Cargo.toml

+4-1
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,10 @@ libafl = { path = "../../../libafl", features = ["tui_monitor"] }
4747
libafl_bolts = { path = "../../../libafl_bolts", features = [
4848
"errors_backtrace",
4949
] }
50-
libafl_qemu = { path = "../../../libafl_qemu", features = ["usermode"] }
50+
libafl_qemu = { path = "../../../libafl_qemu", features = [
51+
"usermode",
52+
"rasan",
53+
] }
5154
libafl_targets = { path = "../../../libafl_targets" }
5255
log = { version = "0.4.22", features = ["release_max_level_info"] }
5356
nix = { version = "0.29.0", features = ["fs"] }

‎fuzzers/binary_only/qemu_launcher/Justfile

+19-2
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,10 @@ harness: libpng
3131

3232
[unix]
3333
run: harness build
34+
#!/bin/bash
35+
36+
source {{ DOTENV }}
37+
CUSTOM_QASAN_PATH={{ BUILD_DIR }}/$CROSS_TARGET/{{ PROFILE_DIR }}/libqasan.so \
3438
{{ FUZZER }} \
3539
--input ./corpus \
3640
--output {{ TARGET_DIR }}/output/ \
@@ -56,7 +60,12 @@ test_inner: harness build
5660

5761
# complie again with simple mgr
5862
cargo build --profile={{PROFILE}} --features="simplemgr,{{ARCH}}" --target-dir={{ TARGET_DIR }} || exit 1
59-
./tests/qasan/test.sh || exit 1
63+
64+
export CUSTOM_QASAN_PATH={{ BUILD_DIR }}/$CROSS_TARGET/{{ PROFILE_DIR }}/libqasan.so
65+
./tests/qasan/qasan_test.sh || exit 1
66+
67+
export CUSTOM_GASAN_PATH={{ BUILD_DIR }}/$CROSS_TARGET/{{ PROFILE_DIR }}/libgasan.so
68+
./tests/qasan/gasan_test.sh || exit 1
6069

6170
[unix]
6271
test:
@@ -72,6 +81,10 @@ single: harness build
7281
{{ HARNESS }}
7382

7483
asan: harness build
84+
#!/bin/bash
85+
86+
source {{ DOTENV }}
87+
CUSTOM_QASAN_PATH={{ BUILD_DIR }}/$CROSS_TARGET/{{ PROFILE_DIR }}/libqasan.so \
7588
{{ FUZZER }} \
7689
--input ./corpus \
7790
--output {{ TARGET_DIR }}/output/ \
@@ -82,6 +95,10 @@ asan: harness build
8295
{{ HARNESS }}
8396

8497
asan_guest: harness build
98+
#!/bin/bash
99+
100+
source {{ DOTENV }}
101+
CUSTOM_GASAN_PATH={{ BUILD_DIR }}/$CROSS_TARGET/{{ PROFILE_DIR }}/libgasan.so \
85102
{{ FUZZER }} \
86103
--input ./corpus \
87104
--output {{ TARGET_DIR }}/output/ \
@@ -93,4 +110,4 @@ asan_guest: harness build
93110

94111
[unix]
95112
clean:
96-
cargo clean
113+
cargo clean

‎fuzzers/binary_only/qemu_launcher/src/client.rs

+14
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,16 @@ impl Client<'_> {
126126
unsafe { AsanModule::builder().env(&env).asan_report().build() }
127127
);
128128

129+
instance_builder.build().run(args, modules, state)
130+
} else if is_asan_guest {
131+
let modules = tuple_list!(
132+
DrCovModule::builder()
133+
.filename(drcov.clone())
134+
.full_trace(true)
135+
.build(),
136+
AsanGuestModule::default(&env),
137+
);
138+
129139
instance_builder.build().run(args, modules, state)
130140
} else {
131141
let modules = tuple_list!(DrCovModule::builder()
@@ -139,6 +149,10 @@ impl Client<'_> {
139149
let modules =
140150
tuple_list!(unsafe { AsanModule::builder().env(&env).asan_report().build() });
141151

152+
instance_builder.build().run(args, modules, state)
153+
} else if is_asan_guest {
154+
let modules = tuple_list!(AsanGuestModule::default(&env));
155+
142156
instance_builder.build().run(args, modules, state)
143157
} else {
144158
let modules = tuple_list!();

‎fuzzers/binary_only/qemu_launcher/src/fuzzer.rs

+2-2
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@ use clap::Parser;
88
#[cfg(feature = "simplemgr")]
99
use libafl::events::SimpleEventManager;
1010
#[cfg(not(feature = "simplemgr"))]
11-
use libafl::events::{EventConfig, Launcher, MonitorTypedEventManager};
11+
use libafl::events::{EventConfig, Launcher, LlmpEventManagerBuilder, MonitorTypedEventManager};
1212
use libafl::{
13-
events::{ClientDescription, LlmpEventManagerBuilder},
13+
events::ClientDescription,
1414
monitors::{tui::TuiMonitor, Monitor, MultiMonitor},
1515
Error,
1616
};

‎fuzzers/binary_only/qemu_launcher/src/instance.rs

+3-2
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,8 @@ use libafl_qemu::{
4444
cmplog::CmpLogObserver,
4545
edges::EdgeCoverageFullVariant,
4646
utils::filters::{HasAddressFilter, NopPageFilter, StdAddressFilter},
47-
EdgeCoverageModule, EmulatorModuleTuple, SnapshotModule, StdEdgeCoverageModule,
47+
AsanGuestModule, EdgeCoverageModule, EmulatorModuleTuple, SnapshotModule,
48+
StdEdgeCoverageModule,
4849
},
4950
Emulator, GuestAddr, Qemu, QemuExecutor,
5051
};
@@ -153,7 +154,7 @@ impl<M: Monitor> Instance<'_, M> {
153154
.map_observer(edges_observer.as_mut())
154155
.build()?;
155156

156-
let mut snapshot_module = SnapshotModule::new();
157+
let mut snapshot_module = SnapshotModule::with_filters(AsanGuestModule::snapshot_filters());
157158

158159
/*
159160
* Since the generics for the modules are already excessive when taking

‎fuzzers/binary_only/qemu_launcher/tests/qasan/.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/qasan

‎fuzzers/binary_only/qemu_launcher/tests/qasan/gasan_test.sh

+75
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
#!/bin/bash
2+
set -e
3+
4+
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
5+
6+
if [[ ! -x "$QEMU_LAUNCHER" ]]; then
7+
echo "env variable QEMU_LAUNCHER does not point to a valid executable"
8+
echo "QEMU_LAUNCHER should point to qemu_launcher"
9+
exit 1
10+
fi
11+
12+
cd "$SCRIPT_DIR"
13+
make
14+
15+
tests=(
16+
"overflow"
17+
"underflow"
18+
"double_free"
19+
"memset"
20+
"uaf"
21+
"test_limits"
22+
)
23+
24+
tests_expected=(
25+
"Segmentation fault"
26+
"Segmentation fault"
27+
"Panic!"
28+
"Panic!"
29+
"Segmentation fault"
30+
"Test-Limits - No Error"
31+
)
32+
33+
tests_not_expected=(
34+
"dummy"
35+
"dummy"
36+
"dummy"
37+
"dummy"
38+
"dummy"
39+
"Context:"
40+
)
41+
42+
# We don't want any core dumps. They can potentially be quite large
43+
ulimit -c 0
44+
45+
for i in "${!tests[@]}"
46+
do
47+
test="${tests[i]}"
48+
expected="${tests_expected[i]}"
49+
not_expected="${tests_not_expected[i]}"
50+
51+
echo "Running $test detection test..."
52+
OUT=$("$QEMU_LAUNCHER" \
53+
-r "inputs/$test.txt" \
54+
--input dummy \
55+
--output out \
56+
--asan-guest-cores 0 \
57+
--cores 0 \
58+
-- qasan 2>&1 | tr -d '\0')
59+
60+
echo "$OUT"
61+
62+
if ! echo "$OUT" | grep -q "$expected"; then
63+
echo "ERROR: Expected: $expected."
64+
echo "Output is:"
65+
echo "$OUT"
66+
exit 1
67+
elif echo "$OUT" | grep -q "$not_expected"; then
68+
echo "ERROR: Did not expect: $not_expected."
69+
echo "Output is:"
70+
echo "$OUT"
71+
exit 1
72+
else
73+
echo "OK."
74+
fi
75+
done

‎fuzzers/binary_only/qemu_launcher/tests/qasan/qasan.c

+1-7
Original file line numberDiff line numberDiff line change
@@ -71,13 +71,7 @@ int main(int argc, char **argv) {
7171

7272
char input = '\0';
7373

74-
// if (read(STDIN_FILENO, &input, 1) < 0) {
75-
76-
// LOG("Failed to read stdin\n");
77-
// return 1;
78-
79-
// }
80-
74+
if (argc > 1) { input = argv[1][0]; }
8175
LLVMFuzzerTestOneInput(&input, 1);
8276

8377
LOG("DONE\n");

‎fuzzers/binary_only/qemu_launcher/tests/qasan/test.sh renamed to ‎fuzzers/binary_only/qemu_launcher/tests/qasan/qasan_test.sh

+8-5
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ tests=(
2222
)
2323

2424
tests_expected=(
25-
"is 0 bytes to the right of the 10-byte chunk"
26-
"is 1 bytes to the left of the 10-byte chunk"
27-
"is 0 bytes inside the 10-byte chunk"
28-
"Invalid 11 bytes write at"
29-
"is 0 bytes inside the 10-byte chunk"
25+
"AddressSanitizer Error"
26+
"AddressSanitizer Error"
27+
"Panic!"
28+
"AddressSanitizer Error"
29+
"AddressSanitizer Error"
3030
"Test-Limits - No Error"
3131
)
3232

@@ -39,6 +39,9 @@ tests_not_expected=(
3939
"Context:"
4040
)
4141

42+
# We don't want any core dumps. They can potentially be quite large
43+
ulimit -c 0
44+
4245
for i in "${!tests[@]}"
4346
do
4447
test="${tests[i]}"

0 commit comments

Comments
 (0)