Merge with upstream/main at 0.15.0. Dependent on the fix of frida glib functions. Only frida_windows_gdiplus tested. Linux not tested

Author: mkravchik

.github/workflows/build_and_test.yml

@@ -276,6 +276,7 @@ jobs:
           - ./fuzzers/forkserver/baby_fuzzer_with_forkexecutor
 
           # Full-system
+          - ./fuzzers/full_system/nyx_launcher
           - ./fuzzers/full_system/nyx_libxml2_standalone
           - ./fuzzers/full_system/nyx_libxml2_parallel
 
@@ -343,7 +344,7 @@ jobs:
               - 'libafl_bolts/**'
               - 'libafl_targets/**'
               - 'libafl_qemu/**'
-              - 'fuzzers/*qemu*/**'
+              - 'fuzzers/**/*qemu*/**'
 
   fuzzers-qemu:
     needs:
@@ -360,8 +361,8 @@ jobs:
 
           # Full-system
           - ./fuzzers/full_system/qemu_baremetal
-          # - ./fuzzers/full_system/qemu_linux_kernel
-          #- ./fuzzers/full_system/qemu_linux_process
+          - ./fuzzers/full_system/qemu_linux_kernel
+          - ./fuzzers/full_system/qemu_linux_process
 
     runs-on: [ self-hosted, qemu ]
     container: registry.gitlab.com/qemu-project/qemu/qemu/ubuntu2204:latest

CONTRIBUTING.md

@@ -27,36 +27,55 @@ Before making your pull requests, try to see if your code follows these rules.
 - `PhantomData` should have the smallest set of types needed. Try not adding `PhantomData` to your struct unless it is really necessary. Also even when you really need `PhantomData`, try to keep the types `T` used in `PhantomData` as smallest as possible 
 - Wherever possible, trait implementations with lifetime specifiers should use '_ lifetime elision.
 - Complex constructors should be replaced with `typed_builder`, or write code in the builder pattern for yourself.
-- Remove generic restrictions at the definitions (e.g., we do not need to specify that types impl `Serialize`, `Deserialize`, or `Debug` anymore at the struct definitions). Therefore, try avoiding code like this unless the contraint is really necessary.
+- Remove generic restrictions at the definitions (e.g., we do not need to specify that types impl `Serialize`, `Deserialize`, or `Debug` anymore at the struct definitions). Therefore, try avoiding code like this unless the constraint is really necessary.
 ```rust
 pub struct X<A> 
     where
         A: P // <- Do not add contraints here
 {
     fn ...
 }
-
 ```
-- Reduce generics to the least restrictive necessary. __Never overspecify the contraints__. There's no automated tool to check the useless constraints, so you have to verify this manually.
+- Reduce generics to the least restrictive necessary. __Never overspecify the constraints__. There's no automated tool to check the useless constraints, so you have to verify this manually.
 ```rust
 pub struct X<A> 
     where
         A: P + Q // <- Try to use the as smallest set of constraints as possible. If the code still compiles after deleting Q, then remove it. 
 {
     fn ...
 }
+```
+
+- Prefer generic to associated types in traits definition as much as possible. They are much easier to use around, and avoid tricky caveats / type repetition in the code. It is also much easier to have unconstrained struct definitions.
 
+Try not to write this:
+```rust
+pub trait X
+{
+    type A;
+    
+    fn a(&self) -> Self::A;
+}
 ```
-- Traits which have an associated type should refer to the associated type, not the concrete/generic. In other words, you should only have the associated type when you can define a getter to it. For example, in the following code, you can define a associate type.
+Try to write this instead:
 ```rust
-pub trait X 
+pub trait X<A>
 {
-    type A; // <- You should(can) define it as long as you have a getter to it.
     fn a(&self) -> A;
 }
+```
 
+- Traits which have an associated type (if you have made sure you cannot use a generic instead) should refer to the associated type, not the concrete/generic. In other words, you should only have the associated type when you can define a getter to it. For example, in the following code, you can define a associate type.
+```rust
+pub trait X 
+{
+    type A; // <- You should(can) define it as long as you have a getter to it.
+    
+    fn a(&self) -> Self::A;
+}
 ```
-- __Ideally__ the types used in the the arguments of methods in traits should have the same as the types defined on the traits.
+
+- __Ideally__ the types used in the arguments of methods in traits should have the same as the types defined on the traits.
 ```rust
 pub trait X<A, B, C> // <- this trait have 3 generics, A, B, and C
 {
@@ -65,11 +84,54 @@ pub trait X<A, B, C> // <- this trait have 3 generics, A, B, and C
     fn do_other_stuff(&self, a: A, b: B); // <- this is not ideal because it does not have C.
 }
 ```
+- Generic naming should be consistent. Do NOT use multiple name for the same generic, it just makes things more confusing. Do:
+```rust
+pub struct X<A> {
+    phantom: PhanomData<A>,
+}
+
+impl<A> X<A> {}
+```
+But not:
+```rust
+pub struct X<A> {
+    phantom: PhanomData<A>,
+}
+
+impl<B> X<B> {} // <- Do NOT do that, use A instead of B
+```
 - Always alphabetically order the type generics. Therefore,
 ```rust
 pub struct X<E, EM, OT, S, Z> {}; // <- Generics are alphabetically ordered
 ```
 But not,
 ```rust
 pub struct X<S, OT, Z, EM, E> {}; // <- Generics are not ordered
-```
+```
+- Similarly, generic bounds in `where` clauses should be alphabetically sorted. Prefer:
+```rust
+pub trait FooA {}
+pub trait FooB {}
+
+pub struct X<A, B>;
+
+impl<A, B> X<A, B>
+where
+    A: FooA,
+    B: FooB,
+{}
+```
+Over:
+```rust
+pub trait FooA {}
+pub trait FooB {}
+
+pub struct X<A, B>;
+
+impl<A, B> X<A, B>
+where
+    B: FooB, // <-|
+             //   | Generic bounds are not alphabetically ordered.
+    A: FooA, // <-|
+{}
+```

Cargo.toml

@@ -49,35 +49,35 @@ exclude = [
 ]
 
 [workspace.package]
-version = "0.14.1"
+version = "0.15.0"
 license = "MIT OR Apache-2.0"
 
 [workspace.dependencies]
 # Internal deps
-libafl = { path = "./libafl", version = "0.14.1", default-features = false }
-libafl_bolts = { path = "./libafl_bolts", version = "0.14.1", default-features = false }
-libafl_cc = { path = "./libafl_cc", version = "0.14.1", default-features = false }
-symcc_runtime = { path = "./libafl_concolic/symcc_runtime", version = "0.14.1", default-features = false }
-symcc_libafl = { path = "./libafl_concolic/symcc_libafl", version = "0.14.1", default-features = false }
-libafl_derive = { path = "./libafl_derive", version = "0.14.1", default-features = false }
-libafl_frida = { path = "./libafl_frida", version = "0.14.1", default-features = false }
-libafl_intelpt = { path = "./libafl_intelpt", version = "0.14.1", default-features = false }
-libafl_libfuzzer = { path = "./libafl_libfuzzer", version = "0.14.1", default-features = false }
-libafl_nyx = { path = "./libafl_nyx", version = "0.14.1", default-features = false }
-libafl_targets = { path = "./libafl_targets", version = "0.14.1", default-features = false }
-libafl_tinyinst = { path = "./libafl_tinyinst", version = "0.14.1", default-features = false }
-libafl_qemu = { path = "./libafl_qemu", version = "0.14.1", default-features = false }
-libafl_qemu_build = { path = "./libafl_qemu/libafl_qemu_build", version = "0.14.1", default-features = false }
-libafl_qemu_sys = { path = "./libafl_qemu/libafl_qemu_sys", version = "0.14.1", default-features = false }
-libafl_sugar = { path = "./libafl_sugar", version = "0.14.1", default-features = false }
-dump_constraints = { path = "./libafl_concolic/test/dump_constraints", version = "0.14.1", default-features = false }
-runtime_test = { path = "./libafl_concolic/test/runtime_test", version = "0.14.1", default-features = false }
-build_and_test_fuzzers = { path = "./utils/build_and_test_fuzzers", version = "0.14.1", default-features = false }
-deexit = { path = "./utils/deexit", version = "0.14.1", default-features = false }
-drcov_utils = { path = "./utils/drcov_utils", version = "0.14.1", default-features = false }
-construct_automata = { path = "./utils/gramatron/construct_automata", version = "0.14.1", default-features = false }
-libafl_benches = { path = "./utils/libafl_benches", version = "0.14.1", default-features = false }
-libafl_jumper = { path = "./utils/libafl_jumper", version = "0.14.1", default-features = false }
+libafl = { path = "./libafl", version = "0.15.0", default-features = false }
+libafl_bolts = { path = "./libafl_bolts", version = "0.15.0", default-features = false }
+libafl_cc = { path = "./libafl_cc", version = "0.15.0", default-features = false }
+symcc_runtime = { path = "./libafl_concolic/symcc_runtime", version = "0.15.0", default-features = false }
+symcc_libafl = { path = "./libafl_concolic/symcc_libafl", version = "0.15.0", default-features = false }
+libafl_derive = { path = "./libafl_derive", version = "0.15.0", default-features = false }
+libafl_frida = { path = "./libafl_frida", version = "0.15.0", default-features = false }
+libafl_intelpt = { path = "./libafl_intelpt", version = "0.15.0", default-features = false }
+libafl_libfuzzer = { path = "./libafl_libfuzzer", version = "0.15.0", default-features = false }
+libafl_nyx = { path = "./libafl_nyx", version = "0.15.0", default-features = false }
+libafl_targets = { path = "./libafl_targets", version = "0.15.0", default-features = false }
+libafl_tinyinst = { path = "./libafl_tinyinst", version = "0.15.0", default-features = false }
+libafl_qemu = { path = "./libafl_qemu", version = "0.15.0", default-features = false }
+libafl_qemu_build = { path = "./libafl_qemu/libafl_qemu_build", version = "0.15.0", default-features = false }
+libafl_qemu_sys = { path = "./libafl_qemu/libafl_qemu_sys", version = "0.15.0", default-features = false }
+libafl_sugar = { path = "./libafl_sugar", version = "0.15.0", default-features = false }
+dump_constraints = { path = "./libafl_concolic/test/dump_constraints", version = "0.15.0", default-features = false }
+runtime_test = { path = "./libafl_concolic/test/runtime_test", version = "0.15.0", default-features = false }
+build_and_test_fuzzers = { path = "./utils/build_and_test_fuzzers", version = "0.15.0", default-features = false }
+deexit = { path = "./utils/deexit", version = "0.15.0", default-features = false }
+drcov_utils = { path = "./utils/drcov_utils", version = "0.15.0", default-features = false }
+construct_automata = { path = "./utils/gramatron/construct_automata", version = "0.15.0", default-features = false }
+libafl_benches = { path = "./utils/libafl_benches", version = "0.15.0", default-features = false }
+libafl_jumper = { path = "./utils/libafl_jumper", version = "0.15.0", default-features = false }
 
 # External deps
 ahash = { version = "0.8.11", default-features = false }     # The hash function already used in hashbrown
@@ -126,12 +126,15 @@ z3 = "0.12.1"
 
 
 [workspace.lints.rust]
+# Deny
+warnings = { level = "deny", priority = -1 }
+
 # Forbid
 unexpected_cfgs = "forbid"
 
 # Allow
 incomplete_features = "allow"
-ambiguous_glob_reexports = "allow"
+# ambiguous_glob_reexports = "allow"
 
 
 [workspace.lints.clippy]
@@ -142,9 +145,10 @@ cargo_common_metadata = "deny"
 
 # Warn
 cargo = { level = "warn", priority = -1 }
-negative_feature_names = "warn"
 
 # Allow
+negative_feature_names = "allow"    # TODO: turn into 'warn' when working
+multiple_crate_versions = "allow"   # TODO: turn into `warn` when working
 unreadable_literal = "allow"
 type_repetition_in_bounds = "allow"
 missing_errors_doc = "allow"

MIGRATION.md

@@ -10,8 +10,18 @@
 - `Qemu` cannot be used to initialize `Emulator` directly anymore. Instead, `Qemu` should be initialized through `Emulator` systematically if `Emulator` should be used.
   - Related: `EmulatorBuilder` uses a single function to provide a `Qemu` initializer: `EmulatorBuilder::qemu_parameters`. For now, it can be either a `Vec<String>` or a `QemuConfig` instance.
   - Related: Qemu's `AsanModule` does not need any special call to `Qemu` init methods anymore. It is now possible to simply initialize `AsanModule` (or `AsanGuestModule`) with a reference to the environment as parameter.
+  - `CustomBufHandlers` has been deleted. Please use `EventManagerHooksTuple` from now on.
+- Trait restrictions have been simplified
+  - The `UsesState` and `UsesInput` traits have been removed in favor of regular Generics.
+  - For the structs/traits that used to use `UsesState`, we bring back the generic for the state.
+  - `Input` is now only accessible through generic. `Input` associated types have been definitely removed.
+  - `HasCorpus` bound has been removed in many places it was unused before.
+  - `StdMutationalStage::transforming` must now explicitly state the Inputs types. As a result, `StdMutationalStage::transforming` must be written `StdMutationalStage::<_, _, FirstInputType, SecondInputType, _, _, _>::transforming`.
+  - The `State` trait is now private in favour of individual and more specific traits
+- Restrictions from certain schedulers and stages that required their inner observer to implement `MapObserver` have been lifted in favor of requiring `Hash`
+  - Related: removed `hash_simple` from `MapObserver`
 
-# 0.14.0 -> 0.14.1
+# 0.14.0 -> 0.15.0
 - Removed `with_observers` from `Executor` trait.
 - `MmapShMemProvider::new_shmem_persistent` has been removed in favour of `MmapShMem::persist`. You probably want to do something like this: `let shmem = MmapShMemProvider::new()?.new_shmem(size)?.persist()?;`
 

README.md

@@ -78,7 +78,13 @@ LibAFL is written and maintained by
  * [Addison Crump](https://github.com/addisoncrump) <[email protected]>
  * [Romain Malmain](https://github.com/rmalmain) <[email protected]>
 
-Please check out [CONTRIBUTING.md](CONTRIBUTING.md) for the contributing guideline.
+## Contributing
+
+Please check out **[CONTRIBUTING.md](CONTRIBUTING.md)** for the contributing guideline.
+
+## Debugging
+
+Your fuzzer doesn't work as expected? Try reading [DEBUGGING.md](./docs/src/DEBUGGING.md) to understand how to debug your problems.
 
 ## Cite
 If you use LibAFL for your academic work, please cite the following paper:

bindings/pylibafl/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "pylibafl"
 description = "Python bindings for LibAFL"
-version = "0.14.1"
+version = "0.15.0"
 license = "MIT OR Apache-2.0"
 repository = "https://github.com/AFLplusplus/LibAFL/"
 keywords = ["fuzzing", "testing", "security", "python"]
@@ -11,15 +11,15 @@ categories = ["development-tools::testing", "emulators", "embedded", "os"]
 [dependencies]
 pyo3 = { workspace = true, features = ["extension-module"] }
 pyo3-log = { version = "0.12.0" }
-libafl_sugar = { path = "../../libafl_sugar", version = "0.14.1", features = [
+libafl_sugar = { path = "../../libafl_sugar", version = "0.15.0", features = [
   "python",
 ] }
-libafl_bolts = { path = "../../libafl_bolts", version = "0.14.1", features = [
+libafl_bolts = { path = "../../libafl_bolts", version = "0.15.0", features = [
   "python",
 ] }
 
 [target.'cfg(target_os = "linux")'.dependencies]
-libafl_qemu = { path = "../../libafl_qemu", version = "0.14.1", features = [
+libafl_qemu = { path = "../../libafl_qemu", version = "0.15.0", features = [
   "python",
 ] }
 
@@ -30,5 +30,7 @@ pyo3-build-config = { workspace = true }
 name = "pylibafl"
 crate-type = ["cdylib"]
 
-[profile.dev]
-panic = "abort"
+# TODO: find a way to fix this when a solution is found
+# https://github.com/rust-lang/cargo/issues/9330
+# [profile.dev]
+# panic = "abort"

bindings/pylibafl/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "PyLibAFL"
-version = "0.14.1"
+version = "0.15.0"
 description = "Advanced Fuzzing Library for Python"
 readme = "README.md"
 requires-python = ">=3.8"

docs/listings/baby_fuzzer/listing-01/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_01"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/listings/baby_fuzzer/listing-02/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_02"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/listings/baby_fuzzer/listing-03/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_03"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/listings/baby_fuzzer/listing-04/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_04"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/listings/baby_fuzzer/listing-05/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_05"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/listings/baby_fuzzer/listing-06/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "baby_fuzzer_listing_06"
-version = "0.14.1"
+version = "0.15.0"
 authors = ["Your Name <[email protected]>"]
 edition = "2018"
 

docs/src/DEBUGGING.md

@@ -25,8 +25,11 @@ You should *never* use the `EDGES_MAP`'s size as this is just the size of the al
 ## Q. I still have problems with my fuzzer.
 Finally, if you really have no idea what is going on, run your fuzzer with logging enabled. (You can use `env_logger`, `SimpleStdoutLogger`, `SimpleStderrLogger` from `libafl_bolts`. `fuzzbench_text` has an example to show how to use it.) (Don't forget to enable stdout and stderr), and you can open an issue or ask us in Discord.
 
-## Q. My fuzzer died of ``Storing state in crashed fuzzer instance did not work''.
+## Q. My fuzzer died of `Storing state in crashed fuzzer instance did not work`.
 If the exit code is zero, then this is because either your harness exited or you are using fuzzer_loop_for and forgot to add `mgr.on_restart` at the end of the fuzzer. In the first case, you should patch your harness not to exit. (or use `utils/deexit`).
 
 ## Q. I can't leave the TUI screen
-Type `q` then you leave TUI.
+Type `q` then you leave TUI.
+
+## Q. I see `QEMU internal SIGSEGV {code=MAPERR, addr=0x48}` and my QEMU fuzzer doesn't run.
+Are you running QEMU fuzzer on WSL? You have to enable vsyscall https://github.com/microsoft/WSL/issues/4694#issuecomment-556095344.

docs/src/message_passing/message_passing.md

@@ -72,7 +72,7 @@ So the outgoing messages flow is like this over the outgoing broadcast `Shmem`:
 [client0]        [client1]    ...    [clientN]
 ```
 
-To use `LLMP` in LibAFL, you usually want to use an `LlmpEventManager` or its restarting variant.
+To use `LLMP` in LibAFL, you usually want to use an `LlmpRestartingEventManager` or its restarting variant.
 They are the default if using LibAFL's `Launcher`.
 
 If you should want to use `LLMP` in its raw form, without any `LibAFL` abstractions, take a look at the `llmp_test` example in [./libafl/examples](https://github.com/AFLplusplus/LibAFL/blob/main/libafl_bolts/examples/llmp_test/main.rs).