Add BTF relocation support in eBPF programs

[v-thakkar]

To be able to support BPF CO-RE, rustc needs to be able to generate the relocations of BTF types across different kernel versions. First part of this work requires adding core::intrinsics::{preserve_access_index, preserve_field_info, preserve_type_info, preserve_enum_value} for the BPF target in rustc. And then making sure/testing that the relocations work well with the userspace part of Aya.

[alessandrod]

Just to clarify: aya supports BTF relocations, see: https://github.com/aya-rs/aya/blob/main/aya/src/obj/btf/relocation.rs. So if you load an ebpf object file with BTF relocations, aya will be able to resolve them.

What doesn't support emitting BTF relocations yet is the rust compiler. So if you compile a rust program for the bpf*-unknown-none targets it will not emit BTF relocations.

[GermanCoding]

What would be needed to get the preserve_access_index, preserve_field_info, preserve_type_info, preserve_enum_value builtins into rustc? I guess some issue/PR over on https://github.com/rust-lang/rust? I would really like to see this feature in Rust, so as to have a really complete CORE solution pretty much equivalent to what you can do in C today. However I don't feel qualified enough to open an issue on the Rust project myself, so maybe someone here has more expertise with this?

[l2dy]

First part of this work requires adding core::intrinsics::{preserve_access_index, preserve_field_info, preserve_type_info, preserve_enum_value} for the BPF target in rustc.

Is it possible to experiment with the new intrinsics with the link_llvm_intrinsics feature in unstable?

What would be needed to get the preserve_access_index, preserve_field_info, preserve_type_info, preserve_enum_value builtins into rustc? I guess some issue/PR over on https://github.com/rust-lang/rust?

Rust language feature requests should be discussed on the internals forum first, and then follow the RFC process if asked to. @GermanCoding If you are really interested in this feature, feel free to ask on the forum.

[v-thakkar]

@l2dy @GermanCoding I think @vadorovsky had some discussions regarding this with the Rust community and there has been some WIP regarding the same. Maybe he can shed more light onto it.

[vadorovsky]

I'm currently trying to achieve that without any changes in Rust compiler - I updated the issue title to generalize it.

Instead, I'm trying to emit these intrinsic calls in https://github.com/aya-rs/bpf-linker, which as a bitcode linker, is alreaady capable of modifying LLVM IR before producing the actual BPF binary.

Clang is producing the @llvm.preserve.array.access.index, @llvm.preserve.struct.access.index and @llvm.preserve.array.access.index intrinsics by performing regular calls of LLVM API, the code is here:

https://github.com/llvm/llvm-project/blob/cb6a62369a353f506a1dde087eeaf5ebea5d5c26/llvm/lib/IR/IRBuilder.cpp#L1260-L1285

The idea is to do the same in bpf-linker whenever we find a getelementptr instruction followed by load which is done on a type, for which we would like to emit BTF relocations.

Clang has two ways of enabling BTF relocations:

• For the whole type, by annotating it with __attribute__((preserve_access_index)), e.g.

struct foo {
  int a;
  int b;
  int c;
  int d;
} __attribute__((preserve_access_index));

int get_a(struct foo *foo) {
  return foo->a;  // <-- this already does BTF relocations
}

• For a single field, if it's being accessed with __builtin_preserve_access_index, e.g.

struct foo {
  int a;
  int b;
  int c;
  int d;
}

int get_a(struct foo *foo) {
  return __builtin_preserve_access_index(foo->a); // <- this does BTF relocation, but the
                                                  // intrinsics are emitted just for field `a`
}

How we could achieve equivalent way of notifying bpf-linker about necessity of emitting the intrinsic for the given type or field? We can use a custom PhantomData field which won't affect the layout of the type.

• For the whole type, we could look for _btf_marker: PhantomData<()>, e.g.

#[repr(C)]
pub struct Foo {
    a: i32,
    b: i32,
    c: i32,
    d: i32,
    _btf_marker: PhantomData<()>,
}

• For a single field, we could look for _btf_marker_FIELD_NAME: PhantomData<()>, e.g.

#[repr(C)]
pub struct Foo {
    a: i32,
    b: i32,
    c: i32,
    d: i32,
    _btf_marker_a: PhantomData<()>,
}

Of course the code examples above are ugly and we don't want users to write such code. We could hide them with macros and make the final code look like:

// This would emit BTF relocs for the whole type
#[repr(C)]
#[derive(Btf)]
pub struct 
    a: i32,
    b: i32,
    c: i32,
    d: i32,
}

// This would emit BTF relocs only for annotated fields
#[repr(C)]
#[derive(Btf)]
pub struct 
    a: i32,
    b: i32,
    c: i32,
    #[btf]
    d: i32,
}

---

To be more explicit, the plan going forward is:

• Necessary change in LLVM-C API which allows to iterate over dbg_records of IR instructions (that's the only reliable way how bpf-linker can figure out that the given getelementptr instruction refers to a certain debug info node)
  • [LLVM-C] Add bindings to Instruction::getDbgRecordRange() llvm/llvm-project#107802
• Preparatory, necessary refactors in bpf-linker (otherwise the code for handling BTF would be horrible)
  • chore: Remove item_stack from DISanitizer bpf-linker#222
  • 🔒🌯 Provide LLVMTypeWrapper trait, add more safe wrappers bpf-linker#223
  • Move diagnostic handler to a separate struct bpf-linker#224
  • Add more safe wrappers bpf-linker#225
• structs <-- I'm working on this step
  • find getelementptr instructions followed by load
  • check if it's a struct
  • check whether the struct, or one of its fields, is annotated with a custom marker _btf_marker: PhantomData<()> or `_btf_marker_FIELD_NAME: PhantomData<()>
  • if found, emit an @llvm.preserve.struct.access.index intrinsic call
• enums
  • find getelementptr instructions followed by load
  • check if it's an enum
  • check whether the enum, or one of its fields, has a variant with a custom marker BtfMarker
    • this ugly marker can be hidden by a macro, details below
• unions
• Provide macros in Aya
  • #[derive(Btf)] macro, which either:
    • adds _btf_marker: PhantomData<()> for the whole type
    • adds _btf_marker_FIELD_NAME: PhantomData<()>
  • #[btf] macro for enums, which adds
  • bpf_core_read! macro BtfMarker

[canoriz]

I'm new to aya. Does this mean an eBPF ELF object compiled from rust may not be CO-RE (because lack of index in ELF), and we are working on it now?

[vadorovsky]

I'm new to aya. Does this mean an eBPF ELF object compiled from rust may not be CO-RE (because lack of index in ELF), and we are working on it now?

Sorry for late reply. Yes, that exactly the case.

[fhilgers]

@vadorovsky Do you mind sharing what's the state on this one? Is there a POC for an implementation? I have seen your branch (https://github.com/vadorovsky/bpf-linker/tree/preserve-access-index-v2), however there has been no updates in a while. Did you hit some roadblock that makes this harder than previously thought.

I'm currently writing all my CO:RE stuff in C, compiling it to llvm bitcode and linking it together with bpf-linker, however thats very cumbersome to maintain. Before I hop into writing proc macros to make writing ebpf programs in a mix of rust and C (for relocations) more convenient, my time might be better spent helping to get this through.

Let me know if there are any updates and how/where one can help, thanks!

[hepek]

I'm currently trying to achieve that without any changes in Rust compiler - I updated the issue title to generalize it.

Instead, I'm trying to emit these intrinsic calls in https://github.com/aya-rs/bpf-linker, which as a bitcode linker, is alreaady capable of modifying LLVM IR before producing the actual BPF binary.

@vadorovsky, any updates on this approach?

I like the macro idea. What do you think about putting all logic in user code? No changes to linkers or LLVM hacking.
Users would declare structs and offsets they use with a macro like this:

btf_use!(sock.__sock_common.skc_family);

Which would declare a static variable:

#[no_mangle]
static mut BTF_OFF_SOCK__SOCK_COMMON_SKC_FAMILY: u32 = 0;

in their ebpf program.

ebpf probe can then use this offset when accessing kernel code via bpf_probe_read_kernel - this can also be wrapped in a macro for easier understanding, something like:

bpf_read_kernel!(&sock, "sock.__sock_common.skc_family").

In the user program we parse the ebpf program ELF binary and find all globals that match our naming scheme. Then read the BTF from "/sys/kernel/btf/vmlinux", get struct offsets and other info and populate the memory pointed to by the globals before we even load the binary. Then load the modified ELF binary with aya::Ebpf::load.

I know it sounds hacky, but it could generate CO-RE ebpf probe that is also free of any generated bindings.

Is this worth trying?

[vadorovsky]

Sorry for slow progress here. I was mostly preoccupied with other things, including BTF maps support (#1117) and statically linking libLLVM to bpf-linker. I think the approach I described is still going to work and I didn't hit anything which would prove me otherwise, it's mostly time issue on my side. 😢

@hepek @fhilgers About the macro idea, I think it could work, although not exactly in the form you described.

btf_use!(sock.__sock_common.skc_family); wouldn't work, because I don't think there is a way you could declare a global variable if you call such macro from inside a function.

However, I think that some kind of combination of:

• derive macro to annotate the struct
• macro for accessing the struct member (we can keep calling it btf_use!)

would work.

Regarding the first derive macro, let's call the macro #[btf]. The code could look like:

#[derive(Btf)]
pub struct kuid_t {
    val: u32,
}

#[derive(Btf)]
pub struct kgid_t {
    val: gid_t,
}

#[derive(Btf)]
pub struct cred {
    suid: kuid_t,
    sgid: kgid_t,
}

#[derive(Btf)]
pub struct task_struct {
    pid: i32,
    tgid: u32,
    cred: *const cred,
}

The macro would iterate over the fields and define globals for each of them:

pub const BTF_OFF__KUID_T__VAL: usize = 0;

pub const BTF_OFF__KGID_T__VAL: usize = 0;

pub const BTF_OFF__CRED__SUID: usize = 0;
pub const BTF_OFF__CRED__SGID: usize = 0;

pub const BTF_OFF__TASK_STRUCT__PID: usize = 0;
pub const BTF_OFF__TASK_STRUCT__TGID: usize = 0;

Then btf_read_kernel! could use these variables to figure out the correct offset.

I think that's worth trying, so if you were thinking to do that go ahead. Thanks for coming up with the idea!

[hepek]

@vadorovsky, definitely. I already have some code that validates the basic idea, however none of the procedural macro niceties is there yet. Will let you know when I come up with something that's usable.

[hepek]

@vadorovsky I have a proof of concept of my idea. You can take a look at an example application using this paradigm in this branch here.

It goes something like this:

In my ebpf program I declare what variables I want to read from the kernel:

bpf_use!(socket.sk);
bpf_use!(sock.sk_protocol);
bpf_use!(sock.__sk_common.__ANON1.__ANON1.skc_rcv_saddr);

This expands to three static variables, one for offset, one for size and one containing the string of the path that defines the variable. You can see me here using socket.sk to get the sock pointer, then reading into sock struct by using a special syntax for anonymous unions (similar to what bindgen spits out).

The tokens are concatenated using crate called paste.
Then in my probes lower in the code I read the values like so:

let skc_rcv_saddr = bpf_read_kernel!(sock_ptr, sock.__sk_common.__ANON1.__ANON1.skc_rcv_saddr, i32)?;

This call expands into a call to aya_ebpf::helpers::gen::bpf_probe_read_kernel internally. Both of those macros are currently in the same file. They'd need to be put somewhere else if this was to become a library.

In userland, we modify the ELF binary before we load it.

   let ebpf_prog = aya::include_bytes_aligned!(concat!(
        env!("OUT_DIR"),
        "/my-prog"
    ));

    let prog = btf_modify_elf_program(&ebpf_prog).expect("failed modifying ELF");
    let mut ebpf = aya::Ebpf::load(&prog)?;

For this I used two crates btf and elf.
We iterate through all symbols in the binary and for those that start with btf_ we perform the following:

1. Look up the string containing the path (btf_<expanded_path>_path), we use this string to read offset and size from the BTF library (there's a function that looks this information up using this syntax.

Then we update the size and offset information in the binary.

What I don't like about this implementation is:

1. Not using the btf parser that comes with aya
2. We have to mention the path of interest twice in the ebpf code, once with btf_use and then again with bpf_read_kernel. Forgetting to declare with btf_use generates an ugly error that might not be obvious to the untrained eye
3. Only works with kernels with BTF info enabled
4. There is platform dependent logic in btf_modify_program where strings are found in .rodata and .rodata.cs16 segments, depending on their length, which is probably not portable across different platforms and compilers.

(edit: sorry for spamming, pressec Comment accidentally the first time).