This is a general-purpose tracking issue for several areas of improvement in Aya's documentation. I'm planning on spending some cycles on some of these over the next couple of weeks. If anyone has any comments/suggestions on something to add or change here, do let me know.

Short-term goals (to help new contributors):

• Add an ARCHITECTURE.md file detailing the architecture of the project
• Add a description of steps needed to 1) implement a new program type; and 2) implement a new map type to CONTRIBUTING.md

Long-term goals (to help library users):

• Document every public-facing part of the userspace API
• Document every public-facing part of aya-bpf
• It probably also would make sense to #![deny(missing_docs)] at that point

It would be helpful to be able to test the try_xdp_stats function of my eBPF program.
This would significantly reduce the dev/test cycle time as a simple cargo test could run tests using the host arch.

For example:

#![cfg_attr(not(test), no_std, no_main)]

use aya_bpf::bindings::xdp_action;
use aya_bpf::cty::c_long;
use aya_bpf::macros::{map, xdp};
use aya_bpf::maps::Array;
use aya_bpf::programs::XdpContext;

use types::XdpData;

#[cfg(not(test))]
#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    unreachable!()
}

#[map]
static mut xdp_stats_map : Array<XdpData> = Array::with_max_entries(1,0);

#[xdp(name = "xdp_stats")]
pub fn xdp_stats(ctx: XdpContext) -> u32 {
    match unsafe { try_xdp_stats(ctx) } {
        Ok(ret) => ret,
        Err(_) => xdp_action::XDP_DROP,
    }
}

unsafe fn try_xdp_stats(_ctx: XdpContext) -> Result<u32, c_long> {
    let data = match xdp_stats_map.get(0) {
        Some(data) => data,
        None => return Err(0)
    };
    let xdp_data = XdpData{
        packet_count: data.packet_count + 1,
    };
    xdp_stats_map.set(0, &xdp_data, 0)?;
    Ok(xdp_action::XDP_PASS)
}

#[cfg(test)]
mod tests {
    use super::*;
    use aya_bpf::programs::XdpContext;

    #[test]
    fn test() {
        let res = unsafe { try_xdp_stats(XdpContext::empty()) };
        match res {
            Ok(action) => assert_eq!(xdp_action::XDP_PASS, action),
            Err(e) => panic!("error: {}", e),
        }
    }
}

There are a couple of things that would be necessary in the context of a test to make this possible:

1. The ability to construct XdpContext (and other contexts) using test data (e.g XdpContext::empty() or XdpContext::from_data(data, metadata, data_len ...)
2. To replace the map implementation with a std::collections::HashMap (or similar compatible structure)
3. To replace helper implementations with ones that don't rely on bpf-helper.h functions

I think this would all be relatively simple, as we should be able using conditional compilation:
#[cfg(any(target_arch = "bpfel", target_arch = "bpfeb")))] or the not() equivalent.

Another potential option would be to create some test helpers that run the program inside the eBPF VM, for example: Load using BPF_PROG_TEST_RUN, provide data, assert on program output. The only downsides to this are that tests would require root, that asserting the Result of the try_* function would probably be more flexible than asserting against the output of the program.

To get around the root issue, I suppose it might be possible to run tests in uBPF or rBPF, but I'm not sure that these VMs would have all the necessary features required.

Hi, I think that a program.load operation is missing before this attach, in fact this example will trigger a NotLoaded error.

Hi, first of all thanks so much for this library, it was so needed.

I've been trying to use it to load a eBPF program which relies on tracepoints. The program has been compiled using clang with:

clang \
	-D__KERNEL__ \
	-D__BPF_TRACING__ \
	-D__TARGET_ARCH_$(linux_arch) \
	-O2 -target bpf \
	-Wno-address-of-packed-member \
	-Wno-compare-distinct-pointer-types \
	-Wno-deprecated-declarations \
	-Wno-gnu-variable-sized-type-not-at-end \
	-Wno-pointer-sign \
	-Wno-pragma-once-outside-header \
	-Wno-unknown-warning-option \
	-Wno-unused-value \
	-Wunused \
	-Wall \
	-fno-stack-protector \
	-fno-jump-tables \
	-fno-unwind-tables \
	-fno-asynchronous-unwind-tables \
	-xc \
	-I . \
	-I /usr/include/${ARCH}-linux-gnu/ \
	-I ${LINUX_HEADERS_ROOT}/include \
	-I ${LINUX_HEADERS_ROOT}/include/uapi \
	-I ${LINUX_HEADERS_ROOT}/include/generated \
	-I ${LINUX_HEADERS_ROOT}/include/generated/uapi \
	-I ${LINUX_HEADERS_ROOT}/tools/testing/selftests \
	-I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include \
	-I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include/generated \
	-I ${LINUX_HEADERS_ROOT}/arch/${linux_arch}/include/generated/uapi \
	-c $< -o $@

While the object file works perfectly with other libraries (goebpf), I was unable to load it with Aya, specifically it fails to find any programs with the specified name (or any programs at all, I tried to enumerate the programs attribute and it's empty).

So I assume there's a specific way to compile the object OR to name functions according to what Aya expects? I'd appreciate if you guys could shed some light on this.

Thanks.

Today I cargo update my project and suddenly I wasn't able to load my probes anymore, neither on x86_64 or ARMv7, in both cases I get a error parsing BPF object. It is caused by this line introduced by d8d3117 and specifically the data.len() > sizeof(bpf_map_def) since in a lot of cases it'll result in something like 280 > 28, which is always true.

If I use aya from a commit before that one, everything works flawlessly.

An important note: on both platforms, all tests run correctly, meaning that loading an eBPF ELF is not really tested.

Hi again, currently aya::generated only supports x86_64 and aarch64.

If I try to compile with cargo build --target=armv7-unknown-linux-gnueabihf for instance (Raspberry Pi 4 target), the build fails with:

error[E0432]: unresolved imports `crate::generated::bpf_map_type`, `crate::generated::AYA_PERF_EVENT_IOC_DISABLE`, `crate::generated::AYA_PERF_EVENT_IOC_ENABLE`, `crate::generated::AYA_PERF_EVENT_IOC_SET_BPF`
  --> /home/evilsocket/.cargo/git/checkouts/aya-92d3024d6eb5b473/c240a2c/aya/src/bpf.rs:13:9
   |
13 |         bpf_map_type::BPF_MAP_TYPE_PERF_EVENT_ARRAY, AYA_PERF_EVENT_IOC_DISABLE,
   |         ^^^^^^^^^^^^                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `AYA_PERF_EVENT_IOC_DISABLE` in `generated`
   |         |
   |         could not find `bpf_map_type` in `generated`
14 |         AYA_PERF_EVENT_IOC_ENABLE, AYA_PERF_EVENT_IOC_SET_BPF,
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `AYA_PERF_EVENT_IOC_SET_BPF` in `generated`
   |         |
   |         no `AYA_PERF_EVENT_IOC_ENABLE` in `generated`

error[E0432]: unresolved import `crate::generated::bpf_map_type`
  --> /home/evilsocket/.cargo/git/checkouts/aya-92d3024d6eb5b473/c240a2c/aya/src/maps/mod.rs:42:5
   |
42 |     generated::bpf_map_type,
   |     ^^^^^^^^^^^^^^^^^^^^^^^ no `bpf_map_type` in `generated`

....
....
a lot more
....
....

Any chance you can either add support for armv7, or alternatively guide me to generate those files and send a PR?

If it is as simple as just running rust-bindgen (on what?), maybe it's also worth adding support for other tier 1 and tier 2 targets?

Thanks.

I was thinking about trying to implement a generic logging solution for eBPF programs.

My potentially crazy solution relies on 2 maps:

1. Hashmap for configuration (e.g log-level)
2. Perf Event Array for logs

The way this might work is as follows:

1. Implement a bpf_logger for the log crate. The log level comes from the config map. Messages higher than the log level are discarded while other messages are written to the perf event array. Alternatively, add a similar crate here given the constraints of writing for bpf targets.
2. The userspace application should be able to poll perf event array, read the logs, and direct to somewhere user accessible - e.g stdout/stderr, a file, systemd journal etc...

Then from within your bpf program you can just use the log macros e.g info!(...)
This would present a much more comprehensive (and production safe) alternative to #39, at (hopefully) a low cost to overall program size.

I appreciate all efforts you went through all the many pieces to the ebpf puzzle interfacing surrounding tools and kernel along with your use of rust workspaces for separating and building all the different aspects.

Please provide an example project to use aya to allow others to wind up faster. The one crate doc example shows to load an ebpf .o, but it didn't really elaborate/clarify enough steps for me to do anything practical with aya yet. I still feel I need to read/master ebpf all C ebpf documentation before taking advantage of aya.

Thank you for listening. Cheers.

It would be nice if maps could be created from user-space before the program is loaded.
This gives more granular control over which maps should (or should not) be pinned than what is available in #45.

As for the API, it's going to be a fairly big change and requires a bit of upfront design.

Typed maps could be able to be instantiated directly, like in the bpf crate, but we'd make the syscalls immediately (or in the case of pinning and the map exists, the use that instead).

let mut my_map  = HashMap<MyPodKey, MyPodValue>::new(255, Some(Path::new("myapp/global")));

The only complication is that created maps need to be available to Bpf::load so we can perform relocations.
In which case we'd need to somehow register created maps with the Bpf struct and make ensure that all load functions have a reference to &self

i noticed that linux 5.15 add time related bpf helper functions, it just changes so fast, maybe checking the kernel version? Or just adding some '#[cfg(some_version)]' so that developer will not misuse helper functions?

It would be useful if Aya could support loading multiple XDP programs on an interface. We have a solution for that in libxdp, which defines a protocol for creating and managing a dispatcher program that calls the component programs.

That protocol is described here: https://github.com/xdp-project/xdp-tools/blob/master/lib/libxdp/protocol.org

Since Aya is doing its own loading, I guess it would make most sense to re-implement the protocol in Rust rather than linking to libxdp. Do you think that would be feasible?

bcc currently has tplist, a tool that outputs a list of available tracepoints / USDT targets and optionally dumps their arguments in a format that can easily be turned into a C data structure.

I think it would be great if Aya had a self-contained tool that could do the above and possibly more. At the moment, I don't know of a good way to find this information besides 1) depending on bcc for the tplist tool, 2) manually examining the contents of these files, or 3) reading the kernel sources.

To do this, we could write a new aya-info tool that similarly parses information from /sys/kernel/debug/tracing. This new tool could be used to get a list of available tracepoints as well as available targets for tracing programs / kprobes. Other features could also be included, such as the ability to query the availability of specific eBPF features in the kernel. Any functionality should probably also be exported as a library (i.e. so that it could be included in a build.rs file or used to dynamically resolve available targets at runtime).

Does anyone have any thoughts on whether this would be worth pursuing?

The documentation of all map and program types should include the minimum kernel version required to use the types.

I'm currently working on PR for both of these. Filing this issue to avoid duplication of effort.

bpfilter is a packet-filtering mechanism similar to iptables and nftables; see https://lwn.net/Articles/747551/ for details. I'd love to have support for bpfilter in aya.

Default behaviour is that the Drop trait calls detach() when the program is terminated (or panics).
For some types of long running programs you may wish to intentionally forget() the fd wrapped by Link, so you can keep the program running after exit. Query and detach can happen out-of-band for some program types (e.g via netlink, or bpf_prog_query)

HashMap::iter is marked as an unsafe method, and it's not immediately clear why. If it is truly unsafe, there should be a safety contract that outlines how the caller should avoid UB. The same applies to other map types.

The list of BPF cgroup program types are as follows -

• BPF_PROG_TYPE_CGROUP_SKB
• BPF_PROG_TYPE_CGROUP_SOCK
• BPF_PROG_TYPE_CGROUP_DEVICE
• BPF_PROG_TYPE_CGROUP_SOCK_ADDR
• BPF_PROG_TYPE_CGROUP_SYSCTL
• BPF_PROG_TYPE_CGROUP_SOCKOPT

The program type BPF_PROG_TYPE_CGROUP_SKB has already been implemented here. This issue will track the development of the remaining program types.

I've tried compiling the Hello XDP bpf program provided at the book, albeit it wont compile:

> cargo xtask build-ebpf
....
....
 Running `rustc --crate-name packetfilter --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=2 -C panic=abort -C embed-bitcode=no -C debuginfo=1 -C debug-assertions=on -C overflow-checks=off -C metadata=e0f528cfa9a6a957 -C extra-filename=-e0f528cfa9a6a957 --out-dir /home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps --target bpfel-unknown-none -C incremental=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/incremental -L dependency=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps -L dependency=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/debug/deps --extern aya_bpf=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps/libaya_bpf-5c6b67f96895e1b5.rlib --extern 'noprelude:compiler_builtins=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps/libcompiler_builtins-f65fb73c67b75f81.rlib' --extern 'noprelude:core=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps/libcore-6d1b02b2c3bc586b.rlib' --extern packetfilter_common=/home/azure/workspace/packetfilter/packetfilter-ebpf/../target/bpfel-unknown-none/debug/deps/libpacketfilter_common-6343852bc3692123.rlib -Z unstable-options`
error: linking with `bpf-linker` failed: signal: 6 (core dumped)

I'm running on Ubuntu 18.4 with kernel version 5.4.0-1063-azure (Its an Azure VM) and llvm-12.

Any help would be greatly appreciated 🙏

Potentially only when using the Cargo dev profile, although it could also be useful for release.
This allows the verifier and bpftool prog dump to produce more useful output

The aya-gen command allows generating bindings from BTF info. We should add support for generating bindings from C header files too. Related to #87 but can be done before.

The command today looks like:

# generate bindings using BTF info from /sys/kernel/btf/vmlinux
aya-gen btf-types ethhdr iphdr

I'm proposing we change it to look like:

# generate bindings using BTF info from /sys/kernel/btf/vmlinux
aya-gen generate ethhdr iphdr

# generate bindings using BTF info from a custom location
aya-gen generate --btf /path/to/btf-info ethhdr iphdr

# generate bindings using a header file
aya-gen generate --header /path/to/file.h ethhdr iphdr

So it would still default to BTF, it would optionally allow passing a custom BTF info path, and it would add the possibility to generate bindings from a C header file using --header.

All variations should additionally allow passing arguments to bindgen directly, eg:

# --blocklist-type and --rustified-enum get passed to bindgen directly
aya-gen generate --header /path/to/file.h ethhdr iphdr -- --blocklist-type foo --rustified-enum bar 

Hi,

I'm trying to use pdu to do some packet parsing from the XdpContext.
This seemed to work at some point in redbpf examples at https://github.com/rebpf/rebpf/blob/50e235721228c1ece2c685f9357a954bd4a322d3/examples/packet_parser/src/kern.rs#L44

[EDIT]
So I ported this function to XdpContext in aya-bpf here
source :
https://sourcegraph.com/github.com/rebpf/rebpf@50e235721228c1ece2c685f9357a954bd4a322d3/-/blob/rebpf/src/libbpf.rs?L547:12#tab=def

I'm trying to access it like so :

#[inline(always)]
fn try_dns_snoop(ctx: XdpContext) -> Result<u32, Error> {

    let buf: &[u8] = if let Some(buf) = ctx.data_buffer() {
        buf
    } else {
        return Ok(xdp_action::XDP_PASS);
    };

    //let ether = EthernetPdu::new(buf)?;
    let first_byte = buf[0];

    let log_entry = PacketLog {
        first_byte: first_byte,
    };
    unsafe {
        EVENTS.output(&ctx, &log_entry, 0);
    }
    Ok(xdp_action::XDP_PASS)
}

But the verifier complains about out of packet access :

0: R1=ctx(id=0,off=0,imm=0) R10=fp0
0: (61) r2 = *(u32 *)(r1 +0)
1: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
1: (15) if r2 == 0x0 goto pc+17
 R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
2: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R10=fp0
2: (61) r3 = *(u32 *)(r1 +4)
3: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
3: (3d) if r2 >= r3 goto pc+15
 R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
4: R1=ctx(id=0,off=0,imm=0) R2_w=pkt(id=0,off=0,r=0,imm=0) R3_w=pkt_end(id=0,off=0,imm=0) R10=fp0
4: (71) r2 = *(u8 *)(r2 +0)
invalid access to packet, off=0 size=1, R2(id=0,off=0,r=0)
R2 offset is outside of the packet
verification time 29 usec
stack depth 0
processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

Am I holding this wrong ?

This would allow for pretty-printing of map values using bpftool.

Current output:

sudo bpftool map dump -p name xdp_stats_map

       "key": ["0x00","0x00","0x00","0x00"
       ],
       "value": ["0x43","0x00","0x00","0x00"
       ]
   }
]

I've noticed that multiple BPF programs loaded by the same userspace process can't share the same host BTF because this method consumes the target_btf object instead of accepting it by reference. However, later in the code, in the only part where target_btf is used (immutably), its reference is used. If i'm not mistaken this means that the very first method could accept a reference instead of a value to begin with.

On my Ubuntu each BTF object is around 13MB (in debug mode) and I'm loading several different eBPF programs, so it'd be nice to be able to share the relocation info among them and save a lot of memory.

A weird issue occurred today that puzzled me a lot. When I used load_file(xxx), everything worked fine. But if I used load(include_bytes!(xxx), None), it produced this error:

Error: error parsing BPF object

Caused by:
    0: error parsing ELF data
    1: Invalid ELF header size or alignment

Anyone knows why? Thanks in advance.

#68 added support for raw tracepoints (raw_tp/). However, there's another variant of raw tracepoints which uses tp_btf/. I don't fully understand how it work but I think it makes it so that you can access the arguments of the traced hook point in a more portable way.

As per the discussion in #71, it would be beneficial to have some kind of an aya_deref! macro to make long chains of raw pointer dereferences more ergonomic (e.g. you can imagine something like aya_deref!(file, f_inode, i_ino) to get the inode number from a file struct). You can imagine that the semantics could be similar to BPF_CORE_READ and friends from libbpf (although with a slightly different purpose).

I'm filing this tracking issue so we have somewhere to discuss possible options and maybe get started on a draft PR soon.

We should setup a CI job to run xtask codgen using the latest libbpf and kernel. Right now we run codegen manually on our dev environments and that's time consuming and error prone.

Tracepoint programs are passed context data according to the layout defined in /sys/kernel/debug/tracing/events/{type}/format eg:

sudo cat /sys/kernel/debug/tracing/events/sched/sched_switch/format
name: sched_switch
ID: 321
format:
        field:unsigned short common_type;       offset:0;       size:2; signed:0;
        field:unsigned char common_flags;       offset:2;       size:1; signed:0;
        field:unsigned char common_preempt_count;       offset:3;       size:1; signed:0;
        field:int common_pid;   offset:4;       size:4; signed:1;

        field:char prev_comm[16];       offset:8;       size:16;        signed:1;
        field:pid_t prev_pid;   offset:24;      size:4; signed:1;
        field:int prev_prio;    offset:28;      size:4; signed:1;
        field:long prev_state;  offset:32;      size:8; signed:1;
        field:char next_comm[16];       offset:40;      size:16;        signed:1;
        field:pid_t next_pid;   offset:56;      size:4; signed:1;
        field:int next_prio;    offset:60;      size:4; signed:1;

We should add support in aya-gen to generate types for this.

Similar to cargo-bpf from RedBPF.
Seeing as setting up the project structure is ~non-trivial it would be nice to be able to initialise a new Aya project quickly.
I think that before this is implemented we'd want to ensure that the "recommend" structure wouldn't change too much.

After some experimentation, this is what I've settled on, but I'd be interested to get input from other users:

.
├── bpf
│   ├── Cargo.lock
│   ├── Cargo.toml
│   ├── build.rs
│   └── src
│       └── xdp
│           └── main.rs
│       └── lib.rs
├── Cargo.lock
├── Cargo.toml
├── README.md
├── types
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── user
    ├── Cargo.lock
    ├── Cargo.toml
    └── src
        └── xdp.rs

• bpf is the bpf program(s)
• types is any shared types between bpf and userspace
• user is the userspace program(s)

Both the user and types package are part of the same workspace as they can both be built with cargo build. bpf however requires rust nightly and some unstable cargo features, and therefore must be in it’s own workspace. With the bpf/.cargo/config.toml setting the necessary flags, the current workflow is currently something like this:

pushd bpf
cargo +nightly build
popd
cargo build

Note: This doesn't factor in any of the ideas around testing of bpf programs from the host... which requires dropping the bpf/.cargo/config.toml 🤔

The PerfEvent docs are pretty minimal. Unfortunately there doesn't seem to be a lot of documentation about the underlying perf API to begin with, especially on the possible events one can hook into.

The perf list command can be used to get a list of event names. That list seems to be generated, see: https://github.com/torvalds/linux/blob/93125562ce385fc0322e010003767d656677bdef/tools/perf/pmu-events/README and https://github.com/torvalds/linux/blob/fb6c79d7261afb7e942251254ea47951c2a9a706/tools/perf/util/parse-events.l#L328

We should come up with something to make working with perf events more approachable.

I don't know how easy/hard this would be, but it would be cool if aya-gen had a --prune-unused flag that would remove unused struct fields and any types they depend on from the generated bindings. The use case for this would be generating an extremely minified vmlinux.rs file for use with BTF programs.

libbpf has the bpf_prog_query() function to get a list of the all attached programs; it is also possible to deattach them and get their names.

I am currently re-writing ir-keytable in rust https://github.com/seanyoung/ir (early stages). For aya to be useful for this work, it would require this functionality.

Following the discussion on #45 and since I started work on #70 it's become clear that the current Map API needs some work.
We have 3 use cases I can think of from userspace

1. I am loading an eBPF file which describes maps and I wish to be able to get a mutable/immutable reference to them in my program (implemented)
2. I wish to create a map from userspace, and register this definition with the BpfLoader such that any loaded programs use the map that I've created (not implemented)
3. I wish to load a map from a pinned location on bpffs for use in my userspace code (not implemented)
   a. As (3) but register the map with the BpfLoader so it can be used when loading my program, which was implemented in #45

Creating a Map from Userspace

The simple case would be to re-implement what we have in aya-bpf

let mymap = HashMap<u32,u32>::with_max_entries(1024, 0);
let my_pinned_map = HashMap<u32,u32>::pinned(1024, 0);

We could however implement a builder style api, which I think is a little more expressive:

let mymap : HashMap<u32, u32> = MapBuilder<HashMap<u32, u32>>::new().max_entries(1024).pinned().build();

Registering Maps to a BpfLoader

use aya::{BpfLoader, Btf};
use std::fs;

let bpf = BpfLoader::new()
    // load the BTF data from /sys/kernel/btf/vmlinux
    .btf(Btf::from_sys_fs().ok().as_ref())
    // load map
    .map(mymapp)
    // finally load the code
    .load_file("file.o")?;

In order for .map() to work, our HashMap<u32,u32> needs to be able to be inserted in to a Vec<Map>.

Given that #70, might need to implement a special case of create - because i believe that we may also need to create dummy inner maps to make the verifier happy - I'm wondering whether it makes sense for Map to become a trait instead.

Loading a Pinned Map from Userspace

let mymap = HashMap<u32,u32>::load("/path/to/map/on/bpffs");

This is a little trickier to implement in practice because we don't have the bpf_map_def ahead of time.

For load:

• We can use bpf_obj_get to get the fd
• We can then use bpf_obj_get_info_by_fd to get the bpf_map_info struct, which appears to be a superset of bpf_map_def

Moving all Map code to a new aya-map module

Seeing as creating maps from userspace will offer an almost identical API to the aya-bpf module, I wonder whether it might be worth moving all map code to it's own module called aya-map 🤔

This would allow us to provide a consistent API from both userspace and kernel space - we can use attributes #[cfg(feature = "user")] or #[cfg(feature = "ebpf")] to implement functions where the implementation needs to differ. In some ways this would make #36 easier, since I believe we already have some mocks in place for maps which could be re-used when testing eBPF code 🤔

Once #43 has been completed, and #47 has merged, I'll move the docs tree to it's own repo called book and update the links/url.

This will allow us to check in the code for the tutorial at the 3 different stages, and refer to sections of the file by anchor, giving us a nifty little button which expands to show the context around the code that's being discussed. Very cool.

We can also add CI to ensure that this sample code compiles.

According to the documentation:

If pid is not None, the program executes only when the target function is triggered by the given pid.

This however doesn't seem to be the case, the kprobe is executed in the context of the calling pid, not the target pid.

For instance, i'm using this probe, trying to block syscalls selectively:

#include <uapi/linux/bpf.h>
#include <bpf_helpers.h>
#include <bpf_map.h>

#define EPERM 1

SEC("kprobe/on_blocked_syscall")
int on_blocked_syscall(void *ctx)
{
	bpf_override_return(ctx, -EPERM);
	return 0;
}

char _license[] SEC("license") = "GPL";

unsigned int _version SEC("version") = 0xFFFFFFFE;

I've compiled this probe with clang and hardcoded the object file as a Vec, that I'm now using like this:

use std::convert::TryInto;
use std::fs;

use aya::{programs::KProbe, Bpf, Btf};

mod probe;

fn main() {
    // target pid
    let target = 4545;
    // load the program
    let data = probe::DATA.to_vec(); // probe::DATA is just a lazy_static Vec<u8>
    let mut bpf = Bpf::load(&data, Btf::from_sys_fs().ok()).unwrap();
    // load the generic kprobe
    let kprobe = bpf.program_mut("on_blocked_syscall").unwrap();
    let kprobe: &mut KProbe = kprobe.try_into().unwrap();

    kprobe.load().unwrap();

    kprobe.attach("__x64_sys_openat", 0, Some(target)).unwrap();

    println!("kprobe attached, now trying to open a file ...");

    fs::read_to_string("/path/to/some/readable/file.txt").unwrap();

    // some blocking operation here
}

Since I'm targeting process 4545 with the attach invocation, I expect the fs::read_to_string call to work, while it instead fails with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/main.rs:29:38
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

And while the program is running, this also seems to apply to all the other processes, for instance:

[  393.127994] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128037] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128069] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128109] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128143] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128178] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128217] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128260] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128300] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  393.128335] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  399.016623] systemd-journald[298]: Failed to open runtime journal: Operation not permitted
[  758.871870] systemd-journald[298]: Failed to open runtime journal: Operation not permitted

Using aya from master:

aya = { git = "https://github.com/alessandrod/aya", branch = "main" }

As part of CI, we should test that we can load and attach all the supported programs. This requires doing two things:

• we need to create small test programs for all the supported types (using aya-bpf and building with bpf-linker)
• we need to create a github action workflow that somehow - probably using qemu or maybe even multipass - loads our own kernel where we can use the bpf syscall (which is otherwise not available in github runners)

I think it would be really cool to have a curated list of projects that use aya. This would help promote the use of the library, plus provide a great repository of practical examples that new users could draw on. Perhaps it could live in its own repo like awesome-aya (e.g. the awesome eBPF repo) or something similar?

Once LSM support gets merged, we've been talking about converting lockc to use aya instead of libbpf-rs, so that might be a nice candidate for such a list. :)

i'm interested in how to use two tracepoints ... i'm following the docs at https://github.com/alessandrod/aya/blob/94b5e2e4e6a535ca74113c5f62c4bd1a7f265469/aya/src/programs/trace_point.rs#L43 and it works with a single tracepoint. Butt when i try to reference a second one, i have an issue with the borrow checker.

This is the code (works fine with a single tp):

fn main() {
    // load the program
    let mut bpf = Bpf::load_file("process_monitor.o").unwrap();
    // tracepoint for sched_process_fork
    let fork_tp: &mut TracePoint = bpf
        .program_mut("on_process_fork")
        .unwrap()
        .try_into()
        .unwrap();
    // tracepoint for sched_process_exit
    let exit_tp: &mut TracePoint = bpf
        .program_mut("on_process_exit")
        .unwrap()
        .try_into()
        .unwrap();

    fork_tp.load().unwrap();
    fork_tp.attach("sched", "sched_process_fork").unwrap();
    exit_tp.load().unwrap();
    exit_tp.attach("sched", "sched_process_exit").unwrap();
    ...
    ...

And this is the error I get:

error[E0499]: cannot borrow `bpf` as mutable more than once at a time
  --> src/main.rs:41:36
   |
35 |     let fork_tp: &mut TracePoint = bpf
   |                                    --- first mutable borrow occurs here
...
41 |     let exit_tp: &mut TracePoint = bpf
   |                                    ^^^ second mutable borrow occurs here
...
47 |     fork_tp.load().unwrap();
   |     ------- first borrow later used here

error: aborting due to previous error

I admit i'm new to rust so this could totally be my fault, but i just copy/pasted the code from the documentation... any help would be appreciated.

Thanks.

Program pinning looks simple enough:
It should be possible to add pub fn pin(&mut self, path: Path ) to the Link trait.
Where Path would be a relative path on /sys/fs/bpf or an absolute path to /sys/fs/bpf.
The implementation would call the bpf syscall and BPF_OBJ_PIN (see: bpf_obj_pin in libbpf for reference).

For Map pinning, the API depends on where the maps are defined.
Aya seems to rely on the eBPF program to define the map, which is the created when the program is loaded:
As such I propose we use the following API:

pub fn load_file<P: AsRef<Path>>(path: P, map_pin_directory: Option<P>) -> Result<Bpf, BpfError>

The map_pin_directory is then passed down through function calls so it's available when the maps are created.
The usage of Path in this API is the same as the one on the Link trait.

Getting information from BPF programs at runtime is hard and bpf_trace_printk is a helpful tool for development (but definitely not production).

I think an ARCHITECTURE.md would also go a long way for getting new contributors up to speed. Even something simple that describes what each crate is for, etc. This looks like an awesome (and very promising) project, and I would love to contribute, but the number of crates and subcrates in this workspace can be a bit overwhelming at first glance. Thanks a lot for your work on the library :)

This is how aya currently defines bpf_map_def: https://github.com/alessandrod/aya/blob/bb595c4e69ff0c72c8327e7f64d43ca7a4bc16a3/aya/src/bpf.rs#L52-L60

Many older eBPF programs use an old definition of bpf_map_def that didn't include the id and pinning fields. Currently aya will fail to parse those programs because of this check: https://github.com/alessandrod/aya/blob/bb595c4e69ff0c72c8327e7f64d43ca7a4bc16a3/aya/src/obj/mod.rs#L512-L517

We could easily support those older programs by relaxing the above check a little - checking that the struct contains up to the map_flags field, and set id and pinning to 0.

I would like to add support for the BPF_MAP_TYPE_LPM_TRIE map.

Some resources on the data-structure:

• https://www.ferrisellis.com/content/ebpf_syscall_and_maps/#bpf_map_type_lpm_trie
• https://blog.apnic.net/2020/10/23/journeying-into-xdp-augmenting-the-dns/
• https://github.com/torvalds/linux/blob/v4.11/samples/bpf/map_perf_test_kern.c#L60

cc: @alessandrod @dave-tucker

Tyring to load a simple XDP program that was built with Aya:

./examples/target/bpfel-unknown-none/debug/xdp: file format elf64-bpf


Disassembly of section xdp/xdp_stats:

0000000000000000 <xdp_stats>:
       0:       b7 00 00 00 02 00 00 00 r0 = 2
       1:       95 00 00 00 00 00 00 00 exit

I've verified that this program can be loaded with sudo bpftool prog load ./examples/target/bpfel-unknown-none/debug/xdp /sys/fs/bpf/xdp_stats

However, loading this from Aya is not possible due to something weird with the BTF relocations.

The snippet of my main.rs that loads the code looks like this:

let mut bpf = Bpf::load_file(path)?;

When run, the program gets stopped by the oom-killer.
Running with strace reveals the following, after reading from /sys/kernel/btf/vmlinux

mmap(NULL, 1863680, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa0e61aa000
brk(0x5619e237b000)                     = 0x5619e237b000
brk(0x5619e239c000)                     = 0x5619e239c000
mmap(NULL, 331776, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa0e6159000
mremap(0x7fa0e6159000, 331776, 659456, MREMAP_MAYMOVE) = 0x7fa0e60b8000
mremap(0x7fa0e60b8000, 659456, 1314816, MREMAP_MAYMOVE) = 0x7fa0e5f77000
mremap(0x7fa0e5f77000, 1314816, 2625536, MREMAP_MAYMOVE) = 0x7fa0e5cf6000
mremap(0x7fa0e5cf6000, 2625536, 5246976, MREMAP_MAYMOVE) = 0x7fa0e57f5000
mremap(0x7fa0e57f5000, 5246976, 10489856, MREMAP_MAYMOVE) = 0x7fa0e4df4000
mremap(0x7fa0e4df4000, 10489856, 20975616, MREMAP_MAYMOVE) = 0x7fa0e39f3000
mremap(0x7fa0e39f3000, 20975616, 41947136, MREMAP_MAYMOVE) = 0x7fa0e11f2000
mremap(0x7fa0e11f2000, 41947136, 83890176, MREMAP_MAYMOVE) = 0x7fa0dc1f1000
mremap(0x7fa0dc1f1000, 83890176, 167776256, MREMAP_MAYMOVE) = 0x7fa0d21f0000
mremap(0x7fa0d21f0000, 167776256, 335548416, MREMAP_MAYMOVE) = 0x7fa0be1ef000
mremap(0x7fa0be1ef000, 335548416, 671092736, MREMAP_MAYMOVE) = 0x7fa0961ee000
mremap(0x7fa0961ee000, 671092736, 1342181376, MREMAP_MAYMOVE) = 0x7fa0461ed000
mremap(0x7fa0461ed000, 1342181376, 2684358656, MREMAP_MAYMOVE) = 0x7f9fa61ec000
mremap(0x7f9fa61ec000, 2684358656, 5368713216, MREMAP_MAYMOVE) = 0x7f9e661eb000
mremap(0x7f9e661eb000, 5368713216, 10737422336, MREMAP_MAYMOVE) = 0x7f9be61ea000
^C--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---

This issue can be worked around by using the following to load the code:

 let data = fs::read(path)?;
 let mut bpf = Bpf::load(&data,None)?;

Running on Fedora 34, Kernel 5.13.4.

I think it's time to move aya to an organization. Ideally I'd also move bpf-linker and my WIP rustc fork there to keep everything in one place.

What name should we use? I created github.com/ebpf-rs a while ago, but I'm not so sure about the name anymore. Suggestions welcome!

We should add support for the new ring buffer API: https://www.kernel.org/doc/html/latest/bpf/ringbuf.html

Apologies for the stupid question, just wanted to know if we can compile Aya userspace programs to WASI?

Use Case: Compile aya userspace programs to WASI and execute them in runtimes like wasmtime.

The cool thing about libbpf is that it defines the BPF_PROG macro which allows to provide typed arguments which are unpacked from the BPF program context:

https://elixir.bootlin.com/linux/v5.14.13/source/tools/lib/bpf/bpf_tracing.h#L381

So far aya doesn't provide such a macro and all aya-based BPF programs are unpacking the context by manually providing offsets, like:

https://github.com/alessandrod/aya-echo-tracepoint/blob/main/echo-ebpf/src/main.rs#L35-L36

It would be nice to have some macro which takes sized types and does the unpacking with offset automatically, without necessity of doing that manually.

I wanted to test out a simple example of using .data, .rodata, and .bss maps in Aya. It seems that .data and .rodata work just fine, but creating the .bss map always seems to fail with an invalid argument error. Inspecting the strace dump (shown below), the offending part of the bpf_attr struct appears to be an incorrect map value size of 0.

Here is my minimal reproduction aya-bpf code:

#![no_std]
#![no_main]

#[used]
static TEST_RODATA: i32 = 0;
#[used]
static mut TEST_DATA: i32 = 1;
#[used]
static mut TEST_BSS: i32 = 0;

Relevant strace dump (offending bpf syscall is the last one, others included for context):

46537 bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=4, max_entries=1, map_flags=0, inner_map_fd=0, map_name=".rodata", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0}, 128) = 3
46537 bpf(BPF_MAP_UPDATE_ELEM, {map_fd=3, key=0x563ce3ec1204, value=0x563ce4ea7020, flags=BPF_ANY}, 128) = 0
46537 bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=4, max_entries=1, map_flags=0, inner_map_fd=0, map_name=".data", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0}, 128) = 4
46537 bpf(BPF_MAP_UPDATE_ELEM, {map_fd=4, key=0x563ce3ec1204, value=0x563ce4eaec90, flags=BPF_ANY}, 128) = 0
46537 bpf(BPF_MAP_CREATE, {map_type=BPF_MAP_TYPE_ARRAY, key_size=4, value_size=0, max_entries=1, map_flags=0, inner_map_fd=0, map_name=".bss", map_ifindex=0, btf_fd=0, btf_key_type_id=0, btf_value_type_id=0, btf_vmlinux_value_type_id=0}, 128) = -1 EINVAL (Invalid argument)