omondi3 / bpftrace

High-level tracing language for Linux eBPF

bpftrace

bpftrace is a high-level tracing language for Linux eBPF available in recent Linux kernels (4.x). bpftrace uses LLVM as a backend to compile scripts to BPF-bytecode and makes use of libbpf and bcc for interacting with the Linux BPF system, as well as existing Linux tracing capabilities: kernel dynamic tracing (kprobes), user-level dynamic tracing (uprobes), tracepoints, etc. The bpftrace language is inspired by awk, C, and predecessor tracers such as DTrace and SystemTap. bpftrace was created by Alastair Robertson.

• How to Install and Build
• Manual / Reference Guide
• Tutorial
• Example One-Liners
• Tools
• Contribute
• Development
• Support
• Probtypes
• Plugins
• License

Example One-Liners

The following one-liners demonstrate different capabilities:

# Files opened by thread name
bpftrace -e 'tracepoint:syscalls:sys_enter_open { printf("%s %s\n", comm, str(args->filename)); }'

# Syscall count by thread name
bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

# Read bytes by thread name:
bpftrace -e 'tracepoint:syscalls:sys_exit_read /args->ret/ { @[comm] = sum(args->ret); }'

# Read size distribution by thread name:
bpftrace -e 'tracepoint:syscalls:sys_exit_read { @[comm] = hist(args->ret); }'

# Show per-second syscall rates:
bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @ = count(); } interval:s:1 { print(@); clear(@); }'

# Trace disk size by PID and thread name
bpftrace -e 'tracepoint:block:block_rq_issue { printf("%d %s %d\n", pid, comm, args->bytes); }'

# Count page faults by thread name
bpftrace -e 'software:faults:1 { @[comm] = count(); }'

# Count LLC cache misses by thread name and PID (uses PMCs):
bpftrace -e 'hardware:cache-misses:1000000 { @[comm, pid] = count(); }'

# Profile user-level stacks at 99 Hertz for PID 189:
bpftrace -e 'profile:hz:99 /pid == 189/ { @[ustack] = count(); }'

# Files opened in the root cgroup-v2
bpftrace -e 'tracepoint:syscalls:sys_enter_openat /cgroup == cgroupid("/sys/fs/cgroup/unified/mycg")/ { printf("%s\n", str(args->filename)); }'

More powerful scripts can easily be constructed. See Tools for examples.

Contribute

Contributions are welcome! Please see the development section below for more information. For new bpftrace tools, please add them to the new user-tools repository. The tools that exist in this repository are a small collection curated by the bpftrace maintainers.

• Bug reports and feature requests: Issue Tracker

• Development IRC: #bpftrace at irc.oftc.net

• Good first issues

Development

• Coding Guidelines
• Development Guide
• Development Roadmap
• Fuzzing
• Nix
• Release Process
• Tests

Support

For additional help / discussion, please use our discussions page.

We are also holding regular office hours open to the public.

Probe types

See the Manual for more details.

Plugins

bpftrace has several plugins/definitions, integrating the syntax into your editor.

• Emacs
• Vim
• VS Code

License

Copyright 2019 Alastair Robertson

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.