Conversation
Set e->sz to fix_sz + dyn_sz in BPF emit macros, so the field now describes the entire ring buffer record including trailing PMU values, stack traces, and pystacks. This eliminates the need for the 8-byte size_t prefix that handle_rb_event() previously wrote before each event in per-worker dump files. Add bpf_event_fix_sz() helper that derives the fixed part size from e->kind, and use it in all dynamic data accessors (bpf_event_pmu_vals, bpf_event_stack_traces_sz, bpf_event_pystack, and stack trace lookup functions) that need to locate trailing data. Simplify bpf_event_iter_next() to read events directly and advance by e->sz, removing the size_t prefix parsing and the now-unnecessary bpf_event_record.sz field. Signed-off-by: Andrii Nakryiko <[email protected]>
When replaying a captured trace (-R), filters from the original capture are not preserved. Context switch events inherently reference two tasks, so the data file contains events touching tasks outside the filter. Without re-specifying filters during replay, those "other side" tasks produce extra incomplete tracks. Add an extras section to the data file format (bumping minor version to 2.1) that persists capture-time filters. During replay, these filters are automatically restored and merged additively with any CLI-specified filters, producing identical trace output without needing to repeat the original filter arguments. Signed-off-by: Andrii Nakryiko <[email protected]>
Add utrace_cfg.c / utrace_cfg.h implementing a DSL parser for
user-defined trace probe definitions specified via -U CLI option.
The DSL format is:
<type>:<target-spec> (<parameters>) { <settings> }
Supported probe types: u (uprobe), uret, usdt, k (kprobe), kret,
tp (tracepoint), raw_tp, uspan, kspan. Generic spans use '<=>' to
join two probe definitions.
Parameters support: stack capture, binary path, PID filter, and
argument capture with type annotation (u8..u64, s8..s64, str) and
optional naming via '->'.
The parser uses a string-view (strv) abstraction for zero-copy
parsing with positional error reporting that highlights the
problematic part of the definition with '^' markers.
Wire -U option into argp, supporting both inline definitions and
@file syntax for reading definitions from a file.
Signed-off-by: Andrii Nakryiko <[email protected]>
Extract string view (struct sview) and all sv_*() helpers from utrace_cfg.c into a new shared strs.h header. Add struct sbuf, a growable string buffer, alongside it. Add utrace_cfg_format() which produces a canonical string representation that round-trips through utrace_cfg_parse(), replacing the old fixed-buffer utrace_cfg_dump() approach. Remove utrace_cfg_dump() and its verbose-mode call site since replay-info now serves that purpose. Persist utrace definitions in the data file's extras section (WEXTRA_UTRACE_DEF) so they are automatically restored during replay without requiring the user to re-specify -U flags. Signed-off-by: Andrii Nakryiko <[email protected]>
Remove SIBLING_MERGE_BEHAVIOR_NONE from all track descriptors to allow Perfetto to merge sibling tracks with the same name. This was previously disabled due to performance issues that may now be resolved. Signed-off-by: Andrii Nakryiko <[email protected]>
Signed-off-by: Andrii Nakryiko <[email protected]>
Add the foundational types shared between BPF and userspace for utrace: - Event kinds: EV_UTRACE_INSTANT, EV_UTRACE_ENTRY, EV_UTRACE_EXIT - Stack trace kind: ST_UTRACE - BPF map config struct: utrace_probe_cfg (per-probe config with args) - BPF event struct: wprof_utrace (cfg_id + arg_len array) - Update bpf_event_fix_sz() for new event kinds Make utrace_cfg.h includeable from BPF code by guarding system includes and pointer-containing structs with #ifndef __bpf__. Allow arg:ret captures on span probes (uspan/kspan), since spans combine entry and exit -- regular args go to entry side, arg:ret to exit side. Signed-off-by: Andrii Nakryiko <[email protected]>
Add BPF programs for utrace probe types: uprobe, uretprobe, kprobe,
kretprobe. All four share a common utrace_handle_probe() inline helper.
The handler reads probe config from the utrace_probe_cfgs BPF array map
(indexed by bpf_get_attach_cookie), captures arguments into a per-CPU
scratch buffer, and emits events via emit_task_event_dyn(). String args
use bpf_probe_read_{kernel,user}_str into the scratch buffer to compute
exact sizes before reserving the ringbuf. Integer args are read inline.
Each arg's byte length is recorded in the event's arg_len[] array, with
negative values indicating read errors.
Signed-off-by: Andrii Nakryiko <[email protected]>
Add utrace.c/utrace.h implementing probe attachment: - utrace_setup_autoload(): enables BPF programs needed based on probe types configured (uprobe, uretprobe, kprobe, kretprobe) - utrace_setup(): populates BPF probe config map, resolves uprobe symbols via elf_find_syms(), and attaches probes with bpf_cookie identifying each probe's map entry Span probes (uspan/kspan) create two map entries sharing the same utrace_id: one for entry (with non-ret args) and one for exit (with arg:ret args). Signed-off-by: Andrii Nakryiko <[email protected]>
Add persisted event format for utrace: - wevent_utrace: stores utrace_id, utrace_stack_id, and trailing variable-size data (arg_refs[] for string/int references, int_vals[] for packed integer values) - Update wevent_fixed_sz() and wevent_kind_name() for new event kinds Add EV_UTRACE_INSTANT/ENTRY/EXIT cases to persist_bpf_event(): - Derives arg count and types from env.utrace_cfgs[utrace_id] - Interns string args into the string pool - Packs integer args into trailing int_vals blob - Resolves stack traces via bpf_event_stack_id(ST_UTRACE) Signed-off-by: Andrii Nakryiko <[email protected]>
Wire utrace into the main wprof pipeline: - Add capture_utrace bitfield to wprof_data_cfg and tristate to env - Add capture_utrace to capture_features[] for replay compatibility - Initialize env.capture_utrace = UNSET so replay picks it up from data - Add -Sutrace / --stacks utrace for enabling utrace stack captures - Validate that utrace configs with 'stack' param require -Sutrace - Call utrace_setup_autoload() before BPF load with proper map sizing - Call utrace_setup() from attach_bpf() to attach probes - Add ST_UTRACE to replay info display with --stacks utrace format - Use consistent 4-space indentation for stacks/PMU sub-items in -RI Signed-off-by: Andrii Nakryiko <[email protected]>
Add emit support for utrace events in both JSON and Perfetto formats. JSON output emits events with type "utrace_instant"/"utrace_entry"/ "utrace_exit", utrace_id, task info, and decoded args (names resolved from utrace config, values from wevent trailing data). Perfetto output creates per-config child tracks under each thread's scheduling track. Span probes emit slice begin/end pairs, non-span probes emit instant events. Args are attached as debug annotations. Stack traces are emitted via emit_callstack() when ST_UTRACE is enabled. Add IID_CAT_UTRACE category for Perfetto event categorization. Signed-off-by: Andrii Nakryiko <[email protected]>
…for spans Replace packed u8 fields in utrace_probe_cfg with proper enum types (utrace_event_type, utrace_type, utrace_arg_type) and widen arg_cnt/idx to int. Replace the bool capture_stack with a flags bitmask (UTRACE_FL_CAPTURE_STACK) and drop the unused is_kernel field (the BPF handler already knows from the SEC type). For kspan/uspan probes, don't set the stack capture flag on the exit map entry since entry and exit fire on the same function, making the exit stack trace redundant. Signed-off-by: Andrii Nakryiko <[email protected]>
Add a blobset that deduplicates arbitrarily-sized byte blobs, similar to strset but content-agnostic. Uses libbpf's hashmap for lookup with offset+size packed into a single long key. Offset 0 is reserved so it can serve as an invalid/sentinel value. Signed-off-by: Andrii Nakryiko <[email protected]>
Add a blob pool section to the wprof.data format alongside the existing string pool, for storing variable-sized binary data with deduplication. Each integer utrace arg is now stored as its own blob in the pool with natural alignment matching its type size (u8->1, u16->2, u32->4, u64->8), and the per-event arg_refs array uniformly contains offsets into either the string pool (for string args) or the blob pool (for integer args). This eliminates the trailing int_vals[] blob that was previously packed after arg_refs in each wevent. Signed-off-by: Andrii Nakryiko <[email protected]>
When the user specifies kprobe/kretprobe/kspan args without an explicit type, resolve the correct type from kernel BTF (/sys/kernel/btf/vmlinux) instead of defaulting to u64. This preserves type information like signedness and width (e.g., int -> s32, ssize_t -> s64). Pointer args to char are auto-detected as strings, other pointers get a new UTRACE_ARG_PTR type that formats as hex in Perfetto/JSON output. Parameter names are also auto-resolved from BTF when not explicitly specified, even if the user provided an explicit type. Signed-off-by: Andrii Nakryiko <[email protected]>
Shorter and easier to type. Signed-off-by: Andrii Nakryiko <[email protected]>
Add `arg:*` shorthand that expands to all positional arguments (and `arg:ret` for return/span probes), with types and names auto-resolved from kernel BTF where possible. `arg:*` coexists with explicit arg definitions (e.g., `arg:1:u32->count`) which take precedence for specific indices; the wildcard fills in the remaining args. Replace `utrace_resolve_btf_types()` with `utrace_augment_args()` that handles both wildcard expansion and type/name resolution in one pass, including for inner probes of generic spans. Normalize param ordering: positional args in ascending order, then arg:ret, then stack/path/pid, using `utrace_param_type` enum values as natural sort keys. Signed-off-by: Andrii Nakryiko <[email protected]>
Relative paths like `path:./utrace_test` would be passed directly to the kernel perf_event_open, which requires absolute paths. Resolve relative paths at parse time using realpath(). Signed-off-by: Andrii Nakryiko <[email protected]>
Fix signedness mismatch for sym_offset (long -> unsigned long) to match updated elf_find_syms signature, and add default case to format_probe switch for UTRACE_PARAM_INVALID. Signed-off-by: Andrii Nakryiko <[email protected]>
When a uprobe definition specifies pid: but no path:, scan the process's /proc/<pid>/maps for executable VMAs and look up the target symbol in each loaded binary. This eliminates the need for an explicit path: param when pid: is provided. Use /proc/<pid>/map_files/<start>-<end> symlinks for ELF symbol lookup and uprobe attachment to correctly handle cross-namespace scenarios, while recording the human-readable VMA path in the utrace config for replay and display. Also fix a pre-existing bug in emit_utrace_args where annotation key names (auto-generated "argN"/"ret" and user-specified names) were stored as stack-local pointers but the protobuf annotation framework only captures the pointer for later encoding. Intern all annotation key names via emit_intern_str() so they remain valid until encoding. Add str_has_suffix() helper to utils.h. Signed-off-by: Andrii Nakryiko <[email protected]>
Pre-compile name_fmt into a flat segment array during config parsing so that format_utrace_name() at emit time is a simple loop over segments with no string matching. Also add settings support (id, name) to utrace probe definitions. Fix kprobe string argument capture for user-space pointers: detect kernel vs user addresses by checking the high bit (negative as s64 means kernel address) and use the appropriate BPF read helper. Signed-off-by: Andrii Nakryiko <[email protected]>
We need bpf_program__clone() Signed-off-by: Andrii Nakryiko <[email protected]>
…ases Instead of having separate case blocks for span vs instant probes, merge them into a single case with two sequential if blocks: one for entry (uprobe/kprobe or span entry) and one for exit (uretprobe/kretprobe or span exit). Span triggers both blocks. Signed-off-by: Andrii Nakryiko <[email protected]>
Add support for tracing BPF programs via fentry/fexit using new probe types: bpf: (instant), bpfret: (return), and bpfspan: (entry+exit span). BPF programs are found by iterating all loaded programs and matching by function name in BTF func_info. Template fentry/fexit programs are prepared with a valid attach target, then cloned per-probe with the actual target's prog_fd and btf_func_id. The prepare→disable autoload→load sequence ensures templates get relocated but never loaded into the kernel. Argument capture reuses the existing utrace arg framework, with bpf_get_func_arg()/bpf_get_func_ret() for reading values instead of pt_regs. BTF-based arg type and name resolution works against the target program's BTF, and wildcard arg expansion is supported. Also refactor utrace initialization into a clean setup/attach split: - utrace_setup(): all pre-load work (autoload, map sizing, BPF target resolution, arg augmentation, BTF cleanup) - utrace_attach(): all post-load work (probe attachment) This eliminates utrace_setup_autoload() and moves map sizing from wprof.c into utrace_setup(). Move cfg_is_span() to utrace_cfg.h as a shared inline helper. Signed-off-by: Andrii Nakryiko <[email protected]>
Move uprobe binary/symbol resolution out of utrace_attach() and into utrace_setup(). Store the resolved attach_path, display_path, and total attach_off on the uprobe cfg, so attach becomes a straight lookup. This separates pre-load setup (resolution) from post-load attach (attachment), matching the existing pattern for BPF target resolution, and keeps attach_path (e.g. /proc/<pid>/map_files/...) distinct from display_path (human-readable VMA name) for logging. While here, consolidate utrace_setup() into a single type-dispatched loop: one pass iterates cfgs and switches on type to handle uprobe resolution + autoload, kprobe autoload, or BPF target lookup + fentry/ fexit need tracking. Map sizing, template-program attach-target setup, arg augmentation, and BTF cleanup stay as separate post-loop steps. Signed-off-by: Andrii Nakryiko <[email protected]>
Add support for generic span definitions of the form "<entry> ~~ <exit>", where the two legs can be any combination of simple probe types (uprobe, uretprobe, kprobe, kretprobe, bpf, bpfret). Both legs share the parent cfg's utrace_id, with entry leg getting UTRACE_ENTRY and exit leg UTRACE_EXIT. Implementation: - cfg_is_span() now includes UTRACE_SPAN, which makes the map_cnt accounting in utrace_setup() pick up the correct "2 map entries per span" automatically. - Introduce utrace_for_each_leg(leg, cfg) macro that yields the cfg itself for non-UTRACE_SPAN configs, or the entry and exit legs for UTRACE_SPAN. This lets utrace_setup() uniformly dispatch per-leg work (autoload, uprobe resolution, BPF target lookup, BTF cleanup) without any UTRACE_SPAN special-casing — the inner type-dispatch switch keys on leg->type, which is always a simple type. - Extract attach_utrace_probe() helper that attaches exactly one probe given explicit event_type, is_exit, and is_retprobe. Callers drive the retprobe selection (for uprobe/kprobe/bpf families it picks the right template BPF program). The attach main loop becomes three small cases: simple types call the helper once, native spans call it twice on the same cfg, UTRACE_SPAN calls it twice on each sub-leg. Signed-off-by: Andrii Nakryiko <[email protected]>
Extend elf_find_syms() with an extra optional out parameter for per-symbol file offsets, computed (libbpf-style) from the symbol's containing section header: st_value - sh_addr + sh_offset. The existing output is renamed to virt_offs and made optional as well. cnt is moved right after syms to keep the input pair together. uprobe attach wants the file offset into the binary, not a virtual-address offset. With some ELFs — notably statically linked binaries where each PT_LOAD segment's (p_vaddr - p_offset) differs by a page — the virt_offs computation (which uses the first PT_LOAD's base) lands at the wrong bytes for symbols in later segments, so uprobes attach to garbage code and never fire. Per-symbol section-based file offsets avoid this entirely. utrace switches its uprobe resolution paths to file_offs. pytrace, pydisc, and inject keep using virt_offs (they add the VMA-derived load base at runtime, which is the correct model for the PIE/shared-lib binaries they target). Signed-off-by: Andrii Nakryiko <[email protected]>
For UTRACE_SPAN, generate the track display name by joining the two legs' probe names with an em-dash (e.g., "PyEval_RestoreThread — PyEval_SaveThread"). Previously this fell through to "?" since utrace_probe_name() didn't handle UTRACE_SPAN. The user can still override with |id:...|, which takes precedence. Signed-off-by: Andrii Nakryiko <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.