(11) feat(acl): crate scaffolding + software reference backend#1575
Open
daniel-noland wants to merge 11 commits into
Open
(11) feat(acl): crate scaffolding + software reference backend#1575daniel-noland wants to merge 11 commits into
daniel-noland wants to merge 11 commits into
Conversation
This was referenced May 31, 2026
daniel-noland
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new dataplane-acl workspace crate that defines the ACL public API surface and implements a pure-software “reference/oracle” backend (typed and dynamic) behind the lookup::Lookup interface.
Changes:
- Add
aclas a workspace member and workspace dependency, plus package metadata for miri/wasm selection. - Introduce
dataplane-aclcrate scaffolding with strict linting and areferencemodule. - Implement
ReferenceTable(typedMatchKey) andDynReferenceTable(runtimeFieldSpec) linear-scan classifiers with unit tests.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
Cargo.toml |
Adds acl to workspace + workspace deps; introduces miri/wasm metadata entry. |
Cargo.lock |
Locks the new dataplane-acl crate and its dependencies. |
acl/Cargo.toml |
Defines the new dataplane-acl package and dependencies. |
acl/src/lib.rs |
Crate-level lint policy and top-level documentation; exports reference module. |
acl/src/reference/mod.rs |
Wires up reference backend modules + re-exports. |
acl/src/reference/table.rs |
Implements typed ReferenceTable/RefRule and Lookup<K, A> integration. |
acl/src/reference/dyn_table.rs |
Implements runtime-shaped DynReferenceTable with validation and byte-slice lookup. |
Comment on lines
+29
to
+35
| pub(crate) fn matches_packed(&self, specs: &[FieldSpec], buf: &[u8]) -> bool { | ||
| debug_assert_eq!(self.fields.len(), specs.len()); | ||
| self.fields | ||
| .iter() | ||
| .zip(specs) | ||
| .all(|(pred, spec)| pred.matches(&buf[spec.offset..spec.offset + spec.size])) | ||
| } |
Comment on lines
+66
to
+73
| fn pack(key: &K) -> Option<[u8; MAX_KEY_BYTES]> { | ||
| if K::KEY_SIZE > MAX_KEY_BYTES { | ||
| return None; | ||
| } | ||
| let mut buf = [0u8; MAX_KEY_BYTES]; | ||
| key.as_key_into(&mut buf[..K::KEY_SIZE]); | ||
| Some(buf) | ||
| } |
Comment on lines
+88
to
+103
| #[must_use] | ||
| pub fn lookup_bytes(&self, key: &[u8]) -> Option<&A> { | ||
| assert_eq!(key.len(), self.key_size, "key length must equal key_size"); | ||
| self.rules | ||
| .iter() | ||
| .find(|rule| rule.matches_packed(&self.specs, key)) | ||
| .map(RefRule::action) | ||
| } | ||
| #[must_use] | ||
| pub fn matches_bytes(&self, key: &[u8]) -> Vec<&RefRule<A>> { | ||
| assert_eq!(key.len(), self.key_size, "key length must equal key_size"); | ||
| self.rules | ||
| .iter() | ||
| .filter(|rule| rule.matches_packed(&self.specs, key)) | ||
| .collect() | ||
| } |
Comment on lines
+265
to
+272
| [workspace.metadata.package.acl] | ||
| package = "dataplane-acl" | ||
| # Default features enable the DPDK `rte_acl` backend, which pulls in | ||
| # `dpdk-sys` (bindgen against the system DPDK headers). miri can't | ||
| # build that path on the cross target, and the reference backend's | ||
| # unit tests run fine outside the miri profile. | ||
| miri = false # hopeless + pointless | ||
| wasm = false # hopeless + pointless |
Comment on lines
+17
to
+18
| //! - [`reference`](mod@reference): linear-scan software classifier; | ||
| //! differential oracle and a mutable cascade front. Always built. |
Add a `dataplane-lifecycle` crate with `Shutdown` and `Subsystem` primitives, signal-handler installation, and a process-wide shutdown watchdog. `Shutdown` bundles a root `CancellationToken` and one `Subsystem` per long-lived component (workers, router, mgmt, metrics). Each subsystem exposes a per-subsystem cancel token, a `TaskTracker`, and a shared fatal flag. Subsystems drain in topological order with per-subsystem deadlines; the detached watchdog enforces an absolute upper bound on total shutdown duration. No consumers yet -- wired up in follow-on commits. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: Daniel Noland <[email protected]>
Plumb lifecycle Subsystems into the routing crate as the first step of the threading rewrite. The rest of `main`'s shutdown signaling (ctrlc + mpsc<i32> + start_mgmt + MetricsServer + old DriverKernel::start) stays in place; follow-on commits migrate each. - `Router::new` takes `(mgmt, mgmt_handle, router)` Subsystems + runtime handle. Plumbed through `packet_processor::start_router`. - `start_rio` takes `&Subsystem`; the IO loop observes its cancel between poll cycles (worst-case exit latency = 1s poll). Adds an ExitGuard so panic-unwind or unexpected loop exit reports fatal. - `RouterCtlMsg::Finish` removed; `RioHandle::finish` becomes idempotent. - `bmp::spawn_background` spawns onto the caller-provided runtime handle tracked under `mgmt`; no more leaked runtime. - `runtime.rs` builds a multi-thread mgmt runtime (only BMP tenants it in this commit) and a `Shutdown` for plumbing into Router::new. - `dataplane` and `mgmt` Cargo.toml gain `lifecycle` dep. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: Daniel Noland <[email protected]>
Move mgmt + metrics from dedicated OS threads with private runtimes to tenants of the multi-thread mgmt runtime introduced in the prior commit. The kernel driver still uses the legacy ctrlc + mpsc<i32> path; the final unification commit migrates that. - `run_mgmt` (renamed from `start_mgmt`): synchronous init on the caller-provided handle, then spawns the three long-lived tasks (config processor, status updater, config watcher) via `Subsystem::spawn_fatal_on_exit` so their unexpected exit flips fatal. Init observes `mgmt.root_token()` so SIGINT during k8s retries returns `LaunchError::Cancelled` within cancel latency. - `LaunchError::Cancelled` is a clean-shutdown signal; the call site in `runtime.rs` forwards the existing mpsc stop channel with code 0 so the legacy shutdown path stays consistent. - `spawn_metrics` replaces `MetricsServer`: HTTP endpoint, upkeep ticker, and stats collector all spawn onto `mgmt_handle` tracked under `metrics`. Uses plain `spawn_on` (not `spawn_fatal_on_exit`) — a dead metrics endpoint should not take down the dataplane. - Drop stale `LaunchError` variants no longer constructed. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: Daniel Noland <[email protected]>
Replace the last legacy shutdown signaling (ctrlc handler + mpsc<i32> exit code + dedicated controller thread) with the single `lifecycle::Shutdown` path, and migrate the kernel driver to scoped threads with cancellation observation. After this commit there is one signaling path: SIGINT/SIGTERM -> shutdown.root, or any subsystem's report_fatal -> shutdown.root, with the watchdog as the absolute upper bound. - `main`: install `spawn_signal_handler` and `spawn_shutdown_watchdog`, run everything inside `concurrency::thread::scope`, block on `root.cancelled()`, then drain subsystems in canonical order (workers -> router -> metrics -> mgmt). Exit code from `is_fatal()`. - `DriverKernel::start`: takes `&Scope` and `&Subsystem`; workers spawn via `spawn_scoped` with an `ExitGuard` Drop pattern that reports fatal on panic-unwind, early `?`-return, and unexpected normal exit. Reader loops observe cancel between reads. Supervisor joins-and-logs. - Drop `dataplane/src/drivers/tokio_util.rs` and its `run_in_local_tokio_runtime` helper (inlined where needed). - Drop `ctrlc` and `mio` from `dataplane` dependencies; drop `ctrlc` from the workspace. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> Signed-off-by: Daniel Noland <[email protected]>
Lets callers derive the rte_acl input-buffer size requirement from a const FIELD_DEFS array at compile time -- useful for feeding into a const generic (e.g. the STRIDE on a DPDK-backed Lookup type alias) rather than rediscovering it at runtime. The runtime path through AclBuildConfig::new still calls into the same helper and caches the result; only the surface broadens from fn to const fn (with the for-loop rewritten as a while-let-i index walk that const-fn-eval can chew on). Existing FieldExtentOverflow validation is preserved; in a const context overflow becomes a compile error instead of UB. Adds one test exercising the const path: const MIN_INPUT_SIZE = AclBuildConfig::compute_min_input_size(&DEFS); plus the assertion that the runtime path returns the same value. just fmt; cargo check -p dataplane-dpdk --all-targets passes.
Introduces shared EAL test scaffolding for downstream crates that
need to exercise rte_acl, mempools, or any other DPDK runtime under
`#[test]`. Two pieces, both behind the new dpdk `test` feature so
production builds remain unchanged:
- dpdk-test-macros: proc-macro crate exposing #[with_eal], which
injects `let _eal = <dpdk_crate>::test_support::start_eal();` at
the top of a #[test] function. Resolves the dpdk crate's path via
proc-macro-crate so the macro works in-tree (`crate`), under the
workspace alias (`::dpdk`), or with the canonical name
(`::dataplane_dpdk`).
- dpdk::test_support: hosts a shared OnceLock<Eal> initialized with
--no-huge / --no-pci / --in-memory and the cpu-affinity-aware
--lcores derivation that was previously inline in acl/mod.rs.
Once-per-process by construction; safe under nextest's per-test
forking and single-process runners alike.
dpdk/src/acl/mod.rs picks up the new macro: every test in the module
loses its `let _eal = start_eal();` prologue and gains a #[with_eal]
attribute above #[test]. The inline start_eal helper and the
module-scoped OnceLock<Eal> static go away.
dpdk/Cargo.toml grows the `test` feature (turns on dpdk-test-macros
plus the id and nix runtime helpers that test_support needs) and a
self-referencing dev-dep `dataplane-dpdk = { path = ".", features =
["test"] }` so dpdk's own tests see the macro surface.
just fmt; cargo check --workspace --all-targets passes.
Introduces a no_std FixedSize trait carrying a known byte width plus a big-endian write. Lives in its own crate so value-providing crates (`net`) and downstream key-packing frameworks (a future `match-action` PR) can both depend on it without depending on each other -- `net` implements FixedSize on its own newtypes without an orphan-rule violation, and the framework doesn't need to know about any particular value supplier. fixed-size/src/lib.rs: - pub trait FixedSize: Copy, with const SIZE: usize and fn write_be(&self, out: &mut [u8]). - Blanket impls for the network-order primitives the framework cares about: u8 / u16 / u32 / u64 / u128 / Ipv4Addr / Ipv6Addr. net/src/fixed_size.rs is the consumer bridge: FixedSize impls for TcpPort, UdpPort, UnicastIpv4Addr, and Vni. Each delegates write_be to the underlying primitive's impl, so wire bytes match what writing the raw integer would produce. Vni is a 24-bit value written into a 4-byte field (high byte zero) because backends model fields in 1/2/4 widths, not 3. net/Cargo.toml grows the fixed-size workspace dep; net/src/lib.rs adds the module behind no cfg gates (it has no further dependencies of its own). just fmt; cargo check --workspace --all-targets passes.
Adds the read-only key/action lookup vocabulary downstream match-action backends will implement. Tiny crate by design (one trait pair, two collection impls, inline tests) so consumer crates depend on this without depending on each other. lookup/src/lib.rs: - pub trait Projection<T>: extracts a key of type T from self. Implemented on `&'a Source` so the lifetime threads into T when T borrows; for owned T the lifetime is unused. The same source can implement Projection<T> for many T -- the call site picks one by inference from a Lookup backend's key type. - impl<K> Projection<Option<K>> for Option<K>: identity, so classify_opt accepts a pre-computed Option<K> directly. Scoped to Option<K> (not a general impl<T> Projection<T> for T) so a backend that implements Lookup<K, A> for every K can't make classify ambiguous. - pub trait Lookup<K, A>: lookup(&K) -> Option<&A>, plus default methods classify (S: Projection<K>) and classify_opt (S: Projection<Option<K>>) that project and look up in one call. Two trait type parameters, so one backend can serve many (K, A) pairs. - Blanket Lookup impls for BTreeMap<K, V> and HashMap<K, V, S> so tests / simple consumers get a working backend out of the box. Inline tests exercise: projection-by-table-type inference at v4 2-tuple and 4-tuple widths, lifetime threading through borrowed projections, miss returning None, classify_opt short-circuit on None projection, the identity Projection<Option<K>> for Option<K>, and HashMap parity with BTreeMap. just fmt; cargo check --workspace --all-targets passes.
Lands the match-action key vocabulary plus its derive. Backends in downstream PRs consume the result via a structural FIELD_SPECS view and a byte-packing as_key_into(); the bolero property-test generators land next as their own feature gate. match-action/src/: - lib.rs publishes the FieldKind enum (Prefix / Mask / Range / Exact), the FieldSpec layout record (name / kind / size / offset), and the MatchKey trait (N, KEY_SIZE, field_specs(), as_key_into()). Trait methods take slices rather than Self::N-sized arrays because Self::N in a trait fn needs unstable generic_const_exprs; the derive emits sized inherent helpers on concrete types instead. - field.rs is a thin re-export of FixedSize from the fixed-size crate, so consumers can refer to it via match_action::FixedSize. - predicate.rs holds the type-erased FieldPredicate form (Prefix, Mask, Range, Exact variants over FieldBytes) plus the Erased backend marker used by the reference oracle. - rule.rs defines the four kind-typed *Spec wrappers (PrefixSpec, MaskSpec, RangeSpec, ExactSpec), the Backend / Accepts / IntoBackendField / IsUniversal traits, and the RuleField envelope carrying name / spec. match-action-derive (proc-macro crate) emits, from a struct annotated with #[prefix] / #[mask] / #[range] / #[exact]: - the MatchKey impl, - a parallel <Name>Rule struct holding the typed *Spec data, - an inherent FIELD_SPECS const for compile-time inspection by downstream backends, and - (for non-generic structs) an inherent as_key() -> [u8; KEY_SIZE] for ergonomic byte packing. tests/derive_roundtrip.rs exercises a 5-tuple-shaped key covering all four predicate kinds: asserts N, KEY_SIZE, the field_specs() content, and that as_key_into / as_key produce the expected big-endian byte layout (with declared field ordering preserved). just fmt; cargo check --workspace --all-targets passes.
Adds property-test generators over the rule wrappers, behind a new `bolero` feature gate so the bolero dep stays out of the default build graph. Downstream consumers (acl property tests, future cascade Upsert-laws harnesses) enable the feature in their dev-deps and draw random matching / non-matching packets for any single rule. match-action/src/generator.rs: - FieldHit / FieldMiss TypeGenerators yielding bytes that satisfy / violate a given field predicate. Cover all four predicate kinds: Prefix (any address sharing the rule's high-order bits / one with flipped bits in that prefix), Mask (value bits match / a flipped in-mask bit), Range (uniform draw in [min, max] / outside the bounds), Exact (the value / a different value). - Miss generators skip universal predicates (range covering all values, mask of all zeros, prefix length zero, etc.); the derive-emitted IsUniversal check on <Name>Rule lets callers detect whole rules that have an empty miss set. match-action/src/lib.rs picks up #[cfg(feature = "bolero")] gates on the generator mod + its FieldHit / FieldMiss re-exports. match-action/Cargo.toml grows the bolero optional dep and the `bolero` feature, which also turns on the matching forwarded feature on match-action-derive (a no-op today, kept for future emit changes in the derive). match-action-derive/Cargo.toml mirrors the `bolero` feature declaration so cargo can forward through. just fmt; cargo check --workspace --all-targets and cargo check -p dataplane-match-action --features bolero pass.
Introduces the dataplane-acl crate with the software reference classifier. The DPDK rte_acl backend lands behind a feature gate in a follow-up PR. The reference backend is a linear-scan software classifier built on the canonical FieldPredicate form from match-action (rule.into_backend_fields::<Erased>()), so it speaks the same four predicate kinds (Prefix / Mask / Range / Exact) as every other backend. Two roles: 1. Differential-testing oracle against rte_acl (a future PR's differential property tests pit both backends against the same random rule + packet draws). 2. Non-lossy substrate for a small-delta cascade front over a slow tail backend. Layout: - src/lib.rs declares the crate-level docs and re-exports the reference module. The dpdk feature gate and dpdk_table_alias! macro land alongside the rte_acl backend itself in the next PR. - src/reference/table.rs is the typed surface: ReferenceTable<K, A> parameterised by a MatchKey and an action; RefRule wraps the lowered Erased predicates plus an action. Inline unit tests cover positional precedence (first match wins) and the four predicate kinds. - src/reference/dyn_table.rs is the runtime-shape twin: DynReferenceTable carries its FieldSpec layout at runtime so property tests can fuzz the schema itself. Returns DynShapeError on shape mismatch. just fmt; cargo check --workspace --all-targets passes.
daniel-noland
force-pushed
the
pr/daniel-noland/match-action
branch
from
c071ee9 to
7d5e7ad
Compare
daniel-noland
force-pushed
the
pr/daniel-noland/acl-reference
branch
from
3220e66 to
710b08b
Compare
Base automatically changed from
pr/daniel-noland/match-action
to
pr/daniel-noland/dpdk-test-eal
June 5, 2026 06:21
daniel-noland
force-pushed
the
pr/daniel-noland/dpdk-test-eal
branch
4 times, most recently
from
df2ad93 to
3741795
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack (11). Base:
pr/daniel-noland/match-action.The ACL crate scaffolding and the software reference backend -- the readable
oracle that defines ACL semantics and the public API surface.
feat(acl): crate scaffolding + software reference backend.The DPDK backend (proven equivalent to this oracle via a differential test)
lands in the next PR.
Review stack (merge bottom -> top):