Reject non-native ABIs in ML seccomp BPF filter

[edsavage]

Summary

Follow-up to a privately reported security finding (details tracked internally): the Linux CSystemCallFilter matched only on seccomp_data.nr, so an x86_64 process could issue int 0x80 (i386) and hit syscall-number collisions — notably i386 socketcall (102) vs allowlisted x86_64 getuid (102). That opened a socket surface the policy intended to block, enabling later HotSpot Attach / JVM escalation in the reported chain.

Fix

Require seccomp_data.arch to match the native ABI (AUDIT_ARCH_X86_64 / AUDIT_ARCH_AARCH64) before any nr allowlist matching. Mismatch returns EACCES immediately. The arch check is a self-contained prefix so existing relative jump offsets in the nr allowlist are unchanged.

Test plan

• Linux CI: existing CSystemCallFilterTest still passes (allowlisted ops work; std::system blocked). Confirmed on CI build 2843 — ml_test_seccomp (test [ML] Reduce peak memory usage and improve estimation for boosted tree training #781) passed on Linux x86_64 and aarch64. The suite was also extended to soft-probe IA32 emulation (int 0x80) and, when available, assert the arch gate denies i386 socketcall after install.
• Manual / security review: confirm i386 int 0x80 syscalls are denied under the new filter. Verified by evaluating the exact BPF program's decision logic against attack vectors (see the verification comment below): the pre-fix filter ALLOWs i386 socketcall(102) (the getuid collision), while the arch-gated filter returns EACCES for it — and for any i386/foreign-arch/x32 syscall — with all native x86_64 allowlisted calls unaffected.

Related: #3078 (__setstate__ pre-load), #3081 (controller PR_SET_DUMPABLE).

[Copilot]

Pull request overview

This PR hardens the Linux seccomp-BPF syscall filter used by ML processes by rejecting non-native/compat ABIs via seccomp_data.arch before any syscall-number allowlist checks, preventing syscall-number collisions (e.g. i386 socketcall vs x86_64 getuid).

Changes:

• Add an ABI/arch gate at the start of the Linux seccomp BPF program (x86_64/aarch64).
• Update public header documentation to describe the new seccomp_data.arch requirement.
• Add a changelog entry and clarify unit-test intent/limitations.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
lib/seccomp/CSystemCallFilter_Linux.cc	Adds seccomp_data.arch check prefix to deny foreign/compat ABIs before syscall allowlist matching.
lib/seccomp/unittest/CSystemCallFilterTest.cc	Documents that the compat-ABI path isn’t currently exercised by the unit test.
include/seccomp/CSystemCallFilter.h	Documents the new native-ABI requirement for the Linux filter.
docs/changelog/3080.yaml	Adds changelog entry for the security hardening.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

lib/seccomp/unittest/CSystemCallFilterTest.cc:219

• Same as above: constrain/zero i386 argument registers for the int $0x80 call so the test remains deterministic and doesn't depend on whatever happened to be in caller-saved registers.

    asm volatile("int $0x80" : "=a"(ret) : "a"(nr) : "memory", "cc");

lib/seccomp/unittest/CSystemCallFilterTest.cc:254

• The IA32 emulation probe currently treats any non-faulting int $0x80 as "i386 entry available". On some kernels/configs, int $0x80 from a 64-bit process can execute the native x86_64 syscall table instead of the i386 compat table; in that case this test would incorrectly fail (expecting socketcall denial) even though no compat ABI is in use. Consider tightening the probe by also issuing syscall number 102 and requiring it to behave like i386 socketcall (i.e. return a negative errno with zeroed args), otherwise skip the compat-ABI assertion.

    long i386GetuidResult{0};
    const bool i386EntryAvailable{tryI386Syscall(I386_NR_GETUID, i386GetuidResult)};
    if (i386EntryAvailable) {
        BOOST_TEST_REQUIRE(i386GetuidResult >= 0);
        LOG_INFO(<< "i386 syscall entry available; will assert arch denial after filter install");

[edsavage]

Test plan item 2 — i386 int 0x80 denial (verification)

Because CI's Apple-Silicon-independent runners can't be reproduced faithfully on my arm64 host (an x86_64 container would run under qemu, where seccomp arch/int 0x80 semantics aren't reliable), I verified the actual BPF program's decision logic deterministically instead: the exact FILTER was compiled with the real x86_64 kernel headers (so __NR_*, AUDIT_ARCH_*, and struct seccomp_data offsets are the true target values) and evaluated in userspace against attack vectors. A pre-fix variant (identical, minus the 3-instruction arch prefix) is included as a control.

AUDIT_ARCH_X86_64=0xc000003e  AUDIT_ARCH_I386=0x40000003  AUDIT_ARCH_AARCH64=0xc00000b7
x86_64 getuid=102  socketcall(i386)=102  execve=59

OLD filter (pre-fix, no arch gate) -- demonstrates the collision:
  [PASS] i386 socketcall(102) [attack]            -> ALLOW          (expect ALLOW)
  [PASS] x86_64 getuid(102)                       -> ALLOW          (expect ALLOW)

FIXED filter (this PR, arch gate) -- attack denied, native preserved:
  [PASS] i386 socketcall(102) [attack]            -> ERRNO(EACCES)  (expect ERRNO(EACCES))
  [PASS] i386 getuid(24)                          -> ERRNO(EACCES)  (expect ERRNO(EACCES))
  [PASS] x86_64 getuid(102)                       -> ALLOW          (expect ALLOW)
  [PASS] x86_64 getpid                            -> ALLOW          (expect ALLOW)
  [PASS] x86_64 read                              -> ALLOW          (expect ALLOW)
  [PASS] x86_64 execve [not allowlisted]          -> ERRNO(EACCES)  (expect ERRNO(EACCES))
  [PASS] x32 getuid (nr|0x40000000)               -> ERRNO(EACCES)  (expect ERRNO(EACCES))
  [PASS] aarch64 arch on x86_64 host              -> ERRNO(EACCES)  (expect ERRNO(EACCES))

RESULT: ALL CHECKS PASSED

Key points:

• Collision reproduced: the pre-fix filter allows i386 socketcall(102) because it collides with allowlisted x86_64 getuid(102) — the #12621 bypass.
• Fix denies it: the arch gate returns EACCES for the i386 call and for any non-native arch; the x32 case remains covered by the pre-existing UPPER_NR_LIMIT check.
• No regression: native x86_64 allowlisted calls still pass; non-allowlisted execve is still denied, so the arch prefix didn't shift the allowlist jump offsets.

This complements the runtime assertion added to CSystemCallFilterTest (which exercises the same denial on CI x86_64 runners that have IA32 emulation).

[edsavage]

Addressed Copilot's latest review in e8c932b: plain // test comments; both int 0x80 probes zero the i386 arg registers (covers suppressed line 219); IA32 detection now skips (not fails) when the compat entry returns -ENOSYS (covers suppressed line 254); header run-on reworded. Inline asm still compiles clean with -Wall -Wextra for x86_64.

[edsavage]

buildkite run_pytorch_tests

[valeriy42]

LGTM. Make sure to backport this and other security changes to 9.5.1, 9.4.5, and 8.19.20.

[github-actions]

💚 All backports created successfully

Status	Branch	Result
✅	8.19
✅	9.4
✅	9.5

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details