jit/interp: tagged-int groundwork, warmup fixes, wasm oversized-module decline#417
Merged
Conversation
|
Warning Review limit reached
Next review available in: 55 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (32)
WalkthroughThis PR introduces tagged-immediate integer representation across pyre-object, pyre-interpreter, and pyre-jit-trace/pyre-jit crates, adding short-circuit checks in type/identity predicates, GC configuration, and JIT-trace guard/unbox emission. It also adds a cached pure-cellvars count to PyCode/PyFrame, wasm backend rejection handling for zero handles, Phase-2 optimizer ownership transfer, and new benchmark scripts. ChangesTagged Integer Support
Cached Pure Cellvars Count
WASM Backend Rejection and Optimizer Ownership
Synthetic FOR_ITER Benchmarks
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
youknowone
changed the title
🤖 Codex parity reviewStatic analysis of this diff vs the local RPython/PyPy sources (commit aeb5d41). 1. Regressions to PyPy parity introduced by this patchNone. 2. Other mismatches introduced by this patchNone. 3. Pre-existing mismatches (already present before this patch)None. 4. Structural adaptations
|
youknowone
force-pushed
the
miframe
branch
2 times, most recently
from
cb5fddb to
1e0a402
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1e0a402e06
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Decline the compile so the metainterp keeps the interpreter fallback — | ||
| // a backend capability limit, reported like any other unsupported shape. | ||
| #[cfg(all(target_arch = "wasm32", not(target_os = "wasi")))] | ||
| if func_handle == 0 { |
There was a problem hiding this comment.
When running the wasm backend through the web glue, this sentinel check is never reached for the oversized/invalid-module case it is meant to handle: majit/majit-backend-wasm/js/jit_glue.js constructs new WebAssembly.Module(bytes) before any catch, and the fallback instantiation can also throw, so a browser CompileError/instantiation error propagates out of glue::compile_module instead of returning 0. The same import is used for bridges, so browser wasm still aborts rather than returning BackendError::Unsupported unless the web glue catches compile/instantiate failures and returns the zero handle.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e5cc9b64d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // code-invariant and cached on the `PyCode` wrapper; `freevars.len()` | ||
| // is a direct read. Falls back to the full walk for stub frames | ||
| // whose `pycode` wrapper carries the `u32::MAX` sentinel. | ||
| let code = unsafe { &*pyframe_get_pycode(self) }; |
There was a problem hiding this comment.
When self.pycode is a wrapper that took the new u32::MAX sentinel path (w_code_new(null) or an unaligned test stub), pyframe_get_pycode(self) returns a null/invalid code pointer, but this line forms &* before w_code_npure_cellvars can return None. That makes the advertised fallback unreachable and turns any ncells()/stack_base() call on such a stub frame into UB or a crash; check the cache/sentinel before dereferencing the raw code pointer.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@majit/majit-backend-wasm/src/lib.rs`:
- Around line 1211-1218: The fail-descriptor registration in the trace/bridge
install flow is happening too early, before the host acceptance checks. Update
the install logic around the func_handle and bridge_slot zero-checks so
register_fail_descrs(...) runs only after those checks pass, and keep it before
any guard cell state is flipped. Apply the same reordering in both affected
paths so unsupported/rejected modules do not publish descriptors for traces or
bridges that will never be installed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 70f85eec-69c0-4284-afb0-b1a06d4a95c7
📒 Files selected for processing (23)
majit/majit-backend-wasm/src/lib.rsmajit/majit-metainterp/src/optimizeopt/unroll.rspyre/bench/synth/foriter_in_while.pypyre/bench/synth/nested_foriter_call.pypyre/bench/synth/nested_foriter_poly.pypyre/bench/synth/nested_foriter_range.pypyre/pyre-interpreter/src/baseobjspace.rspyre/pyre-interpreter/src/objspace/descroperation.rspyre/pyre-interpreter/src/pycode.rspyre/pyre-interpreter/src/pyframe.rspyre/pyre-interpreter/src/typedef.rspyre/pyre-jit-trace/src/helpers.rspyre/pyre-jit-trace/src/jitcode_dispatch.rspyre/pyre-jit-trace/src/state.rspyre/pyre-jit/src/eval.rspyre/pyre-object/src/celldict.rspyre/pyre-object/src/dictmultiobject.rspyre/pyre-object/src/identitydict.rspyre/pyre-object/src/intobject.rspyre/pyre-object/src/iterobject.rspyre/pyre-object/src/listobject.rspyre/pyre-object/src/pyobject.rspyre/pyre-object/src/sliceobject.rs
youknowone
force-pushed
the
miframe
branch
3 times, most recently
from
25bba27 to
f230969
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f23096963a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+3071
to
+3072
| fn is_tagged_immediate(&self, addr: usize) -> bool { | ||
| MiniMarkGC::is_tagged_immediate(self, addr) |
There was a problem hiding this comment.
When taggedpointers is enabled, exposing is_tagged_immediate here is not enough because this same allocator still lets check_is_object call get_actual_typeid, whose foreign-object fallback dereferences gcref.0 at offset 0. Any path that reaches majit_gc::check_is_object or get_actual_typeid without doing the new precheck (for example speculative GC-type validation) will crash or read from an odd tagged-int address; the collector should reject tagged immediates before attempting type-id/classptr reads.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f10b512baa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+1217
to
+1218
| if func_handle == 0 { | ||
| return Err(BackendError::Unsupported( |
There was a problem hiding this comment.
When this new Unsupported branch is taken on the browser wasm backend, register_fail_descrs(&fail_descrs) has already appended this rejected trace's exits to the global fail-index registry above. The loop compile error path is explicitly non-terminal (pyjitpl.rs:5960-5964), so a hot loop whose module is consistently rejected can retry and leak another batch of dead fail descriptors each time before falling back; the bridge path has the same ordering with bridge_descrs. Instantiate/check the zero sentinel before publishing descriptors so rejected modules do not consume global fail-index entries.
Useful? React with 👍 / 👎.
- nested_foriter_range: for(range) × for(range), homogeneous - nested_foriter_poly: for(range) × for(list), polymorphic type flip - nested_foriter_call: for(range) × for(range) + function call - foriter_in_while: while > for(range), the common real-world pattern All four verify JIT correctness against PYRE_JIT=0 oracle. Assisted-by: Claude
…rands numeric_operand_overrides / needs_numeric_unaryop_dispatch ran lookup_where_with_method_cache(type, "__add__"/…) for every numeric binop/unary operand, including exact int/float. That lookup interns the dunder name via box_str_constant(Wtf8::new(name)) (a UTF-8 validation) on each call, up to 4x per binop, on every pre-threshold iteration. Extract numeric_base_type_of_overriding_subclass, gated on is_exact_builtin_instance — the same exact-builtin guard already opening subclass_special_override and is_true. An exact builtin numeric instance cannot override a special method, so it returns None without the string-keyed lookup. Measured on `while i < N: total += i + 1; i += 1` (PYRE_JIT=0, 3M iters, aarch64): ~700-808 -> ~287-300 ns/iter (~2.4x), result byte-identical. 30-fn warmup: ~1.12 -> ~0.72-0.99 ms/fn. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude
Phase 2 of the unroll optimizer is the last reader of self.phase1_emit_ops, snapshot_boxes, snapshot_frame_sizes, snapshot_vable_boxes, snapshot_vref_boxes, snapshot_frame_pcs, and call_pure_results, so hand them to opt_p2 via std::mem::take instead of clone. The GC re-root (replace_compile_snapshot_roots over collect_snapshot_const_ptr_slots) stays after the in-place remap and before optimize, preserving the raw *mut OpRef slot addresses. trace_inputargs stays a clone (re-read at the Phase 2 debug_assert and p2_inputarg_types). Snapshot clone cost measured at ~13us on small traces, so this is not a warmup lever; the change is a strictly-cheaper, safer form. Mirrors the InvalidLoop simple_opt move precedent in pyjitpl.rs. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude
try_inplace_special ran lookup_type_special(lhs, "__iadd__"/…) for every augmented numeric assignment (total += …), a string-keyed method-cache probe that interns the dunder name via box_str_constant. int/long/float/ complex/bool define no in-place special, so the lookup is always a miss. Gate on is_exact_numeric_builtin (is_exact_builtin_instance + numeric storage): exact numerics skip straight to the binary op. Builtin sequences (list/bytearray) define __iadd__/__imul__ and subclasses may override the in-place slot, so both fall through to the lookup unchanged. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~296-300 -> ~240-257 ns/iter, cumulative from baseline ~700-808 -> ~3.0x. Parity verified vs py3.14 (JIT on/off): list/bytearray += keep identity, int/float/bigint/complex += correct, IAddInt/list-subclass __iadd__ overrides dispatch, int += float reflected fall-through. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude
PyFrame::ncells() ran npure_cellvars(code) — an O(cellvars × varnames) overlap count — on every pop_value/peek_at (twice per binop). The count is code-invariant, so compute it once in w_code_new_with_hidden_applevel and store it as a u32 on the PyCode wrapper (u32::MAX sentinel for null/unaligned code_ptr stubs). PyFrame::ncells() reads it via w_code_npure_cellvars and adds freevars.len(), falling back to the full walk for stub frames. The new field sits after fast_natural_arity, before the cache pointers; CODE_PTR_OFFSET / CODE_W_GLOBALS_OFFSET (the only JIT-baked offsets) are unaffected, and a u32 is not a GC-walked reference. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~240-257 -> ~227-229 ns/iter; npure_cellvars gone from the profile. Cumulative from baseline ~700-808 -> ~3.2x. Parity verified vs py3.14 including a closure loop (non-empty cellvars). check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude
py_type_check, ll_isinstance, is_exact_type, and is_exact_builtin_instance short-circuit on a tagged immediate before the ob_type deref. Gated on tagged_int::CAN_BE_TAGGED (const false), so the guards fold away and the deref path is unchanged until enablement. Assisted-by: Claude
…tagged ints - is_seq_iter short-circuits on a tagged immediate before the ob_type deref, so the GC value-stack walker (walk_raw_seq_iter_roots) never dereferences one; the other walk_raw_* predicates route through the already-tag-guarded py_type_check/ll_isinstance. - w_int_new returns a tagged immediate for values that fit, instead of allocating; w_int_new_unique documented as never-tag (subclass identity); the JIT virtual-reconstruction allocators documented as stay-boxed. - The GC constructor mirrors CAN_BE_TAGGED into GcConfig.taggedpointers so the collector-core immediate guards go live in lockstep with the maker. All gated on tagged_int::CAN_BE_TAGGED (const false); behavior unchanged until enablement. dynasm+cranelift 187/187. Assisted-by: Claude
is_w's int branch already handles tagged immediates without change: two equal-valued immediates match the ptr::eq fast path, and an immediate vs a boxed int of the same value compares equal by value through the tag-aware w_int_get_value. No behavioral change (CAN_BE_TAGGED still false). Assisted-by: Claude
jit_compile_wasm returns handle 0 when the wasm runtime rejects the emitted module (e.g. a function body exceeding the parser's size limit, which a trace within trace_limit can still produce after the optimizer peels/unrolls the loop). Return BackendError::Unsupported in compile_loop and compile_bridge when compile_module returns 0, so execute_token does not dispatch dead table slot 0 and leave frame[0] unwritten. Assisted-by: Claude
Eight object→type accessors and dict/slice value-readers dereferenced (*obj).ob_type / .w_class after only a null check, faulting on a tagged immediate. A PYRE_JIT=0 flip of CAN_BE_TAGGED=true SIGSEGVs at stdlib bootstrap through these sites, which the type-probe primitives hardened earlier do not cover (they route through py_type_check; these do a direct field read). Guarded, all gated on CAN_BE_TAGGED (default false, inert): - typedef::r#type returns gettypefor(&INT_TYPE) for a tagged int - celldict::unwrap_cell returns the immediate unchanged - listobject::is_plain_int1 returns true (always an exact int) - dictmultiobject::key_compares_by_identity / identitydict::is_correct_type return false (an int compares by value) - sliceobject::is_slice returns false - baseobjspace::object_getattr_miss reads the class via typedef::r#type typedef::init_type_type and object_getattr_miss error messages now name the value's type via the tag-safe typedef::r#type instead of a raw (*obj).ob_type read. Verified with CAN_BE_TAGGED=true, PYRE_JIT=0: all 177 synth scripts run crash-free and byte-identical to the boxed build (object-repr addresses aside). Inert build: check.py 187/187 dynasm + cranelift. Assisted-by: Claude
is_trace_plain_int dereferences (*obj).w_class after a py_type_check that already returns true for a tagged immediate, faulting when the trace-entry value classifier (ConcreteValue::from_pyobj) inspects a tagged int on the value stack. A tagged immediate is always an exact int, so return true before the deref. Gated on CAN_BE_TAGGED (default false, inert). Assisted-by: Claude
walk_module_value_slot dereferenced (*w_value).ob_type without the tagged-immediate skip its sibling unwrap_cell (celldict.rs:206) already carries. A tagged int in a module-dict value slot faulted the ob_type read during a minor-GC root walk (walk_pyframe_roots -> walk_module_dicts_gc -> w_module_dict_walk_gc_cells). Forward the slot as-is on a tagged immediate. Gated on CAN_BE_TAGGED (default false). Assisted-by: Claude
The walker and MIFrame int-unbox paths emitted GuardClass + GetfieldGcI unconditionally, dereferencing the operand header. A tagged immediate has no header, so those loads fault once ints tag. Before the class guard, dispatch on the operand's observed concrete: a tagged immediate emits CastPtrToInt + IntAnd(1) + GuardTrue then IntRshift(1) arithmetic untag (rtagged.py ll_unboxed_to_int) and skips the class guard/getfield entirely; a boxed operand emits the same low-bit test with GuardFalse, then falls through to the unchanged GuardClass + GetfieldGcI. int_or_bool_unbox_type_descr takes the INT arm for a tagged immediate before the is_bool header deref. All gated on CAN_BE_TAGGED (default false); no new opcodes, no backend changes. Assisted-by: Claude
…ated CAN_BE_TAGGED) Assisted-by: Claude
…f (gated taggedpointers) Add a public `majit_gc::is_tagged_immediate(addr)` free shim delegating through a new `ACTIVE_IS_TAGGED_IMMEDIATE` thread-local callback, wired into `ActiveGcGuardHooks` and installed by the dynasm, cranelift, and wasm backends alongside `check_is_object`. The installed callbacks reach the active GC's `GcAllocator::is_tagged_immediate`, which MiniMarkGC answers from `config.taggedpointers && (addr & 1 == 1)` and other backends default to false. In `PtrInfo::get_known_class`'s `Constant` arm, return `None` for a tagged-immediate constant before the `check_is_object` / `cls_of_gcref` offset-0 reads, so the odd-valued address is not dereferenced as an object header. Inert while taggedpointers is off. Assisted-by: Claude
…ntAddOvf(x,x),1),1) -> x (gated taggedpointers) Add an `IntRshift` arm to `OptRewrite::propagate_forward` that, when the active GC has `taggedpointers` enabled, folds the box/unbox round-trip `IntRshift(IntOr(IntAddOvf(x, x), 1), 1)` to `x` via `make_equal_to` + `Remove`. The arm peels `IntOr` then `IntAddOvf` with `get_producing_op`, requires both the `IntOr` and `IntRshift` second args to be the constant `1`, and checks the `IntAddOvf` args with `same_box`. The box ops stay; only the redundant unbox is removed. Gated on the new `majit_gc::taggedpointers_enabled()` accessor (probes the installed `is_tagged_immediate` callback with an odd sentinel), so it returns `PassOn` and is byte-identical with the config off. Add `majit_gc::taggedpointers_enabled()` free shim. Assisted-by: Claude
…_payload, guard_int_object_value) (gated CAN_BE_TAGGED) Assisted-by: Claude
…s on a tagged-immediate operand (gated CAN_BE_TAGGED) Assisted-by: Claude
…ated taggedpointers) Assisted-by: Claude
x86_64-windows enforces the stack guard page strictly: a trace body whose prologue moves %rsp past the 4096-byte guard in a single `sub` (deep nested-FOR_ITER bodies reach 150KB-540KB fixed_frame_storage) skips the guard page, so the first write below it faults 0xC0000005. Linux/macOS grow the stack on the fault instead. Enable inline probestack in Compiler::new so the prologue touches each page. Scoped to Windows via the runtime `cfg!(windows)` macro rather than a `#[cfg]` attribute so the flag_builder.set calls compile on every platform; non-Windows codegen is unchanged (verified cranelift check.py 159/159 on aarch64/x86_64 darwin). probestack_size_log2 defaults to 12 (= the 4096 guard page). Assisted-by: Claude
…ed CAN_BE_TAGGED) Assisted-by: Claude
…nters) Assisted-by: Claude
youknowone
added a commit
that referenced
this pull request
Jul 9, 2026
…e decline (#417) * synth: add nested FOR_ITER test coverage - nested_foriter_range: for(range) × for(range), homogeneous - nested_foriter_poly: for(range) × for(list), polymorphic type flip - nested_foriter_call: for(range) × for(range) + function call - foriter_in_while: while > for(range), the common real-world pattern All four verify JIT correctness against PYRE_JIT=0 oracle. Assisted-by: Claude * interp: skip dunder method-cache lookup for exact-builtin numeric operands numeric_operand_overrides / needs_numeric_unaryop_dispatch ran lookup_where_with_method_cache(type, "__add__"/…) for every numeric binop/unary operand, including exact int/float. That lookup interns the dunder name via box_str_constant(Wtf8::new(name)) (a UTF-8 validation) on each call, up to 4x per binop, on every pre-threshold iteration. Extract numeric_base_type_of_overriding_subclass, gated on is_exact_builtin_instance — the same exact-builtin guard already opening subclass_special_override and is_true. An exact builtin numeric instance cannot override a special method, so it returns None without the string-keyed lookup. Measured on `while i < N: total += i + 1; i += 1` (PYRE_JIT=0, 3M iters, aarch64): ~700-808 -> ~287-300 ns/iter (~2.4x), result byte-identical. 30-fn warmup: ~1.12 -> ~0.72-0.99 ms/fn. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * jit: move (not clone) Phase 2 snapshot/emit buffers in unroll optimizer Phase 2 of the unroll optimizer is the last reader of self.phase1_emit_ops, snapshot_boxes, snapshot_frame_sizes, snapshot_vable_boxes, snapshot_vref_boxes, snapshot_frame_pcs, and call_pure_results, so hand them to opt_p2 via std::mem::take instead of clone. The GC re-root (replace_compile_snapshot_roots over collect_snapshot_const_ptr_slots) stays after the in-place remap and before optimize, preserving the raw *mut OpRef slot addresses. trace_inputargs stays a clone (re-read at the Phase 2 debug_assert and p2_inputarg_types). Snapshot clone cost measured at ~13us on small traces, so this is not a warmup lever; the change is a strictly-cheaper, safer form. Mirrors the InvalidLoop simple_opt move precedent in pyjitpl.rs. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * interp: skip in-place special lookup for exact-builtin numeric lhs try_inplace_special ran lookup_type_special(lhs, "__iadd__"/…) for every augmented numeric assignment (total += …), a string-keyed method-cache probe that interns the dunder name via box_str_constant. int/long/float/ complex/bool define no in-place special, so the lookup is always a miss. Gate on is_exact_numeric_builtin (is_exact_builtin_instance + numeric storage): exact numerics skip straight to the binary op. Builtin sequences (list/bytearray) define __iadd__/__imul__ and subclasses may override the in-place slot, so both fall through to the lookup unchanged. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~296-300 -> ~240-257 ns/iter, cumulative from baseline ~700-808 -> ~3.0x. Parity verified vs py3.14 (JIT on/off): list/bytearray += keep identity, int/float/bigint/complex += correct, IAddInt/list-subclass __iadd__ overrides dispatch, int += float reflected fall-through. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * interp: cache npure_cellvars on PyCode to avoid per-stack-op recompute PyFrame::ncells() ran npure_cellvars(code) — an O(cellvars × varnames) overlap count — on every pop_value/peek_at (twice per binop). The count is code-invariant, so compute it once in w_code_new_with_hidden_applevel and store it as a u32 on the PyCode wrapper (u32::MAX sentinel for null/unaligned code_ptr stubs). PyFrame::ncells() reads it via w_code_npure_cellvars and adds freevars.len(), falling back to the full walk for stub frames. The new field sits after fast_natural_arity, before the cache pointers; CODE_PTR_OFFSET / CODE_W_GLOBALS_OFFSET (the only JIT-baked offsets) are unaffected, and a u32 is not a GC-walked reference. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~240-257 -> ~227-229 ns/iter; npure_cellvars gone from the profile. Cumulative from baseline ~700-808 -> ~3.2x. Parity verified vs py3.14 including a closure loop (non-empty cellvars). check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * object: tag-guard type-probe primitives ahead of tagged-int enablement py_type_check, ll_isinstance, is_exact_type, and is_exact_builtin_instance short-circuit on a tagged immediate before the ob_type deref. Gated on tagged_int::CAN_BE_TAGGED (const false), so the guards fold away and the deref path is unchanged until enablement. Assisted-by: Claude * object/jit: tag-aware GC walker, int maker, and GC config wiring for tagged ints - is_seq_iter short-circuits on a tagged immediate before the ob_type deref, so the GC value-stack walker (walk_raw_seq_iter_roots) never dereferences one; the other walk_raw_* predicates route through the already-tag-guarded py_type_check/ll_isinstance. - w_int_new returns a tagged immediate for values that fit, instead of allocating; w_int_new_unique documented as never-tag (subclass identity); the JIT virtual-reconstruction allocators documented as stay-boxed. - The GC constructor mirrors CAN_BE_TAGGED into GcConfig.taggedpointers so the collector-core immediate guards go live in lockstep with the maker. All gated on tagged_int::CAN_BE_TAGGED (const false); behavior unchanged until enablement. dynasm+cranelift 187/187. Assisted-by: Claude * objspace: document tagged-int identity in is_w is_w's int branch already handles tagged immediates without change: two equal-valued immediates match the ptr::eq fast path, and an immediate vs a boxed int of the same value compares equal by value through the tag-aware w_int_get_value. No behavioral change (CAN_BE_TAGGED still false). Assisted-by: Claude * majit-wasm: decline compile when host rejects the emitted module jit_compile_wasm returns handle 0 when the wasm runtime rejects the emitted module (e.g. a function body exceeding the parser's size limit, which a trace within trace_limit can still produce after the optimizer peels/unrolls the loop). Return BackendError::Unsupported in compile_loop and compile_bridge when compile_module returns 0, so execute_token does not dispatch dead table slot 0 and leave frame[0] unwritten. Assisted-by: Claude * object/interp: tag-guard interpreter raw-deref sites for tagged ints Eight object→type accessors and dict/slice value-readers dereferenced (*obj).ob_type / .w_class after only a null check, faulting on a tagged immediate. A PYRE_JIT=0 flip of CAN_BE_TAGGED=true SIGSEGVs at stdlib bootstrap through these sites, which the type-probe primitives hardened earlier do not cover (they route through py_type_check; these do a direct field read). Guarded, all gated on CAN_BE_TAGGED (default false, inert): - typedef::r#type returns gettypefor(&INT_TYPE) for a tagged int - celldict::unwrap_cell returns the immediate unchanged - listobject::is_plain_int1 returns true (always an exact int) - dictmultiobject::key_compares_by_identity / identitydict::is_correct_type return false (an int compares by value) - sliceobject::is_slice returns false - baseobjspace::object_getattr_miss reads the class via typedef::r#type typedef::init_type_type and object_getattr_miss error messages now name the value's type via the tag-safe typedef::r#type instead of a raw (*obj).ob_type read. Verified with CAN_BE_TAGGED=true, PYRE_JIT=0: all 177 synth scripts run crash-free and byte-identical to the boxed build (object-repr addresses aside). Inert build: check.py 187/187 dynasm + cranelift. Assisted-by: Claude * jit-trace: tag-guard is_trace_plain_int trace-entry classifier is_trace_plain_int dereferences (*obj).w_class after a py_type_check that already returns true for a tagged immediate, faulting when the trace-entry value classifier (ConcreteValue::from_pyobj) inspects a tagged int on the value stack. A tagged immediate is always an exact int, so return true before the deref. Gated on CAN_BE_TAGGED (default false, inert). Assisted-by: Claude * object: tag-guard walk_module_value_slot GC root walker walk_module_value_slot dereferenced (*w_value).ob_type without the tagged-immediate skip its sibling unwrap_cell (celldict.rs:206) already carries. A tagged int in a module-dict value slot faulted the ob_type read during a minor-GC root walk (walk_pyframe_roots -> walk_module_dicts_gc -> w_module_dict_walk_gc_cells). Forward the slot as-is on a tagged immediate. Gated on CAN_BE_TAGGED (default false). Assisted-by: Claude * jit-trace: emit primitive-IR tag branch in int unbox paths The walker and MIFrame int-unbox paths emitted GuardClass + GetfieldGcI unconditionally, dereferencing the operand header. A tagged immediate has no header, so those loads fault once ints tag. Before the class guard, dispatch on the operand's observed concrete: a tagged immediate emits CastPtrToInt + IntAnd(1) + GuardTrue then IntRshift(1) arithmetic untag (rtagged.py ll_unboxed_to_int) and skips the class guard/getfield entirely; a boxed operand emits the same low-bit test with GuardFalse, then falls through to the unchanged GuardClass + GetfieldGcI. int_or_bool_unbox_type_descr takes the INT arm for a tagged immediate before the is_bool header deref. All gated on CAN_BE_TAGGED (default false); no new opcodes, no backend changes. Assisted-by: Claude * jit-trace: box int-binop result as a tagged immediate when it fits (gated CAN_BE_TAGGED) Assisted-by: Claude * optimizer: recognize a tagged immediate's class without a header deref (gated taggedpointers) Add a public `majit_gc::is_tagged_immediate(addr)` free shim delegating through a new `ACTIVE_IS_TAGGED_IMMEDIATE` thread-local callback, wired into `ActiveGcGuardHooks` and installed by the dynasm, cranelift, and wasm backends alongside `check_is_object`. The installed callbacks reach the active GC's `GcAllocator::is_tagged_immediate`, which MiniMarkGC answers from `config.taggedpointers && (addr & 1 == 1)` and other backends default to false. In `PtrInfo::get_known_class`'s `Constant` arm, return `None` for a tagged-immediate constant before the `check_is_object` / `cls_of_gcref` offset-0 reads, so the odd-valued address is not dereferenced as an object header. Inert while taggedpointers is off. Assisted-by: Claude * optimizer: fold the tagged-int box/unbox round-trip IntRshift(IntOr(IntAddOvf(x,x),1),1) -> x (gated taggedpointers) Add an `IntRshift` arm to `OptRewrite::propagate_forward` that, when the active GC has `taggedpointers` enabled, folds the box/unbox round-trip `IntRshift(IntOr(IntAddOvf(x, x), 1), 1)` to `x` via `make_equal_to` + `Remove`. The arm peels `IntOr` then `IntAddOvf` with `get_producing_op`, requires both the `IntOr` and `IntRshift` second args to be the constant `1`, and checks the `IntAddOvf` args with `same_box`. The box ops stay; only the redundant unbox is removed. Gated on the new `majit_gc::taggedpointers_enabled()` accessor (probes the installed `is_tagged_immediate` callback with an odd sentinel), so it returns `PassOn` and is byte-identical with the config off. Add `majit_gc::taggedpointers_enabled()` free shim. Assisted-by: Claude * jit-trace: tag-guard the int-payload unbox helpers (trace_guarded_int_payload, guard_int_object_value) (gated CAN_BE_TAGGED) Assisted-by: Claude * jit-trace: decline the spec_ii tuple and int-storage list-append folds on a tagged-immediate operand (gated CAN_BE_TAGGED) Assisted-by: Claude * jit: tag-guard the cls_of_gcref typeptr read for tagged immediates (gated taggedpointers) Assisted-by: Claude * cranelift: enable inline stack probes on Windows x86_64-windows enforces the stack guard page strictly: a trace body whose prologue moves %rsp past the 4096-byte guard in a single `sub` (deep nested-FOR_ITER bodies reach 150KB-540KB fixed_frame_storage) skips the guard page, so the first write below it faults 0xC0000005. Linux/macOS grow the stack on the fault instead. Enable inline probestack in Compiler::new so the prologue touches each page. Scoped to Windows via the runtime `cfg!(windows)` macro rather than a `#[cfg]` attribute so the flag_builder.set calls compile on every platform; non-Windows codegen is unchanged (verified cranelift check.py 159/159 on aarch64/x86_64 darwin). probestack_size_log2 defaults to 12 (= the 4096 guard page). Assisted-by: Claude * interp: read binop-error operand type names via tag-safe ll_type (gated CAN_BE_TAGGED) Assisted-by: Claude * gc: return false from can_move for tagged immediates (gated taggedpointers) Assisted-by: Claude
youknowone
added a commit
that referenced
this pull request
Jul 9, 2026
…e decline (#417) * synth: add nested FOR_ITER test coverage - nested_foriter_range: for(range) × for(range), homogeneous - nested_foriter_poly: for(range) × for(list), polymorphic type flip - nested_foriter_call: for(range) × for(range) + function call - foriter_in_while: while > for(range), the common real-world pattern All four verify JIT correctness against PYRE_JIT=0 oracle. Assisted-by: Claude * interp: skip dunder method-cache lookup for exact-builtin numeric operands numeric_operand_overrides / needs_numeric_unaryop_dispatch ran lookup_where_with_method_cache(type, "__add__"/…) for every numeric binop/unary operand, including exact int/float. That lookup interns the dunder name via box_str_constant(Wtf8::new(name)) (a UTF-8 validation) on each call, up to 4x per binop, on every pre-threshold iteration. Extract numeric_base_type_of_overriding_subclass, gated on is_exact_builtin_instance — the same exact-builtin guard already opening subclass_special_override and is_true. An exact builtin numeric instance cannot override a special method, so it returns None without the string-keyed lookup. Measured on `while i < N: total += i + 1; i += 1` (PYRE_JIT=0, 3M iters, aarch64): ~700-808 -> ~287-300 ns/iter (~2.4x), result byte-identical. 30-fn warmup: ~1.12 -> ~0.72-0.99 ms/fn. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * jit: move (not clone) Phase 2 snapshot/emit buffers in unroll optimizer Phase 2 of the unroll optimizer is the last reader of self.phase1_emit_ops, snapshot_boxes, snapshot_frame_sizes, snapshot_vable_boxes, snapshot_vref_boxes, snapshot_frame_pcs, and call_pure_results, so hand them to opt_p2 via std::mem::take instead of clone. The GC re-root (replace_compile_snapshot_roots over collect_snapshot_const_ptr_slots) stays after the in-place remap and before optimize, preserving the raw *mut OpRef slot addresses. trace_inputargs stays a clone (re-read at the Phase 2 debug_assert and p2_inputarg_types). Snapshot clone cost measured at ~13us on small traces, so this is not a warmup lever; the change is a strictly-cheaper, safer form. Mirrors the InvalidLoop simple_opt move precedent in pyjitpl.rs. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * interp: skip in-place special lookup for exact-builtin numeric lhs try_inplace_special ran lookup_type_special(lhs, "__iadd__"/…) for every augmented numeric assignment (total += …), a string-keyed method-cache probe that interns the dunder name via box_str_constant. int/long/float/ complex/bool define no in-place special, so the lookup is always a miss. Gate on is_exact_numeric_builtin (is_exact_builtin_instance + numeric storage): exact numerics skip straight to the binary op. Builtin sequences (list/bytearray) define __iadd__/__imul__ and subclasses may override the in-place slot, so both fall through to the lookup unchanged. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~296-300 -> ~240-257 ns/iter, cumulative from baseline ~700-808 -> ~3.0x. Parity verified vs py3.14 (JIT on/off): list/bytearray += keep identity, int/float/bigint/complex += correct, IAddInt/list-subclass __iadd__ overrides dispatch, int += float reflected fall-through. check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * interp: cache npure_cellvars on PyCode to avoid per-stack-op recompute PyFrame::ncells() ran npure_cellvars(code) — an O(cellvars × varnames) overlap count — on every pop_value/peek_at (twice per binop). The count is code-invariant, so compute it once in w_code_new_with_hidden_applevel and store it as a u32 on the PyCode wrapper (u32::MAX sentinel for null/unaligned code_ptr stubs). PyFrame::ncells() reads it via w_code_npure_cellvars and adds freevars.len(), falling back to the full walk for stub frames. The new field sits after fast_natural_arity, before the cache pointers; CODE_PTR_OFFSET / CODE_W_GLOBALS_OFFSET (the only JIT-baked offsets) are unaffected, and a u32 is not a GC-walked reference. Measured on `while i < n: total += i + 1; i += 1` (PYRE_JIT=0, aarch64): ~240-257 -> ~227-229 ns/iter; npure_cellvars gone from the profile. Cumulative from baseline ~700-808 -> ~3.2x. Parity verified vs py3.14 including a closure loop (non-empty cellvars). check.py dynasm 187/187, cranelift 187/187. Assisted-by: Claude * object: tag-guard type-probe primitives ahead of tagged-int enablement py_type_check, ll_isinstance, is_exact_type, and is_exact_builtin_instance short-circuit on a tagged immediate before the ob_type deref. Gated on tagged_int::CAN_BE_TAGGED (const false), so the guards fold away and the deref path is unchanged until enablement. Assisted-by: Claude * object/jit: tag-aware GC walker, int maker, and GC config wiring for tagged ints - is_seq_iter short-circuits on a tagged immediate before the ob_type deref, so the GC value-stack walker (walk_raw_seq_iter_roots) never dereferences one; the other walk_raw_* predicates route through the already-tag-guarded py_type_check/ll_isinstance. - w_int_new returns a tagged immediate for values that fit, instead of allocating; w_int_new_unique documented as never-tag (subclass identity); the JIT virtual-reconstruction allocators documented as stay-boxed. - The GC constructor mirrors CAN_BE_TAGGED into GcConfig.taggedpointers so the collector-core immediate guards go live in lockstep with the maker. All gated on tagged_int::CAN_BE_TAGGED (const false); behavior unchanged until enablement. dynasm+cranelift 187/187. Assisted-by: Claude * objspace: document tagged-int identity in is_w is_w's int branch already handles tagged immediates without change: two equal-valued immediates match the ptr::eq fast path, and an immediate vs a boxed int of the same value compares equal by value through the tag-aware w_int_get_value. No behavioral change (CAN_BE_TAGGED still false). Assisted-by: Claude * majit-wasm: decline compile when host rejects the emitted module jit_compile_wasm returns handle 0 when the wasm runtime rejects the emitted module (e.g. a function body exceeding the parser's size limit, which a trace within trace_limit can still produce after the optimizer peels/unrolls the loop). Return BackendError::Unsupported in compile_loop and compile_bridge when compile_module returns 0, so execute_token does not dispatch dead table slot 0 and leave frame[0] unwritten. Assisted-by: Claude * object/interp: tag-guard interpreter raw-deref sites for tagged ints Eight object→type accessors and dict/slice value-readers dereferenced (*obj).ob_type / .w_class after only a null check, faulting on a tagged immediate. A PYRE_JIT=0 flip of CAN_BE_TAGGED=true SIGSEGVs at stdlib bootstrap through these sites, which the type-probe primitives hardened earlier do not cover (they route through py_type_check; these do a direct field read). Guarded, all gated on CAN_BE_TAGGED (default false, inert): - typedef::r#type returns gettypefor(&INT_TYPE) for a tagged int - celldict::unwrap_cell returns the immediate unchanged - listobject::is_plain_int1 returns true (always an exact int) - dictmultiobject::key_compares_by_identity / identitydict::is_correct_type return false (an int compares by value) - sliceobject::is_slice returns false - baseobjspace::object_getattr_miss reads the class via typedef::r#type typedef::init_type_type and object_getattr_miss error messages now name the value's type via the tag-safe typedef::r#type instead of a raw (*obj).ob_type read. Verified with CAN_BE_TAGGED=true, PYRE_JIT=0: all 177 synth scripts run crash-free and byte-identical to the boxed build (object-repr addresses aside). Inert build: check.py 187/187 dynasm + cranelift. Assisted-by: Claude * jit-trace: tag-guard is_trace_plain_int trace-entry classifier is_trace_plain_int dereferences (*obj).w_class after a py_type_check that already returns true for a tagged immediate, faulting when the trace-entry value classifier (ConcreteValue::from_pyobj) inspects a tagged int on the value stack. A tagged immediate is always an exact int, so return true before the deref. Gated on CAN_BE_TAGGED (default false, inert). Assisted-by: Claude * object: tag-guard walk_module_value_slot GC root walker walk_module_value_slot dereferenced (*w_value).ob_type without the tagged-immediate skip its sibling unwrap_cell (celldict.rs:206) already carries. A tagged int in a module-dict value slot faulted the ob_type read during a minor-GC root walk (walk_pyframe_roots -> walk_module_dicts_gc -> w_module_dict_walk_gc_cells). Forward the slot as-is on a tagged immediate. Gated on CAN_BE_TAGGED (default false). Assisted-by: Claude * jit-trace: emit primitive-IR tag branch in int unbox paths The walker and MIFrame int-unbox paths emitted GuardClass + GetfieldGcI unconditionally, dereferencing the operand header. A tagged immediate has no header, so those loads fault once ints tag. Before the class guard, dispatch on the operand's observed concrete: a tagged immediate emits CastPtrToInt + IntAnd(1) + GuardTrue then IntRshift(1) arithmetic untag (rtagged.py ll_unboxed_to_int) and skips the class guard/getfield entirely; a boxed operand emits the same low-bit test with GuardFalse, then falls through to the unchanged GuardClass + GetfieldGcI. int_or_bool_unbox_type_descr takes the INT arm for a tagged immediate before the is_bool header deref. All gated on CAN_BE_TAGGED (default false); no new opcodes, no backend changes. Assisted-by: Claude * jit-trace: box int-binop result as a tagged immediate when it fits (gated CAN_BE_TAGGED) Assisted-by: Claude * optimizer: recognize a tagged immediate's class without a header deref (gated taggedpointers) Add a public `majit_gc::is_tagged_immediate(addr)` free shim delegating through a new `ACTIVE_IS_TAGGED_IMMEDIATE` thread-local callback, wired into `ActiveGcGuardHooks` and installed by the dynasm, cranelift, and wasm backends alongside `check_is_object`. The installed callbacks reach the active GC's `GcAllocator::is_tagged_immediate`, which MiniMarkGC answers from `config.taggedpointers && (addr & 1 == 1)` and other backends default to false. In `PtrInfo::get_known_class`'s `Constant` arm, return `None` for a tagged-immediate constant before the `check_is_object` / `cls_of_gcref` offset-0 reads, so the odd-valued address is not dereferenced as an object header. Inert while taggedpointers is off. Assisted-by: Claude * optimizer: fold the tagged-int box/unbox round-trip IntRshift(IntOr(IntAddOvf(x,x),1),1) -> x (gated taggedpointers) Add an `IntRshift` arm to `OptRewrite::propagate_forward` that, when the active GC has `taggedpointers` enabled, folds the box/unbox round-trip `IntRshift(IntOr(IntAddOvf(x, x), 1), 1)` to `x` via `make_equal_to` + `Remove`. The arm peels `IntOr` then `IntAddOvf` with `get_producing_op`, requires both the `IntOr` and `IntRshift` second args to be the constant `1`, and checks the `IntAddOvf` args with `same_box`. The box ops stay; only the redundant unbox is removed. Gated on the new `majit_gc::taggedpointers_enabled()` accessor (probes the installed `is_tagged_immediate` callback with an odd sentinel), so it returns `PassOn` and is byte-identical with the config off. Add `majit_gc::taggedpointers_enabled()` free shim. Assisted-by: Claude * jit-trace: tag-guard the int-payload unbox helpers (trace_guarded_int_payload, guard_int_object_value) (gated CAN_BE_TAGGED) Assisted-by: Claude * jit-trace: decline the spec_ii tuple and int-storage list-append folds on a tagged-immediate operand (gated CAN_BE_TAGGED) Assisted-by: Claude * jit: tag-guard the cls_of_gcref typeptr read for tagged immediates (gated taggedpointers) Assisted-by: Claude * cranelift: enable inline stack probes on Windows x86_64-windows enforces the stack guard page strictly: a trace body whose prologue moves %rsp past the 4096-byte guard in a single `sub` (deep nested-FOR_ITER bodies reach 150KB-540KB fixed_frame_storage) skips the guard page, so the first write below it faults 0xC0000005. Linux/macOS grow the stack on the fault instead. Enable inline probestack in Compiler::new so the prologue touches each page. Scoped to Windows via the runtime `cfg!(windows)` macro rather than a `#[cfg]` attribute so the flag_builder.set calls compile on every platform; non-Windows codegen is unchanged (verified cranelift check.py 159/159 on aarch64/x86_64 darwin). probestack_size_log2 defaults to 12 (= the 4096 guard page). Assisted-by: Claude * interp: read binop-error operand type names via tag-safe ll_type (gated CAN_BE_TAGGED) Assisted-by: Claude * gc: return false from can_move for tagged immediates (gated taggedpointers) Assisted-by: Claude
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Branch
miframe— 9 commits ahead ofmain, grouped by theme.Tagged-int enablement groundwork (inert,
CAN_BE_TAGGED=false)Slices 1–4 of the tagged-int epic. All land harmlessly behind the disabled flag until a later enablement flip; no behavior change yet.
object: tag-guard type-probe primitives ahead of tagged-int enablementobject/jit: tag-aware GC walker, int maker, and GC config wiring for tagged intsobjspace: document tagged-int identity in is_wInterpreter warmup fixes (follow-up to gh#394)
interp: cache npure_cellvars on PyCode to avoid per-stack-op recomputeinterp: skip in-place special lookup for exact-builtin numeric lhsinterp: skip dunder method-cache lookup for exact-builtin numeric operandsJIT
jit: move (not clone) Phase 2 snapshot/emit buffers in unroll optimizersynth: add nested FOR_ITER test coveragemajit-wasm: decline compile when host rejects the emitted module—jit_compile_wasmreturns handle 0 when the wasm runtime rejects an emitted module (e.g. a function body exceeding the parser's size limit, which a trace withintrace_limitcan still produce after the optimizer peels/unrolls the loop).compile_loop/compile_bridgenow returnBackendError::Unsupportedon handle 0 soexecute_tokendoes not dispatch dead table slot 0 and leaveframe[0]unwritten (was surfacing asconsume_vable_info: get_total_size=15 != vable_size-1=10onnested_foriter_call).Verification
check.py: dynasm 187/187, cranelift 187/187, wasm 187/187nested_foriter_call→ 9940050000 (was panicking on wasm)Summary by CodeRabbit
Bug Fixes
New Features
Performance