Skip to content

Make module subtyping checks proportional to the expected signature, cache repeated field checks - #22281

Open
JasonGross wants to merge 3 commits into
rocq-prover:masterfrom
JasonGross:fix-22279-include-self-subtyping
Open

Make module subtyping checks proportional to the expected signature, cache repeated field checks#22281
JasonGross wants to merge 3 commits into
rocq-prover:masterfrom
JasonGross:fix-22279-include-self-subtyping

Conversation

@JasonGross

@JasonGross JasonGross commented Jul 16, 2026

Copy link
Copy Markdown
Member

Fixes #22279 (quadratic time when chaining module types with <+).

Written by Claude (Anthropic AI) at the request of and under the supervision of @JasonGross.

Problem

Every <+ of a functor module type triggers the "Include Self" subtyping check of the whole accumulated signature against the functor's parameter signature, twice (once with universe inference in Declaremods, once in the kernel). Subtyping.check_subtypes did two things proportional to the size of the subtype side, even when the expected signature needs few or no fields checked:

  1. strengthen eagerly rebuilt every accumulated constant field;
  2. check_signatures eagerly built a label map (make_labmap) over the whole accumulated signature.

For a chain of n interfaces this is O(n²) with significant constants. Instrumented at n=800 (10 parameters per interface, empty parameter signature): 3.3s of CPU in the single Module Type ... := I1 <+ ... <+ In command, ~1.6s in strengthen, ~2.3s in make_labmap, across 1600 check_subtypes calls.

Fix

Commit 1 — make each check proportional to the expected signature. All non-checker callers of check_subtypes guarantee that the subtype side's fields are already in the environment, unsubstituted (the invariant documented in check_structure for the nargs = 0 case). A new direct flag makes check_signatures look fields up in the environment by name on demand, strengthening constant and submodule fields individually at lookup time (mirroring exactly what Modops.strengthen produces per field). Lookups fall back to the lazily-built label map whenever the label does not name a field (constructor names, secondary inductive types of a block), preserving existing behavior, error messages, the checker paths, and the recursive cases where the environment holds substituted fields.

Commit 2 — cache successful field checks. Prefix-shaped chains (each stage parameterized by the whole chain so far) and repeated applications of the same functor legitimately perform quadratically many field checks. Successful checks of a constant field pair are now cached under conditions that make the verdict a function of the pair alone, all checked at run time: both substitutions leave the bodies physically unchanged, and the check produced no new universe constraints (the returned state is physically the input one — true of the kernel checking mode always, and of the inference mode exactly when the needed constraints are already entailed). Cached verdicts stay valid while the environment grows monotonically, so the cache is purely functional and lives alongside the environment it was computed in: it is threaded through Mod_typing/Subtyping and stored in a field of the kernel's safe_environment on the checking side, and in a summary-registered reference on the inference side (Declaremods). Discarding or rolling back the state (Undo, Reset, document navigation) thereby discards or rolls back the cache with it — there is no global mutable state in the kernel and no invalidation obligation on callers. The cache is indexed by field label with a short capped list of physically-compared pairs, so lookups are cheap and retained memory is bounded.

Measurements

(Module Type command time from -time, or wall time for whole files; baseline is current master)

shape baseline this PR
#22279 benchmark: 800 chained interfaces, 10 params each, empty Args 3.33 s 0.15 s
800 chained interfaces, 2 params each, shared 3-field Args 0.46 s 0.05 s
prefix chain, 400 stages (~160k field re-checks) 6.8 s ~6.9 s, checks reduced to 400 cache misses + 160k hits totaling <0.2 s (the rest is materializing the quadratic amount of signature content, unrelated to subtyping)
100 applications of one functor to a 1000-field module 2.5 s 0.5 s

Tests

test-suite/modules/IncludeSelfChain.v covers chains over empty and non-empty parameter signatures (including definitions checked up to conversion, inductive types and submodules in the parameter signature), rejection of ill-typed chains, prefix-shaped chains, repeated functor applications, and a Reset-and-redo of a check whose universe constraints must be reproduced (exercising the cache flush). The modules, success, failure, bugs, and output test-suite subsystems pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_019ttctspSoVoquHLQtbPVZw

@coqbot-app coqbot-app Bot added the needs: full CI The latest GitLab pipeline that ran was a light CI. Say "@coqbot run full ci" to get a full CI. label Jul 16, 2026
@JasonGross
JasonGross force-pushed the fix-22279-include-self-subtyping branch from 189a113 to 821fb37 Compare July 16, 2026 14:57
Comment thread kernel/subtyping.mli Outdated
must be called whenever the global state is rolled back in time
(e.g. Undo, Reset, document navigation), as rolling back may remove
universe constraints that cached verdicts rely on. *)
val flush_cache : unit -> unit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want such a footgun in the kernel

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had fable thread a cache instead

@JasonGross
JasonGross force-pushed the fix-22279-include-self-subtyping branch from 821fb37 to e4a7c10 Compare July 16, 2026 20:33
@JasonGross JasonGross added kind: performance Improvements to performance and efficiency. part: kernel request: full CI Use this label when you want your next push to trigger a full CI. labels Jul 16, 2026
@JasonGross
JasonGross force-pushed the fix-22279-include-self-subtyping branch from e4a7c10 to 139ee27 Compare July 16, 2026 21:00
@coqbot-app coqbot-app Bot removed request: full CI Use this label when you want your next push to trigger a full CI. needs: full CI The latest GitLab pipeline that ran was a light CI. Say "@coqbot run full ci" to get a full CI. labels Jul 16, 2026
@JasonGross
JasonGross marked this pull request as ready for review July 16, 2026 23:05
@JasonGross
JasonGross requested review from a team as code owners July 16, 2026 23:05
@ppedrot

ppedrot commented Jul 17, 2026

Copy link
Copy Markdown
Member

This still doesn't look like the correct way to fix this, even though I haven't given a lot of thought about a potential solution yet.

@JasonGross JasonGross added the needs: full CI The latest GitLab pipeline that ran was a light CI. Say "@coqbot run full ci" to get a full CI. label Jul 21, 2026
JasonGross and others added 3 commits July 21, 2026 00:04
Subtyping.check_subtypes used to eagerly strengthen the whole signature
of the subtype side and to eagerly build a label map over all its
fields, even when the expected signature only requires checking a few
fields (or none). Chaining functor module types with <+ re-runs this
check on the whole accumulated signature at every stage (twice: once
for universe inference and once in the kernel), which made such chains
quadratic in the total number of fields (rocq-prover#22279).

All non-checker callers guarantee that the fields of the subtype side
are already part of the environment, unsubstituted (this is the
invariant documented in check_structure for the non-functor case).
Expose a [direct] flag on check_subtypes for those callers: fields are
then looked up in the environment by name on demand, and constant and
submodule fields are strengthened individually at lookup time
(mirroring what Modops.strengthen produces), instead of walking the
whole signature. Lookups fall back to a lazily-built label map when a
label does not name a field of the module (e.g. constructor names or
secondary inductive types of a block), preserving behavior in those
cases, in the checker, and in the recursive cases where the
environment holds substituted fields.

On the benchmark from rocq-prover#22279 (800 chained interfaces of 10 parameters
each over an empty parameter signature), the Module Type command goes
from 3.33s to 0.15s; with a shared 3-field parameter signature it goes
from 0.46s to 0.05s (n=800, 2 fields per interface).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019ttctspSoVoquHLQtbPVZw
Some module constructions legitimately perform quadratically many field
subtype checks of the same pairs of fields: chains of module types each
parameterized by a prefix of the chain, or repeated applications of the
same functor. Cache successful checks of a constant field pair under
conditions, checked at run time, that make the verdict a function of
the pair alone: both substitutions leave the bodies physically
unchanged, and the check produced no new universe constraints (the
returned state is physically the input one — always true of the kernel
checking mode, and true of the inference mode exactly when the needed
constraints are already entailed).

Cached verdicts stay valid as long as the ambient environment evolves
monotonically, so the cache is purely functional and lives alongside
the environment it was computed in: threaded through the module
translation functions and stored in a field of the kernel's
safe_environment on the checking side, and in a summary-registered
reference on the inference side (Declaremods). Discarding or rolling
back the state (Undo, Reset, document navigation) thereby discards or
rolls back the cache with it — there is no global mutable state and no
invalidation obligation.

The cache is indexed by field label with a short capped list of
physically-compared pairs, so lookups are cheap and retained memory is
bounded.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_019ttctspSoVoquHLQtbPVZw
@JasonGross
JasonGross force-pushed the fix-22279-include-self-subtyping branch from 139ee27 to f58b5e5 Compare July 21, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind: performance Improvements to performance and efficiency. needs: full CI The latest GitLab pipeline that ran was a light CI. Say "@coqbot run full ci" to get a full CI. part: kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chaining module <+ operator shows quadratic time increase

3 participants