Fix a round-trip fidelity bug for documents that use indented MDX-JSX…#218
Merged
Conversation
…ion help (#1988) The sample-regression script printed `gh workflow run nightly.yml` and `gh workflow run weekly.yml`, but both workflows were retired in #549 (the perf-regression / parse-health / elevated-PBT tiers). A developer running `bun run sample-regression` and following the help would hit "workflow not found". Remove the two dead lines and the now-orphaned "+ perf regressions" header clause; the measure:fuzz / measure:stress commands and the specs/2026-04-19-ci-signal-quality pointer remain accurate. GitOrigin-RevId: 224e053382290e668c80b6ebae739e4b2a49935e
* docs(ok): add MDX CRDT bridge conformance hardening spec (Tier 2 A+C) * feat(ok): add step-7g JSX-container boundary-blank comparator fold Workstream A of the MDX CRDT-bridge conformance hardening (Tier 2). normalizeBridge gains a surgical step-7g pre-pass (foldJsxContainerBoundaryBlanks) that drops the serializer-inserted boundary blank inside a Capitalized-JSX container (depth >= 1, lowercase details excluded), converting an indented child under a blank into one under a non-blank tag so the existing 7f folds it through its unchanged block-construct guards. This makes the dirty-path indented-children MDX-JSX class reach the bridge fixed point within tolerance. Comparator-side only (precedent #38); to-markdown-handlers.ts and the index.ts gate are unchanged. Register the jsx-container-boundary-blank tolerance class at all five sites (the class tuple, detectAppliedToleranceClasses, SEVERITY_BY_CLASS, and the two exhaustiveness tests). Unit tests cover the positive folds, the three AC-A4 must-stay-divergent guards (doc-level indented code, blockquote and loose-list continuations nested in a Step), the lowercase details exclusion, fence-interior preservation, and the residual (inside-container deep indent folds correctly since container-relative indent is consumed at parse). 189 bridge unit tests, 1302 fidelity tests, and 127 server-bridge tests pass. * feat(ok): de-indent JSX container tags in step-7g for neighbor-independent fold A closing container tag that follows a block construct (a fenced code block or ATX heading) was not folded by the existing 7f (its previous-line guard rejects a fence close), leaving a Step that ends in a code block beyond tolerance. Extend step-7g to de-indent the structural container tag lines themselves (open and close) inside a container, which is always safe (a JSX tag is never content, code, or a list marker) and neighbor-independent. Full matrix stays green: 174 normalize unit tests including the new fence-close-tag case, 1302 fidelity tests, 127 server-bridge tests, and all 10 indented-jsx corpus fixtures fold and are idempotent. * feat(ok): add shared indented-JSX + large-embed corpus and PRD-6955 regression fixtures Workstream C corpus foundation (AC-C1). indented-jsx.json (10 shapes: the real github-sync 4-Step D5 canonical, M2/M3 variants, heading-less, flush-left, Tabs/Tab, depths 1-5, wide-6 siblings, a fenced-code-child intersection) and large-embed.json (the dual html-preview PRD-6955 trigger shape) are the single corpus source of truth. The byte-exact PRD-6955 forensic captures (prd-6955-before.md 225 lines / 2 valid scripts; prd-6955-corrupted-triplicated.md 661 lines / 3 triplicated copies / 3 brace-injected scripts) ship with a README marking them never-open-in-editor. Loaders added to fixtures/index.ts; a corpus-coverage ratchet test (the US-012 recurrence guard, modeled on I14 CORPUS_FALLBACK_FLOOR) pins the container shape, the github-sync fixture, an html-preview embed, and the intact regression captures. No new md-audit construct: <Steps>/<Step> are generic user JSX already covered by jsx-component; the registry's mechanical Tier-S/M classification would reject a recognizer-less jsx-container, so the corpus-ratchet test is the architecturally-correct US-012 protection (which SPEC 8.1 itself models on CORPUS_FALLBACK_FLOOR). * [US-003] extend i13 over full dirty-path corpus + O2 repeated-drain + O3 child-edit Append loadIndentedJsxFixtures() (10 shared-corpus shapes incl. the real github-sync.mdx 4-Step, the heading-less/blank-line-inside/multi-sibling reconstructed shapes, depths 4-5, wide-6 siblings, fenced-code child) to INDENTED_JSX_CLASS alongside the 4 hand-written shapes, so the fidelity suite exercises the full indented-children class and can never regress to the heading-only coverage. Add the O2 repeated-drain oracle to the per-case loop: six chained dirty drains, each normalizing equal to the authored source and every drain after the first byte-identical (the tightest no-byte-growth statement, the PRD-7110 amplifier signature). The class spans nesting depths 1-5. Add the O3 structural child-edit describe block: swap, delete, insert at the PM-JSON level plus a closing-tag edit via componentName rename (re-emits both open and close tag), each asserting a byte-stable within-tolerance bridge fixed point with every sibling marker present once in the intended order. The DETAILS_CANONICAL_CLASS flush-left anti-regression stays green. * [US-001] re-render P-B-002 tolerance-set enumeration for jsx-container-boundary-blank The step-7g comparator fold added jsx-container-boundary-blank to BRIDGE_TOLERANCE_CLASSES (now 16). The md-conformance P-B-002 contract pins its formalStatement to enumerate the live tolerance set exactly (count, membership, order) so growing the set fails until the prose is deliberately re-reviewed and re-rendered. Update the formalStatement to 16 classes with jsx-container-boundary-blank in array order, then regenerate the committed conformance-catalog.json (enumerate:conformance) and OK-MARKDOWN-CONFORMANCE-SUITE.md (conformance:render). This tier was not run when the tolerance class first landed, so the gate was left red. * [US-004] G2 forward-regression guard + O7 comparator forward-guards Add a PBT over the shared JSX/embed corpus (loadIndentedJsxFixtures + loadLargeEmbedFixtures) asserting that the dirty serializer output for every fixture is a within-tolerance fixed point AND that any surviving byte difference is explained by a NAMED tolerance class via detectAppliedToleranceClasses (never a silent over-fold or bloat). This is locus-independent: it pins the step-7g widening against over-fold drift and the delegated indentation against an mdast-util-mdx-jsx bump. A permanent planted-positive control injects real content the serializer never emitted and asserts the guard rejects it (discriminating, not tautological). Add O7 normalizeBridge idempotence over the corpus, and O7c MDX normalizing-construct convergence re-asserted against the real normalizeBridge (not the trimEnd-based helpers normalize): a CommonMark heading->paragraph tight separator and the headline indented-children <Steps> dirty round-trip, each asserted to BE a normalizing construct (round-trip differs) AND to converge under the comparator, so the convergence assertion is never vacuous. * [US-005] growth/duplication oracles (O1) + embed JS-parse oracle (O6) growth-engine.test.ts (new): O1 single-drain byte budget + no-duplication over the corpus, the AC-C3 static occurrence-count oracle that pins the PRD-6955 triplication (clean capture < 3x, corrupted >= 3x), and a pinned-today assertion that assertContentPreservation is blind to pure duplication plus a skipped landing-site test for the deferred B4 growth guard. embed-script-fidelity.test.ts (new): O6 extracts every <script>, asserts valid embeds parse and survive a round-trip with the script head intact and idempotent on second pass, the brace-injection fingerprint (const->{onst) is absent from clean inputs, and the forensic CORRUPTED capture carries it where BEFORE does not. Finding: the injected head is NOT a SyntaxError (JS block grammar + the IIFE wrapper absorb the stray brace, making onst a runtime undefined-reference), so the fingerprint is the reliable detector, not a parse exception. * [US-006] integration C14/C15 + map-driven EXTEND + O4 undo oracle C14 (new): 2 clients + an agent write edit a 4-Step indented-JSX doc across a pause/resume divergence window; on reconnect the bridge settles to a bounded, in-order fixed point (bridge invariant + O1 byte budget + O3 no Step reorder/duplication). C15 (new): dual html-preview embeds under a divergence window survive intact (scripts parse, no brace-injection signature, const DATA= head preserved, no triplication, byte-budget). map-driven EXTEND: a third test drives the indented-JSX construct (not just plain paragraphs) with the O1 byte-budget + no-duplication assertion. O4 (new jsx-undo-roundtrip): real applyAgentUndo via /api/agent-undo on an indented multi-Step doc returns the source within normalizeBridge tolerance, a subsequent settle does not re-dirty the undone region, and a concurrent peer keystroke survives the agent undo. No redo companion: agents have no redo product surface and the post-undo settle clears the UndoManager redo stack. The B3a/B3b production watchers are the deferred Workstream B; only the O4 oracle is in scope. All 7 tests green; deterministic seeds with 30s per-test timeouts for CI contention. typecheck + biome + oxlint clean. * [US-007] fuzz JSX-block + large-embed op kinds + O1 byte-budget oracle bridge-convergence.fuzz.test.ts: add jsx-block (indented <Steps>/<Step>) and large-embed (html preview <script>) op kinds at all five sites (Op union, generateOps ladder at 3% each, applyOp dispatch via the agent-write surface, ALL_OP_KINDS, WRITE_SURFACE_TO_OP_KIND). The D18 coverage gate now fails if either construct op kind is lost (closes G2 of the gap map). Both are excluded from oracle (d)/(e) marker tracking exactly like the existing chunked-source-paste large-payload op (their marker is multi-line-embedded). oracle-e-expectations.test-helper.ts OracleEOp widened to match. O1 byte-budget oracle wired into the fuzz post-drain (cumulative-authored x3 + slack; an unbounded-growth doc now fails the seed instead of passing as converged-late) and server-authoritative-stress (per-client converged bytes <= authored x2 + slack). Verified: D18 gate green, fuzz seed-7 smoke passed, stress seed-7 smoke passed (88 edits, no byte-budget false-fail), typecheck + biome clean. Randomized search stays ad-hoc per repo policy; the D18 gate + byte-budget are the CI-deterministic part. * [US-008] refresh ng-anchors catalog + fix fixtures comment-discipline Make the full `bun run check` gate green end to end (AC-C4). - ng-anchors-catalog.json: the conformance test files added across US-003/006/007 bumped the mechanization seam testFileCount from 174 to 177. Regenerated via enumerate-ng-anchors so the committed catalog matches a fresh build. Only testFileCount and generatedAt changed; the @floor anchor extraction was already bidirectionally fresh. - fixtures/index.ts: removed a US-012 story-id citation from a doc comment per the comment-discipline rule (story ids belong in the PR and commit, not in source). Kept the substance: why the corpus and the coverage ratchet exist. md-audit container-JSX coverage is carried by the existing jsx-component construct plus the shape-specific corpus-coverage ratchet; a separate container construct would have no distinct mdast/schema/promoter key and be taxonomic pollution. Decision recorded for no-skimp review. * fixup! local-review: baseline (pre-review state) * fixup! local-review: address findings (pass 1) * fixup! local-review: address findings (pass 2) * fixup! local-review: baseline (pre-review state) * harden step-7g scan perf, tolerance-count citations, and fuzz faults Address findings from the post-implementation local review. - normalize.ts step-7g: the boundary-blank scan re-walked the entire blank run for every blank line, O(K^2) in K consecutive blanks on the uncached per-drain bridge hot path with no live size guard. Replace with a two-sweep nearest-non-blank precompute (O(N), behavior-identical) and add a large-K correctness test. - Add the patch changeset this behavior-changing bridge fix was missing. - Conformance contracts: sweep four stale "14 tolerance classes" prose citations to 16 (P-A-104, P-A-404, P-X-003, and a test comment) so the regenerated tier-S catalog no longer asserts two cardinalities for one frozen set; broaden the cardinality lock-step guard to the tolerance-context prose forms; re-render the catalog and suite. - Add a mechanical parity test asserting the md-audit BRIDGE_TOLERANCE_CLASSES hand-copy stays in lock-step with core (the one mirror with only a comment). - Fuzz: the jsx-block/large-embed apply path swallowed every write fault; it now re-throws non-409 errors so a genuine delivery fault fails the seed loudly and only the legitimate 409 doc-in-conflict refusal is treated as not-applied. agentWriteMd surfaces the HTTP status to enable the distinction. * fixup! local-review: address findings (pass 1) * no-skimp: sweep stale tolerance-count comments in the meaning-oracle to 16 The meaning-oracle subsystem (scorecard, candidate-meaning-relation + test, o2-byte-roundtrip-identity test) carried 9 comments calling normalizeBridge the "14-class bridge lens" / "14 tolerance classes". The bridge tolerance set is now 16 (this PR added jsx-container-boundary-blank). These are the same-shape sibling sites of the stale-citation defect already fixed in the conformance contracts + suite narrative this round; bumping them now keeps the documentation consistent rather than leaving a latent inventory for the next audit. Comments and one test name only; no logic change. * Optimize normalizeBridge common case and strengthen bridge tests Address claude[bot] review suggestions on the bridge comparator: - Short-circuit foldJsxContainerBoundaryBlanks when the document has no Capitalized JSX tag, so the common case skips the pre-pass entirely (zero-cost on the per-drain hot path) instead of allocating six length-proportional arrays and scanning fences twice. - Hoist the redundant split in detectAppliedToleranceClasses so the jsx-boundary-blank probe and the ordered-list-marker scan share one split of each input rather than two. - Pin the self-closing-tag depth guard with a depth-0 discriminating test. The guard's observable effect is at depth 0, where a false open would flip a preserved blank to a folded one; inside a container the guard cannot be isolated since every depth stays at least 1. - Strengthen the O4 undo round-trip test with the full bridge invariant after settle (assertBridgeInvariant), proving the bridge settles with no re-derive loop rather than only checking the Y.Text round-trip. GitOrigin-RevId: 9c31587faa4c85f1b664e3d46078600903e26d04
![inkeep-internal-ci[bot]](https://avatars.githubusercontent.com/u/123133689?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Automated approval from agents-private public-mirror-sync (run: https://github.com/inkeep/agents-private/actions/runs/27850168565). Source of truth is the monorepo; direct edits on inkeep/open-knowledge are overwritten on next sync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix a round-trip fidelity bug for documents that use indented MDX-JSX container components (
<Steps>,<Tabs>, and similar nested components). Editing one of these documents no longer risks silent indentation rewrites, content reordering, or duplication: the editor's bridge now recognizes the serializer's container formatting as equivalent to the authored source, so the document settles to a stable state in a single pass and what you typed is what gets stored.