Memoize schema expansion + fix #69/#70 follow-ups (issue #58)#71
Merged
Conversation
Implements SchemaContext with document-level memoization to avoid re-expanding shared sub-schemas at every use site. Key changes: - Add SchemaContext struct for tracking expanded schema references - Add write_schema_table_with_context for memoized schema rendering - Add collect_schema_rows_with_memo that avoids duplicate expansion - Add render_deferred_schemas for rendering schema definitions at end - Preserve original functions for backward compatibility This addresses the output/token bloat issue where shared schemas (like Error models) were being fully re-expanded at every reference. The implementation uses: - HashSet for tracking expanded references - Markdown links for cross-references to already-expanded schemas - Deferred rendering of schema definitions in a dedicated section - Proper cycle detection with links to existing schemas Note: The new functions are implemented but not yet integrated into the main rendering pipeline. The original functions remain functional and all existing tests pass. Co-authored-by: nrynss <[email protected]>
Issue #58: schema rendering is now memoized per document. Component schemas reached through a `$ref` are expanded once into a trailing "Schema Definitions" section and linked from every use site instead of being re-inlined at each occurrence, cutting output and token cost for specs with shared schemas. `--inline-schemas` restores the fully self-contained expansion (with cycle detection) for callers that want it. The previous commit only added unused, unformatted infrastructure to schema.rs (CI red on `cargo fmt --check`, nothing wired in). That is replaced with a clean, fully integrated implementation: - `SchemaContext` threads through each body view → `write_endpoint` → `write_schema_table`; `render_schema_definitions` emits the shared section once per document. Self/mutual references resolve to links, so cycles need no special-casing in linked mode. Anchors are allocated collision-free and the section is rendered in deterministic first-encounter order. Follow-ups to the previously merged PRs: - #69: the "no effect" warning now reports the detail level in the lowercase spelling the user types (`--detail standard`), not the Debug-derived `Standard`. `--inline-schemas` without `--include-schemas` warns too. - #70: endpoint heading anchors are scoped by service in the service view, so a multi-tag operation rendered under several sections gets a unique anchor each time and its TOC links stay unambiguous. Tests: linked-default expansion + dedup, self-reference-as-link, inline mode, inline-mode cycle detection, warning casing, and multi-tag anchor uniqueness (new `multi_tag_oas3.json` fixture). 43 pass; fmt and clippy clean. Co-Authored-By: Claude Opus 4.8 <[email protected]>
nrynss
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #58 (schema re-expansion / token bloat) and folds in two follow-ups to the previously merged #69 and #70.
By default at
--detail full --include-schemas, component schemas reached through a$refare now expanded once into a trailing## Schema Definitionssection and linked from every use site ([Name](#schema-name)), instead of being re-inlined at each occurrence. Shared schemas (a common error/envelope, or a type referenced many times) collapse to a single definition, and self/mutual references become links rather than triggering "cycle detected" or deep re-expansion.--inline-schemasrestores the previous fully self-contained behaviour (every$refexpanded inline at each use site, with per-chain cycle detection) for callers who want one copy-pasteable blob.What changed since the draft
The first commit added ~310 lines of unused, unformatted scaffolding to
schema.rs— CI was red oncargo fmt --checkand nothing was wired into the render pipeline. That is replaced with a clean, fully integrated implementation:schema::SchemaContextcarries document-level memoization (seen refs → stable anchors, in first-encounter order). Each body view creates one context, threads&mutit throughwrite_endpoint→write_schema_table, then callsrender_schema_definitionsto emit the shared section once.output_is_deterministicinvariant).$refalways resolves to a link.Follow-ups to merged PRs
--detail standard), not the Debug-derivedStandard. Added a warning when--inline-schemasis used without--include-schemas.Tests
Net +7 tests (43 pass),
cargo fmt --checkandcargo clippy --all-targets -- -D warningsclean:--inline-schemasfull expansion + inline-mode cycle detection--inline-schemasmisuse warningmulti_tag_oas3.jsonfixtureDocs (README, CLAUDE.md) updated for the new default and
--inline-schemas.