spec+docs: Content Platform CMS support (increment 1 of cms-routes-in-sitemap-for-faststore) - #189
Open
renatomaurovtex wants to merge 14 commits into
Open
Conversation
Add SDD for SFS-3123 covering Business Context, Arch Decisions and Technical Contract for including CMS/hCMS routes in the sitemap served by vtex.store-sitemap for FastStore stores. Co-authored-by: Cursor <[email protected]>
Co-authored-by: Cursor <[email protected]>
Per the spec for SFS-3123, introduce the rollout-gated `enableCmsRoutes` setting (default `false`), the `CMS_ROUTES_INDEX` index filename, and the protocol-driven `CMS_ROUTES_MAX_URLS_PER_FILE` / `*_MAX_BYTES_PER_FILE` ceilings (50k URLs / 50 MB) that drive the chunking step. The setting is wired into the existing INDEX_MAP so that `enabledIndexFiles` includes the new bucket only when the flag is on, preserving backwards compatibility for stores that have not opted in. Co-authored-by: Cursor <[email protected]>
New middleware (`generateCmsRoutes`) that materializes the CMS-routes sitemap source for FastStore stores per the spec for SFS-3123: - Pulls Rewriter Internals via the existing `listInternalsWithRetry` - Classifies routes as CMS-origin by excluding framework-owned types (`product`, `department`, `category`, `subcategory`, `brand`) and `notFound*` error pages - Mandatorily excludes types containing `login` or `error` substrings (FR-3); hCMS metadata join (noindex / canonical) is deferred per Decision 3 — defensive default is "not excluded" - Honors the CMS toggle via Rewriter `disableSitemapEntry` - Honors the existing `disableRoutesTerm` substring exclusion - Groups by binding and writes per-binding entry files plus the `CMS_ROUTES_INDEX` to a dedicated `cms-routes_<hash>` bucket - Chunks each binding's routes at the protocol ceiling (CMS_ROUTES_MAX_URLS_PER_FILE / CMS_ROUTES_MAX_BYTES_PER_FILE) - Short-circuits to a no-op when `enableCmsRoutes` is false so the rollout is reversible (invariant 9) 7 jest tests cover binding partitioning, exclusion filters (framework types, disableSitemapEntry, notFound/login/error, disableRoutesTerm), chunking at the URL ceiling, and the off-by-default short-circuit. Co-authored-by: Cursor <[email protected]>
Add `getCmsRoutes` to `services/routes.ts` mirroring the shape of
`getUserRoutes` and `getAppsRoutes` (flat path list across bindings),
reusing the `isValidCmsRoute` filter exported from `generateCmsRoutes`
so the JSON API and the XML pipeline are guaranteed to apply the same
classification (invariant 6 — determinism).
Wire it into `generateCustomRoutes` so the VBase-cached payload returned
by `GET /_v/public/sitemap/custom-routes` now contains a third entry:
{ name: 'cms-routes', routes: ['/our-story', '/black-friday', ...] }
The `customRoutes` middleware filters the response based on both
`enableAppsRoutes` and `enableCmsRoutes` so each section disappears when
its corresponding rollout flag is off (invariant 9). Two new tests cover
the on/off filter cases.
Co-authored-by: Cursor <[email protected]>
… (US-3, FR-5) Per Decision 5 of the spec, multi-locale stores now declare every locale alternate (including self) and a `hreflang="x-default"` pointing at the default binding for each `<url>` entry. Single-binding stores keep the prior behavior and emit no alternates (preserves invariant 7 — backwards compatibility). Per FR-5, the `Route` type gains optional `changefreq`, `priority` and `lastmod` fields. URLEntry emits the corresponding `<changefreq>` and `<priority>` tags only when the route declares them, keeping existing sources (product / navigation / apps) byte-compatible with their current XML output. `generateCmsRoutes` writes routes with default values (`weekly` / `0.5`) so every CMS-origin entry carries the protocol tags as required. Two existing localization tests are updated to the new spec-compliant shape; four new tests cover single-binding suppression, current binding not being the default, presence of `<changefreq>`/`<priority>` when set, and their absence when omitted. The 3 pre-existing failures in this test file are in the unrelated `catalogSitemapEntry` path and not affected by this change. Co-authored-by: Cursor <[email protected]>
…on 6)
The `robots` middleware now appends `Sitemap: https://{host}/sitemap.xml`
to the served body when no `Sitemap:` directive is already present. The
detection is case-insensitive and tolerant of leading whitespace
(invariant 8 of the spec — robots.txt idempotency), so merchants who
declared their sitemap manually are unaffected and the directive is
never duplicated.
Logic is extracted into a pure `ensureSitemapDirective` helper covered
by 7 unit tests: append on missing, no-op on present, case-insensitive
match, indented match, empty body, repeated-call idempotency, and the
"existing different URL" case (we honor the merchant's manual choice
rather than override it).
Co-authored-by: Cursor <[email protected]>
…back `sitemap` middleware now reads the CMS routes index from the dedicated `cms-routes_<bindingHash>` bucket and appends its entries to the served `<sitemapindex>` when `enableCmsRoutes` is on (invariant 9). The read is nullable — a missing CMS index simply means no extra sub-sitemaps to emit, preserving the "no empty regression" invariant. `sitemapEntry` middleware's legacy path now falls back to the cms-routes bucket when the requested file is not found in the production bucket, so that crawler requests for `/sitemap/cms-routes-N.xml` resolve correctly. Two new tests in `sitemap.test.ts` cover the on/off rollout cases. As a positive side effect of adding `isCrossBorder: true` to the shared mock context (required to exercise the legacy path that my changes touch), the 7 previously-failing tests in the suite that mis-routed to `catalogSitemap` are also now green — they were testing the legacy behavior all along but the default path had changed. Co-authored-by: Cursor <[email protected]>
All five user stories are implemented and tested. Closing the loop on the spec lifecycle. Co-authored-by: Cursor <[email protected]>
New `docs/CMS_ROUTES.md` covering the full lifecycle of the CMS routes feature introduced in SFS-3123: how it works, enabling via Admin UI or CLI, smoke-test verification steps, per-page exclusion options (toggle, Rewriter mutation, disableRoutesTerm), multi-locale hreflang output, the JSON endpoint migration path from next-sitemap, robots.txt integration, and a known-limitations table for the pending hCMS cross-team dependencies. `docs/README.md` gains a short "CMS routes (FastStore)" section that cross-links to the new document. Co-authored-by: Cursor <[email protected]>
The middleware existed and had unit tests but was never connected to the pipeline: GENERATE_CMS_ROUTES_EVENT was missing, service.json had no handler entry, index.ts did not register the event handler, and generateSitemap never fired the event when enableCmsRoutes was on. Co-authored-by: Cursor <[email protected]>
Stop routing CMS routes through enabledIndexFiles (wrong VBase bucket) and group Rewriter internals by page id so persisted routes carry cross-binding alternates for US-3 hreflang output. Co-authored-by: Cursor <[email protected]>
Add support for the new VTEX CMS (Content Platform) as a parallel route source to the existing hCMS (legacy) integration, woven across the existing sections of the spec. Highlights: - New CMS source via the VTEX CMS Data Plane REST API (Decision 7), pinned to the production branch with ETag (`If-None-Match`) caching. - Mutual exclusivity between hCMS and Content Platform per generation; Content Platform wins when both flags are set (Decision 8 / FR-10). - Schema-driven content-type discovery: every type whose schema declares a `path` field is ingested, no hardcoded allowlist (Decision 9). - Opt-out driven exclusively by existing SEO fields (`noindex`, `canonical`) with no new schema field required (Decision 10 / FR-11). - Locale fidelity: alternates emitted only for actually-published locales, no synthetic fallback URLs (Decision 11). - Phases P6, P7, P8 added to the Implementation Plan; planned for `[email protected]`. Motivated by the [progressive migration of hCMS stores to the new VTEX CMS][1] underway since March 2026, which would otherwise regress sitemap coverage for every FastStore v3 store that flips to the new CMS. [1]: https://help.vtex.com/announcements/2026-03-30-headless-cms-stores-will-be-upgraded-to-the-new-vtex-cms Co-authored-by: Cursor <[email protected]>
…uide Update CMS_ROUTES.md to reflect the Content Platform increment specified in cms-routes-in-sitemap-for-faststore.md. The doc continues to be the single source of truth for both CMS surfaces; the existing hCMS sections are clarified as legacy and two new top-level sections are added: - "Content Platform support" — how the new ingestion path works (Data Plane API, production-branch pinning, schema-driven discovery, SEO-field opt-out, ETag caching), mutual-exclusivity matrix between the two flags, enable/verify/exclude flows specific to Content Platform, and the observability events emitted by the new generator. - "Migrating from hCMS to Content Platform" — step-by-step migration guide (snapshot, flip flags, validate, spot-check, rollback) for stores being progressively upgraded by the CMS team. Known limitations and the Reference section are updated accordingly. The new behavior ships in `[email protected]` (P6-P8 of the spec Implementation Plan); this commit documents the contract ahead of code so that customer-success and integrators can prepare in parallel. Co-authored-by: Cursor <[email protected]>
Contributor
|
Hi! I'm VTEX IO CI/CD Bot and I'll be helping you to publish your app! 🤖 Please select which version do you want to release:
And then you just need to merge your PR when you are ready! There is no need to create a release commit/tag.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is an increment on top of the
cms-routes-in-sitemap-for-faststorespec/feature (PR #188). It adds support for the new VTEX CMS (the Content Platform) as a parallel route source to the existing hCMS (legacy) integration.Why now
VTEX is progressively migrating Headless CMS (legacy) stores to the new VTEX CMS (Content Platform) since March 30, 2026. Without an explicit Content Platform path in this app, every store that flips loses its CMS-page coverage in the sitemap — defeating the purpose of the hCMS work already in #188.
What's in this PR
`specs/cms-routes-in-sitemap-for-faststore.md` (commit `2d1de61`)
The spec is woven to cover both CMS surfaces (not appended as a separate section). Key additions:
`docs/CMS_ROUTES.md` (commit `aa5f07e`)
The customer-facing doc gains two new top-level sections:
The existing hCMS-specific content is clarified as legacy (with a pointer to the new section). Known Limitations and Reference sections are updated.
What's NOT in this PR
Test plan
This is a documentation-only PR. Validation is by review:
Related
Made with Cursor