feat: Content Platform CMS routes in sitemap (Increment 2 of cms-routes-in-sitemap-for-faststore)

[renatomaurovtex]

Summary

Implements the Content Platform increment (P6–P9) of specs/cms-routes-in-sitemap-for-faststore.md, wiring the new VTEX CMS (Content Platform) Data Plane REST API as a parallel route source alongside the existing hCMS/Rewriter integration. Builds on PRs #187 (spec), #188 (hCMS implementation) and #189 (Content Platform spec increment).

Note — the original commits in this branch were written against a hypothetical Data Plane ({{account}}.myvtex.com/api/cms/data-plane, schema-driven discovery, noindex field, production-branch resolution). Mid-PR we reverse-engineered the real contract against pm2023team2 and rewired the integration. The current state is described below; older commit messages still reference the original design.

Real Data Plane contract used

• Host: https://{account}.vtexcommercestable.com.br/api/content-platform/data/*
• Production-only by design: the Data Plane returns only published data; drafts live on the Control Plane and are unreachable from this app — there is no branch parameter on the REST surface.
• No schema-listing endpoint: routable content types are configured via a new app setting (contentPlatformContentTypes) rather than discovered. Auto-discovery via vtex.admin-cms-graphql is documented as a follow-up.
• Methods used: listEntries (cursor-paginated, returns id / name / updatedAt / searchKeywords, no slug) and getEntry (returns full seo.slug, seo.canonical, seo.title, locale metadata). getEntryBySlug is also exposed on the client for ad-hoc lookups but not used by the sitemap path.
• Locale model: getEntry is called once per (entry, binding.defaultLocale). The Data Plane returns 404 Not Found for locales that were never published — the binding is silently skipped, so we never synthesise locales (Decision 5).
• ETag caching: the Data Plane sends ETag headers and honors If-None-Match / 304 Not Modified. Cached payloads are persisted in VBase across generations.

hCMS (legacy) source — FastStore correction

FastStore Headless CMS pages live in the CMS builder data layer and are delivered to the storefront via the REST API (/_v/cms/api/{projectId}/{contentType}) — they are never registered as Rewriter Internals. The previous generateCmsRoutes read from rewriter.listInternals, so hCMS landing pages of FastStore stores never reached the sitemap. A CmsBuilder client + shared fetchEligibleHcmsSlugs helper now source these pages directly (published-only, requires seo.slug, excludes home /, honors seo.canonical opt-out and disableRoutesTerm); project id and routable content types are settings-driven (hcmsProjectId, hcmsContentTypes).

Per user story

• US-1 (Content Platform pages discoverable) — generateContentPlatformRoutes iterates contentPlatformContentTypes (default ["landingPage", "home"]), paginates listEntries, and fans out getEntry calls per binding-locale. Routes are emitted from seo.slug.
• US-2 (SEO opt-out) — A page is excluded when seo.canonical is non-empty AND points to a URL different from its own seo.slug (Decision 10). noindex is not available in the current Content Platform seo schema, so the opt-out is canonical-only for now. Login/error paths are excluded centrally (FR-3).
• US-3 (Multi-locale & alternates) — URL entries include xhtml:link tags for every locale where the entry actually returned 200, plus an x-default. Bindings whose getEntry returns 404 are silently dropped — fallback locales are never synthesised.
• US-5 (custom-routes endpoint) — /_v/public/sitemap/custom-routes exposes a content-platform-routes (or cms-routes) section. Mutual exclusivity is enforced at serve time: only the active source's section is returned.
• US-6 (Mutual exclusivity) — resolveActiveCmsSource(settings) returns 'hcms' | 'content-platform' | 'none'. When both flags are on, Content Platform wins (Decision 8) and cms-routes-ignored-by-mutual-exclusivity is logged once per generation. Enforced at four layers: event firing, generation middlewares, served <sitemapindex> / entry files, and customRoutes.

Single-binding serving (catalog proxy path)

Single-binding stores (the common FastStore case, e.g. isCrossBorder === false) delegate both /sitemap.xml and /sitemap/:path to the catalog proxy (catalog.getSitemap), which never reads VBase. As a result the generated CMS routes were correctly persisted but invisible in the served XML — /sitemap.xml only listed brand / category / product, and /sitemap/cms-routes-N.xml returned 400 (the catalog has no such file). Only multi-binding (legacySitemap) stores and the custom-routes JSON endpoint surfaced them.

This increment closes that gap on the catalog path:

• catalogSitemap (in node/middlewares/sitemap.ts) now reads the active CMS index from VBase (getCmsIndexFor) and merges its sub-sitemap entries into the catalog <sitemapindex>, with a defensive fallback (catalog XML returned unchanged when it has no <sitemapindex> or no CMS index exists).
• catalogSitemapEntry (in node/middlewares/sitemapEntry.ts) now serves CMS entry files (/sitemap/cms-routes-N.xml, /sitemap/hcms-routes-N.xml) directly from the active CMS bucket in VBase before falling back to the catalog proxy.
• Both paths honor resolveActiveCmsSource (mutual exclusivity) and are fully gated by the existing settings — behavior is unchanged when no CMS source is enabled. Single-binding entries carry no xhtml:link alternates (preserved via buildLocalization).

Key code additions

• node/clients/cmsDataPlane.ts — external client against vtexcommercestable.com.br/api/content-platform/data/* with cursor-based listEntries, getEntry (with If-None-Match), getEntryBySlug, 5xx retry, and graceful 404 handling ({ notFound: true }).
• node/clients/cmsBuilder.ts — external client against {account}.myvtex.com/_v/cms/api/* for the FastStore hCMS legacy source (paginated listPages, 5xx retry).
• node/services/hcmsRoutes.ts — shared fetchEligibleHcmsSlugs / isEligibleHcmsPage filter used by both the XML generator and the custom-routes endpoint (determinism, invariant 6).
• node/middlewares/generateMiddlewares/generateContentPlatformRoutes.ts — settings-driven ingestion, per-locale fan-out, canonical-based SEO opt-out, disableRoutesTerm filtering, multi-locale alternates, chunked persistence (50k URLs / 50 MB per file — Decision 2) under a dedicated content-platform-routes VBase bucket with its own index (Decision 7).
• node/services/contentPlatformCache.ts — VBase-backed ETag cache (content-platform-data-cache bucket) with base64url-safe filenames keyed by (contentType, entryId, locale).
• node/services/routes.ts — resolveActiveCmsSource + resolveCmsBucket + getContentPlatformRoutes; getCmsRoutes early-returns when hCMS is not active and now sources from the CMS builder API.
• node/middlewares/{sitemap,sitemapEntry,customRoutes}.ts and generateMiddlewares/{generateSitemap,generateCustomRoutes,generateCmsRoutes}.ts — wire the active-source resolver through both pipelines, including the single-binding catalog path (see above).
• node/globals.ts — Route.source: 'hcms' | 'content-platform' | 'apps' | 'user'.
• manifest.json — enableContentPlatformRoutes (boolean, default false), contentPlatformStoreId (string, default "faststore"), contentPlatformContentTypes (string[], default ["landingPage", "home"]), hcmsProjectId / hcmsContentTypes, and outbound-access policies for {{account}}.vtexcommercestable.com.br/api/content-platform/data/* and {{account}}.myvtex.com/_v/cms/api/*.

Tests

• generateContentPlatformRoutes.test.ts — settings allowlist honored / falls back to default, multi-locale fan-out emits one URL per actually-published locale with grouped alternates, 404 silently skips unpublished locales (no synthesised alternates), single-binding stores write only the en-US bucket, ETag is persisted on fresh 200, cached PublishedEntry is reused on 304, seo.canonical-based exclusion, login/error exclusion, disableRoutesTerm filter, mutual-exclusivity logging.
• generateCmsRoutes.test.ts — hCMS pages sourced from the CMS builder API: published-only, canonical opt-out, disableRoutesTerm, chunking, off-by-default and mutual-exclusivity short-circuits.
• node/services/routes.test.ts — exhaustive matrix for resolveActiveCmsSource (all flag combinations, Content Platform tie-breaker).
• sitemap.test.ts / sitemapEntry.test.ts — single-binding catalog path: index merge for hCMS and Content Platform, catalog XML unchanged when no CMS source is enabled, mutual exclusivity (only the active source merged), entry served from VBase before the catalog fallback, and 404/proxy fallback when the CMS file is absent or the flag is off.
• Augmented suites — generateSitemap.test.ts, customRoutes.test.ts cover event firing, bucket reads, and mutual exclusivity at every layer.

Assumptions

• contentPlatformStoreId / hcmsProjectId default to "faststore", matching the typical FastStore contentSource.project. Stores using a different project must override the setting before enabling the flag.
• The shared disableRoutesTerm setting (already used by hCMS) applies identically to Content Platform paths — same mental model for merchants migrating between sources.
• ETag persistence is best-effort: if VBase write fails, the next generation simply re-fetches the entry. We do not surface cache writes as user-visible errors.

Deviations from spec (carried into the spec text)

• Decision 7 — Data Plane is production-only by design (no branch parameter exists on the REST surface).
• Decision 9 — routable content type discovery replaced by a settings-driven allowlist (contentPlatformContentTypes); auto-discovery via vtex.admin-cms-graphql is a follow-up.
• Decision 10 — SEO opt-out is canonical-only today (noindex is not in the current Content Platform seo schema). The original noindex OR canonical-mismatch rule is preserved in the decision text for traceability and will be re-enabled when the field is added.

These changes are reflected in specs/cms-routes-in-sitemap-for-faststore.md (Decisions 7/9/10/11 + Implementation Plan P6–P9) and in docs/CMS_ROUTES.md.

Validation notes

• The single-binding serving was verified end-to-end on a FastStore workspace (vendemo): with enableCmsRoutes on, /sitemap.xml now lists …/sitemap/cms-routes-0.xml alongside the catalog entries, and /sitemap/cms-routes-0.xml serves the hCMS landing page.
• XML generation depends on the IO event bus. The generateCmsRoutes / generateContentPlatformRoutes middlewares run as event handlers. In a flaky vtex link dev environment the event server can drop these events (Connection to event server has failed), leaving the CMS bucket empty and the serving path falling back to the catalog proxy. This is a dev-link limitation, not a serving bug — the custom-routes JSON endpoint (which generates synchronously) is unaffected, and events work normally on a deployed app.

Follow-ups

• noindex opt-out — re-enable as a second leg of the SEO opt-out as soon as the Content Platform seo schema exposes the field.
• Auto-discovery of routable types — query vtex.admin-cms-graphql to auto-populate contentPlatformContentTypes instead of forcing a manual setting.
• Pre-existing test failures (in utils.test.ts, prepare.test.ts, generateRewriterRoutes.test.ts) were already red on master — out of scope here, worth a separate cleanup PR.
• TSLint baseline — repo carries ~49 pre-existing lint errors; the new files in this PR introduce zero additional lint errors.

Spec & related PRs

• Spec: specs/cms-routes-in-sitemap-for-faststore.md — status updated from Approved (Content Platform increment) to Done in this PR.
• spec: cms-routes-in-sitemap-for-faststore #187 — original spec
• feat: cms-routes-in-sitemap-for-faststore #188 — hCMS legacy implementation (Increment 1)
• spec+docs: Content Platform CMS support (increment 1 of cms-routes-in-sitemap-for-faststore) #189 — Content Platform spec increment

Made with Cursor

[vtex-io-ci-cd]

Hi! I'm VTEX IO CI/CD Bot and I'll be helping you to publish your app! 🤖

Please select which version do you want to release:

• Patch (backwards-compatible bug fixes)

• Minor (backwards-compatible functionality)

• Major (incompatible API changes)

And then you just need to merge your PR when you are ready! There is no need to create a release commit/tag.

• No thanks, I would rather do it manually 😞

[vtex-io-docs-bot]

Beep boop 🤖

Thank you so much for keeping our documentation up-to-date ❤️