Skip to content

[WIP] docs: clear docs/ for the Mintlify docs source#601

Open
ShawnChen-Sirius wants to merge 6 commits into
chdb-io:mainfrom
ShawnChen-Sirius:docs/restructure-for-mintlify
Open

[WIP] docs: clear docs/ for the Mintlify docs source#601
ShawnChen-Sirius wants to merge 6 commits into
chdb-io:mainfrom
ShawnChen-Sirius:docs/restructure-for-mintlify

Conversation

@ShawnChen-Sirius

@ShawnChen-Sirius ShawnChen-Sirius commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Why

Following the docs-platform alignment, chdb-io/chdb/docs will become the aggregation source for the official chDB documentation (synced into the main ClickHouse docs build — the same model clickhouse-connect adopted in ClickHouse/clickhouse-connect#778). This PR clears the directory, stages the published docs tree in it, and wires up tooling and CI ahead of the 7/17 platform cutover — without switching the source of truth yet (see the staging notice in docs/README.md; content edits keep flowing through the live pipeline until the cutover stabilizes).

What

1. Three-way split of the old docs/ (references updated):

  • Sphinx site → sphinx/ (.readthedocs.yml, Makefile, ruff exclude, CONTRIBUTING follow)
  • Engineering references → dev-docs/ (README/CONTRIBUTING/llms.txt links and the llms-full generator follow)
  • docs/_static/ stays put: its images are pinned by absolute raw URLs from the org profile, every published PyPI release page, and the example notebooks — frozen public URLs.

2. ReadTheDocs retirement banner — canonical docs are clickhouse.com/docs/chdb; RTD stops updating September 17, 2026.

3. Staged docs snapshot — the docs/products/chdb slice of the aggregated docs in ClickHouse/ClickHouse (44 Mintlify .mdx pages + navigation.json, snapshot of 2026-07-08) imported into docs/, with a README marking it a staging copy, not the source of truth.

4. llms-full generator: dual source--docs-source local builds from the staged tree (aggregator-internal /products/chdb/… links resolved to published slugs); default stays remote until the source-of-truth switch. Both modes emit identical URLs for all 44 pages. Also fixes a real bug in the shipped llms-full.txt: Python/Go import lines inside code examples were being stripped by the MDX-import regex (20 import chdb lines were missing); regenerated.

5. Docs verification CI — clickhouse-connect's docs_verify.yml adapted: the aggregator's products/chdb slice is replaced with this repo's staged pages and the scoped Mintlify checks run on every PR touching docs/. README/_static are filtered out of the staged slice — the same whitelist the future sync workflow will use.

6. Content diff verdict (dev-docs vs docs pages): every overlapping dev-docs file contains substantial content the docs pages lack (22–69 headings each; PANDAS_COMPATIBILITY and FUNCTIONS are ~3× the size of their counterparts). Nothing is deleted; they stay in dev-docs/ as source material for enriching the docs pages later.

After the cutover stabilizes (not in this PR)

Final re-sync of the snapshot → flip the generator default to local → replace the staging notice with contributor instructions → sync workflow + pre-merge preview with the docs team. At that point docs/ becomes the single source of truth.

🤖 Generated with Claude Code

Note

Move Sphinx docs to sphinx/ and populate docs/ with Mintlify source

  • Relocates all existing Sphinx documentation files from docs/ to sphinx/ so that docs/ can be used exclusively for the new Mintlify-based docs source.
  • Adds a large set of new Mintlify .mdx pages under docs/ covering installation guides (Python, Node.js, Go, Rust, Bun, C/C++), DataStore reference, configuration, debugging, guides, and API reference.
  • Moves internal developer documents (architecture, migration guides, etc.) from docs/ to dev-docs/ and updates all cross-references in README.md, CONTRIBUTING.md, llms.txt, and llms-full.txt.
  • Updates build tooling: Makefile, .readthedocs.yml, pyproject.toml, and .gitignore now point to sphinx/ instead of docs/.
  • Adds a CI workflow (.github/workflows/docs_verify.yml) that runs Mintlify docs validation against the docs/ tree on PRs and pushes to main.
  • Behavioral Change: make docs and make docs-md now build from sphinx/; Read the Docs builds from sphinx/conf.py and sphinx/requirements.txt.

Macroscope summarized f9662ac.

ShawnChen-Sirius and others added 2 commits July 8, 2026 10:21
docs/ will become the aggregation source for the official chDB docs
(synced into the main ClickHouse docs build), so its current mixed
content moves out:

- Sphinx site (rst sources, conf.py, requirements.txt, logo) ->
  sphinx/. The ReadTheDocs build, Makefile 'docs'/'docs-md' targets,
  ruff exclude, and CONTRIBUTING instructions now point there.
- Engineering references (ARCHITECTURE, FUNCTIONS, PANDAS_*,
  EXPLAIN_METHOD, FACTORY_METHODS, PROFILING, REMOTE_SESSION_DESIGN,
  XFAIL_ANALYSIS) -> dev-docs/. README/CONTRIBUTING/llms.txt links and
  the llms-full generator follow.
- docs/_static/ stays put: its images are referenced by absolute
  raw.githubusercontent URLs from the org profile, PyPI release pages
  (README is the long_description), and example notebooks — the paths
  are effectively frozen public URLs and must not move.
- /research/ added to .gitignore so local strategy notes can't be
  committed into the future docs source by accident.

Co-Authored-By: Claude Fable 5 <[email protected]>
The canonical chDB documentation is clickhouse.com/docs/chdb. Add a
banner to the ReadTheDocs landing page announcing that this site stops
updating on September 17, 2026 (two months after the docs-platform
cutover) and will then redirect to the official docs.

Co-Authored-By: Claude Fable 5 <[email protected]>
@ShawnChen-Sirius ShawnChen-Sirius changed the title docs: clear docs/ for the Mintlify docs source [WIP] docs: clear docs/ for the Mintlify docs source Jul 7, 2026
ShawnChen-Sirius and others added 4 commits July 8, 2026 11:59
Import the docs/products/chdb slice of the aggregated docs in
ClickHouse/ClickHouse (44 .mdx pages + navigation.json, snapshot of
2026-07-08) into docs/. This is a staging copy so structure, tooling,
and CI verification can be prepared ahead of the docs-platform cutover
— it is NOT the source of truth yet, and docs/README.md says so.
Content edits keep flowing through the current live pipeline until the
cutover stabilizes; a final re-sync happens before this directory takes
over as the canonical source.

Co-Authored-By: Claude Fable 5 <[email protected]>
…tree

--docs-source local builds llms-full.txt from this repo's docs/ staging
tree (Mintlify .mdx + aggregator-internal '/products/chdb/…' links,
resolved to published slugs through the frontmatter slug map). The
default stays remote — the live docs pipeline remains the source of
truth until after the platform cutover; flipping the default is the
source-of-truth switch.

Both modes produce identical published URLs for all 44 pages; byte
difference is format-conversion noise only.

Also fixes a real bug in the shipped llms-full.txt: the MDX import
stripper matched any line starting with 'import', which deleted the
import lines from Python and Go code examples (20 'import chdb' lines
were missing). It now only strips ESM imports (a 'from "…"' clause or
a bare string import). Regenerated.

Co-Authored-By: Claude Fable 5 <[email protected]>
Adopt the docs verification workflow from clickhouse-connect: fetch the
check driver from ClickHouse/ClickHouse, replace the aggregator's
docs/products/chdb slice with this repo's docs/ content, and run the
scoped Mintlify checks (docs.json schema, navigation, MDX parse,
snippet imports). Runs on any PR touching docs/.

Only page content is staged for the check — README.md (the staging
notice) and _static/ (images pinned to frozen public URLs) are
filtered out, the same whitelist a future sync workflow will use.

Co-Authored-By: Claude Fable 5 <[email protected]>
A legacy .gitignore rule for Sphinx autodoc output (docs/api/*)
silently dropped api/python.mdx from the staged docs snapshot, which
the new docs-verify CI caught on its first run: navigation references
products/chdb/api/python but the page was absent. Point the Sphinx
build-output rules at sphinx/ (where that build now lives) and commit
the page.

Co-Authored-By: Claude Fable 5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant