docs: llms-full.txt link integrity, MCP entry, and package layering#600
Merged
ShawnChen-Sirius merged 2 commits intoJul 7, 2026
Merged
Conversation
…ge layering
llms-full.txt / generator:
- Rewrite relative link targets to absolute URLs: site-absolute doc
paths get the clickhouse.com/docs prefix, file-relative targets
resolve through a source-path -> published-slug map (filenames and
slugs diverge, e.g. guides/querying-pandas.md publishes at
/chdb/guides/pandas), and repo-relative targets in ARCHITECTURE.md
and binding READMEs point at github.com blob / raw URLs.
- Replace YouTube iframe embeds with plain watch links and drop
<Image> tags whose imports were already stripped.
- Leave file:/data: connection-string examples untouched.
- Verified: all 59 clickhouse.com/docs links and all GitHub links in
the regenerated file resolve (one upstream-content exception:
JeffSackmann/tennis_atp, a third-party dataset repo referenced by
the JupySQL guide, is gone from GitHub).
llms.txt:
- Add MCP entry: chDB is served through the official mcp-clickhouse
server ('pip install mcp-clickhouse[chdb]', CHDB_ENABLED=true,
run_chdb_select_query tool).
- Explain package layering: chdb-core builds the libchdb engine
binaries; chdb (Python) and the node/bun/go/rust packages are thin
bindings over the same C ABI.
Co-Authored-By: Claude Fable 5 <[email protected]>
- Full sweep of all 132 unique link targets in llms-full.txt found two genuinely dead ones: the JeffSackmann/tennis_atp dataset repo (404) and jupysql.ploomber.io (domain no longer resolves). The generator now unwraps links matching DEAD_LINK_PREFIXES to their anchor text; both are removed from the regenerated file. - The six 403 targets (npmjs, crates.io, clickpy, badge.fury.io) are bot walls, not dead pages — the packages were verified alive via the npm registry and crates.io APIs — so 403 stays warning-only. - The CI link check now covers llms-full.txt as well as llms.txt (markdown targets plus '**URL:**' provenance lines) and treats unreachable-after-retries (000) as dead, since a vanished domain never returns 404. Known upstream content issue, out of scope here: the JupySQL guide's code sample still downloads from the removed tennis_atp repo, so the tutorial itself is broken in clickhouse-docs and needs a new dataset. Co-Authored-By: Claude Fable 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #599 (these two commits were pushed after it merged, so they land here).
llms-full.txt: every link is now absolute and alive
A full sweep of all 132 unique link targets found four kinds of broken links, all fixed in the generator:
/interfaces/formats) now carry theclickhouse.com/docsprefix.guides/querying-pandas.md) resolve through a source-path → published-slug map, because filenames and slugs diverge (that file publishes at/chdb/guides/pandas).[Watch the video](watch URL)links; dangling<Image>tags (their MDX imports are stripped) are removed;file:/data:connection-string examples are left untouched.Two targets were verified genuinely dead and are unwrapped to plain text via a
DEAD_LINK_PREFIXESlist: theJeffSackmann/tennis_atpdataset repo (404) andjupysql.ploomber.io(domain no longer resolves). The six 403 targets (npmjs, crates.io, clickpy, badge.fury.io) are bot walls — packages verified alive via the npm registry and crates.io APIs.Result: 130 unique links in the regenerated file, zero dead.
Known upstream issue, out of scope: the JupySQL guide's code sample still downloads from the removed tennis_atp repo, so that tutorial is broken in clickhouse-docs itself.
llms.txt: two additions
pip install 'mcp-clickhouse[chdb]',CHDB_ENABLED=true, agents get arun_chdb_select_querytool.libchdbengine binaries;chdb(Python) and the node/bun/go/rust packages are thin bindings over the same C ABI — plus routing guidance for where issues belong.CI: dead-link gate now covers both files
The link check walks llms.txt and llms-full.txt (markdown targets plus
**URL:**provenance lines), and treats unreachable-after-retries (000) as dead — a vanished domain never returns 404. 403/429 stay warning-only.🤖 Generated with Claude Code
Note
Fix link integrity and add MCP entry in
llms.txtandllms-full.txthttps://clickhouse.com/docs/...URLs, convert iframe embeds to YouTube watch links, remove<Image />tags, and unwrap known-dead external links to plain text.github.comblob orraw.githubusercontent.comURLs depending on whether they point to images.llms.txtandllms-full.txt, treat HTTP 000 as dead alongside 404/410, and log 403/429 as warnings.Macroscope summarized 3fcefde.