Feature Request: Add support for generating `llms.txt` files

[IndrajeetPatil]

The llms.txt format provides a standardized Markdown document designed for LLMs and AI coding assistants. Several Python packages now include llms.txt files on their documentation sites (e.g., PydanticAI, LangGraph/LangChain) to improve discoverability and usage by AI tools.

Proposal:
Should {pkgdown} automatically generate llms.txt files for R packages?
This would help R packages become more accessible to LLM-based development workflows.

Benefits:

• Standardized format for AI tools to understand package functionality
• Improved package discoverability in AI-assisted coding
• Alignment with emerging best practices in documentation

[e-kotov]

Great idea! Perhaps https://github.com/e-kotov/rdocdump ( https://cran.r-project.org/package=rdocdump ) could serve as a backend for this. It can extract the package docs (optionally with source code) and vignettes into a single txt file.

To run in current R package source folder:

rdd_to_txt(
  pkg = ".",
  file = "llms.txt",
  content = c("docs", "vignettes")
)

[hadley]

I think there are two pieces here. Firstly, I think we need to generate .md versions of all reference pages and articles (that will require a little thought to avoid evaluating examples multiple times). Then we can create an llm.txt that includes the README (possibly with some convention to include/exclude specific chunks only for llms) and then an index to the reference pages and articles.

We might also want to consider generating an llms-full.txt that inlines all reference pages & articles, but this is likely to be problematically large for decent sized packages.

[maelle]

that will require a little thought to avoid evaluating examples multiple times

Could we call Pandoc on the HTML pages to transform them back to Markdown?

[hadley]

@maelle I was thinking we'd just reimplement the HTML conversion with markdown, but maybe using pandoc would be easier?

[maelle]

#2917

[maelle]

The "testing" articles of pkgdown don't look too informative in the llms.txt 😅

[maelle]

If a website has an RSS feed, it is referenced in the metadata as

<link rel="alternate" type="application/rss+xml" title="title"
    href='link' />

I wonder whether there is such a standard for

• llms.txt
• <url>.md

Probably not yet as llms.txt is quite new?

[hadley]

@maelle yeah, I think it's currently more like robots.txt where it's just looked for in an expected location, no other tooling yet.