diff --git a/AGENTS.md b/AGENTS.md index 771f398..747116b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -126,6 +126,20 @@ If none fit, write a custom lens with three fields: - `description`: what kind of beliefs to extract. - `question`: the guiding question doxa should ask of each source. +## Offer a starter pack (optional) + +Before building from scratch, offer a ready-made base -- the fastest path to value, +and entirely optional: + +```bash +doxa packs list # the available curated bases +doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes from + # Lenny's Podcast, View From The Top, Paul Graham, YC +``` + +`doxa packs install` creates a base if there isn't one. Ask before installing, and +skip it if the user would rather curate their own sources (the First Source flow below). + ## First Source Ask the user for the first trusted source, then ingest it: diff --git a/README.md b/README.md index d07c6b2..34b49f7 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,11 @@ Example: Keyword search works with no API key, database, embedding model, or network. +> **Want value instantly?** `doxa packs install startup-wisdom` installs an optional, +> ready-mined base of ~14k founder/product/growth beliefs from Lenny's Podcast, View +> From The Top, Paul Graham, and Y Combinator -- each pinned to a verbatim quote + +> source link. Browse packs with `doxa packs list`. + --- ## Docs @@ -81,80 +86,20 @@ Keyword search works with no API key, database, embedding model, or network. - [Retrieval](docs/retrieval.md) · [Writing a lens](docs/writing-a-lens.md) · [Schema](docs/schema.md) · [Architecture](docs/architecture.md) - [Agent skill](docs/skill.md) · [AGENTS.md](AGENTS.md) · [skill/SKILL.md](skill/SKILL.md) - [Example configs gallery](examples/README.md) -- copy-ready `doxa.yaml` templates +- [Example Q&A](docs/examples-qa.md) -- grounded answers on the demo base --- ## Example questions & answers -These examples are from the bundled public-domain demo and were generated with -`--search keyword`. Keyword is the default, so the reproduce lines use the -shorter command form. - -
-Should I trust my own judgment over the crowd? - -The demo retrieves Emerson's belief that self-trust is necessary, but it also surfaces a real Socratic tension: confidence in judgment should be held alongside intellectual humility. - -`source=Self-Reliance / Ralph Waldo Emerson / 1841` -`quote="Ralph Waldo Emerson: Trust thyself: every heart vibrates to that iron string."` - -`source=Apology / Plato, translated by Benjamin Jowett / 1892` -`quote="Socrates: I know that I have no wisdom, small or great."` - -Reproduce: `doxa query "Should I trust my own judgment over the crowd?"` - -
- -
-Is it rational to fear death? - -The demo grounds Socrates' answer in the belief that a good person need not fear ultimate harm from death. - -`source=Apology / Plato, translated by Benjamin Jowett / 1892` -`quote="Socrates: Wherefore, O judges, be of good cheer about death, and know of a certainty, that no evil can happen to a good man, either in life or after death."` - -Reproduce: `doxa query "Is it rational to fear death?"` - -
- -
-Should a republic eliminate liberty to stop faction? - -Madison complicates the idea: liberty feeds faction, but the retrieved belief says liberty is not a condition a republic can simply extinguish. - -`source=Federalist No. 10 / James Madison / 1787` -`quote="James Madison: Liberty is to faction what air is to fire, an aliment without which it instantly expires."` - -`source=Federalist No. 10 / James Madison / 1787` -`quote="James Madison: The latent causes of faction are thus sown in the nature of man."` - -Reproduce: `doxa query "Should a republic eliminate liberty to stop faction?"` - -
- -
-What is the highest standard for the mind? - -The demo answers through Emerson: the highest retrieved standard is the integrity of one's own mind. - -`source=Self-Reliance / Ralph Waldo Emerson / 1841` -`quote="Ralph Waldo Emerson: Nothing is at last sacred but the integrity of your own mind."` - -Reproduce: `doxa query "What is the highest standard for the mind?"` - -
- -
-What makes a life worth living? - -The demo retrieves Socrates' belief that a worthy life requires examination. - -`source=Apology / Plato, translated by Benjamin Jowett / 1892` -`quote="Socrates: The unexamined life is not worth living."` - -Reproduce: `doxa query "What makes a life worth living?"` +A walkthrough of grounded answers on the bundled demo (Emerson, Plato, Madison) -- +including a real Socratic tension doxa surfaces rather than flattens -- is in +[docs/examples-qa.md](docs/examples-qa.md). Or just run it: -
+```bash +doxa demo +doxa query "Should I trust my own judgment over the crowd?" +``` --- @@ -181,6 +126,9 @@ read the same source and produce different belief bases. ## doxa vs. a RAG app +
+Short version: it's RAG plumbing wrapped around a curated belief + quote graph. Expand for the full answer. + Under the hood, doxa *is* RAG plumbing -- BM25 + pgvector hybrid over an embedded store. The difference is what sits in the index and what happens around it. @@ -210,6 +158,8 @@ The honest one-liner: doxa is RAG plumbing wrapped around a curated belief-and-q graph with typed epistemic honesty, queried as a point of view rather than a document search. The retriever was never the moat; the data model and the discipline on top are. +
+ --- ## How it works @@ -240,49 +190,17 @@ rebuilt from JSONL at any time. ## Install -Core install is intentionally small. It supports config loading, demo data, -keyword retrieval, evaluation, and plain text / URL ingestion: - -```bash -python -m pip install -e . -``` - -Install every optional runtime integration: - -```bash -python -m pip install -e ".[all]" -``` - -Install by requirements file instead: - -```bash -python -m pip install -r requirements.txt -``` - -For test/dev work: - -```bash -python -m pip install -r requirements-dev.txt -python -m pytest -q -``` - -Extras are also available individually: +Core is intentionally small (config, demo, keyword query, eval, text/URL ingest): ```bash -python -m pip install -e ".[embeddings,postgres]" -python -m pip install -e ".[pdf,youtube]" -python -m pip install -e ".[openai,anthropic]" +python -m pip install -e . # core +python -m pip install -e ".[all]" # every optional integration ``` -| Extra | Adds | Used by | -| --- | --- | --- | -| core | `PyYAML` | config, demo, keyword query, eval | -| `embeddings` | `fastembed` | semantic vectors | -| `postgres` | `psycopg2-binary`, `pgvector` | pgvector indexing/search | -| `pdf` | `PyMuPDF` | PDF ingestion | -| `youtube` | `yt-dlp` | YouTube transcript ingestion | -| `openai` | `openai` | OpenAI and OpenAI-compatible mining | -| `anthropic` | `anthropic` | Anthropic mining | +Install only the extras you need -- `embeddings` (semantic vectors), `postgres` +(pgvector), `pdf`, `youtube`, `openai`, `anthropic`, e.g. +`pip install -e ".[pdf,youtube]"`. Full extras matrix + dev setup: +[docs/configuration.md](docs/configuration.md). --- diff --git a/docs/configuration.md b/docs/configuration.md index 3471c7b..a43c8c7 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -92,3 +92,33 @@ doxa domains export ``` All domain subcommands accept `--config`. + +## Install extras + +Core install (`pip install -e .`) ships only `PyYAML` and covers config, demo, +keyword query, eval, and text/URL ingestion. Add extras as needed: + +```bash +python -m pip install -e ".[all]" # every integration +python -m pip install -e ".[embeddings,postgres]" # semantic + hybrid search +python -m pip install -e ".[pdf,youtube]" # PDF + YouTube ingestion +python -m pip install -e ".[openai,anthropic]" # API mining providers +python -m pip install -r requirements.txt # or pin via requirements file +``` + +| Extra | Adds | Used by | +| --- | --- | --- | +| core | `PyYAML` | config, demo, keyword query, eval | +| `embeddings` | `fastembed` | semantic vectors | +| `postgres` | `psycopg2-binary`, `pgvector` | pgvector indexing/search | +| `pdf` | `PyMuPDF` | PDF ingestion | +| `youtube` | `yt-dlp` | YouTube transcript ingestion | +| `openai` | `openai` | OpenAI and OpenAI-compatible mining | +| `anthropic` | `anthropic` | Anthropic mining | + +Dev/test setup: + +```bash +python -m pip install -r requirements-dev.txt +python -m pytest -q +``` diff --git a/docs/examples-qa.md b/docs/examples-qa.md new file mode 100644 index 0000000..15f1ba6 --- /dev/null +++ b/docs/examples-qa.md @@ -0,0 +1,70 @@ +# Example questions & answers + +These examples are from the bundled public-domain demo and were generated with +`--search keyword` (the default), so the reproduce lines use the shorter form. + +
+Should I trust my own judgment over the crowd? + +The demo retrieves Emerson's belief that self-trust is necessary, but it also surfaces a real Socratic tension: confidence in judgment should be held alongside intellectual humility. + +`source=Self-Reliance / Ralph Waldo Emerson / 1841` +`quote="Ralph Waldo Emerson: Trust thyself: every heart vibrates to that iron string."` + +`source=Apology / Plato, translated by Benjamin Jowett / 1892` +`quote="Socrates: I know that I have no wisdom, small or great."` + +Reproduce: `doxa query "Should I trust my own judgment over the crowd?"` + +
+ +
+Is it rational to fear death? + +The demo grounds Socrates' answer in the belief that a good person need not fear ultimate harm from death. + +`source=Apology / Plato, translated by Benjamin Jowett / 1892` +`quote="Socrates: Wherefore, O judges, be of good cheer about death, and know of a certainty, that no evil can happen to a good man, either in life or after death."` + +Reproduce: `doxa query "Is it rational to fear death?"` + +
+ +
+Should a republic eliminate liberty to stop faction? + +Madison complicates the idea: liberty feeds faction, but the retrieved belief says liberty is not a condition a republic can simply extinguish. + +`source=Federalist No. 10 / James Madison / 1787` +`quote="James Madison: Liberty is to faction what air is to fire, an aliment without which it instantly expires."` + +`source=Federalist No. 10 / James Madison / 1787` +`quote="James Madison: The latent causes of faction are thus sown in the nature of man."` + +Reproduce: `doxa query "Should a republic eliminate liberty to stop faction?"` + +
+ +
+What is the highest standard for the mind? + +The demo answers through Emerson: the highest retrieved standard is the integrity of one's own mind. + +`source=Self-Reliance / Ralph Waldo Emerson / 1841` +`quote="Ralph Waldo Emerson: Nothing is at last sacred but the integrity of your own mind."` + +Reproduce: `doxa query "What is the highest standard for the mind?"` + +
+ +
+What makes a life worth living? + +The demo retrieves Socrates' belief that a worthy life requires examination. + +`source=Apology / Plato, translated by Benjamin Jowett / 1892` +`quote="Socrates: The unexamined life is not worth living."` + +Reproduce: `doxa query "What makes a life worth living?"` + +
diff --git a/doxa/_assets/skill/SKILL.md b/doxa/_assets/skill/SKILL.md index 51faaf2..96b2b2e 100644 --- a/doxa/_assets/skill/SKILL.md +++ b/doxa/_assets/skill/SKILL.md @@ -56,16 +56,21 @@ attribution, source, or belief the CLI did not return. ## 3. Start or grow a base -### Fastest start: install a curated pack +### Fastest start: offer a curated pack (optional) -If the user wants value immediately, install a starter pack instead of building -from scratch (`doxa packs install` creates a base if there isn't one): +If the user wants value immediately, offer a ready-made starter pack instead of +building from scratch. It's optional -- ask first, and skip it if they'd rather +curate their own base. `doxa packs install` creates a base if there isn't one: ```bash -doxa packs list # curated, ready-made bases -doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes +doxa packs list # the available curated bases +doxa packs install startup-wisdom # OPTIONAL: ~14k founder/product/growth beliefs + + # quotes from Lenny's Podcast, View From The Top, + # Paul Graham, and Y Combinator (each with a source link) ``` +Prefer the user's own sources? Skip the pack and use the lens + ingest flow below. + ### Pick a lens first (don't make the user invent one) A lens is the question doxa asks of every source -- it shapes every belief diff --git a/skill/SKILL.md b/skill/SKILL.md index 51faaf2..96b2b2e 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -56,16 +56,21 @@ attribution, source, or belief the CLI did not return. ## 3. Start or grow a base -### Fastest start: install a curated pack +### Fastest start: offer a curated pack (optional) -If the user wants value immediately, install a starter pack instead of building -from scratch (`doxa packs install` creates a base if there isn't one): +If the user wants value immediately, offer a ready-made starter pack instead of +building from scratch. It's optional -- ask first, and skip it if they'd rather +curate their own base. `doxa packs install` creates a base if there isn't one: ```bash -doxa packs list # curated, ready-made bases -doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes +doxa packs list # the available curated bases +doxa packs install startup-wisdom # OPTIONAL: ~14k founder/product/growth beliefs + + # quotes from Lenny's Podcast, View From The Top, + # Paul Graham, and Y Combinator (each with a source link) ``` +Prefer the user's own sources? Skip the pack and use the lens + ingest flow below. + ### Pick a lens first (don't make the user invent one) A lens is the question doxa asks of every source -- it shapes every belief