Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,20 @@ If none fit, write a custom lens with three fields:
- `description`: what kind of beliefs to extract.
- `question`: the guiding question doxa should ask of each source.

## Offer a starter pack (optional)

Before building from scratch, offer a ready-made base -- the fastest path to value,
and entirely optional:

```bash
doxa packs list # the available curated bases
doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes from
# Lenny's Podcast, View From The Top, Paul Graham, YC
```

`doxa packs install` creates a base if there isn't one. Ask before installing, and
skip it if the user would rather curate their own sources (the First Source flow below).

## First Source

Ask the user for the first trusted source, then ingest it:
Expand Down
132 changes: 25 additions & 107 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,11 @@ Example:

Keyword search works with no API key, database, embedding model, or network.

> **Want value instantly?** `doxa packs install startup-wisdom` installs an optional,
> ready-mined base of ~14k founder/product/growth beliefs from Lenny's Podcast, View
> From The Top, Paul Graham, and Y Combinator -- each pinned to a verbatim quote +
> source link. Browse packs with `doxa packs list`.

---

## Docs
Expand All @@ -81,80 +86,20 @@ Keyword search works with no API key, database, embedding model, or network.
- [Retrieval](docs/retrieval.md) · [Writing a lens](docs/writing-a-lens.md) · [Schema](docs/schema.md) · [Architecture](docs/architecture.md)
- [Agent skill](docs/skill.md) · [AGENTS.md](AGENTS.md) · [skill/SKILL.md](skill/SKILL.md)
- [Example configs gallery](examples/README.md) -- copy-ready `doxa.yaml` templates
- [Example Q&A](docs/examples-qa.md) -- grounded answers on the demo base

---

## Example questions & answers

These examples are from the bundled public-domain demo and were generated with
`--search keyword`. Keyword is the default, so the reproduce lines use the
shorter command form.

<details>
<summary>Should I trust my own judgment over the crowd?</summary>

The demo retrieves Emerson's belief that self-trust is necessary, but it also surfaces a real Socratic tension: confidence in judgment should be held alongside intellectual humility.

`source=Self-Reliance / Ralph Waldo Emerson / 1841`
`quote="Ralph Waldo Emerson: Trust thyself: every heart vibrates to that iron string."`

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: I know that I have no wisdom, small or great."`

Reproduce: `doxa query "Should I trust my own judgment over the crowd?"`

</details>

<details>
<summary>Is it rational to fear death?</summary>

The demo grounds Socrates' answer in the belief that a good person need not fear ultimate harm from death.

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: Wherefore, O judges, be of good cheer about death, and know of a certainty, that no evil can happen to a good man, either in life or after death."`

Reproduce: `doxa query "Is it rational to fear death?"`

</details>

<details>
<summary>Should a republic eliminate liberty to stop faction?</summary>

Madison complicates the idea: liberty feeds faction, but the retrieved belief says liberty is not a condition a republic can simply extinguish.

`source=Federalist No. 10 / James Madison / 1787`
`quote="James Madison: Liberty is to faction what air is to fire, an aliment without which it instantly expires."`

`source=Federalist No. 10 / James Madison / 1787`
`quote="James Madison: The latent causes of faction are thus sown in the nature of man."`

Reproduce: `doxa query "Should a republic eliminate liberty to stop faction?"`

</details>

<details>
<summary>What is the highest standard for the mind?</summary>

The demo answers through Emerson: the highest retrieved standard is the integrity of one's own mind.

`source=Self-Reliance / Ralph Waldo Emerson / 1841`
`quote="Ralph Waldo Emerson: Nothing is at last sacred but the integrity of your own mind."`

Reproduce: `doxa query "What is the highest standard for the mind?"`

</details>

<details>
<summary>What makes a life worth living?</summary>

The demo retrieves Socrates' belief that a worthy life requires examination.

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: The unexamined life is not worth living."`

Reproduce: `doxa query "What makes a life worth living?"`
A walkthrough of grounded answers on the bundled demo (Emerson, Plato, Madison) --
including a real Socratic tension doxa surfaces rather than flattens -- is in
[docs/examples-qa.md](docs/examples-qa.md). Or just run it:

</details>
```bash
doxa demo
doxa query "Should I trust my own judgment over the crowd?"
```

---

Expand All @@ -181,6 +126,9 @@ read the same source and produce different belief bases.

## doxa vs. a RAG app

<details>
<summary>Short version: it's RAG plumbing wrapped around a curated belief + quote graph. Expand for the full answer.</summary>

Under the hood, doxa *is* RAG plumbing -- BM25 + pgvector hybrid over an embedded
store. The difference is what sits in the index and what happens around it.

Expand Down Expand Up @@ -210,6 +158,8 @@ The honest one-liner: doxa is RAG plumbing wrapped around a curated belief-and-q
graph with typed epistemic honesty, queried as a point of view rather than a document
search. The retriever was never the moat; the data model and the discipline on top are.

</details>

---

## How it works
Expand Down Expand Up @@ -240,49 +190,17 @@ rebuilt from JSONL at any time.

## Install

Core install is intentionally small. It supports config loading, demo data,
keyword retrieval, evaluation, and plain text / URL ingestion:

```bash
python -m pip install -e .
```

Install every optional runtime integration:

```bash
python -m pip install -e ".[all]"
```

Install by requirements file instead:

```bash
python -m pip install -r requirements.txt
```

For test/dev work:

```bash
python -m pip install -r requirements-dev.txt
python -m pytest -q
```

Extras are also available individually:
Core is intentionally small (config, demo, keyword query, eval, text/URL ingest):

```bash
python -m pip install -e ".[embeddings,postgres]"
python -m pip install -e ".[pdf,youtube]"
python -m pip install -e ".[openai,anthropic]"
python -m pip install -e . # core
python -m pip install -e ".[all]" # every optional integration
```

| Extra | Adds | Used by |
| --- | --- | --- |
| core | `PyYAML` | config, demo, keyword query, eval |
| `embeddings` | `fastembed` | semantic vectors |
| `postgres` | `psycopg2-binary`, `pgvector` | pgvector indexing/search |
| `pdf` | `PyMuPDF` | PDF ingestion |
| `youtube` | `yt-dlp` | YouTube transcript ingestion |
| `openai` | `openai` | OpenAI and OpenAI-compatible mining |
| `anthropic` | `anthropic` | Anthropic mining |
Install only the extras you need -- `embeddings` (semantic vectors), `postgres`
(pgvector), `pdf`, `youtube`, `openai`, `anthropic`, e.g.
`pip install -e ".[pdf,youtube]"`. Full extras matrix + dev setup:
[docs/configuration.md](docs/configuration.md).

---

Expand Down
30 changes: 30 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,33 @@ doxa domains export
```

All domain subcommands accept `--config`.

## Install extras

Core install (`pip install -e .`) ships only `PyYAML` and covers config, demo,
keyword query, eval, and text/URL ingestion. Add extras as needed:

```bash
python -m pip install -e ".[all]" # every integration
python -m pip install -e ".[embeddings,postgres]" # semantic + hybrid search
python -m pip install -e ".[pdf,youtube]" # PDF + YouTube ingestion
python -m pip install -e ".[openai,anthropic]" # API mining providers
python -m pip install -r requirements.txt # or pin via requirements file
```

| Extra | Adds | Used by |
| --- | --- | --- |
| core | `PyYAML` | config, demo, keyword query, eval |
| `embeddings` | `fastembed` | semantic vectors |
| `postgres` | `psycopg2-binary`, `pgvector` | pgvector indexing/search |
| `pdf` | `PyMuPDF` | PDF ingestion |
| `youtube` | `yt-dlp` | YouTube transcript ingestion |
| `openai` | `openai` | OpenAI and OpenAI-compatible mining |
| `anthropic` | `anthropic` | Anthropic mining |

Dev/test setup:

```bash
python -m pip install -r requirements-dev.txt
python -m pytest -q
```
70 changes: 70 additions & 0 deletions docs/examples-qa.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Example questions & answers

These examples are from the bundled public-domain demo and were generated with
`--search keyword` (the default), so the reproduce lines use the shorter form.

<details>
<summary>Should I trust my own judgment over the crowd?</summary>

The demo retrieves Emerson's belief that self-trust is necessary, but it also surfaces a real Socratic tension: confidence in judgment should be held alongside intellectual humility.

`source=Self-Reliance / Ralph Waldo Emerson / 1841`
`quote="Ralph Waldo Emerson: Trust thyself: every heart vibrates to that iron string."`

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: I know that I have no wisdom, small or great."`

Reproduce: `doxa query "Should I trust my own judgment over the crowd?"`

</details>

<details>
<summary>Is it rational to fear death?</summary>

The demo grounds Socrates' answer in the belief that a good person need not fear ultimate harm from death.

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: Wherefore, O judges, be of good cheer about death, and know of a certainty, that no evil can happen to a good man, either in life or after death."`

Reproduce: `doxa query "Is it rational to fear death?"`

</details>

<details>
<summary>Should a republic eliminate liberty to stop faction?</summary>

Madison complicates the idea: liberty feeds faction, but the retrieved belief says liberty is not a condition a republic can simply extinguish.

`source=Federalist No. 10 / James Madison / 1787`
`quote="James Madison: Liberty is to faction what air is to fire, an aliment without which it instantly expires."`

`source=Federalist No. 10 / James Madison / 1787`
`quote="James Madison: The latent causes of faction are thus sown in the nature of man."`

Reproduce: `doxa query "Should a republic eliminate liberty to stop faction?"`

</details>

<details>
<summary>What is the highest standard for the mind?</summary>

The demo answers through Emerson: the highest retrieved standard is the integrity of one's own mind.

`source=Self-Reliance / Ralph Waldo Emerson / 1841`
`quote="Ralph Waldo Emerson: Nothing is at last sacred but the integrity of your own mind."`

Reproduce: `doxa query "What is the highest standard for the mind?"`

</details>

<details>
<summary>What makes a life worth living?</summary>

The demo retrieves Socrates' belief that a worthy life requires examination.

`source=Apology / Plato, translated by Benjamin Jowett / 1892`
`quote="Socrates: The unexamined life is not worth living."`

Reproduce: `doxa query "What makes a life worth living?"`

</details>
15 changes: 10 additions & 5 deletions doxa/_assets/skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,21 @@ attribution, source, or belief the CLI did not return.

## 3. Start or grow a base

### Fastest start: install a curated pack
### Fastest start: offer a curated pack (optional)

If the user wants value immediately, install a starter pack instead of building
from scratch (`doxa packs install` creates a base if there isn't one):
If the user wants value immediately, offer a ready-made starter pack instead of
building from scratch. It's optional -- ask first, and skip it if they'd rather
curate their own base. `doxa packs install` creates a base if there isn't one:

```bash
doxa packs list # curated, ready-made bases
doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes
doxa packs list # the available curated bases
doxa packs install startup-wisdom # OPTIONAL: ~14k founder/product/growth beliefs +
# quotes from Lenny's Podcast, View From The Top,
# Paul Graham, and Y Combinator (each with a source link)
```

Prefer the user's own sources? Skip the pack and use the lens + ingest flow below.

### Pick a lens first (don't make the user invent one)

A lens is the question doxa asks of every source -- it shapes every belief
Expand Down
15 changes: 10 additions & 5 deletions skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,16 +56,21 @@ attribution, source, or belief the CLI did not return.

## 3. Start or grow a base

### Fastest start: install a curated pack
### Fastest start: offer a curated pack (optional)

If the user wants value immediately, install a starter pack instead of building
from scratch (`doxa packs install` creates a base if there isn't one):
If the user wants value immediately, offer a ready-made starter pack instead of
building from scratch. It's optional -- ask first, and skip it if they'd rather
curate their own base. `doxa packs install` creates a base if there isn't one:

```bash
doxa packs list # curated, ready-made bases
doxa packs install startup-wisdom # ~14k founder/product/growth beliefs + quotes
doxa packs list # the available curated bases
doxa packs install startup-wisdom # OPTIONAL: ~14k founder/product/growth beliefs +
# quotes from Lenny's Podcast, View From The Top,
# Paul Graham, and Y Combinator (each with a source link)
```

Prefer the user's own sources? Skip the pack and use the lens + ingest flow below.

### Pick a lens first (don't make the user invent one)

A lens is the question doxa asks of every source -- it shapes every belief
Expand Down
Loading