Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ __pycache__/
.pytest_cache/
.mypy_cache/
.ruff_cache/
.coverage
htmlcov/
.venv/
.pip-cache/
venv/
Expand Down
16 changes: 16 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ users often don't know what's available.
| Ingest | `doxa ingest <file\|url\|pdf\|youtube\|->` |
| Hard pages | `doxa ingest <url> --via jina\|firecrawl\|brightdata\|claude\|codex\|hermes` `[--mode browser\|extract --prompt ...] [--yolo]` |
| Answer | `doxa query "<q>" --answer` / `--json` / `--domain <slug>` / `--search hybrid` |
| Voice (optional) | `doxa query "<q>" --present hawking` -- answer as a quest; `doxa present --list` |
| Honest | `doxa eval` · `doxa sources list\|remove <id>` · `doxa domains set <slug> <0-10>` |
| Agent skill | `doxa skill install --harness claude-code\|codex\|hermes\|openclaw` |

Expand Down Expand Up @@ -223,6 +224,21 @@ sources:
argv: ["my-mcp-fetch", "{url}"]
```

## Presentation Modes

By default doxa returns evidence and you write the answer in your own voice. For
reflective or big-question prompts, offer the optional `hawking` voice:

```bash
doxa present --list # available voices
doxa query "<user question>" --answer --present hawking
```

A non-plain mode prepends a composition directive to the query output. Follow it
for voice and shape only -- it never relaxes the grounding rule in Discipline
below. Set `presentation.default` in `doxa.yaml` to make a mode sticky. See
[docs/presentation.md](docs/presentation.md).

## Install As A Harness Skill

Offer to register doxa as a skill for the user's harness:
Expand Down
30 changes: 29 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Keyword search works with no API key, database, embedding model, or network.
## Docs

- [Configuration](docs/configuration.md) · [Providers](docs/providers.md) · [Ingestion](docs/ingestion.md)
- [Retrieval](docs/retrieval.md) · [Writing a lens](docs/writing-a-lens.md) · [Schema](docs/schema.md) · [Architecture](docs/architecture.md)
- [Retrieval](docs/retrieval.md) · [Writing a lens](docs/writing-a-lens.md) · [Schema](docs/schema.md) · [Architecture](docs/architecture.md) · [Presentation modes](docs/presentation.md)
- [Agent skill](docs/skill.md) · [AGENTS.md](AGENTS.md) · [skill/SKILL.md](skill/SKILL.md)
- [Example configs gallery](examples/README.md) -- copy-ready `doxa.yaml` templates
- [Example Q&A](docs/examples-qa.md) -- grounded answers on the demo base
Expand All @@ -103,6 +103,34 @@ doxa query "Should I trust my own judgment over the crowd?"

---

## Presentation modes (optional)

doxa returns evidence; the agent reading it writes the prose answer. A
**presentation mode** is an optional voice for that final step. The default is
`plain` (output unchanged). The flagship optional mode is `hawking`, distilled
from how Stephen Hawking presented evidence in the first two chapters of *A Brief
History of Time*: open on the ancient question rather than the apparatus, let the
evidence arrive as a procession of minds, say the largest thing in the plainest
sentence, keep the genuine strangeness intact, and turn the answer into a
question about what we can know.

```bash
doxa present --list # list modes
doxa present hawking # read the composition directive
doxa query "did time have a beginning?" --answer --present hawking
```

With a non-plain mode, `doxa query` prints a `=== doxa presentation directive ===`
block before the evidence; an agent reads it and composes in that voice. The mode
changes voice and shape only -- it never loosens the verbatim-grounding rule, so
the answer is still built only from returned beliefs and quotes. With `--json`, a
non-plain mode wraps output as `{"presentation": {...}, "results": [...]}`; `plain`
stays a bare list. Make a mode sticky in `doxa.yaml` with `presentation.default`,
or add your own by registering a `PresentationProfile` in `doxa/present.py`. See
[docs/presentation.md](docs/presentation.md).

---

## Why doxa exists

Fluency is where hallucinations hide: a summarizer can compress, overstate,
Expand Down
11 changes: 11 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,18 @@ doxa is deliberately small in the core path:
5. Verify each quote is a real substring of the source after whitespace normalization.
6. Store beliefs, quotes, and source text as JSONL.
7. Retrieve with keyword BM25, optional pgvector semantic search, or hybrid RRF.
8. Optionally emit a presentation directive so the consuming agent composes the answer in a chosen voice.

JSONL is the source of truth. Postgres/pgvector is an optional index, not the
canonical store.

## Where answers are composed

doxa retrieves evidence; the agent reading the output composes the prose answer
(`doxa query --answer` is the built-in deterministic renderer). That split is
where presentation modes live. A `PresentationProfile` (`doxa/present.py`) is an
optional composition directive the CLI prints alongside the evidence at query
time. It shapes voice and structure only -- the retrieved beliefs and verbatim
quotes remain the sole evidence, so a mode can never become a license to
embellish. The default is `plain` (no directive). See [presentation.md](presentation.md).

89 changes: 89 additions & 0 deletions docs/presentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Presentation modes

doxa retrieves evidence; the agent reading the output composes the prose answer.
A **presentation mode** is an optional voice for that final step. It changes the
voice and shape of the answer, never the grounding.

The default is `plain`: no directive, output unchanged. The flagship optional
mode is `hawking`.

## Use it

```bash
doxa present --list # list modes with one-line summaries
doxa present hawking # print the composition directive
doxa present hawking --json # machine-readable profile

doxa query "did time have a beginning?" --present hawking
doxa query "did time have a beginning?" --answer --present hawking # pairs with the evidence brief
```

With a non-plain mode, `doxa query` prints a directive block before the
evidence:

```text
=== doxa presentation directive: hawking ===
...
=== end presentation directive ===

1. <belief> ...
```

An agent reads the directive, then writes the answer in that voice using only
the retrieved beliefs and quotes.

### Output shapes

- Text: the directive is prepended, then the usual numbered results.
- `--json` with `plain`: a bare list of results (unchanged, backward compatible).
- `--json` with a non-plain mode: `{"presentation": {...}, "results": [...]}`.

### Make it the default

```yaml
# doxa.yaml
presentation:
default: hawking
```

A `--present` flag on `doxa query` overrides the configured default.

## The hawking mode

It is distilled from how Stephen Hawking presented evidence in the first two
chapters of *A Brief History of Time*. The directive carries an arc, a set of
moves (each with a short verbatim exemplar), and hard constraints:

- Open on the old question -- wonder, not the apparatus.
- Stage a procession of minds: the retrieved beliefs are the cast, and the
reader watches the picture change.
- Say the largest thing in the plainest sentence.
- Keep the strangeness intact; never simplify into a comfortable falsehood.
- Make the answer epistemology: name what kind of claim this is and how far it
can be trusted.
- Hold humility and audacity together.
- Close on the human stakes of the question.

**It is voice, not license.** The directive restates the inviolable rule: every
belief and quote must still come from doxa's retrieved records, conviction is
still reported honestly, and nothing is invented. Rigor under the lyricism is
itself the Hawking move.

## Add a mode

Modes are data. Register a `PresentationProfile` in `doxa/present.py`:

```python
MY_PROFILE = PresentationProfile(
name="my-voice",
title="...",
summary="...",
arc=("...",),
moves=(Move(name="...", directive="...", exemplar="..."),),
constraints=("...",),
)
```

Add it to the `_PROFILES` registry and it becomes available to `doxa present`,
`doxa query --present`, and the `presentation.default` config key. A profile with
no `arc`/`moves`/`constraints` is treated as plain (no directive).
9 changes: 9 additions & 0 deletions doxa.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,15 @@ retrieval:
bm25_b: 0.75
rrf_k: 60

presentation:
# Optional voice for composing answers from retrieved evidence.
# plain - no directive; the agent answers in its own voice (default).
# hawking - emit a composition directive that shapes the answer as a human
# quest (see `doxa present hawking`). Voice only; never changes the
# verbatim-grounding guarantee.
# Override per query with `doxa query ... --present hawking`.
default: plain

preferences:
# Domain weights run from 0 to 10. Matching domain:<slug> tags get a small
# retrieval boost and are also shown to the mining prompt as tagging hints.
Expand Down
18 changes: 18 additions & 0 deletions doxa/_assets/skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ proactively offer the capabilities below -- users often don't know they exist.
| --- | --- |
| See where things stand | `doxa status` (config, counts, provider, semantic on/off) |
| Answer, grounded | `doxa query "<q>" --answer` (or `--json` for tools) |
| Answer as a quest (optional) | `doxa query "<q>" --present hawking` -- Hawking-style voice; see `doxa present --list` |
| Head start (curated base) | `doxa packs install startup-wisdom` -- browse with `doxa packs list` |
| Start your own base | `doxa init --lens-template <name>` -- browse with `doxa lenses list` |
| Ingest a source | `doxa ingest <file\|url\|pdf\|youtube\|->` |
Expand Down Expand Up @@ -54,6 +55,23 @@ Humanize only the surrounding prose. Never alter the bytes, punctuation,
capitalization, or whitespace inside a returned quote, and never invent a quote,
attribution, source, or belief the CLI did not return.

### Optional: presentation voice

For a reflective or big-question prompt, ask for an optional voice:

```bash
doxa query "<user question>" --answer --present hawking
```

A non-plain profile prints a `=== doxa presentation directive ===` block before
the evidence. Read it and compose the answer in that voice -- the `hawking`
profile shapes the answer as a human quest (open on the old question, let the
evidence arrive as a procession of minds, say the largest thing plainly, keep the
strangeness intact, turn the answer toward what we can know). It changes voice
and shape only and never relaxes the grounding rule below. `doxa present --list`
shows profiles; `doxa present hawking` prints the directive. Default is `plain`
(no directive), so skip it unless the moment calls for it.

## 3. Start or grow a base

### Fastest start: offer a curated pack (optional)
Expand Down
57 changes: 54 additions & 3 deletions doxa/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
user_lens_dir,
)
from .packs import export_pack, get_pack, install_pack, load_registry
from .present import DEFAULT_PROFILE, PresentationProfile, available_profiles, get_profile
from .resources import demo_config_path, skill_text
from .schema import DoxaError, RetrievalResult
from .sources import load_source
Expand Down Expand Up @@ -222,7 +223,7 @@ def _prompt_lens() -> dict[str, Any]:
return {}


def _init_answers(args: argparse.Namespace, *, interactive: bool) -> dict[str, str]:
def _init_answers(args: argparse.Namespace, *, interactive: bool) -> dict[str, Any]:
provider_default = _normalize_provider(args.provider)
provider = _prompt_provider(provider_default) if interactive else provider_default
defaults = PROVIDER_DEFAULTS[provider]
Expand Down Expand Up @@ -278,7 +279,7 @@ def _init_answers(args: argparse.Namespace, *, interactive: bool) -> dict[str, s
}


def _build_init_config(answers: dict[str, str], *, full: bool = False) -> dict[str, Any]:
def _build_init_config(answers: dict[str, Any], *, full: bool = False) -> dict[str, Any]:
if full:
config: dict[str, Any] = deepcopy(DEFAULT_CONFIG)
else:
Expand Down Expand Up @@ -478,8 +479,17 @@ def _store_has_beliefs(config: dict[str, Any]) -> bool:
return True


def _resolve_presentation(args: argparse.Namespace, config: dict[str, Any]) -> PresentationProfile:
"""Pick the presentation profile: CLI flag wins, else config default."""

flag = getattr(args, "present", None)
configured = str(config.get("presentation", {}).get("default", DEFAULT_PROFILE))
return get_profile(flag if flag is not None else configured)


def cmd_query(args: argparse.Namespace) -> int:
config = load_config(args.config)
profile = _resolve_presentation(args, config)
if not args.json and _is_demo_fallback(config):
_hint("note: no doxa.yaml here -- querying the bundled demo base. Run `doxa init` to build your own.")
domains = parse_domain_selectors(args.domain, args.domains)
Expand All @@ -494,7 +504,12 @@ def cmd_query(args: argparse.Namespace) -> int:
for warning in warnings:
print(f"warning: {warning}", file=sys.stderr)
if args.json:
print(json.dumps([result.to_dict() for result in results], indent=2, ensure_ascii=False))
payload: Any = [result.to_dict() for result in results]
# Plain stays a bare list (backward compatible); a non-plain profile
# wraps the list so the directive travels with the evidence.
if not profile.is_plain():
payload = {"presentation": profile.to_dict(), "results": payload}
print(json.dumps(payload, indent=2, ensure_ascii=False))
return 0
if not results:
print("No matching beliefs found.")
Expand All @@ -504,6 +519,11 @@ def cmd_query(args: argparse.Namespace) -> int:
else:
_hint("hint: try broader terms, raise --top, or --search hybrid if you've run `doxa index`.")
return 0
directive = profile.render_directive()
if directive:
# The directive shapes voice only; the evidence below stays the ground truth.
print(directive)
print("")
if args.answer:
print(render_terminal_answer(args.query, results))
return 0
Expand All @@ -512,6 +532,25 @@ def cmd_query(args: argparse.Namespace) -> int:
return 0


def cmd_present(args: argparse.Namespace) -> int:
if args.list:
for name in available_profiles():
profile = get_profile(name)
marker = " (default)" if name == DEFAULT_PROFILE else ""
print(f"{name}{marker}: {profile.summary}")
return 0
profile = get_profile(args.name)
if args.json:
print(json.dumps(profile.to_dict(), indent=2, ensure_ascii=False))
return 0
directive = profile.render_directive()
if directive:
print(directive)
else:
print(f"{profile.name}: {profile.summary}")
return 0


def cmd_eval(args: argparse.Namespace) -> int:
config = load_config(args.config)
if not args.json and _is_demo_fallback(config):
Expand Down Expand Up @@ -1002,9 +1041,21 @@ def build_parser() -> argparse.ArgumentParser:
query.add_argument("--domains", default="", help="Comma-separated domain slugs to boost.")
query.add_argument("--no-domain-boost", action="store_true", help="Disable configured domain preference boosts.")
query.add_argument("--answer", action="store_true", help="Format results as a clean evidence brief (claim + grounding quotes).")
query.add_argument(
"--present",
choices=available_profiles(),
default=None,
help="Optional presentation voice. Emits a composition directive with the evidence. Default: presentation.default in config (plain).",
)
query.add_argument("--json", action="store_true")
query.set_defaults(func=cmd_query)

present = subparsers.add_parser("present", help="Show an optional presentation profile (composition directive).")
present.add_argument("name", nargs="?", default=DEFAULT_PROFILE, help="Profile name. Default: plain.")
present.add_argument("--list", action="store_true", help="List available presentation profiles.")
present.add_argument("--json", action="store_true")
present.set_defaults(func=cmd_present)

eval_parser = subparsers.add_parser("eval", help="Run faithfulness and link-integrity checks.")
eval_parser.add_argument("--config")
eval_parser.add_argument("--json", action="store_true")
Expand Down
5 changes: 5 additions & 0 deletions doxa/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,11 @@
"bm25_b": 0.75,
"rrf_k": 60,
},
"presentation": {
# Optional voice for composing answers. "plain" leaves output unchanged;
# "hawking" emits a composition directive (see `doxa present --list`).
"default": "plain",
},
"preferences": {
"domains": {
"general": 2,
Expand Down
6 changes: 4 additions & 2 deletions doxa/packs.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ def _read_pack_lines(location: str, filename: str) -> list[str]:
url = location.rstrip("/") + "/" + filename
try:
req = urllib.request.Request(url, headers={"User-Agent": _USER_AGENT})
with urllib.request.urlopen(req, timeout=120) as resp:
# scheme validated to http(s) above, so file:/custom schemes can't reach here
with urllib.request.urlopen(req, timeout=120) as resp: # nosec B310
text = resp.read().decode("utf-8")
except Exception as exc: # noqa: BLE001
raise DoxaError(f"could not fetch {url}: {exc}") from exc
Expand Down Expand Up @@ -154,7 +155,8 @@ def export_pack(
kept_quotes.append(Quote.from_dict(raw))

sources: dict[tuple[str, str], SourceRecord] = {}
for obj in (*kept_beliefs, *kept_quotes):
records: list[Belief | Quote] = [*kept_beliefs, *kept_quotes]
for obj in records:
ref = obj.source
if not ref.title:
continue
Expand Down
Loading
Loading