feat: configurable curation engine (Claude or local model)#11
Open
mhellevang wants to merge 11 commits into
Open
feat: configurable curation engine (Claude or local model)#11mhellevang wants to merge 11 commits into
mhellevang wants to merge 11 commits into
Conversation
Add an opt-in, agent-free curation engine that runs the full pipeline with a
local model via Ollama — selectable with `engine: local` in
.openpaper/config.yaml. Default stays `claude` (existing skill flow unchanged).
Design is hybrid: all editorial arithmetic (interest weights, source/recency
bonuses, 30%/40%/20% caps, role assignment) is deterministic Python in
curate.py mirroring curation-guide.md and covered by tests; the local model
only does per-article semantic matching and role-calibrated summaries. That
split is what keeps a small (4B) model reliable.
- scripts/curate.py local curation engine (config + preferences parsing,
scoring, model layer, edition assembly)
- scripts/make_paper.py agent-free fetch -> curate -> render orchestrator
- tests/test_curate.py first unit tests in the repo (deterministic curation)
- references/local-engine.md + config.example.yaml, README + SKILL.md notes
Ingest (fetch_all.py) and Present (render.py) are reused unchanged.
make_paper.py now ends by launching serve.py --latest after render, so the local engine opens the finished edition in a browser just like the Claude flow instead of leaving an HTML file in .openpaper/. Add --no-serve for headless/cron runs. Points the server at the localised template's assets when present. Fixes found while verifying: - serve.py: use errno.EADDRINUSE for the port fallback (was hardcoded 98, the Linux value; on macOS it is 48, so the fallback never triggered and the server crashed on a busy port). - curate.py: guard generated headlines against the model echoing the "OpenPaper" masthead from the reading profile; fall back to the real source title. Add _bad_title + unit test.
Add bench.py to time the curation pipeline (matching + summaries) across models and emit editions for side-by-side comparison. Benchmark result (M-series Mac, 57-article corpus): - gemma4:e4b-it-q4_K_M ~4.2m, full bodies, fluent bokmål — recommended - gemma4:12b-it-q4_K_M ~7.5m AND broken at this quant (empty bodies, title corruption, untranslated text) — not recommended - Claude default ~1.8m wall-clock (parallel subagents), best quality Also: gemma4:e4b is byte-identical to gemma4:e4b-it-q4_K_M (the base tag was already instruction-tuned), so config now uses the explicit -it tag for clarity only. Update local-engine.md model guidance accordingly.
…wer) Tested higher precision to see if it fixed q4's empty/garbled output. On a 48GB M-series Mac it was far worse: ~2.4 tok/s, heavy swap at 32k context, matching alone ~37 min, and a runaway generation forced an abort. No 12B build tested is viable; stay on e4b-it-q4_K_M.
Extract the editorial arithmetic (scoring, topic/source/serendipity caps, role assignment) out of curate.py into curation_core.py so both engines share one unit-tested implementation instead of drifting apart. Add rank.py: a deterministic ranker for engine: claude + ranking: deterministic. Claude scores per-article relevance (its strength), then the same core pipeline picks the slate and assigns roles — reproducible, and the caps are enforced automatically instead of by hand. Claude still writes the summaries. Default ranking: agent keeps the existing freehand flow. Document the new config key and flow; add tests/test_rank.py.
…ngine # Conflicts: # skills/openpaper/SKILL.md
falense
reviewed
Jun 8, 2026
|
|
||
| ```bash | ||
| ollama pull gemma4:e4b | ||
| uv run skills/openpaper/scripts/make_paper.py --data-dir .openpaper |
Owner
There was a problem hiding this comment.
for this to work .openpaper/config.yaml needs to exist
Owner
There was a problem hiding this comment.
It fails from a new install when Claude hasnt made the scaffolding
FileNotFoundError: [Errno 2] No such file or directory: '/home/sondre/Repositories/OpenPaper/.openpaper/sources/_base.py'
The engine: local path (make_paper.py) was documented as standalone but
depended on .openpaper/ scaffolding only the Claude setup wizard creates.
From a fresh clone it crashed in deploy_base_module with
FileNotFoundError on sources/_base.py (no sources/ dir), or exited
pointing at a config.yaml that didn't exist.
- make_paper.py now bootstraps on first run: creates the data dirs, a
config.yaml (engine: local), a starter preferences.md, and installs
shipped default news fetchers. Idempotent; never overwrites user files.
An existing engine: claude config still defers to the Claude flow.
- fetch_all.deploy_base_module mkdirs sources/ before copying _base.py.
- ship skills/openpaper/fetchers/news/{bbc,hackernews}.py as defaults.
- docs: README + local-engine.md describe the bootstrap and the
'run Claude once to tailor sources' path; fix the model name to the
instruction-tuned gemma4:e4b-it-q4_K_M (base e4b ignores instructions).
make_paper.py now brings up the prerequisites itself so one command works on a fresh machine with no manual setup: it installs Playwright's Chromium on first run, and ensures Ollama is running (starting `ollama serve` if needed) with the configured model pulled before curating. Add --skip-setup to skip these checks for cron/headless runs where everything is present. Also trim the README: the 'Curation engine' section was ~40 lines of internal design detail; reduce it to a tiny 'Run fully local' step (install Ollama, run one command) that mirrors the Install step, and leave the design, model notes, and limits to the local engine guide.
It's already the default in make_paper.py, so the local-engine command is just `uv run skills/openpaper/scripts/make_paper.py`.
_ensure_ollama now returns the server process only when it had to start one; main() stops exactly that process in a finally (covering success, error, and Ctrl+C), so a run leaves the machine as it found it. A server that was already running is never touched. The model unloads on Ollama's own idle keep_alive either way.
Contributor
Author
Whops! Ja jeg hadde ikke gått opp boostrap-løypa 🙈 Nå skal det bare være å kjøre Om du har tid og lyst kan du jo ta en titt og se hvordan det funker med lokal modell på din maskin, om enn bare for å sjekke ut, så kan du føle på det 😄 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two related changes to the Curate stage:
pipeline with a local model via Ollama — selectable with
engine: localin.openpaper/config.yaml.(
curation_core.py), so the Claude engine can opt into the samedeterministic selection with
ranking: deterministic— Claude scoresrelevance and writes the prose, but the slate and role tiers come from the
same reproducible code.
The defaults are unchanged:
engine: claude+ranking: agentis exactlytoday's freehand skill flow.
Why
Today the Curate stage requires a Claude Code session. A local engine gives
users a private, offline, free option that doesn't depend on Claude Code — at
the cost of some editorial polish. (Ties into #9's interest in making scoring
explicit.)
Once the arithmetic existed, it was a short step to let the Claude flow reuse
it: hand-curation drifts from the project's own caps (e.g. ending up over the
per-source limit), whereas
ranking: deterministicenforces them automaticallywhile keeping Claude's relevance judgement and summaries.
Design: code does the judgment, the model does the prose
A small local model is a poor editor (asked for a holistic score it skews to
surface topicality and undervalues big human-interest stories). So the work is
split along what each side is good at:
curation_core.py) does all editorial arithmetic —interest weights, source/recency bonuses, feedback decay, the 30%/40%/20%
caps, and role assignment — mirroring
curation-guide.md, and is unit-tested.semantic matching against the reader's interests, and the role-calibrated
summaries (with role-length enforced in code).
curation_core.pyis engine-agnostic, so the only difference between the twoengines is who scores relevance and writes the summaries — not the caps or
role tiers:
engine: localcuration_coreengine: claude+ranking: deterministiccuration_core(viarank.py)engine: claude+ranking: agent(default)Included
scripts/curation_core.py— the shared, engine-agnostic arithmetic(config +
preferences.mdparsing, scoring, the caps, role assignment).scripts/curate.py— theengine: localengine:curation_core+ the localmodel layer (semantic match, summaries) + edition assembly.
scripts/rank.py— theranking: deterministicranker for the Claude flow:takes Claude-supplied match scores, runs the same pipeline, emits the slate +
roles.
--print-prefsreports the interests and caps to score against.scripts/make_paper.py— agent-freefetch_all → curate → renderorchestrator.Ends by launching the preview server and opening the finished edition in a
browser — the same UX as the Claude flow — with
--no-servefor headless/cron.scripts/bench.py— benchmark harness for comparing local models on a corpus.tests/test_curate.py+tests/test_rank.py— first unit tests in the repo(deterministic curation + the ranker glue).
19 passing.references/local-engine.md+references/config.example.yaml; README andSKILL.md notes (incl. the deterministic-ranking flow).
Ingest (
fetch_all.py) and Present (render.py/serve.py) are reused — thelocal flow now chains
serve.pyfor preview just like the agent does.Try it
Local engine:
Deterministic ranking under the Claude engine — set
ranking: deterministicin.openpaper/config.yaml; the/openpaperskill then scores relevance and runsselection through
rank.py(see SKILL.md → Stage 2 → Deterministic ranking).Model notes (benchmarked — see
bench.py)gemma4:e4b-it-q4_K_M— recommended. Instruction-tuned; faithful, fluentbokmål, ~4 min for a 14-article edition on an M-series Mac. Use an
-it(instruction-tuned) tag — base
gemma4:e4b/e2bignore the JSON/summaryinstructions and tend to translate articles verbatim.
12b-it-q4_K_Mwas ~1.8× slower and unreliable(empty story bodies, title corruption, untranslated text);
12b-it-q8_0wasmemory-bound (~12× slower, heavy swap). Bigger is not better here.
Fixes folded in
serve.pyport fallback now useserrno.EADDRINUSEinstead of ahardcoded
98(the Linux value). On macOSEADDRINUSEis48, so thefallback never fired and the server crashed on a busy port — surfaced once
make_paper.pystarted auto-launching the server."OpenPaper"masthead(mentioned in the reading profile) into a story's
title. Such titles are nowrejected in favour of the real source headline (
_bad_title, unit-tested).Scope / known limits
compensate but won't match Claude's instinct for the most interesting lead.
ranking: deterministiclikewise can't catch cross-source duplicates or thinsources on its own — the Claude flow keeps an editorial override pass on top.
cap length — a per-locale post-check is a follow-up.
model from selection entirely), a feedback-writing loop, and non-Ollama
runtimes (llama.cpp/MLX).
Happy to iterate on any of this if the direction is welcome.