All notable changes to WhisperForge will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Unused vendored Fabric
patterns/tree — deleted 256 pattern files that no code,Makefile, CI job, or doc referenced. The app's prompts and knowledge base live underprompts/<user>/viawhisperforge_core/prompts.pyand are unaffected. - Superseded historical docs — retired
docs/DOCUMENTATION-AUDIT-2026-05-17.md,docs/AUDIO-REPO-CONSOLIDATION-AUDIT-2026-05-18.md,docs/WHISPERFORGE-AUDIT-AND-ROADMAP-2026-05-18.md, anddocs/WHISPERFORGE-MASTER-PLAN-2026-05-18.md. Each already redirected readers to the livingSTATUS.md/ROADMAP.md/NEXT-ROUND-PLANset; git history preserves the full record.
- Short-horizon week plan —
docs/WEEK-PLAN-2026-07-01.mdsequences the rest of the week (land the cleanup, run fresh dogfood evidence, scope one next lane) and is linked fromSTATUS.md.
- Doc cross-references rewired —
STATUS.md,ROADMAP.md,docs/LINEAR-GITHUB-PIPELINE.md,.company-os/project.yaml, andops/roadmap/features.jsonnow point at the living docs after the historical snapshots were retired.make docs-checkstays green.
- Phase 0 modernization guardrails — CI now runs roadmap JSON validation,
docs truth checks, high-signal lint, and unit tests on pull requests;
.dockerignorekeeps local env/cache/venv state out of Docker build contexts;.env.exampledocuments the required runtime knobs without carrying secrets. - Audio repo consolidation audit —
docs/AUDIO-REPO-CONSOLIDATION-AUDIT-2026-05-18.mdrecords the canonicalspektorAIdecision, legacy repo salvage matrix, and archive plan. - Presentation-readiness quality rail —
make lintnow provides a dependency-light syntax check, rendered UI smoke skips live Ollama discovery by default, and run manifests are normalized through a versionedartifact_schema_version=1contract. - Run story timeline — the Review tab now renders a compact capture ->
context -> composition -> review -> export -> handoff timeline from the run
manifest, backed by pure
whisperforge_core.run_storytests. Review rendering now lives inui/review.pysoui/output.pycan focus on output tabs and export/save orchestration. - Current local-first workplan —
docs/NEXT-ROUND-PLAN-2026-05-19.mdandops/roadmap/features.jsonnow track the next presentation-polish and local-first reliability wave. - Presentation demo fixture pack —
scripts/seed_demo_dataset.pynow seeds a completed article/handoff run, a SongForge creative-pack run, and a partial/failed run so cold-start demos have useful Runs-dialog material without live credentials. - Review tab polish — Run Story steps now show readable statuses and UTC timestamps, Review sections surface clearer empty states, and handoff previews can be downloaded as Markdown before any tracker routing.
- Import-folder capture helpers —
whisperforge_core.capturescan now import text/markdown notes and audio pointers from files or folders, dedupe obvious repeats, and ignore generated chunk/temp files. - Report-only weekly and topic synthesis — resurfacing now includes deterministic weekly recaps and topic-evolution summaries from capture/run metadata without external credentials.
- Obsidian-friendly vault export — markdown export now has a vault mode
that writes date-organized notes and updates a lightweight
index.md. - Large-file router evaluation —
docs/LARGE-FILE-ROUTER-EVALUATION-2026-05-19.mdevaluates FFmpeg-style large-file handling as provider-router work instead of a legacy code import. - Next round plan —
docs/NEXT-ROUND-PLAN-2026-05-19.mdrecords the next recommended sequence: dogfood a real Wispr Flow capture, add end-to-end UI coverage, and choose a release target. - Brand and license closeout — repo metadata now points to this canonical WhisperForge implementation, and root docs explicitly state that no open-source license is granted until the owner adds one.
- Presentation runbook —
docs/PRESENTATION-RUNBOOK-2026-05-19.mdprovides a deterministic collaborator demo flow, claim boundaries, reviewer prompts, and the next post-review implementation slice. - 2026 master plan —
docs/WHISPERFORGE-MASTER-PLAN-2026-05-18.mdresets WhisperForge around Wispr Flow-era capture, knowledge-base intelligence, source-grounded composition, agentic handoffs, recovery, and a bounded SongForge creative lane. - Capture inbox foundation — Wispr Flow paste, notes, uploads, and
recordings now create durable capture records under
.cache/captures/and link those records into run artifacts and export receipts. - Knowledge-base health audit — profile KB files now have inventory data, conservative stale/duplicate/private-looking warnings, and a visible health summary in the KB dialog.
- Retrieval inspector metadata — RAG retrieval can now report per-stage hits, scores, context roles, and voice anchors, with run-artifact persistence and a UI inspection path from the benchmark dialog.
- Prompt recipe command palette — built-in and profile-level recipe manifests can declare inputs, stages, defaults, outputs, eval checks, and handoff targets; selected recipes are persisted into run artifacts.
- Composition review surface — the Output card now includes a source- grounded review tab that keeps the draft beside receipts, excerpts, claim flags, revision notes, compare variants, and persona variants.
- Transcription provider matrix —
docs/TRANSCRIPTION-PROVIDER-MATRIX-2026-05-18.mdrecords current backends, candidate providers, privacy/cost tradeoffs, and the smallest next diarization integration path. - Advisory scorecards — deterministic, credential-free scorecards now rate voice, grounding, usefulness, recipe compliance, and handoff readiness; the verdict is stored with run artifacts/history and rendered in markdown/Notion exports without blocking saves.
- Agent handoff routing — the Review tab now supports preview-first draft
generation plus explicit approve-and-create routing to GitHub/Linear via
whisperforge_core.handoff_router; routing defaults to dry-run when config is missing and supportsWHISPERFORGE_HANDOFF_DRY_RUN=1as a kill switch. - Run workspace recovery — the Runs dialog now reads local run manifests, shows partial/error/export state, and can reopen completed outputs so safe downstream exports can be retried without rerunning generation.
- Primary-loop integration coverage —
tests/test_primary_loop_flow.pynow exercises paste input -> pipeline execution -> run artifact persistence -> Notion export receipt -> Runs dialog reopen restoration in one local flow. - Rendered primary-loop UI smoke —
tests/test_ui_primary_loop_smoke.pydrives paste input and Run interaction, clicks markdown export, confirms run-history selection for reopen, and validates run-artifact export receipts from the rendered UI path. - Browser localhost smoke command —
scripts/browser_e2e_smoke.pyplusmake browser-e2edrives a real browser flow (runs dialog reopen + markdown export) against a local Streamlit instance and checks run-artifact exports. - Fresh browser smoke fixtures —
make browser-e2e-freshnow usesWHISPERFORGE_E2E_FIXTURE_PATHwithtests/fixtures/browser_e2e_fresh_run.jsoninstead of runtime monkeypatching, preserving a deterministic credential-free smoke while following normal adapter wiring. - Services transcription detail parity — the transcription service now
returns
text,segments, andlanguage;HttpTranscriber.transcribe_detailed()now forwards these fields instead of dropping segment metadata. - Run reopen integration coverage —
tests/test_run_reopen_flow.pypins reopen behavior for completed and partial runs, including restore of core output fields and last Notion export URL. - Profile operating-system metadata —
profile.yamlcan now describe project context, defaults, KB packs, privacy notes, and preferred handoff targets, with validation surfaced in the KB dialog. - Resurfacing digest —
make digestnow writes a local report-only digest from captures and run artifacts, grouping notable captures, unresolved follow-ups, strong outputs, stale drafts, and reusable source nuggets. - SongForge creative lane — the built-in SongForge recipe now produces original lyric drafts, spoken-word variants, service-agnostic music prompt packs, and source notes from capture plus KB context.
- Agent-facing Makefile surface —
make help,make test,make smoke,make app,make services-run,make services-smoke, andmake services-downnow provide the standard local operations entrypoints. - Service contract tests —
tests/test_services_contract.pypins modern transcription/processing/storage payload fields for cleanup, chapters, segments, images, article length, RAG, compare, personas, fact-check output, and run metrics. - HTTP adapter contract tests —
tests/test_http_adapters.pypins the frontend HTTP clients against the same modern processing/storage payloads. - Profile manifests and user personas —
prompts/<user>/profile.yamlcan define prompt/persona overrides, andprompts/<user>/personas/*.mdjoins the built-in persona list. - Markdown source receipts rendering — markdown export can render source receipt fixtures from bundle metadata or run metrics.
- Notion source receipts rendering — saved Notion pages now carry source receipts alongside run metrics when a bundle includes them.
- Credential-free editorial eval —
make eval-fixturechecks fixture source receipts and fact-check rendering without API keys. - Durable run artifacts — direct-mode runs now write manifests and stage
checkpoints under
.cache/runs/<run_id>/. - Documentation audit artifact —
docs/DOCUMENTATION-AUDIT-2026-05-17.mdcaptures the current sync state, verification commands, stale-doc fixes, and remaining blockers. - Current audit and roadmap artifact —
docs/WHISPERFORGE-AUDIT-AND-ROADMAP-2026-05-18.mdrecords the completed issue wave, full repo audit findings, from-here roadmap, and human decision questions. - KB governance review controls — profile KB audits now support canonical voice anchors, intentionally ignored files, reviewer actions, and pre-generation governance warnings.
- Transcription router media plan — the audio router now has fixture-backed ffprobe inspection, planned-only FFmpeg normalization, output contracts, and privacy/cost receipts without changing runtime defaults.
- Approved resurfacing digest routing —
make digestremains report-only by default, while explicit CLI approval can route to the local follow-up queue or local Notion page/task draft files. - SongForge export polish — SongForge packs now include deterministic structure variants and originality guardrails, with fixture coverage and markdown/vault export preservation.
- Documentation truth check —
make docs-checknow verifies project markdown links, documentedmaketargets, and freshness fields for current status/roadmap docs. Pull requests that touch docs run the same check in CI.
- Detailed transcription now reuses the chunk/cache-aware text path for non-WhisperX backends, so the primary UI and transcription service routes preserve large-file behavior instead of bypassing the chunker.
make lintnow pairs compile checks with high-signal Ruff rules, leaving full repo-wide style adoption as a deliberate follow-up instead of a noisy one-shot migration.- The services frontend image now copies
ui/alongsideapp.py, matching the Streamlit entrypoint's actual imports. - Runs dialog reopen now restores the run manifest
capture_id, and the Review tab feeds capture metadata into the Run Story timeline so reopened runs keep their source receipts and friendlier capture labels. - Audio unit fixtures now use WAV so
make testdoes not require machine-levelffmpegjust to generate test audio. - Root docs now describe the current prompt profile layout, Ollama as a first-class provider lane, rendered UI smoke testing, and the current services-mode parity boundary.
- Root docs now describe the post-consolidation baseline: no open issues or
PRs remain, only
origin/mainremains remotely, stale detached worktrees were removed, and the active verification baseline lives inSTATUS.md. ROADMAP.mdandSTATUS.mdnow describe the post-swarm state instead of the pre-implementation planning baseline.ROADMAP.mdis now a concise index into the 2026 master plan and active issue registry, instead of a stale long-form status dump.STATUS.mdand the active next-round plan now reflect the closed#49through#52queue and merged PRs#53through#56.- User-defined personas now flow through both the Generation Settings selector and the pipeline resolver instead of being limited to built-ins at run time.
- Run history now upserts by stable
run_idand stores local artifact links alongside Notion and markdown export paths.
whisperforge_core/http_adapters.pynow forwards modern pipeline options and round-trips the fullPipelineResultfields exposed by the processing service.HttpStorage.save()now forwards modernContentBundlefields such as chapters, compare/persona variants, critique, fact-check flags, and run metrics.
- KB benchmark tool — new
whisperforge_core/rag/benchmark.pycompares legacy KB-dump vs RAG top-K retrieval for a specific query, reporting char/token counts, cost at the provider's input rate, which voice-anchor doc RAG picked, and percentage savings. Pure measurement — no LLM calls, so running it is free and deterministic. - 📊 Benchmark button in the sidebar opens a dialog where you pick a stage (or "All stages"), edit the sample query (defaults to the current transcript), and get a table comparing legacy vs RAG across whichever stage(s) you selected.
benchmark_all_stages()helper — one call returns the full stage-by-stage comparison so you can see at a glance whether RAG wins everywhere or just on specific stages.
- RAG Phase 1–3 shipped the pipeline. Phase 4 is the tuning tool: KK's KB shows a 58.9% input-token drop on
wisdom_extraction(6,546 → 2,691 tokens) — data-driven evidence that RAG is the right default for anything bigger than 25 chunks. Phase 4 makes that number visible before the first run, so you can decide between Auto / Always / Never with real numbers instead of vibes. - Note on real-world cost: these numbers are un-cached list-price. Anthropic prompt caching on the stable legacy block closes the gap ~10× (cache reads are 10% of input rate). The UI's summary line flags this explicitly.
tests/test_rag_benchmark.py— 7 new (shape, empty-KB no-crash, legacy-chars > raw KB due to framing, known-model cost is non-zero, unknown-model cost is zero but tokens count, delta math, all-stages returns one row per stage). 148 total.
-
Per-run metrics block in Notion — every saved page now carries its own receipt: total
$, LLM $ , ASR $, cache savings, call count, input/output/cache-read/cache-write tokens, wall-clock duration, transcription backend, and which features fired (agentic, fact-check, chapters, images, RAG, compare, personas). Renders as aRun metricstoggle (gray) between the last content section and the Metadata heading so it reads like the footer of a receipt. -
Markdown mirror — same fields appear as a
## Run metricsbullet list in the exported.md, sitting in the same slot above## Metadata. -
Duration tracking —
ui/pipeline._execute_run()stampspipeline_started_atbefore stage 0 and freezespipeline_ended_atright after pipeline completion (before the auto-save), so the duration number captures the pipeline itself, not the Notion roundtrip. - ContentBundle.run_metrics — optional dict field. When None, both Notion and markdown rendering skip the block entirely so manual/test bundles stay clean.
- Previously every run dumped cost data into
history.json— useful for📜 Runsbut invisible on the saved Notion page itself. Now each page is self-contained: you can eyeball "this run cost $0.04, saved $0.01 to cache, agentic + images on, ran 2m 14s" without cross-referencing.
tests/test_notion.py— 5 new (no-metrics → no block; full metrics render with $/tokens/duration/flags; sub-minute formats as34s, 134s formats as2m 14s; empty flag set showsnone; metrics sits above Metadata heading).tests/test_export.py— 2 new (markdown rendering with full metrics; no metrics → no section). 141 total.
- Multi-voice personas — new built-in persona directives in
whisperforge_core/config.PERSONAS(Punchy podcast host, Industry analyst, Twitter/X thread, Essay — long reflective, Newsletter — first person). Pipeline Stage 7.2 re-runsarticle_writingonce per selected persona with the directive appended; variants land onPipelineResult.persona_articlesas[{"name", "text"}]. - Persona multiselect in ⚙ Generation Settings → Personas. Empty list = off (the default).
- Notion rendering — each persona variant becomes its own color-cycled toggle block (blue → green → yellow → orange → red → purple) beneath the main article, matching the A/B compare card style.
- Markdown export — persona variants appear as
## Persona · <name>sections in the exported.md. - Output tabs — one
🎭 <name>tab per persona variant in the Output card, mirroring the Notion layout so screenshots line up.
- Re-runs cost only the final article stage (transcript + wisdom + outline are already cached), so generating 3 persona variants is ~1.2× the cost of a single article draft, not 3×. Makes it cheap to surface "what if I said this as a tweet thread vs. an essay" without a fresh pipeline run.
- A/B provider compare — optional alt-provider article side-by-side.
PipelineResultgainsarticle_compare+compare_label;run()gainscompare_provider+compare_modelkwargs. When both are set and a primary article exists, Stage 7.25 fires a secondarticle_writingcall with the alt provider/model using the same transcript + wisdom + outline + length directive — only the voice varies. - Compare picker in ⚙ Generation Settings → A/B compare. Empty = off.
- Notion + markdown rendering — alternate article becomes an
⚖ Article · <alt>toggle block with its own color band; markdown export writes a## Article · <label>section after the primary. - Output tab —
⚖ Comparetab in the Output card with a caption explaining the alt voice.
- Lets you decide "promote Haiku draft to Sonnet?" without spending a full fresh pipeline run. Failures on the compare call are logged, not raised — the main article still ships.
- Markdown export — new
whisperforge_core/export.pyrenders anyContentBundleas a single.mdfile with YAML frontmatter (title, created, source, models, tags, optional notion_url). Obsidian-compatible. Chapters get[MM:SS]/[H:MM:SS]prefixes. - 💾 Markdown button in the Output card next to Save-to-Notion. Saves to
.cache/exports/YYYY-MM-DD-<slug>.md; a download button appears after export. auto_export_markdowntoggle in ⚙ Generation Settings → Output — when on, every Notion save also writes the markdown copy alongside.ui/output._build_bundle()extracted as shared ContentBundle constructor so Notion save and markdown export stay in lockstep.
tests/test_export.py— 12 new (frontmatter shape, section ordering, chapter timestamps, fact-check rendering, duplicate-filename disambiguation, slug unicode handling). 131 total.
- RAG on knowledge base — new
whisperforge_core/rag/package. When a user's KB grows past 25 chunks (orWF_RAG=1/ KB mode = "Always" in the UI), the pipeline switches from dumping the whole KB on every call to per-stage top-K retrieval with an always-included voice anchor.chunker.py— heading-aware split (top-3 markdown levels) with sliding-window fallback for oversized sections and .txt docs.embedder.py— lazysentence-transformers/all-MiniLM-L6-v2(cached, MPS-accelerated on Apple Silicon). Override viaWF_EMBED_MODEL.store.py— numpy cosine + on-disk persistence at.cache/rag/<user>/<embed_hash>/with mtime-based invalidation.retriever.py— stage-specific query augmentation map, voice-anchor heuristic (filename match on voice/style/tone/writing/persona), block rendering.
- KB mode segmented control in ⚙ Generation Settings: Auto / Always / Never.
- Live verified: wisdom stage dropped from ~6000 → ~2300 input tokens (62%) with RAG engaged on KK's KB. Auto mode keeps small KBs on the legacy "dump + cache" path since prompt caching already wins there.
sentence-transformers>=2.7,<6— pinned below v6 because v6 adds a hard torchcodec dep that breaks on macOS without exact FFmpeg libavutil versions.
- Auto-save to Notion at end of pipeline (default ON, opt-out via ⚙ Generation Settings). Restores the old Record-tab "I'm Feeling Lucky" auto-save behavior as a flag. Lazy-imports
ui.output._save_to_notionfromui.pipelineto avoid circular dependency.st.toastconfirms or warns. - Article length targeting — segmented control (Brief / Standard / Long-form) maps to ~500 / 1500 / 3000 words. Routed into both
article_writingandarticle_revisestages withmax_tokensbudget (1.8× target words for headroom) AND a_user_prefixride-along that injects"Target length: approximately N words"into the user content.
- Live A/B: same transcript at Brief vs Long-form produced 579 vs 3113 words (5.4× ratio, both within 16% of targets).
- Prompt-cache hit confirmed on every stage after the 0.6.1 default-user fix landed:
cache_read=5821per call. - 103 tests still pass.
- Scanner line sweep — the 1px-too-subtle scanner from pre-refactor was rebuilt at 2px with stronger glow + a proper 10s sweep that fades in/out at the edges.
- Primary buttons ("I'm Feeling Lucky", "Save to Notion") get a teal→purple→magenta gradient with a slow 6s sheen animation — replaces the old
.lucky-buttonenergy. - Ambient background drift — 60s subtle gradient hue-shift on
.stAppso the canvas feels alive. - Section-header underline wipes in on first render (
scaleX 0→1over 0.6s). - Pulse on active
sac.stepsstep — borrowed@keyframes pulsefrom the old.process-indicator .dot. - Bottom bar gets a top-edge shimmer that breathes on an 8s cycle.
- Bordered cards have a hover-lift transition.
- CSS selectors that didn't match Streamlit 1.56's actual DOM (discovered by grepping
streamlit/static/*.js):st.segmented_controlrenders asstButtonGroup, NOTstSegmentedControl. The provider picker was unstyled before this fix.stVerticalBlockBorderWrapperdoesn't exist; bordered containers now match via:has()+style*="border"attribute selectors.st.statusshares thestExpandertestid; both selectors applied as a pair.
- Default user regression — pass-1 of the refactor defaulted the sidebar to alphabetically-first user (
Caroline_Hiltonin Kris's tree), which has no KB. Result: every pipeline run since 0.6.0 ran with no KB injection →cache_read=0everywhere → ~67% prompt-cache savings silently disabled. Nowui/session._resolve_default_user()picks: persisted last-selection → first KB-having user → alphabetically first. The sidebar persists changes to.cache/ui_preferences.json.
- Three-zone app shell (header / sidebar + main / fixed bottom bar), Cursor/Claude-Code style.
app.pyshrunk from 970 → 55 LoC; feature code moved to a newui/package. ui/package split by concern:session.py— singleinit_all_state()entrypoint; every session-state key lives in one place. Newclear_run()preserves settings, zeros per-run state. Per-provider model memory viaremember_model_for_provider().shell.py— header +render_bottom_bar()wrapped instreamlit-extras.bottom_container+@st.fragment(run_every="2s")for live cost/cache/calls/model readout without rerunning the pipeline.sidebar.py— compact profile / provider (st.segmented_control) / model pickers + four dialog-opening action buttons.dialogs.py— four@st.dialogmodals for prompt editor (selectbox + one text_area, replacing 5 stacked editors), knowledge base manager, run history (st.data_editor+LinkColumnon Notion URLs), and clear-run confirm.input.py— three tabs (Upload / Record / Paste) that all write to onepending_input, replacing ~400 LoC of near-duplicate handler code.pipeline.py—sac.stepshorizontal stage indicator plusst.statuscollapsible progress container with per-stagewrite()lines. Single entrypoint routes both Transcribe-only and Full-pipeline modes.output.py— one bordered card per section viast.container(border=True),st.feedback("thumbs")per section, single unified Save-to-Notion path (replaced three).
- Dependencies:
streamlit-extras>=0.4.0,streamlit-antd-components>=0.3.2. Streamlit pin bumped to>=1.37for@st.fragment. - New CSS components:
.bottom-bar/.bottom-metric,.sidebar-status,.status-dot.*variants, segmented-control palette overrides, sac.steps palette overrides, dialog + popover styling — all matching the existing cyberpunk vibe.
- Recording-tab auto-save-to-Notion (old behavior) replaced by the single explicit "Save to Notion" button in the unified Output card. All three input tabs now behave identically after submit.
st.spinner+st.empty+ manual markdown replaced byst.status("…", expanded=True)streaming per-stage log lines.st.successbanners replaced byst.toastephemeral notifications.
.lucky-button,.quick-access,.quick-button,.content-section,.terminal-output,.process-indicator(+@keyframes pulse),.notion-button,.scanner-line(+@keyframes scanner-move+ its broken inline HTML),.toast-notification(+@keyframes toast-in)..status-container/.status-card/.status-value— the old "System Status" three-card row whose JS tried toinnerHTMLstatic elements Streamlit just rerendered..app-footer/.footer-content/.footer-status/.footer-status-dot/.status-secure/.status-sovereignty/.status-offline .footer-status-dot— replaced by the compact.sidebar-statusline.
pytest tests/ -q→ 103 pass (no regressions; UI isn't unit-tested by design).tests/smoke.sh→ HTTP 200 on/_stcore/health.- Every
.classinstyles.pyhas a matching emitter inapp.pyorui/*.py(verified by a grep-both-ways script). - Live boot:
streamlit run app.py→ three zones render, no exceptions in the server log, index HTML clean.
-
Image generation via Google Nano Banana (Gemini 2.5 Flash Image). Ported from
kk-ai-ecosystem/tools/image-gen/generate.py— just the core generator, not the CLI or markdown-file parser. Newwhisperforge_core/images.pywithgenerate_image(),generate_images()batch,extract_prompts()parser, and a YAML-backed style registry (styles/image_styles.yaml— kk, hopecode, bcai, upgrade). Pipeline gainsgenerate_images=Trueflag that turns theimage_promptsstage output into real PNGs saved to.cache/images/<run>/. Cost tracked per-image at ~$0.039 (flash) / ~$0.10 (pro preview). - Settings dashboard in the sidebar: one "⚙️ Generation Settings" expander consolidates every pipeline knob (cleanup, chapters, agentic, fact-check, image generation + style + aspect ratio + model).
- Image gallery in the main area: after a run, generated images render as a 3-column grid with per-image download button; failures show a warning with the API error.
- Tightened the
image_promptsDEFAULT_PROMPT to produce a numbered list with bold labels, which made the extractor reliable (5/5 prompts parsed in live test vs 1 before).
extract_prompts()now tries numbered-list → blockquote → numbered-heading → whole-text-fallback in priority order, tolerating multiple LLM output formats.whisperforge_core/__init__.pyre-exports theimagesmodule.
- In-browser recording via
st.audio_input(native in Streamlit 1.35+). New "Record" tab alongside "Audio Upload" and "Text Input" with Transcribe + I'm Feeling Lucky buttons that wire straight through to the existing pipeline. No separate recording app needed. - Cost tracker — new
whisperforge_core.costmodule with a session usage ledger, per-provider/model pricing table, and Anthropic cache-semantics-awareestimate_cost(). Everyllm._callrecords token counts on success. Sidebar shows "Est. cost" + "Cache saved" metrics for the session. - Run history — new
whisperforge_core.historymodule writes a JSONL record to.cache/history.jsonlafter each successful Notion save (title, link, provider/model, cost, cache savings, flags, audio filename). Sidebar "Recent runs" expander shows the last 8 with one-click Notion links.
whisperforge_core/__init__.pyre-exports the newcostandhistorymodules.
- Agentic article drafting (opt-in via
agentic=Trueor the "Agentic drafting" sidebar toggle). Single-shotarticle_writingbecomes a three-pass flow: draft → critique → revise. NewDEFAULT_PROMPTSentries forarticle_critiqueandarticle_revise. Verified live on a 3-topic clip: Haiku 4.5 produced 13 specific editorial notes addressing voice drift, invented framing, length discipline, and incomplete thoughts; revised article addressed each. - Fact-check pass (opt-in via
fact_check=Trueor the "Fact-check article" toggle). Reads the final article against the transcript, returns JSON list of claims not grounded in the source. Verified live: caught 4 real fabrications (invented launch dates, unsourced tool-behavior claims). New "Fact Check" toggle in Notion (green when clean, red with per-claim details when flagged). PipelineResultgainsarticle_draft,article_critique,fact_check_flags.ContentBundlemirrors these so the editorial trail is auditable from Notion.
- Fact-check (when combined with agentic) runs against the revised article, not the draft — so we're grounding the final product.
- Timestamp-aware chapters end-to-end. WhisperX segments now thread UI → pipeline → chapter prompt → ContentBundle → Notion
[M:SS]prefix. Newaudio.TranscriptionDetailsdataclass andaudio.transcribe_audio_detailed(); newadapters.Transcriber.transcribe_detailed()protocol method; newchapters_timestampedcontent_type that formats input as[SSSS.S]lines and asks the model to copystart_secondsper chapter. Verified live: 3-topic clip → chapters at 4.0s / 15.8s / 27.9s matching audio boundaries.
- Anthropic prompt caching on KB + system blocks. KB goes first with
cache_control: {"type": "ephemeral"}; per-stage prompt follows uncached. Verified: stages 2-5 of a pipeline run read ~5800 cached tokens at 0.1x input cost. ~67% input-token cost reduction per run. - Post-ASR cleanup stage (opt-out via
cleanup=False). New content_typetranscript_cleanupruns as pipeline stage 0, strips fillers/false-starts/typos before downstream stages see the transcript. Steals Wispr Flow's signature move. - Chapterization stage (opt-out via
chapters=False). New content_typechaptersproduces[{title, summary, start_quote}]via JSON schema.llm.generate_chapters()helper parses defensively. Notion gets a new "Chapters" toggle above Transcription with bulleted entries and optional[MM:SS]prefix. - Structured Outputs on title/summary/tags via OpenAI JSON schema mode. All regex fragility gone. Helpers upgraded from gpt-3.5-turbo to gpt-4o-mini.
- WhisperX backend (
TRANSCRIPTION_BACKEND=whisperx). faster-whisper + wav2vec2 alignment + optional pyannote diarization. Model cache is per-process for fast repeat runs. Diarization labels each segment with[SPEAKER_XX]whenWHISPERX_DIARIZATION=1andWHISPERX_HF_TOKENis set. - Silero VAD chunker (
CHUNKER=vad). Cuts on silences instead of bytes, drops silent segments entirely, falls back to size-based on failure. - New env knobs:
WHISPER_MODEL,CHUNKER,WHISPERX_MODEL,WHISPERX_DEVICE,WHISPERX_COMPUTE,WHISPERX_DIARIZATION,WHISPERX_HF_TOKEN.
LLM_MODELScatalog refreshed again. Claude Haiku 4.5 added as new session-state default (fastest + in-voice). Sonnet 4.5 and Opus 4.5 kept as premium tiers.- Cloud transcription default →
gpt-4o-mini-transcribe(waswhisper-1). Still pinnable viaWHISPER_MODEL. _call()split for Anthropic to use structuredsystemblocks (KB first with cache_control, prompt second uncached). OpenAI/Ollama paths unchanged, still flat string.PipelineResultgainedraw_transcript,cleaned_transcript,chaptersfields.ContentBundlegainedchapterslist field.
readme.mdenv-var reference table expanded with all new knobs.
- Local model lanes: Ollama provider (OpenAI-compatible,
localhost:11434) with auto-discovery of installed models viallm.discover_ollama_models(). MLX Whisper and whisper.cpp backends selectable viaTRANSCRIPTION_BACKEND=mlx|whisper_cpp. - Cache wiring —
WHISPERFORGE_CACHE=1enables sha256-keyed result caching acrossaudio.transcribe_audioandllm.generate. Never caches empty/None (so failures retry). 20 new unit tests covering the cache behavior.
LLM_MODELScatalog refreshed. Dropped Claude 3.x (retired from Anthropic's API). Added Claude Haiku 4.5 (new default), Sonnet 4.5, Opus 4.5. OpenAI adds gpt-4o and gpt-4o-mini; gpt-4 and gpt-4-turbo kept as legacy entries.- Default provider/model in
main()session-state init → Anthropicclaude-haiku-4-5(fast, cheap, in-voice). Override in sidebar for premium or local. transcribe_audiosmall-file fast path now routes throughtranscribe_chunkso all three backends (openai / mlx / whisper_cpp) behave consistently.
- Key harvest from
kk-ai-ecosystem/.envauto-populates.env; target Notion database (WhisperForge DB) auto-discovered via integration search.
monitoring/,scripts/,experiments/,services/auth/, rootDockerfile,setup.py,shared-requirements.txt,.DS_Store(all dead).
- Full
readme.mdrewrite: architecture diagram, "Running fully local" section, expanded env-var reference table.
whisperforge_core/package — shared business logic (audio, llm, notion, prompts, pipeline, cache, logging, adapters) used by both the monolith and the FastAPI microservices.DEPLOY_MODE=direct|servicesenv flag selecting between local (in-process) and containerized (HTTP) backends viawhisperforge_core.adapters.styles.py— all CSS extracted fromapp.py.- Text-input tab can now save generated content to Notion (previously dead-ended).
requirements-services.txtfor the FastAPI stack;audioop-ltsshim for Python 3.13 (pydub compat).
app.pyshrunk 2302 → ~705 lines (thin UI shell overwhisperforge_core).- Services under
services/rewritten as thin wrappers onwhisperforge_core;services/frontend/app.pydeleted in favor of a single canonicalapp.pythat runs in both modes. docker-compose.ymlrebuilt: adds missingstorageservice, removes orphanedauth/postgres/admin. Auth deferred for single-user deployments.shared/security.pysimplified to anX-API-Key: SERVICE_TOKENcheck (JWT deferred).- Whisperforge CLI (
whisperforge.py) is now a 45-line wrapper onwhisperforge_core.audio.
requirements.txtwas the FastAPI/JWT stack instead of Streamlit — now lists the correct runtime deps.shared/config.pyNameError on undefinedCLAUDE_API_KEY.- Deprecated
st.experimental_rerun()→st.rerun(). - Three bare
except:clauses replaced withexcept OSError:for cleanup paths.
- Grok provider support (endpoint unverified, only
grok-1listed). - Legacy
old_app.pyafter salvaging logging + cache patterns. services/admin/(stubs, no Dockerfile),services/shared/(dup of/shared/), stale test files.
- Automatic audio file chunking for files over 25MB
- Detailed progress logging in transcription service
- Extended timeout handling for large files
- Reduced chunk size to 10MB for faster processing
- Transcription service timeout increased to 300s
- Frontend request timeout increased to 300s
- Improved error handling and logging
- Added bitrate compression for audio chunks
- Timeout issues with larger audio files
- Memory handling for large file processing
- Temporary file cleanup reliability
- Chunk size: 10MB with 128k bitrate compression
- Max file size remains at 200MB
- Added progress tracking per chunk
- Enhanced error logging across services
- Improved temp file management
- Limited progress feedback in frontend during chunking
- No automatic retry on chunk failure
- Processing time increases linearly with file size
- Initial release
- Audio file upload and transcription
- Notion integration
- Insights extraction
- Language detection
- Chunking support for large files
- Notion emoji validation issue
- Audio processing compatibility with Python 3.11
- Monitoring infrastructure with Prometheus and Grafana
- Basic Notion integration in storage service
- Health check endpoints across all services
- Updated project structure with monitoring directory
- Enhanced error handling in storage service