Problem
The layered AI fallback in feat/ai-backend-codex-sdk is unit-tested with mocked subprocesses, but the real handoffs have never been observed firing:
- codex SDK → codex CLI subprocess (when the SDK raises)
- codex → claude cross-backend hop (
AI_FALLBACK_CHAIN)
The logic is proven per-branch in tests/test_ai.py (TestRunCodexFallback, TestFallbackChain), but no test forces a real SDK failure and watches codex exec / claude -p actually catch it end-to-end.
Why it matters
Fallback is the whole safety story for using the beta openai-codex package. If the real handoff has a bug the mocks don't model (e.g. the CLI subprocess env, auth, or output shape differs), the "safe fallback" claim is unproven.
Fix
Add a live smoke test (manual or gated integration test) that:
- Monkeypatches
_run_codex_sdk to raise, runs a real book, confirms codex exec returns valid metadata.
- Sets an unusable codex +
AI_FALLBACK_CHAIN=claude, confirms claude -p catches it.
Context
codex SDK path and claude resolve() path ARE live-verified on real books. Only the failure-handoff paths are mock-only.
File: src/audiobook_pipeline/ai.py — _run_codex, _run_cli_prompt
Problem
The layered AI fallback in
feat/ai-backend-codex-sdkis unit-tested with mocked subprocesses, but the real handoffs have never been observed firing:AI_FALLBACK_CHAIN)The logic is proven per-branch in
tests/test_ai.py(TestRunCodexFallback,TestFallbackChain), but no test forces a real SDK failure and watchescodex exec/claude -pactually catch it end-to-end.Why it matters
Fallback is the whole safety story for using the beta
openai-codexpackage. If the real handoff has a bug the mocks don't model (e.g. the CLI subprocess env, auth, or output shape differs), the "safe fallback" claim is unproven.Fix
Add a live smoke test (manual or gated integration test) that:
_run_codex_sdkto raise, runs a real book, confirmscodex execreturns valid metadata.AI_FALLBACK_CHAIN=claude, confirmsclaude -pcatches it.Context
codex SDK path and claude
resolve()path ARE live-verified on real books. Only the failure-handoff paths are mock-only.File:
src/audiobook_pipeline/ai.py—_run_codex,_run_cli_prompt