fix(test-runtime): isolate run-graph Codex logs per run to stop flaky live-run detection#274
Merged
Merged
Conversation
Run-graph Department executions inherited the ambient FKST_RUNTIME_LOG_DIR, so fkst.codex_runs could see user-global live or recent records and suppress mocked Codex spawns. Give each run_graph invocation a tempdir-scoped Codex log directory shared by all of its Department steps. Cover ambient isolation, same-graph status visibility, and repeated-run isolation. Tests: - cargo build -p fkst-framework - cargo test -p fkst-framework --test run_graph_cli - cargo test -p fkst-framework --test sdk_codex codex_runs The full crate test was attempted but the managed sandbox denies the observe integration test Unix socket bind with EPERM. ⟦AI:FKST⟧
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
ElonSG
added a commit
that referenced
this pull request
Jul 14, 2026
Generalize #274's per-run_graph Codex log isolation to each top-level Lua test. This closes the producer-side ambient FKST_RUNTIME_LOG_DIR leak that let observation tests pollute later run_graph consumers in the same harness process. Reuse the existing Department env guard for error-safe restoration, and cover cross-test status isolation plus error and panic cleanup. ⟦AI:FKST⟧ Co-authored-by: Loning <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
fkst.test.run_graphinvoked every department withDeptRunOptions::default_env(), supplying no per-runFKST_RUNTIME_LOG_DIR.fkst.codex_runs(live-run detection) therefore read the ambient user-global Codex log stream (~/Library/Logs/fkst) viasdk_codex's fallback. A matching running record — or a recent completed record from a preceding test invocation — makes a department (e.g.github-devloop-pr.fix) takeskip-idempotent(live-exec-ref)after its driving delivery is accepted, so the graph quiesces without spawning the mocked Codex.Effect: any run-graph test that asserts a Codex spawn is flaky — it fails whenever any ambient
codexprocess runs on the host (e.g. concurrent operator workers) or a prior test seeded the global log dir, and passes in clean CI. This was diagnosed from agithub-devloop-prcomposed run-graph test (run_graph_polluted_merge_gate_stream_reaches_fix) that failed locally (ambient codex running) but was green in CI; controlled reproduction (seeding the global log dir) flips PASS→FAIL, andmax_stepsis irrelevant (a passing graph spawns Codex at step 5 of 11; cap 20 still fails with a seeded dir).Fix (test-runtime only; production untouched)
run_graphnow creates a per-invocationruntime_log_dirunder the temp dir it already allocates and threads it throughrun_recordto every department execution viaFKST_RUNTIME_LOG_DIR(RUNTIME_LOG_DIR_ENV). One shared dir perrun_graphcall ⇒ codex-run statuses created within a graph stay visible to later steps of that same graph; a fresh dir per call ⇒ graphs are isolated from each other and from the ambient/global stream.sdk_codex(read_codex_status_records,runtime_log_dir/default_runtime_log_dir) has zero diff — real runs are unaffected.max_stepsunchanged; no production live-run dedup change.Test evidence (operator-verified outside sandbox)
run_graph_isolates_codex_logs_per_run_and_preserves_intra_graph_visibility: seeds a matching ambient codex-run record, asserts the in-graph spawn is NOT suppressed, and asserts intra-graph status visibility + inter-run isolation. RED without the fix (exited 101, "ambient codex status suppressed in-graph spawn", run_graph exceeded max_steps with pending=1); GREEN with it.run_graph_clisuite: 6 passed / 0 failed.cargo build -p fkst-framework: clean.Review record
Operator diff-review: production isolation confirmed (no
sdk_codex.rsdiff),with_env_varhelper ispub(crate)test-infra, per-graph tempdir scoping guarantees both isolation and intra-graph visibility, no existing test weakened. Diagnosed via an independent codex investigation (controlled flaky reproduction) as the sole root cause of the local-red/CI-green divergence.🤖 Generated with Claude Code
https://claude.ai/code/session_01C8iuJ16FXCA7ejVruPEoZG
⟦AI:FKST⟧