Discovered incidentally while hammering cargo test --all --features subprocess -- --test-threads=64 under heavy host load (load average ~50-130 on a 32-core box, from concurrent unrelated builds) trying to reproduce #135.
test_background_job_status_transitions (crates/kaish-kernel/tests/background_execution_tests.rs:172-187) does:
kernel.execute("sleep 1 &").await.unwrap();
...
let status = wait_for_job(&kernel, 1, Duration::from_secs(1)).await;
assert_eq!(status, "done:0", "expected done:0 status");It waits on a background sleep 1 job with a wait_for_job timeout of exactly 1 second — zero margin between the sleep duration and the deadline waiting on it. Under real CPU contention (many concurrent processes competing for scheduling), the child sleep 1 plus the polling/wakeup path can easily take just over 1 wall-clock second, so wait_for_job times out before the job transitions to done:0.
Observed failure (one instance, full log available on request, not preserved beyond this report):
thread 'test_background_job_status_transitions' (690699) panicked at crates/kaish-kernel/tests/background_execution_tests.rs:44:13:
Job 1 did not complete within 1s
test result: FAILED. 18 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.18s
Suggest widening the wait_for_job timeout for this test (e.g. 3-5s) or using a shorter background command so the margin isn't razor-thin under load. Not otherwise investigated further — this was collateral evidence from #135's reproduction attempt, not the focus of that investigation.
Discovered incidentally while hammering
cargo test --all --features subprocess -- --test-threads=64under heavy host load (load average ~50-130 on a 32-core box, from concurrent unrelated builds) trying to reproduce #135.test_background_job_status_transitions(crates/kaish-kernel/tests/background_execution_tests.rs:172-187) does:It waits on a background
sleep 1job with await_for_jobtimeout of exactly 1 second — zero margin between the sleep duration and the deadline waiting on it. Under real CPU contention (many concurrent processes competing for scheduling), the childsleep 1plus the polling/wakeup path can easily take just over 1 wall-clock second, sowait_for_jobtimes out before the job transitions todone:0.Observed failure (one instance, full log available on request, not preserved beyond this report):
Suggest widening the
wait_for_jobtimeout for this test (e.g. 3-5s) or using a shorter background command so the margin isn't razor-thin under load. Not otherwise investigated further — this was collateral evidence from #135's reproduction attempt, not the focus of that investigation.