fix(cli): reject --max-steps < 1 (silent empty run read as a clean pass) - #25
Conversation
A --max-steps of 0 or negative made the agent loop a no-op: the trace was empty and the summary reported hijack=0 / leak=0, which reads as the model resisting the attack when nothing actually ran. Validate after parsing and exit 2 with the existing [leakgauge] error style. Fixes bamdadd#24
|
Thanks for this, Dipak — the fix itself is spot on. The only thing red is a formatting nit, not your logic. CI's (or |
|
Merged — thank you, Dipak. The formatting nit was the only thing standing between this and green, so I ran That's three landed now — really nice run. If leakgauge is useful to you, a star helps others find it and Watch will surface new releases; would love to keep seeing your contributions. |
Problem
--max-steps 0(or negative) makesfor step in range(max_steps)a no-op — the agent never runs, and the benchmark reports hijack=0 / leak=0 / utility=0, which looks like the model resisted the attack when in fact nothing ran.Fix
Validate after arg parsing, matching the existing error style:
with exit code 2, before any adapter/model work happens.
Tests
--max-steps 0→ rc 2, nothing written--max-steps -3→ rc 2--max-steps 1still runs and writes a summarypython -m pytest tests/passes.Fixes #24