docs: retire two post-#707 residuals (stale 0.431 board, dead run_one_job.py)#709
Merged
Merged
Conversation
…_job.py) Two residuals survived PR #707's results reseal. architecture/evaluation.rst: the benchmark section still carried a stale 0.431 Fmax board on the superseded GOA 220 to 229 frame under a "Provisional results, pending final recompute" admonition, plus its leakage tables. That board is already preserved verbatim in historical/pre-v227.rst under a superseded banner, so the duplicate is removed from the protocol page and replaced with a pointer to the sealed board (/results) and the retained historical record. The admonition was resolved by declaring the frame superseded (not provisional): the numbers were a different metric and window, never the sealed board, so there was nothing to "recompute". The temporal leakage caveat is kept as a protocol-level point (why external comparisons leak, why references freeze at t0) without the stale figures. The seealso that framed /results as "GOA 220 to 229" is corrected to the sealed v227 to v230 board, with a link to the historical figures. complexity/measuring.rst: dropped three citations of scripts/run_one_job.py, which does not exist (removed in 80ed10e). Documented the real supported path to measure a single job: POST /jobs to enqueue, then run scripts/worker.py for that queue under scalene (the one bundled profiler) or stdlib cProfile, stopping the worker with Ctrl+C after the job reaches SUCCEEDED (it exits cleanly, so the report is written). Removed pyinstrument/snakeviz commands (neither is a project dependency, so they were unverifiable). Added a "Known gap" note pointing at the internal entry point BaseWorker.handle_job in protea/workers/base_worker.py and flagging the missing single-job CLI.
check_openapi_refs treats a documented route absent from the OpenAPI
schema as a phantom and fails lint. The page wrote GET /jobs/<job-uuid>
with a literal placeholder; the schema exposes /jobs/{job_id}.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #709 +/- ##
===========================================
- Coverage 88.90% 88.85% -0.05%
===========================================
Files 237 245 +8
Lines 17692 18481 +789
===========================================
+ Hits 15729 16422 +693
- Misses 1963 2059 +96 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR #707 resealed the results docs on the sealed headline (
f_micro_w0.4063, first in 7 of 9 cells, frame v227 to v230). Two residuals survived it, both flagged in #707's own "known residuals" note.Defect 1:
architecture/evaluation.rstThe stale 0.431 board is removed, not moved, because the record already exists. The full superseded board (the k-ablation, scoring configs, the three re-ranker generations, the external-tool benchmark carrying the
0.431Fmax, and the leakage analysis) is already preserved verbatim inhistorical/pre-v227.rstunder a dated "superseded, retained for provenance only" banner created by #707. The copy inevaluation.rstwas a duplicate of that record. Deleting the duplicate and pointing to the historical page preserves the record while removing the superseded number from a page a reader would read as current. The protocol page keeps the temporal-leakage caveat as a protocol-level point (why external comparisons leak, why references freeze at t0), minus the specific stale figures.Discrepancy reported: the task described two "Provisional" admonitions on this page. There is only one literal
.. admonition:: Provisional results, pending final recompute(and #707's own PR body refers to it in the singular). I resolved that one and also corrected a related staleseealsothat framed/resultsas the "GOA 220 to 229 temporal holdout";/resultsis now the sealed v227 to v230 board, so that bullet was retargeted and a link to the historical figures added.The one admonition read:
Resolution: it is no longer "provisional pending recompute". That board is a different metric on a different, superseded frame; it was never going to become the sealed board, so there is nothing to recompute into it. The honest state is superseded, which is exactly how the historical page frames it. The admonition (and its dangling pointer to a "full provisional notice" on
/results, which no longer exists) was removed together with the duplicated board, and the page now points to the sealed board and the retained historical record.Defect 2:
complexity/measuring.rstscripts/run_one_job.pywas cited three times as a profiling target. It does not exist. Git history: it was added in the initial commit, migrated inb08f536, and deleted in80ed10e("feat: CI/CD, linting, ngrok expose, and test fixes"). Its old body calledworker.handle_job(job_id)directly against a hardcoded 3-op registry, bypassing the queue, the exact ad-hoc pattern the house rules now forbid.There is no single-job CLI wrapper. The task named the internal method
execute_single(job_id); that name does not exist anywhere in the repo. The real internal entry point isBaseWorker.handle_job(job_id)inprotea/workers/base_worker.py, which is what the page now points at.What replaced the dead command (all verified to exist):
POST /jobswith{operation, queue_name, payload}(the only sanctioned dispatch path; the request model and{"id", "status"}response were read fromprotea/api/routers/jobs.py).poetry run scalene ... scripts/worker.py --queue <queue>.scaleneis a real project dependency (pyproject.toml, locked at 2.3.0);scripts/worker.py --queueis the documented worker command; the worker catchesKeyboardInterrupt, logsWorker stopped., and exits cleanly (read inscripts/worker.py), so scalene's on-exit report is written afterCtrl+C.cProfile+pstatswrapping the same worker.pyinstrumentandsnakevizcommands: neither is a project dependency, sopoetry run pyinstrument/snakevizwere unverifiable and I would not print them.scripts/run_one_job.pywas removed in80ed10e; the internal entry point isBaseWorker.handle_job(job_id); until a wrapper lands, profile the continuous worker or read per-step timings from the JobEvent log (which needs no profiler).Validation
scripts/check_no_em_dashes.py: OK, 0 violations on both files.scripts/check_doc_refs.py: 199 broken refs, unchanged from the clean-developbaseline. All 5 contributed byevaluation.rstare pre-existing autodoc:module:/:func:roles I did not touch;measuring.rstcontributes 0. My new:doc:cross-refs (/results,/historical/pre-v227,/operate/reproduce-0.4063) all resolve.Things I was told that turned out false
evaluation.rst: there is only one.base_worker.pyhas an internalexecute_single(job_id)method": no such method exists; the real entry point ishandle_job(job_id). Documented the real one; fabricated nothing.