Fix/offline analysis mq - #611
Closed
VinCinx wants to merge 3 commits into
Closed
Conversation
下游离线分析收到 item-complete(success) 后立即反查 standard eval output, 却常读到 running/空。根因是发送时机早于读侧就绪: 链路A(CompleteItemRun) 只写了 run_log(result_state=Logged) 就发 MQ, 而读侧三张表(turn_result / evaluator_result_ref / item_result)要等链路B(scheduler daemon 扫 Logged → RecordItemRunLogs)在下一个 tick 才写, 竞态窗口约一个 tick(BOE 实测 ~22.8s)。 将 item-complete(success) 唯一发送点从链路A 后移到链路B 的 recordEvalItemRunLogs: 在 RecordItemRunLogs 写完读侧三张表 + 置 result_state=Resulted 之后再发, 此刻 下游反查必读到就绪结果, 竞态消除。 - 删除链路A CompleteItemRun 里的 PublishItemComplete 发送块, 仅保留写 Logged - ExptSchedulerImpl 注入 itemCompletePublisher + exptItemRefRepo - recordEvalItemRunLogs 循环外按归属集(expt_item_ref)分组批量补 EvalSetItem (ItemKey / per-item 版本), 循环内 State==Success 守卫后组装并发送 - per-item DatasetVersionID 取 expt_item_ref.EvalSetVersionID(多集非主集也正确), 不用 ExptEvalItem.EvalSetVersionID(主集硬编码) - 抽 buildItemCompleteEventFromScheduler 复用 buildItemCompleteEvent 语义, 单一实现 - 幂等靠 result_state Logged→Resulted 状态位; 下游按 (expt_id,expt_run_id,item_id) 去重 - 发送失败只 CtxWarn 不阻断 Co-Authored-By: Claude Opus 4.8 <[email protected]>
链路B item-complete 发送不可靠: 发失败只告警且 result_state 已置 Resulted 终态, tick 下轮扫描(result_state=Logged)不再命中 → 消息永久丢失。 改为: - 新增终态 ExptItemResultStateSent; Resulted 降级为"读侧已写待发送" - tick 扫描条件纳入 Resulted(Logged→Logged|Resulted), 使待发送 item 下轮被重扫 - 发送成功才 MarkItemResultSent 翻 Sent; 失败留 Resulted 下轮重发 - 非成功行/开源 nil publisher 直接翻 Sent; 卡 Resulted 超 30min 强制翻 Sent + 告警 - 不重复写表: 靠 RecordItemRunLogs 既有幂等门(!=Logged 短路), 统计不重复累加 - sendItemComplete 成功打印完整 event JSON(便于验证与排障) Co-Authored-By: Claude Opus 4.8 <[email protected]>
- Test_buildItemCompleteEvent_LinkAB_Equivalence: 钉死链路A/B组装一致契约 - Test_resolveItemCompleteMeta: 单集/多集/ref缺失/主集缺失/MGet失败/BatchGet失败 (0→100%) - Test_MarkItemResultSent (0→100%) - Test_findEvalSetForItem: nil/单集/多集命中不命中 (50→100%) - Test_sendItemComplete_nilMeta: meta 缺失分支 (→100%) Co-Authored-By: Claude Opus 4.8 <[email protected]>
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## main #611 +/- ##
==========================================
+ Coverage 78.17% 78.21% +0.04%
==========================================
Files 691 691
Lines 84608 84699 +91
==========================================
+ Hits 66139 66248 +109
+ Misses 14538 14524 -14
+ Partials 3931 3927 -4
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Check the PR title
(Optional) Translate the PR title into Chinese
(Optional) More detailed description for this PR(en: English/zh: Chinese)
en:
zh(optional):
(Optional) Which issue(s) this PR fixes