Skip to content

Commit 4856a04

Browse files
authored
Merge pull request #382 from Integration-Automation/feat/step-repair-batch
Add step_repair: repair-tactic policy for failed / no-effect actions
2 parents d9b362c + ddde1f8 commit 4856a04

15 files changed

Lines changed: 346 additions & 0 deletions

File tree

README/WHATS_NEW_zh-CN.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# 本次更新 — AutoControl
22

3+
## 本次更新 (2026-06-24) — 失败 / 无效果动作的修复策略引擎
4+
5+
当动作没效果时选择下一个修复战术——并驱动重试循环。完整参考:[`docs/source/Zh/doc/new_features/v170_features_doc.rst`](../docs/source/Zh/doc/new_features/v170_features_doc.rst)
6+
7+
- **`plan_repair` / `next_tactic` / `run_with_repair`**(`AC_plan_repair`):`self_healing`/`locator_repair` 只修复*无法解析*的定位器;`loop_guard`*检测*卡住循环而无战术选择。本功能消费效果判定(例如来自 `action_effect`)并返回有序战术——`wait_retry` / `relocate` / `nudge` / `scroll_into_view` / `escalate`——接着 `run_with_repair` 以注入的 `act` / `verify` / `apply_tactic` / `verdict_for` / `sleep` 接缝驱动有界重试循环,返回 `RepairOutcome`。纯标准库状态机;不导入 `PySide6`。与 `action_effect` + `postcondition` 完成自我修正三件套。
8+
39
## 本次更新 (2026-06-24) — 声明式动作后置条件
410

511
以 JSON 规格断言动作的预期结果,并对照 before 帧做差异。完整参考:[`docs/source/Zh/doc/new_features/v169_features_doc.rst`](../docs/source/Zh/doc/new_features/v169_features_doc.rst)

README/WHATS_NEW_zh-TW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# 本次更新 — AutoControl
22

3+
## 本次更新 (2026-06-24) — 失敗 / 無效果動作的修復策略引擎
4+
5+
當動作沒效果時選擇下一個修復戰術——並驅動重試迴圈。完整參考:[`docs/source/Zh/doc/new_features/v170_features_doc.rst`](../docs/source/Zh/doc/new_features/v170_features_doc.rst)
6+
7+
- **`plan_repair` / `next_tactic` / `run_with_repair`**(`AC_plan_repair`):`self_healing`/`locator_repair` 只修復*無法解析*的定位器;`loop_guard`*偵測*卡住迴圈而無戰術選擇。本功能消費效果判定(例如來自 `action_effect`)並回傳有序戰術——`wait_retry` / `relocate` / `nudge` / `scroll_into_view` / `escalate`——接著 `run_with_repair` 以注入的 `act` / `verify` / `apply_tactic` / `verdict_for` / `sleep` 接縫驅動有界重試迴圈,回傳 `RepairOutcome`。純標準函式庫狀態機;不匯入 `PySide6`。與 `action_effect` + `postcondition` 完成自我修正三件套。
8+
39
## 本次更新 (2026-06-24) — 宣告式動作後置條件
410

511
以 JSON 規格斷言動作的預期結果,並對照 before 幀做差異。完整參考:[`docs/source/Zh/doc/new_features/v169_features_doc.rst`](../docs/source/Zh/doc/new_features/v169_features_doc.rst)

WHATS_NEW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# What's New — AutoControl
22

3+
## What's new (2026-06-24) — Repair-Tactic Policy for Failed / No-Effect Actions
4+
5+
Pick the next repair tactic when an action does nothing — and drive the retry loop. Full reference: [`docs/source/Eng/doc/new_features/v170_features_doc.rst`](docs/source/Eng/doc/new_features/v170_features_doc.rst).
6+
7+
- **`plan_repair` / `next_tactic` / `run_with_repair`** (`AC_plan_repair`): `self_healing`/`locator_repair` only fix a locator that *didn't resolve*; `loop_guard` only *detects* a stuck loop with no tactic selection. This consumes an effect verdict (e.g. from `action_effect`) and returns the ordered tactics to try — `wait_retry` / `relocate` / `nudge` / `scroll_into_view` / `escalate` — then `run_with_repair` drives a bounded retry loop with injected `act` / `verify` / `apply_tactic` / `verdict_for` / `sleep` seams, returning a `RepairOutcome`. Pure-stdlib state machine; no `PySide6`. Completes the self-correction trio with `action_effect` + `postcondition`.
8+
39
## What's new (2026-06-24) — Declarative Action Postconditions
410

511
Assert an action's expected outcome as a JSON spec, diffed against the before-frame. Full reference: [`docs/source/Eng/doc/new_features/v169_features_doc.rst`](docs/source/Eng/doc/new_features/v169_features_doc.rst).
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
Repair-Tactic Policy for Failed / No-Effect Actions
2+
===================================================
3+
4+
When an action does nothing or lands wrong, the agent needs a *policy* for what to try next —
5+
re-locate and retry, nudge the coordinate, scroll the target into view, wait and retry, or give
6+
up and escalate. ``self_healing`` / ``locator_repair`` only repair a locator that *did not
7+
resolve* (element not found); they do nothing when the element was found and clicked but had no
8+
effect. ``loop_guard`` only *detects* a stuck loop — it has no tactic selection or backoff.
9+
``step_repair`` is that missing controller: it consumes an effect verdict (e.g. from
10+
``action_effect``) and drives a bounded retry loop, choosing the next untried tactic each round.
11+
12+
Pure-stdlib state machine; every side effect — performing the action, verifying it, applying a
13+
tactic, sleeping — is an injected callable, so the loop is fully deterministic and
14+
unit-testable with no device. Imports no ``PySide6``.
15+
16+
Headless API
17+
------------
18+
19+
.. code-block:: python
20+
21+
from je_auto_control import (plan_repair, run_with_repair, RepairPolicy,
22+
classify_effect)
23+
24+
# just the plan
25+
plan_repair("no_op") # ['wait_retry', 'relocate', 'nudge']
26+
plan_repair("changed_elsewhere") # ['escalate']
27+
28+
# drive the loop with injected seams
29+
outcome = run_with_repair(
30+
act=lambda: click(*target),
31+
verify=lambda: not is_no_op(before(), after()),
32+
apply_tactic=apply, # e.g. relocate / nudge the target
33+
verdict_for=lambda: classify_effect(before(), after(), action).effect,
34+
policy=RepairPolicy(max_attempts=3))
35+
print(outcome.ok, outcome.tactics_used)
36+
37+
``plan_repair`` returns the ordered tactics for a verdict (a string like ``no_op`` /
38+
``changed_elsewhere`` or an ``EffectVerdict`` dict), capped at ``max_attempts``;
39+
``next_tactic`` returns the next untried one. ``run_with_repair`` runs ``act`` then ``verify``;
40+
on failure it applies tactics until success or exhaustion, returning a ``RepairOutcome``
41+
(``ok`` / ``attempts`` / ``tactics_used`` / ``detail``). ``RepairPolicy`` caps attempts and
42+
lists the allowed tactics.
43+
44+
Executor command
45+
----------------
46+
47+
``AC_plan_repair`` (``verdict`` / ``max_attempts`` → ``{count, tactics}``) is exposed as the
48+
MCP tool ``ac_plan_repair`` (read-only) and as the Script Builder command **Plan Repair
49+
Tactics** under **Native UI**. (The live ``run_with_repair`` loop is driven from Python, since
50+
it takes injected callables.)

docs/source/Eng/eng_index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,7 @@ Comprehensive guides for all AutoControl features.
192192
doc/new_features/v167_features_doc
193193
doc/new_features/v168_features_doc
194194
doc/new_features/v169_features_doc
195+
doc/new_features/v170_features_doc
195196
doc/ocr_backends/ocr_backends_doc
196197
doc/observability/observability_doc
197198
doc/operations_layer/operations_layer_doc
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
失敗 / 無效果動作的修復策略引擎
2+
================================
3+
4+
當動作沒有效果或落點錯誤時,代理需要一個*策略*決定下一步該試什麼——重新定位重試、微調座標、
5+
把目標捲入視野、等待重試,或放棄並升級。``self_healing`` / ``locator_repair`` 只修復*無法
6+
解析*的定位器(找不到元素);當元素找到並點擊了卻無效果時,它們無能為力。``loop_guard`` 只
7+
*偵測*卡住的迴圈——沒有戰術選擇或退避。``step_repair`` 正是缺少的控制器:它消費一個效果判定
8+
(例如來自 ``action_effect``),並驅動有界的重試迴圈,每輪選擇下一個尚未嘗試的戰術。
9+
10+
純標準函式庫狀態機;每個副作用——執行動作、驗證、套用戰術、睡眠——都是注入的可呼叫物件,
11+
因此迴圈完全確定性、可在無裝置下單元測試。不匯入 ``PySide6``。
12+
13+
無頭 API
14+
--------
15+
16+
.. code-block:: python
17+
18+
from je_auto_control import (plan_repair, run_with_repair, RepairPolicy,
19+
classify_effect)
20+
21+
# 只要規劃
22+
plan_repair("no_op") # ['wait_retry', 'relocate', 'nudge']
23+
plan_repair("changed_elsewhere") # ['escalate']
24+
25+
# 以注入接縫驅動迴圈
26+
outcome = run_with_repair(
27+
act=lambda: click(*target),
28+
verify=lambda: not is_no_op(before(), after()),
29+
apply_tactic=apply, # 例如 relocate / nudge 目標
30+
verdict_for=lambda: classify_effect(before(), after(), action).effect,
31+
policy=RepairPolicy(max_attempts=3))
32+
print(outcome.ok, outcome.tactics_used)
33+
34+
``plan_repair`` 回傳某判定(字串如 ``no_op`` / ``changed_elsewhere`` 或 ``EffectVerdict``
35+
字典)的有序戰術,截到 ``max_attempts``;``next_tactic`` 回傳下一個尚未試過的。
36+
``run_with_repair`` 執行 ``act`` 然後 ``verify``;失敗時套用戰術直到成功或耗盡,回傳
37+
``RepairOutcome``(``ok`` / ``attempts`` / ``tactics_used`` / ``detail``)。``RepairPolicy``
38+
限制嘗試次數並列出允許的戰術。
39+
40+
執行器指令
41+
----------
42+
43+
``AC_plan_repair``(``verdict`` / ``max_attempts`` → ``{count, tactics}``)以 MCP 工具
44+
``ac_plan_repair``(唯讀)及 Script Builder 指令 **Plan Repair Tactics**(位於 **Native UI**
45+
分類下)形式提供。(實際的 ``run_with_repair`` 迴圈因接受注入可呼叫物件,由 Python 驅動。)

docs/source/Zh/zh_index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,7 @@ AutoControl 所有功能的完整使用指南。
192192
doc/new_features/v167_features_doc
193193
doc/new_features/v168_features_doc
194194
doc/new_features/v169_features_doc
195+
doc/new_features/v170_features_doc
195196
doc/ocr_backends/ocr_backends_doc
196197
doc/observability/observability_doc
197198
doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -323,6 +323,10 @@
323323
from je_auto_control.utils.postcondition import (
324324
PostconditionReport, check_postcondition, compile_postcondition,
325325
)
326+
# Repair-tactic policy for failed / no-effect actions (self-correction loop)
327+
from je_auto_control.utils.step_repair import (
328+
RepairOutcome, RepairPolicy, next_tactic, plan_repair, run_with_repair,
329+
)
326330
# Locate on-screen regions by colour (mask + connected components)
327331
from je_auto_control.utils.color_region import (
328332
find_color_region, find_color_regions,
@@ -1267,6 +1271,11 @@ def start_autocontrol_gui(*args, **kwargs):
12671271
"PostconditionReport",
12681272
"check_postcondition",
12691273
"compile_postcondition",
1274+
"RepairPolicy",
1275+
"RepairOutcome",
1276+
"plan_repair",
1277+
"next_tactic",
1278+
"run_with_repair",
12701279
"find_color_region",
12711280
"find_color_regions",
12721281
"ssim_compare",

je_auto_control/gui/script_builder/command_schema.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3205,6 +3205,15 @@ def _add_set_of_marks_specs(specs: List[CommandSpec]) -> None:
32053205
),
32063206
description="Check expected outcome clauses against after/before frames.",
32073207
))
3208+
specs.append(CommandSpec(
3209+
"AC_plan_repair", "Native UI", "Plan Repair Tactics",
3210+
fields=(
3211+
FieldSpec("verdict", FieldType.STRING,
3212+
placeholder="no_op / changed_elsewhere / changed"),
3213+
FieldSpec("max_attempts", FieldType.INT, optional=True, default=3),
3214+
),
3215+
description="Ordered repair tactics for a failed/no-effect action verdict.",
3216+
))
32083217
specs.append(CommandSpec(
32093218
"AC_validate_action", "Native UI", "Validate / Snap Action",
32103219
fields=(

je_auto_control/utils/executor/action_executor.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4144,6 +4144,17 @@ def _check_postcondition(after: Any, spec: Any, before: Any = None) -> Dict[str,
41444144
return check_postcondition(after, spec, before=before).to_dict()
41454145

41464146

4147+
def _plan_repair(verdict: Any, max_attempts: Any = 3) -> Dict[str, Any]:
4148+
"""Adapter: ordered repair tactics for an effect verdict (no_op / changed_…)."""
4149+
import json
4150+
from je_auto_control.utils.step_repair import RepairPolicy, plan_repair
4151+
if isinstance(verdict, str) and verdict.strip().startswith("{"):
4152+
verdict = json.loads(verdict)
4153+
tactics = plan_repair(verdict,
4154+
policy=RepairPolicy(max_attempts=int(max_attempts)))
4155+
return {"count": len(tactics), "tactics": tactics}
4156+
4157+
41474158
def _validate_action(action: Any, screen: Any = None,
41484159
targets: Any = None) -> Dict[str, Any]:
41494160
"""Adapter: validate a coordinate action (bounds + optional snap-to-target)."""
@@ -6025,6 +6036,7 @@ def __init__(self):
60256036
"AC_classify_effect": _classify_effect,
60266037
"AC_effect_near_point": _effect_near_point,
60276038
"AC_check_postcondition": _check_postcondition,
6039+
"AC_plan_repair": _plan_repair,
60286040
"AC_validate_action": _validate_action,
60296041
"AC_replay_trace": _replay_trace,
60306042
"AC_match_elements": _match_elements,

0 commit comments

Comments
 (0)