Skip to content

Commit c5dc4d3

Browse files
committed
Add selection_view: container selection + view switching
select_control_item selects one item, but the container-level SelectionPattern answers what is currently selected and whether multiple selection is allowed - the assertion target after selecting. MultipleViewPattern switches a control between its views (Explorer list/details/tile). get_selection / list_views / set_view extend the backend ABC + Windows UIA backend via the same fake-backend seam.
1 parent b27b15f commit c5dc4d3

13 files changed

Lines changed: 416 additions & 0 deletions

File tree

WHATS_NEW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# What's New — AutoControl
22

3+
## What's new (2026-06-25) — Container Selection + View Switching (Selection / MultipleView)
4+
5+
Read what's selected in a listbox/grid, and switch Explorer-style views. Full reference: [`docs/source/Eng/doc/new_features/v201_features_doc.rst`](docs/source/Eng/doc/new_features/v201_features_doc.rst).
6+
7+
- **`get_selection` / `list_views` / `set_view`** (`AC_get_selection`, `AC_list_views`, `AC_set_view`): `select_control_item` selects *one* item, but the container-level `SelectionPattern` answers "what is currently selected, and may it select multiple?" — the assertion target after selecting. `MultipleViewPattern` switches a control between its views (Explorer's list / details / tile / thumbnail), a precondition that otherwise needs fragile menu clicking. `get_selection` returns `{items, can_select_multiple, is_required}`, `list_views` returns `{current, views}`, and `set_view` switches by view name. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA in the Windows backend). No `PySide6`.
8+
39
## What's new (2026-06-25) — Advanced TextPattern (find / select / read attributes)
410

511
Search a control's text, select a match to replace it, and read font/colour formatting. Full reference: [`docs/source/Eng/doc/new_features/v200_features_doc.rst`](docs/source/Eng/doc/new_features/v200_features_doc.rst).
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
Container Selection + View Switching (Selection / MultipleView)
2+
===============================================================
3+
4+
``select_control_item`` (SelectionItemPattern) selects *one* item; the
5+
container-level ``SelectionPattern`` answers the natural follow-up — **what is
6+
currently selected** in a listbox / grid / tab, and **may it select multiple?** —
7+
the assertion target after selecting. ``MultipleViewPattern`` switches a control
8+
between its views (Explorer's list / details / tile / thumbnail), a common
9+
precondition that otherwise needs fragile menu clicking.
10+
11+
* :func:`get_selection` — ``{items, can_select_multiple, is_required}``,
12+
* :func:`list_views` — ``{current, views: [...]}``,
13+
* :func:`set_view` — switch to a named view.
14+
15+
Each is a thin dispatch onto the injectable ``accessibility.backends.get_backend()``
16+
seam — headless-testable via a fake backend; the real UIA calls live in the
17+
Windows backend. Imports no ``PySide6``.
18+
19+
Headless API
20+
------------
21+
22+
.. code-block:: python
23+
24+
from je_auto_control import get_selection, list_views, set_view
25+
26+
get_selection(name="File List")
27+
# {"items": ["report.pdf", "notes.txt"], "can_select_multiple": True,
28+
# "is_required": False}
29+
30+
list_views(name="File List")
31+
# {"current": "Details", "views": ["List", "Details", "Tiles"]}
32+
set_view("Tiles", name="File List") # switch the view
33+
34+
The control is located by ``name`` / ``role`` / ``app_name`` / ``automation_id``
35+
(same as the other native-control actions). ``get_selection`` / ``list_views``
36+
return their dict (or ``None`` if the control or pattern isn't found);
37+
``set_view`` returns ``bool`` (False when the named view isn't supported).
38+
39+
Executor commands
40+
-----------------
41+
42+
``AC_get_selection`` (``{found, selection}``), ``AC_list_views`` (``{found,
43+
views}``) and ``AC_set_view`` (``view``). They are exposed as the matching
44+
``ac_*`` MCP tools (the reads read-only, ``set_view`` destructive) and as Script
45+
Builder commands under **Native UI**.
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
容器選取 + 檢視切換(Selection / MultipleView)
2+
==============================================
3+
4+
``select_control_item``(SelectionItemPattern)選取*單一*項目;容器層級的 ``SelectionPattern``
5+
回答自然的後續問題——listbox / grid / tab 中**目前選了什麼**,以及**是否可多選?**——這正是選取
6+
之後的斷言目標。``MultipleViewPattern`` 在控制項的各檢視之間切換(檔案總管的清單 / 詳細資料 /
7+
並排 / 縮圖),這是個常見前置條件,否則就得靠脆弱的選單點擊。
8+
9+
* :func:`get_selection` ——``{items, can_select_multiple, is_required}``,
10+
* :func:`list_views` ——``{current, views: [...]}``,
11+
* :func:`set_view` ——切換到具名的檢視。
12+
13+
每個都是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派——可透過注入 fake
14+
backend 進行無頭測試;真正的 UIA 呼叫位於 Windows 後端。不匯入 ``PySide6``。
15+
16+
無頭 API
17+
--------
18+
19+
.. code-block:: python
20+
21+
from je_auto_control import get_selection, list_views, set_view
22+
23+
get_selection(name="File List")
24+
# {"items": ["report.pdf", "notes.txt"], "can_select_multiple": True,
25+
# "is_required": False}
26+
27+
list_views(name="File List")
28+
# {"current": "Details", "views": ["List", "Details", "Tiles"]}
29+
set_view("Tiles", name="File List") # 切換檢視
30+
31+
控制項以 ``name`` / ``role`` / ``app_name`` / ``automation_id`` 定位(與其他原生控制動作相同)。
32+
``get_selection`` / ``list_views`` 回傳其字典(找不到控制項或模式則為 ``None``);``set_view``
33+
回傳 ``bool``(具名檢視不支援時為 False)。
34+
35+
執行器指令
36+
----------
37+
38+
``AC_get_selection``(``{found, selection}``)、``AC_list_views``(``{found, views}``)與
39+
``AC_set_view``(``view``)。皆以對應的 ``ac_*`` MCP 工具(讀取類為唯讀、``set_view`` 為破壞性)
40+
及 Script Builder 指令(位於 **Native UI** 分類下)形式提供。

je_auto_control/__init__.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,10 @@
8585
from je_auto_control.utils.legacy_accessible import (
8686
legacy_default_action, legacy_info,
8787
)
88+
# Container selection state + view switching (Selection / MultipleView patterns)
89+
from je_auto_control.utils.selection_view import (
90+
get_selection, list_views, set_view,
91+
)
8892
# Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
8993
from je_auto_control.utils.clipboard_rich_formats import (
9094
build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1692,6 +1696,7 @@ def start_autocontrol_gui(*args, **kwargs):
16921696
"move_element", "resize_element", "set_window_state",
16931697
"window_interaction_state",
16941698
"legacy_info", "legacy_default_action",
1699+
"get_selection", "list_views", "set_view",
16951700
"build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
16961701
"set_clipboard_rtf", "get_clipboard_rtf",
16971702
"set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1676,6 +1676,21 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
16761676
fields=fields,
16771677
description="Fire an old control's MSAA default action (fallback).",
16781678
))
1679+
specs.append(CommandSpec(
1680+
"AC_get_selection", "Native UI", "Get Container Selection",
1681+
fields=fields,
1682+
description="Read a container's selection (SelectionPattern).",
1683+
))
1684+
specs.append(CommandSpec(
1685+
"AC_list_views", "Native UI", "List Control Views",
1686+
fields=fields,
1687+
description="List a control's selectable views (MultipleViewPattern).",
1688+
))
1689+
specs.append(CommandSpec(
1690+
"AC_set_view", "Native UI", "Set Control View",
1691+
fields=(FieldSpec("view", FieldType.STRING),) + fields,
1692+
description="Switch a control to the named view (MultipleViewPattern).",
1693+
))
16791694
specs.append(CommandSpec(
16801695
"AC_get_control_text", "Native UI", "Get Control Text",
16811696
fields=fields,

je_auto_control/utils/accessibility/backends/base.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,30 @@ def legacy_default_action(self, name: Optional[str] = None,
257257
success — the fallback when Value / Invoke / Toggle all do nothing."""
258258
self._unsupported("legacy_default_action")
259259

260+
# --- container selection + views (Selection / MultipleView patterns) ----
261+
262+
def get_selection(self, name: Optional[str] = None, role: Optional[str] = None,
263+
app_name: Optional[str] = None,
264+
automation_id: Optional[str] = None,
265+
) -> Optional[Dict[str, Any]]:
266+
"""Return a container's selection state — ``{items, can_select_multiple,
267+
is_required}`` (SelectionPattern), or None."""
268+
self._unsupported("get_selection")
269+
270+
def list_views(self, name: Optional[str] = None, role: Optional[str] = None,
271+
app_name: Optional[str] = None,
272+
automation_id: Optional[str] = None,
273+
) -> Optional[Dict[str, Any]]:
274+
"""Return a control's selectable views — ``{current, views: [...]}``
275+
(MultipleViewPattern: list / details / tile / …), or None."""
276+
self._unsupported("list_views")
277+
278+
def set_view(self, view: str = "", name: Optional[str] = None,
279+
role: Optional[str] = None, app_name: Optional[str] = None,
280+
automation_id: Optional[str] = None) -> bool:
281+
"""Switch a control to the named view (MultipleViewPattern); True on success."""
282+
self._unsupported("set_view")
283+
260284
def _unsupported(self, operation: str):
261285
"""Raise a clear error for an action this backend can't perform."""
262286
raise AccessibilityNotAvailableError(

je_auto_control/utils/accessibility/backends/windows_backend.py

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,8 @@
3737
_UIA_TRANSFORM_PATTERN_ID = 10016
3838
_UIA_WINDOW_PATTERN_ID = 10009
3939
_UIA_LEGACYIACCESSIBLE_PATTERN_ID = 10018
40+
_UIA_SELECTION_PATTERN_ID = 10001
41+
_UIA_MULTIPLEVIEW_PATTERN_ID = 10008
4042
_UIA_AUTOMATIONID_PROPERTY = 30011
4143
_EXPAND_STATES = {0: "collapsed", 1: "expanded", 2: "partial", 3: "leaf"}
4244
_WINDOW_VISUAL_STATES = {"normal": 0, "maximized": 1, "minimized": 2}
@@ -370,6 +372,54 @@ def legacy_default_action(self, name=None, role=None, app_name=None,
370372
"IUIAutomationLegacyIAccessiblePattern",
371373
lambda pattern: pattern.DoDefaultAction())
372374

375+
def get_selection(self, name=None, role=None, app_name=None,
376+
automation_id=None) -> Optional[Dict[str, Any]]:
377+
raw = self._find_raw(name, role, app_name, automation_id)
378+
pattern = self._pattern(raw, _UIA_SELECTION_PATTERN_ID,
379+
"IUIAutomationSelectionPattern") if raw else None
380+
if pattern is None:
381+
return None
382+
try:
383+
items = _header_names(pattern.GetCurrentSelection())
384+
can_multiple = bool(pattern.CurrentCanSelectMultiple)
385+
required = bool(pattern.CurrentIsSelectionRequired)
386+
except (OSError, AttributeError):
387+
return None
388+
return {"items": items, "can_select_multiple": can_multiple,
389+
"is_required": required}
390+
391+
def _multiple_view(self, name, role, app_name, automation_id):
392+
raw = self._find_raw(name, role, app_name, automation_id)
393+
return self._pattern(raw, _UIA_MULTIPLEVIEW_PATTERN_ID,
394+
"IUIAutomationMultipleViewPattern") if raw else None
395+
396+
def list_views(self, name=None, role=None, app_name=None,
397+
automation_id=None) -> Optional[Dict[str, Any]]:
398+
pattern = self._multiple_view(name, role, app_name, automation_id)
399+
if pattern is None:
400+
return None
401+
try:
402+
view_ids = list(pattern.GetCurrentSupportedViews())
403+
current = int(pattern.CurrentCurrentView)
404+
except (OSError, AttributeError, ValueError, TypeError):
405+
return None
406+
return {"current": _view_name(pattern, current),
407+
"views": [_view_name(pattern, view_id) for view_id in view_ids]}
408+
409+
def set_view(self, view="", name=None, role=None, app_name=None,
410+
automation_id=None):
411+
pattern = self._multiple_view(name, role, app_name, automation_id)
412+
if pattern is None:
413+
return False
414+
try:
415+
for view_id in pattern.GetCurrentSupportedViews():
416+
if _view_name(pattern, view_id) == str(view):
417+
pattern.SetCurrentView(int(view_id))
418+
return True
419+
except (OSError, AttributeError, ValueError, TypeError):
420+
return False
421+
return False
422+
373423
def get_table_headers(self, name=None, role=None, app_name=None,
374424
automation_id=None) -> Optional[Dict[str, Any]]:
375425
raw = self._find_raw(name, role, app_name, automation_id)
@@ -511,6 +561,14 @@ def _read_row(pattern, row: int, cols: int):
511561
return cells
512562

513563

564+
def _view_name(pattern, view_id) -> str:
565+
"""Return a MultipleViewPattern view's name, or '' on failure."""
566+
try:
567+
return str(pattern.GetViewName(int(view_id)) or "")
568+
except (OSError, AttributeError, ValueError, TypeError):
569+
return ""
570+
571+
514572
def _header_names(array) -> List[str]:
515573
"""Read an IUIAutomationElementArray of header elements into name strings."""
516574
names: List[str] = []

je_auto_control/utils/executor/action_executor.py

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2573,6 +2573,35 @@ def _legacy_default_action(name: Optional[str] = None, role: Optional[str] = Non
25732573
automation_id=automation_id)
25742574

25752575

2576+
def _get_selection(name: Optional[str] = None, role: Optional[str] = None,
2577+
app_name: Optional[str] = None,
2578+
automation_id: Optional[str] = None) -> Dict[str, Any]:
2579+
"""Adapter: a container's selection state (SelectionPattern)."""
2580+
from je_auto_control.utils.selection_view import get_selection
2581+
selection = get_selection(name=name, role=role, app_name=app_name,
2582+
automation_id=automation_id)
2583+
return {"found": selection is not None, "selection": selection}
2584+
2585+
2586+
def _list_views(name: Optional[str] = None, role: Optional[str] = None,
2587+
app_name: Optional[str] = None,
2588+
automation_id: Optional[str] = None) -> Dict[str, Any]:
2589+
"""Adapter: a control's selectable views (MultipleViewPattern)."""
2590+
from je_auto_control.utils.selection_view import list_views
2591+
views = list_views(name=name, role=role, app_name=app_name,
2592+
automation_id=automation_id)
2593+
return {"found": views is not None, "views": views}
2594+
2595+
2596+
def _set_view(view: str, name: Optional[str] = None, role: Optional[str] = None,
2597+
app_name: Optional[str] = None,
2598+
automation_id: Optional[str] = None) -> bool:
2599+
"""Adapter: switch a control to a named view (MultipleViewPattern)."""
2600+
from je_auto_control.utils.selection_view import set_view
2601+
return set_view(str(view), name=name, role=role, app_name=app_name,
2602+
automation_id=automation_id)
2603+
2604+
25762605
def _get_control_text(name: Optional[str] = None, role: Optional[str] = None,
25772606
app_name: Optional[str] = None,
25782607
automation_id: Optional[str] = None) -> Dict[str, Any]:
@@ -6559,6 +6588,9 @@ def __init__(self):
65596588
"AC_window_interaction_state": _window_interaction_state,
65606589
"AC_legacy_info": _legacy_info,
65616590
"AC_legacy_default_action": _legacy_default_action,
6591+
"AC_get_selection": _get_selection,
6592+
"AC_list_views": _list_views,
6593+
"AC_set_view": _set_view,
65626594
"AC_get_control_text": _get_control_text,
65636595
"AC_find_control_text": _find_control_text,
65646596
"AC_select_control_text": _select_control_text,

je_auto_control/utils/mcp_server/tools/_factories.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1283,6 +1283,33 @@ def a11y_control_tools() -> List[MCPTool]:
12831283
handler=h.legacy_default_action,
12841284
annotations=DESTRUCTIVE,
12851285
),
1286+
MCPTool(
1287+
name="ac_get_selection",
1288+
description=("Read a container's selection via SelectionPattern: "
1289+
"{found, selection:{items:[names], can_select_multiple, "
1290+
"is_required}} — what's selected in a listbox/grid/tab."),
1291+
input_schema=schema(dict(_M)),
1292+
handler=h.get_selection,
1293+
annotations=READ_ONLY,
1294+
),
1295+
MCPTool(
1296+
name="ac_list_views",
1297+
description=("List a control's selectable views via "
1298+
"MultipleViewPattern: {found, views:{current, views:"
1299+
"[names]}} — e.g. Explorer list / details / tile."),
1300+
input_schema=schema(dict(_M)),
1301+
handler=h.list_views,
1302+
annotations=READ_ONLY,
1303+
),
1304+
MCPTool(
1305+
name="ac_set_view",
1306+
description=("Switch a control to the named 'view' via "
1307+
"MultipleViewPattern. Returns True on success."),
1308+
input_schema=schema({"view": {"type": "string"}, **_M},
1309+
required=["view"]),
1310+
handler=h.set_view,
1311+
annotations=DESTRUCTIVE,
1312+
),
12861313
MCPTool(
12871314
name="ac_get_control_text",
12881315
description=("Read a control's full text via TextPattern: "

je_auto_control/utils/mcp_server/tools/_handlers.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -885,6 +885,21 @@ def legacy_default_action(name=None, role=None, app_name=None,
885885
return _legacy_default_action(name, role, app_name, automation_id)
886886

887887

888+
def get_selection(name=None, role=None, app_name=None, automation_id=None):
889+
from je_auto_control.utils.executor.action_executor import _get_selection
890+
return _get_selection(name, role, app_name, automation_id)
891+
892+
893+
def list_views(name=None, role=None, app_name=None, automation_id=None):
894+
from je_auto_control.utils.executor.action_executor import _list_views
895+
return _list_views(name, role, app_name, automation_id)
896+
897+
898+
def set_view(view, name=None, role=None, app_name=None, automation_id=None):
899+
from je_auto_control.utils.executor.action_executor import _set_view
900+
return _set_view(view, name, role, app_name, automation_id)
901+
902+
888903
def get_selected_text(name=None, role=None, app_name=None, automation_id=None):
889904
from je_auto_control.utils.executor.action_executor import _get_selected_text
890905
return _get_selected_text(name, role, app_name, automation_id)

0 commit comments

Comments
 (0)