Merge pull request #421 from Integration-Automation/feat/ax-text-advanced-batch

Add advanced TextPattern: find / select / read text attributes

Author: JE-Chen

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-25) — Advanced TextPattern (find / select / read attributes)
+
+Search a control's text, select a match to replace it, and read font/colour formatting. Full reference: [`docs/source/Eng/doc/new_features/v200_features_doc.rst`](docs/source/Eng/doc/new_features/v200_features_doc.rst).
+
+- **`find_control_text` / `select_control_text` / `control_text_attributes`** (`AC_find_control_text`, `AC_select_control_text`, `AC_control_text_attributes`): `ax_text` shipped the three whole-range *reads*, but couldn't search for a substring, select a found range, or read text formatting — needed to assert "the error word is red and bold" or to place the selection at matched text before typing. This rounds out TextPattern: `find_control_text` searches the real content (not OCR) via `FindText`, `select_control_text` finds + selects a range so the next keystrokes replace it, and `control_text_attributes` reads `{font_name, font_size, bold, italic, foreground_color}`. Dispatched through the injectable accessibility backend seam (headless-testable via a fake backend; real UIA in the Windows backend). No `PySide6`.
+
 ## What's new (2026-06-25) — MSAA Bridge for Legacy Controls (LegacyIAccessible)
 
 Automate the long tail of old Win32 controls that expose nothing via modern UIA. Full reference: [`docs/source/Eng/doc/new_features/v199_features_doc.rst`](docs/source/Eng/doc/new_features/v199_features_doc.rst).

docs/source/Eng/doc/new_features/v200_features_doc.rst

@@ -0,0 +1,48 @@
+Advanced TextPattern — Find / Select / Read Attributes
+======================================================
+
+``ax_text`` shipped the three whole-range *reads* (document / selection / visible
+text). It could not **search** for a substring, **select** a found range, or read
+text **formatting attributes** — needed to assert "the error word is red and
+bold" or to place the caret / selection at matched text before typing. This rounds
+TextPattern out from "dump the text" to "interrogate and manipulate" it.
+
+* :func:`find_control_text` — whether ``text`` occurs in the control
+  (TextPattern.FindText, searches the real content, not OCR),
+* :func:`select_control_text` — find ``text`` and select its range, so the next
+  keystrokes replace it (FindText + Select),
+* :func:`control_text_attributes` — the selection's font / colour formatting.
+
+Each is a thin dispatch onto the injectable ``accessibility.backends.get_backend()``
+seam — headless-testable via a fake backend; the real UIA calls live in the
+Windows backend. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import (find_control_text, select_control_text,
+                                 control_text_attributes, type_text)
+
+    if find_control_text("TODO", name="Editor"):
+        select_control_text("TODO", name="Editor")    # selection now spans "TODO"
+        type_text("DONE")                              # replaces it
+
+    control_text_attributes(name="Editor")
+    # {"font_name": "Consolas", "font_size": 11.0, "bold": True,
+    #  "italic": False, "foreground_color": 16711680}
+
+``ignore_case`` (default ``True``) controls the search. The control is located by
+``name`` / ``role`` / ``app_name`` / ``automation_id`` (same as the other
+TextPattern reads). ``find_control_text`` / ``select_control_text`` return
+``bool``; ``control_text_attributes`` returns the formatting dict (values may be
+``None`` where the range spans mixed formatting) or ``None`` if not found.
+
+Executor commands
+-----------------
+
+``AC_find_control_text`` / ``AC_select_control_text`` (``text`` / ``ignore_case``)
+and ``AC_control_text_attributes`` (``{found, attributes}``). They are exposed as
+the matching ``ac_*`` MCP tools (find / attributes read-only, select destructive)
+and as Script Builder commands under **Native UI**.

docs/source/Zh/doc/new_features/v200_features_doc.rst

@@ -0,0 +1,44 @@
+進階 TextPattern——搜尋 / 選取 / 讀取屬性
+========================================
+
+``ax_text`` 先前提供三種整段*讀取*(document / selection / visible 文字)。它無法**搜尋**子字串、
+**選取**找到的範圍,或讀取文字的**格式屬性**——而這些正是斷言「錯誤字是紅色且粗體」或在輸入前
+把游標 / 選取定位到匹配文字所需。本次把 TextPattern 從「傾印文字」擴充為「查詢與操作」文字。
+
+* :func:`find_control_text` ——``text`` 是否出現在控制項中(TextPattern.FindText,搜尋真正的
+  內容,而非 OCR),
+* :func:`select_control_text` ——找到 ``text`` 並選取其範圍,讓接下來的按鍵取代它
+  (FindText + Select),
+* :func:`control_text_attributes` ——選取範圍的字型 / 顏色格式。
+
+每個都是對可注入的 ``accessibility.backends.get_backend()`` 接縫的薄分派——可透過注入 fake
+backend 進行無頭測試;真正的 UIA 呼叫位於 Windows 後端。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import (find_control_text, select_control_text,
+                                 control_text_attributes, type_text)
+
+    if find_control_text("TODO", name="Editor"):
+        select_control_text("TODO", name="Editor")    # 選取範圍現在涵蓋 "TODO"
+        type_text("DONE")                              # 取代它
+
+    control_text_attributes(name="Editor")
+    # {"font_name": "Consolas", "font_size": 11.0, "bold": True,
+    #  "italic": False, "foreground_color": 16711680}
+
+``ignore_case``(預設 ``True``)控制搜尋。控制項以 ``name`` / ``role`` / ``app_name`` /
+``automation_id`` 定位(與其他 TextPattern 讀取相同)。``find_control_text`` /
+``select_control_text`` 回傳 ``bool``;``control_text_attributes`` 回傳格式字典(範圍跨越混合
+格式時某些值可能為 ``None``),找不到則回傳 ``None``。
+
+執行器指令
+----------
+
+``AC_find_control_text`` / ``AC_select_control_text``(``text`` / ``ignore_case``)與
+``AC_control_text_attributes``(``{found, attributes}``)。皆以對應的 ``ac_*`` MCP 工具
+(find / attributes 為唯讀、select 為破壞性)及 Script Builder 指令(位於 **Native UI** 分類下)
+形式提供。

je_auto_control/__init__.py

@@ -53,9 +53,10 @@
     collapse_control, control_expand_state, control_range, expand_control,
     scroll_control_into_view, select_control_item, set_control_range,
 )
-# Native text reads via UIA TextPattern (document / selection / visible)
+# Native text via UIA TextPattern (read / find / select / attributes)
 from je_auto_control.utils.ax_text import (
-    get_control_text, get_selected_text, get_visible_text,
+    control_text_attributes, find_control_text, get_control_text,
+    get_selected_text, get_visible_text, select_control_text,
 )
 # Readable / addressable a11y-tree post-processing (role names + node paths)
 from je_auto_control.utils.ax_tree_walk import (
@@ -1681,6 +1682,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "select_control_item", "control_range", "set_control_range",
     "scroll_control_into_view",
     "get_control_text", "get_selected_text", "get_visible_text",
+    "find_control_text", "select_control_text", "control_text_attributes",
     "control_type_name", "humanize_role", "humanize_tree",
     "assign_node_paths", "find_by_path",
     "is_interactive_role", "tab_order", "audit_focus_order", "focus_control",

je_auto_control/gui/script_builder/command_schema.py

@@ -1681,6 +1681,25 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
         fields=fields,
         description="Read full text via TextPattern (multiline / document safe).",
     ))
+    specs.append(CommandSpec(
+        "AC_find_control_text", "Native UI", "Find Text in Control",
+        fields=(FieldSpec("text", FieldType.STRING),
+                FieldSpec("ignore_case", FieldType.BOOL, optional=True,
+                          default=True)) + fields,
+        description="Whether text occurs in a control (TextPattern.FindText).",
+    ))
+    specs.append(CommandSpec(
+        "AC_select_control_text", "Native UI", "Select Text in Control",
+        fields=(FieldSpec("text", FieldType.STRING),
+                FieldSpec("ignore_case", FieldType.BOOL, optional=True,
+                          default=True)) + fields,
+        description="Find + select text in a control (FindText + Select).",
+    ))
+    specs.append(CommandSpec(
+        "AC_control_text_attributes", "Native UI", "Get Text Attributes",
+        fields=fields,
+        description="Read selection formatting (font/size/bold/italic/colour).",
+    ))
     specs.append(CommandSpec(
         "AC_get_selected_text", "Native UI", "Get Selected Text",
         fields=fields,

je_auto_control/utils/accessibility/backends/base.py

@@ -123,6 +123,28 @@ def visible_text(self, name: Optional[str] = None, role: Optional[str] = None,
         """Return only the on-screen text of the control (TextPattern), or None."""
         self._unsupported("visible_text")
 
+    def find_text(self, text: str = "", ignore_case: bool = True,
+                  name: Optional[str] = None, role: Optional[str] = None,
+                  app_name: Optional[str] = None,
+                  automation_id: Optional[str] = None) -> bool:
+        """Return whether ``text`` occurs in the control (TextPattern.FindText)."""
+        self._unsupported("find_text")
+
+    def select_text(self, text: str = "", ignore_case: bool = True,
+                    name: Optional[str] = None, role: Optional[str] = None,
+                    app_name: Optional[str] = None,
+                    automation_id: Optional[str] = None) -> bool:
+        """Find ``text`` and select its range (TextPattern.FindText + Select)."""
+        self._unsupported("select_text")
+
+    def text_attributes(self, name: Optional[str] = None,
+                        role: Optional[str] = None, app_name: Optional[str] = None,
+                        automation_id: Optional[str] = None,
+                        ) -> Optional[Dict[str, Any]]:
+        """Return formatting of the control's selection — ``{font_name, font_size,
+        bold, italic, foreground_color}`` (TextPattern attributes), or None."""
+        self._unsupported("text_attributes")
+
     # --- keyboard focus ----------------------------------------------------
 
     def set_focus(self, name: Optional[str] = None, role: Optional[str] = None,

je_auto_control/utils/accessibility/backends/windows_backend.py

@@ -445,6 +445,48 @@ def visible_text(self, name=None, role=None, app_name=None,
         except (OSError, AttributeError):
             return None
 
+    def _find_range(self, text, ignore_case, name, role, app_name, automation_id):
+        """Find ``text`` in the control's document range (TextPattern.FindText)."""
+        pattern = self._text_pattern(name, role, app_name, automation_id)
+        if pattern is None:
+            return None
+        try:
+            return pattern.DocumentRange.FindText(str(text), False,
+                                                  bool(ignore_case))
+        except (OSError, AttributeError):
+            return None
+
+    def find_text(self, text="", ignore_case=True, name=None, role=None,
+                  app_name=None, automation_id=None) -> bool:
+        return self._find_range(text, ignore_case, name, role, app_name,
+                                automation_id) is not None
+
+    def select_text(self, text="", ignore_case=True, name=None, role=None,
+                    app_name=None, automation_id=None) -> bool:
+        found = self._find_range(text, ignore_case, name, role, app_name,
+                                 automation_id)
+        if not found:
+            return False
+        try:
+            found.Select()
+            return True
+        except (OSError, AttributeError):
+            return False
+
+    def text_attributes(self, name=None, role=None, app_name=None,
+                        automation_id=None) -> Optional[Dict[str, Any]]:
+        pattern = self._text_pattern(name, role, app_name, automation_id)
+        if pattern is None:
+            return None
+        try:
+            selection = pattern.GetSelection()
+            text_range = (selection.GetElement(0)
+                          if selection and int(selection.Length or 0) > 0
+                          else pattern.DocumentRange)
+        except (OSError, AttributeError):
+            return None
+        return _read_text_attributes(text_range)
+
     def set_focus(self, name=None, role=None, app_name=None,
                   automation_id=None) -> bool:
         raw = self._find_raw(name, role, app_name, automation_id)
@@ -512,6 +554,33 @@ def _as_text(value) -> str:
     return str(value or "")
 
 
+# UIA TextPattern attribute ids (UIAutomationClient AttributeId range).
+_TEXT_ATTR_FONT_NAME = 40005
+_TEXT_ATTR_FONT_SIZE = 40006
+_TEXT_ATTR_FONT_WEIGHT = 40007
+_TEXT_ATTR_FOREGROUND = 40008
+_TEXT_ATTR_IS_ITALIC = 40014
+
+
+def _attr(text_range, attribute_id, cast):
+    try:
+        return cast(text_range.GetAttributeValue(attribute_id))
+    except (OSError, AttributeError, ValueError, TypeError):
+        return None
+
+
+def _read_text_attributes(text_range) -> Dict[str, Any]:
+    """Read font / colour formatting of a TextRange into a plain dict."""
+    weight = _attr(text_range, _TEXT_ATTR_FONT_WEIGHT, int)
+    return {
+        "font_name": _attr(text_range, _TEXT_ATTR_FONT_NAME, _as_text),
+        "font_size": _attr(text_range, _TEXT_ATTR_FONT_SIZE, float),
+        "bold": (weight >= 700) if isinstance(weight, int) else None,
+        "italic": _attr(text_range, _TEXT_ATTR_IS_ITALIC, bool),
+        "foreground_color": _attr(text_range, _TEXT_ATTR_FOREGROUND, int),
+    }
+
+
 # (key, LegacyIAccessiblePattern attribute, cast) for the MSAA bridge read.
 _LEGACY_READS = (
     ("name", "CurrentName", _as_text),

je_auto_control/utils/ax_text/__init__.py

@@ -1,8 +1,10 @@
-"""Native text reading via the UI Automation TextPattern (document/selection/visible)."""
+"""Native text via the UI Automation TextPattern (read / find / select / attributes)."""
 from je_auto_control.utils.ax_text.ax_text import (
-    get_control_text, get_selected_text, get_visible_text,
+    control_text_attributes, find_control_text, get_control_text,
+    get_selected_text, get_visible_text, select_control_text,
 )
 
 __all__ = [
     "get_control_text", "get_selected_text", "get_visible_text",
+    "find_control_text", "select_control_text", "control_text_attributes",
 ]

je_auto_control/utils/ax_text/ax_text.py

@@ -17,7 +17,7 @@
 platform by injecting a fake backend; the real UIA calls live in the Windows
 backend. Imports no ``PySide6``.
 """
-from typing import Optional
+from typing import Any, Dict, Optional
 
 
 def _backend():
@@ -59,3 +59,34 @@ def get_visible_text(name: Optional[str] = None, role: Optional[str] = None,
     """
     return _backend().visible_text(name=name, role=role, app_name=app_name,
                                    automation_id=automation_id)
+
+
+def find_control_text(text: str, *, ignore_case: bool = True,
+                      name: Optional[str] = None, role: Optional[str] = None,
+                      app_name: Optional[str] = None,
+                      automation_id: Optional[str] = None) -> bool:
+    """Return whether ``text`` occurs in the control (TextPattern.FindText)."""
+    return _backend().find_text(str(text), bool(ignore_case), name=name,
+                                role=role, app_name=app_name,
+                                automation_id=automation_id)
+
+
+def select_control_text(text: str, *, ignore_case: bool = True,
+                        name: Optional[str] = None, role: Optional[str] = None,
+                        app_name: Optional[str] = None,
+                        automation_id: Optional[str] = None) -> bool:
+    """Find ``text`` and select its range — position the caret / selection before
+    typing to replace it (TextPattern.FindText + Select); True on success."""
+    return _backend().select_text(str(text), bool(ignore_case), name=name,
+                                  role=role, app_name=app_name,
+                                  automation_id=automation_id)
+
+
+def control_text_attributes(name: Optional[str] = None, role: Optional[str] = None,
+                            app_name: Optional[str] = None,
+                            automation_id: Optional[str] = None,
+                            ) -> Optional[Dict[str, Any]]:
+    """Return the control selection's formatting — ``{font_name, font_size, bold,
+    italic, foreground_color}`` (TextPattern attributes), or None if not found."""
+    return _backend().text_attributes(name=name, role=role, app_name=app_name,
+                                      automation_id=automation_id)

je_auto_control/utils/executor/action_executor.py

@@ -2582,6 +2582,38 @@ def _get_control_text(name: Optional[str] = None, role: Optional[str] = None,
                                      automation_id=automation_id)}
 
 
+def _find_control_text(text: str, ignore_case: Any = True,
+                       name: Optional[str] = None, role: Optional[str] = None,
+                       app_name: Optional[str] = None,
+                       automation_id: Optional[str] = None) -> bool:
+    """Adapter: whether text occurs in a control (TextPattern.FindText)."""
+    from je_auto_control.utils.ax_text import find_control_text
+    return find_control_text(str(text), ignore_case=bool(ignore_case), name=name,
+                             role=role, app_name=app_name,
+                             automation_id=automation_id)
+
+
+def _select_control_text(text: str, ignore_case: Any = True,
+                         name: Optional[str] = None, role: Optional[str] = None,
+                         app_name: Optional[str] = None,
+                         automation_id: Optional[str] = None) -> bool:
+    """Adapter: find + select text in a control (TextPattern.FindText + Select)."""
+    from je_auto_control.utils.ax_text import select_control_text
+    return select_control_text(str(text), ignore_case=bool(ignore_case),
+                               name=name, role=role, app_name=app_name,
+                               automation_id=automation_id)
+
+
+def _control_text_attributes(name: Optional[str] = None, role: Optional[str] = None,
+                             app_name: Optional[str] = None,
+                             automation_id: Optional[str] = None) -> Dict[str, Any]:
+    """Adapter: read a control selection's font/colour formatting (TextPattern)."""
+    from je_auto_control.utils.ax_text import control_text_attributes
+    attrs = control_text_attributes(name=name, role=role, app_name=app_name,
+                                    automation_id=automation_id)
+    return {"found": attrs is not None, "attributes": attrs}
+
+
 def _get_selected_text(name: Optional[str] = None, role: Optional[str] = None,
                        app_name: Optional[str] = None,
                        automation_id: Optional[str] = None) -> Dict[str, Any]:
@@ -6528,6 +6560,9 @@ def __init__(self):
             "AC_legacy_info": _legacy_info,
             "AC_legacy_default_action": _legacy_default_action,
             "AC_get_control_text": _get_control_text,
+            "AC_find_control_text": _find_control_text,
+            "AC_select_control_text": _select_control_text,
+            "AC_control_text_attributes": _control_text_attributes,
             "AC_get_selected_text": _get_selected_text,
             "AC_get_visible_text": _get_visible_text,
             "AC_read_table": _read_table,