Skip to content

Commit ac60881

Browse files
committed
Document AnyDesk-style Quick Connect + Phase 4/5 features
README.md / README_zh-TW.md / README_zh-CN.md and the corresponding Sphinx new-features pages now cover: - Quick Connect screen as the AnyDesk-style default landing view - parse_remote_desktop_target() coordinator - on_pending_viewer approval callback + view-only mode - ip_allowlist (CIDR + exact IPs) - single_use_tokens (one-shot share codes) - RFC 6238 TOTP 2FA (totp_secret host param + viewer totp_code) - list_host_monitors() + monitor_index - Remote cursor overlay + enable_cursor_broadcast - broadcast_chat / send_chat / on_chat (Phase 5.2) - broadcast_viewer_cursor / on_viewer_cursor (Phase 5.1) - mouse_move_relative input action - Motion-aware capture (frame-hash dedup) - viewer.stats() rolling FPS/kbps snapshot - JpegSequenceRecorder (no PyAV) + RelayServer - host_service install/uninstall CLI per platform Sphinx RST parses without new warnings on top of the existing pre-existing role / Chinese-punctuation noise.
1 parent d123310 commit ac60881

5 files changed

Lines changed: 775 additions & 43 deletions

File tree

README.md

Lines changed: 163 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -577,24 +577,175 @@ viewer.send_input({"action": "type", "text": "hello"})
577577
viewer.disconnect()
578578
```
579579

580-
GUI: **Remote Desktop** tab with two sub-tabs.
581-
582-
- **Host** — token field with a *Generate* button, security warning
583-
about the bind address, start / stop controls, refreshing port +
584-
viewer-count status, and a 4 fps preview pane below the controls so
585-
the user being remoted sees what viewers see.
586-
- **Viewer** — address / port / token form, *Connect* / *Disconnect*,
587-
and a custom frame-display widget that paints incoming JPEG frames
588-
scaled with `KeepAspectRatio`. Mouse / wheel / key events on the
589-
display are remapped from widget coordinates back to the remote
590-
screen's pixel space using the latest frame's dimensions, then
591-
forwarded as `INPUT` messages.
580+
GUI: **Remote Desktop** tab opens to the **Quick Connect** screen
581+
(AnyDesk-style) by default — huge Host ID on one side, a single input
582+
that accepts `host:port`, `ws://`, `wss://`, or a 9-digit Host ID on
583+
the other, with *Connect* and *Start hosting* as the two primary
584+
buttons. Recent connections are remembered across sessions. Advanced
585+
per-transport sub-tabs (legacy TCP / WS host + viewer, WebRTC host +
586+
viewer with manual SDP / custom codecs / TLS pinning) stay one click
587+
away. WebRTC sub-tabs lazy-load so a stock install without the
588+
`[webrtc]` extra still opens the tab.
592589

593590
> ⚠️ Anyone with the host:port and token gets full mouse / keyboard
594591
> control of the host machine. Default bind is `127.0.0.1`; expose
595592
> externally only via SSH tunnel or TLS front-end. The token is the
596593
> only line of defence — treat it like a password.
597594
595+
**Quick Connect headless API.** The transport coordinator that backs
596+
the GUI input box is also exported, so scripts can dispatch the same
597+
way:
598+
599+
```python
600+
from je_auto_control import parse_remote_desktop_target
601+
parse_remote_desktop_target("192.168.1.10:5555")
602+
# ConnectTarget(kind='tcp', host='192.168.1.10', port=5555, ...)
603+
parse_remote_desktop_target("ws://hub:8765/desk")
604+
# ConnectTarget(kind='ws', host='hub', port=8765, path='/desk')
605+
parse_remote_desktop_target("123-456-789")
606+
# ConnectTarget(kind='webrtc_id', host_id='123456789')
607+
```
608+
609+
**Connection approval + view-only mode.** Optional callback gates
610+
every incoming session AnyDesk-style. Returning `"view_only"` admits
611+
the viewer but drops their `INPUT` messages; returning a falsy value
612+
(or raising) sends `AUTH_FAIL` "rejected by host":
613+
614+
```python
615+
from je_auto_control import RemoteDesktopHost, PendingViewer
616+
617+
def gate(p: PendingViewer) -> str:
618+
if p.address[0].startswith("10."):
619+
return "view_only"
620+
return "full" # or True
621+
622+
host = RemoteDesktopHost(token="tok", on_pending_viewer=gate)
623+
```
624+
625+
**IP allowlist (CIDR + exact IPs).** Reject peers outside the
626+
configured ranges *before* TLS / auth runs, so attackers can't probe
627+
further:
628+
629+
```python
630+
host = RemoteDesktopHost(
631+
token="tok", ip_allowlist=["10.0.0.0/8", "192.168.1.100"],
632+
)
633+
```
634+
635+
**One-time share codes** — extra tokens that self-destruct on first
636+
successful auth, ideal for client-support workflows:
637+
638+
```python
639+
host = RemoteDesktopHost(token="tok", single_use_tokens=["abc123"])
640+
host.add_single_use_token("9k4ndx") # rotate at runtime
641+
host.revoke_single_use_token("abc123") # cancel before it's used
642+
```
643+
644+
**TOTP 2FA (RFC 6238, stdlib only).** Layer a 6-digit OTP on top of
645+
the token; host accepts ±1 step of clock drift:
646+
647+
```python
648+
from je_auto_control.utils.remote_desktop.totp import (
649+
generate_secret, generate_code, provisioning_uri,
650+
)
651+
secret = generate_secret()
652+
print(provisioning_uri(secret, account="alice")) # otpauth:// URI for QR
653+
654+
host = RemoteDesktopHost(token="tok", totp_secret=secret)
655+
viewer = RemoteDesktopViewer(
656+
host=..., token="tok", totp_code=generate_code(secret),
657+
)
658+
```
659+
660+
**Multi-monitor selection.** Capture one specific monitor instead of
661+
the combined virtual desktop:
662+
663+
```python
664+
from je_auto_control import list_host_monitors, RemoteDesktopHost
665+
print(list_host_monitors())
666+
# [{'index': 0, 'is_combined': True, ...},
667+
# {'index': 1, 'left': 0, 'top': 0, ...},
668+
# {'index': 2, 'left': 1920, ...}]
669+
host = RemoteDesktopHost(token="tok", monitor_index=1)
670+
```
671+
672+
**Remote cursor overlay.** Host broadcasts cursor position at 30 Hz
673+
(deduped on still desktops); the viewer's popup window draws an arrow
674+
on top of the JPEG stream so you can see exactly where the host's
675+
pointer is. Disable via `enable_cursor_broadcast=False`.
676+
677+
**Multi-viewer collaborative cursors + chat.** Two new message types
678+
(`CHAT` and `CURSOR` with `viewer_id`). Use a `MultiViewerHost` to
679+
relay one viewer's pointer to the others; pair with the chat channel
680+
for ad-hoc text between operators:
681+
682+
```python
683+
host = RemoteDesktopHost(
684+
token="tok", on_chat=lambda sender, text: print(sender, ":", text),
685+
)
686+
host.broadcast_chat("session starts in 30s")
687+
host.broadcast_viewer_cursor("alice", 200, 300)
688+
689+
viewer = RemoteDesktopViewer(
690+
host=..., on_chat=lambda s, t: ...,
691+
on_viewer_cursor=lambda vid, x, y: ...,
692+
)
693+
viewer.send_chat("ack")
694+
```
695+
696+
**Relative mouse mode (FPS / CAD).** New input action that sends
697+
deltas instead of absolute coordinates:
698+
699+
```python
700+
viewer.send_input({"action": "mouse_move_relative", "dx": 5, "dy": -3})
701+
```
702+
703+
**Motion-aware capture.** The capture loop now hashes each encoded
704+
JPEG; identical frames are skipped, so a static desktop produces
705+
~zero bandwidth. New viewers are seeded with the latest frame on auth
706+
so they never see a black popup.
707+
708+
**Live stats** (FPS / kbps / totals over a 3-second window):
709+
710+
```python
711+
viewer.stats()
712+
# {'fps': 24.3, 'kbps': 4801.2, 'frames': 720.0, 'bytes': 1.8e7, 'uptime': 30.2}
713+
```
714+
715+
**JPEG sequence recorder (no PyAV needed).** TCP-path session
716+
capture: each frame written to disk plus `manifest.json` so it can
717+
be replayed at original cadence:
718+
719+
```python
720+
from je_auto_control.utils.remote_desktop.jpeg_recorder import (
721+
JpegSequenceRecorder,
722+
)
723+
rec = JpegSequenceRecorder("~/recordings/2026-05-23")
724+
rec.start()
725+
viewer = RemoteDesktopViewer(host=..., on_frame=rec.record_frame)
726+
# ... session ...
727+
rec.stop() # writes manifest.json next to the .jpg files
728+
```
729+
730+
**TCP relay (WebRTC fallback).** When P2P fails (strict NAT, mobile
731+
CGNAT, hotel Wi-Fi), both peers connect outbound to a relay and
732+
exchange a shared 32-byte session ID; the relay pipes bytes between
733+
them. Same module ships an `encode_handshake(role, session_id)`
734+
helper for clients:
735+
736+
```python
737+
from je_auto_control.utils.remote_desktop.relay import RelayServer
738+
relay = RelayServer(bind="0.0.0.0", port=9000) # NOSONAR # public relay
739+
relay.start()
740+
```
741+
742+
**Service installer (unattended host).** `python -m
743+
je_auto_control.utils.remote_desktop.host_service ...`
744+
exposes `configure` / `init` / `run` plus per-platform installers:
745+
`install-windows-service` / `uninstall-windows-service` (pywin32),
746+
`generate-launchd` / `uninstall-launchd`, `generate-systemd` /
747+
`uninstall-systemd`.
748+
598749
**Encrypted transports + alternate protocols.** Pass an `ssl_context`
599750
to either `RemoteDesktopHost` or `RemoteDesktopViewer` to wrap every
600751
connection in TLS. For firewall-friendly access, use the in-tree

README/README_zh-CN.md

Lines changed: 124 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -552,13 +552,133 @@ viewer.send_input({"action": "type", "text": "hello"})
552552
viewer.disconnect()
553553
```
554554

555-
GUI:**Remote Desktop** 分页,内含两个子分页。
556-
557-
- **Host**(被远程的本机)— Token 字段附 *生成* 按钮、bind 地址安全提示、启动 / 停止控制、实时刷新的 port + viewer 数量状态栏,以及 4fps 预览面板让被远程的人看到 viewer 看到的画面。
558-
- **Viewer**(控制他机)— 地址 / port / token 表单、*连接* / *断开*、自绘 frame display widget,会把 JPEG 等比缩放绘入。display 上的鼠标 / 滚轮 / 键盘事件会用最新 frame 的尺寸映射回原始远程屏幕的像素坐标,再用 `INPUT` 消息发回。
555+
GUI:**Remote Desktop** 分页默认打开的是 **快速连线**(AnyDesk 风格)— 一边是超大本机 Host ID,另一边一个输入框接受 `host:port``ws://``wss://` 或 9 位数字 Host ID,搭配 *连接**开始被远程* 两个主要按钮。近期连线会跨 session 记住。进阶的逐传输子分页(既有 TCP / WS host + viewer、WebRTC host + viewer 含手动 SDP / 自定义编码器 / TLS pinning)仍只差一个 click。WebRTC 子分页采延迟载入,没装 `[webrtc]` extra 也能正常开启整个分页。
559556

560557
> ⚠️ 取得 host:port 与 token 的人,等同拥有本机完整鼠标 / 键盘控制权。默认仅绑 `127.0.0.1`;要对外暴露请务必搭配 SSH tunnel 或 TLS 前端。Token 是唯一防线 — 请当作密码保管。
561558
559+
**快速连线的 headless API**。撑起 GUI 输入框的 transport coordinator 也对外开放,脚本可以走同样的解析路径:
560+
561+
```python
562+
from je_auto_control import parse_remote_desktop_target
563+
parse_remote_desktop_target("192.168.1.10:5555")
564+
# ConnectTarget(kind='tcp', host='192.168.1.10', port=5555, ...)
565+
parse_remote_desktop_target("ws://hub:8765/desk")
566+
# ConnectTarget(kind='ws', host='hub', port=8765, path='/desk')
567+
parse_remote_desktop_target("123-456-789")
568+
# ConnectTarget(kind='webrtc_id', host_id='123456789')
569+
```
570+
571+
**连接审批 + 仅检视模式**。可选 callback 守住每一个 incoming session,AnyDesk 风格。返回 `"view_only"` admit 但丢掉 viewer 的 `INPUT`;返回 falsy(或 raise)就送 `AUTH_FAIL "rejected by host"`
572+
573+
```python
574+
from je_auto_control import RemoteDesktopHost, PendingViewer
575+
576+
def gate(p: PendingViewer) -> str:
577+
if p.address[0].startswith("10."):
578+
return "view_only"
579+
return "full" # 或 True
580+
581+
host = RemoteDesktopHost(token="tok", on_pending_viewer=gate)
582+
```
583+
584+
**IP 白名单(CIDR + 单一 IP)**。在 TLS / auth 之前就拒绝范围外的对端,攻击者连探测都不行:
585+
586+
```python
587+
host = RemoteDesktopHost(
588+
token="tok", ip_allowlist=["10.0.0.0/8", "192.168.1.100"],
589+
)
590+
```
591+
592+
**一次性分享码** — 额外的 token,认证成功一次后自毁;客服支援流程很好用:
593+
594+
```python
595+
host = RemoteDesktopHost(token="tok", single_use_tokens=["abc123"])
596+
host.add_single_use_token("9k4ndx") # 运行时加
597+
host.revoke_single_use_token("abc123") # 还没被用就先撤销
598+
```
599+
600+
**TOTP 2FA(RFC 6238,纯 stdlib)**。在 token 之上加一层 6 位数字 OTP;host 接受 ±1 时间步的 clock drift:
601+
602+
```python
603+
from je_auto_control.utils.remote_desktop.totp import (
604+
generate_secret, generate_code, provisioning_uri,
605+
)
606+
secret = generate_secret()
607+
print(provisioning_uri(secret, account="alice")) # 给 QR code 用的 otpauth:// URI
608+
609+
host = RemoteDesktopHost(token="tok", totp_secret=secret)
610+
viewer = RemoteDesktopViewer(
611+
host=..., token="tok", totp_code=generate_code(secret),
612+
)
613+
```
614+
615+
**多屏幕选择**。指定某一屏幕截取,而非合并虚拟桌面:
616+
617+
```python
618+
from je_auto_control import list_host_monitors, RemoteDesktopHost
619+
print(list_host_monitors())
620+
# [{'index': 0, 'is_combined': True, ...},
621+
# {'index': 1, ...},
622+
# {'index': 2, ...}]
623+
host = RemoteDesktopHost(token="tok", monitor_index=1)
624+
```
625+
626+
**远程光标 overlay**。host 每秒 30 Hz 广播 cursor 位置(静止桌面去重);viewer 的弹出窗口会在 JPEG 流上叠一个箭头,看得到 host 鼠标位置。可用 `enable_cursor_broadcast=False` 关掉。
627+
628+
**多 viewer 协作光标 + 文字 chat**。两个新 message type(`CHAT``CURSOR``viewer_id`)。搭配 `MultiViewerHost` 把一个 viewer 的指针 echo 给其他人;chat channel 给操作者之间临时对话用:
629+
630+
```python
631+
host = RemoteDesktopHost(
632+
token="tok", on_chat=lambda sender, text: print(sender, ":", text),
633+
)
634+
host.broadcast_chat("session starts in 30s")
635+
host.broadcast_viewer_cursor("alice", 200, 300)
636+
637+
viewer = RemoteDesktopViewer(
638+
host=..., on_chat=lambda s, t: ...,
639+
on_viewer_cursor=lambda vid, x, y: ...,
640+
)
641+
viewer.send_chat("ack")
642+
```
643+
644+
**相对鼠标模式(FPS / CAD)**。新输入 action 送 delta 而非绝对坐标:
645+
646+
```python
647+
viewer.send_input({"action": "mouse_move_relative", "dx": 5, "dy": -3})
648+
```
649+
650+
**动态截取**。capture loop 会 hash 每张编码后的 JPEG;重复 frame 直接跳过,所以静止桌面几乎零带宽。新 viewer 在 auth 后立即拿到最新 frame,不会看到一片黑。
651+
652+
**即时统计**(FPS / kbps / 累计 — 3 秒滑动窗口):
653+
654+
```python
655+
viewer.stats()
656+
# {'fps': 24.3, 'kbps': 4801.2, 'frames': 720.0, 'bytes': 1.8e7, 'uptime': 30.2}
657+
```
658+
659+
**JPEG 序列录影(不需要 PyAV)**。TCP path 的 session 录影:每张 frame 写到磁盘,再加一份 `manifest.json` 让播放器可以原速重放:
660+
661+
```python
662+
from je_auto_control.utils.remote_desktop.jpeg_recorder import (
663+
JpegSequenceRecorder,
664+
)
665+
rec = JpegSequenceRecorder("~/recordings/2026-05-23")
666+
rec.start()
667+
viewer = RemoteDesktopViewer(host=..., on_frame=rec.record_frame)
668+
# ... session ...
669+
rec.stop() # 在 .jpg 旁边写出 manifest.json
670+
```
671+
672+
**TCP relay(WebRTC fallback)**。当 P2P 失败(严格 NAT、移动 CGNAT、酒店 Wi-Fi),两端都向 relay 主动连线、交换一个 32-byte session ID,relay 在中间互转 bytes。同一模块附 `encode_handshake(role, session_id)` 给 client 用:
673+
674+
```python
675+
from je_auto_control.utils.remote_desktop.relay import RelayServer
676+
relay = RelayServer(bind="0.0.0.0", port=9000) # NOSONAR # 对外 relay
677+
relay.start()
678+
```
679+
680+
**服务安装器(无人值守 host)**`python -m je_auto_control.utils.remote_desktop.host_service ...` 提供 `configure` / `init` / `run`,以及每个平台的安装命令:`install-windows-service` / `uninstall-windows-service`(需 pywin32)、`generate-launchd` / `uninstall-launchd``generate-systemd` / `uninstall-systemd`
681+
562682
**加密传输与替代协议**:传 `ssl_context``RemoteDesktopHost``RemoteDesktopViewer` 即套上 TLS。要穿墙/给浏览器接,用内置的 WebSocket 版本(无额外依赖),加 `ssl_context``wss://`
563683

564684
```python

0 commit comments

Comments
 (0)