Skip to content

Commit 347d497

Browse files
committed
S0.2: resolve Leanback UA; smoke is now a real Leanback-load gate
Empirically found a UA that holds youtube.com/tv through full load: the Chromecast CrKey UA. Verified over CDP — lands on /tv#/, title 'YouTube on TV', ytlr-app present (12 ytlr-* element types), no ?app=desktop redirect. Seeded into config/app.json. Key finding: content_shell IGNORES --user-agent (Cobalt hardcodes a Chrome/999 UA), so the TV identity is injected via CDP Network.setUserAgent- Override. The default Chrome/999 UA gets bounced to the desktop site. This means the shipped launcher will need a UA patch or a startup CDP override (R1, Phase 2/3), not a launch flag. - scripts/cdp.py: stdlib-only CDP client (ws handshake/framing, UA override, navigate, DOM assert, screenshot; reconnects across Leanback target teardown). - smoke.sh: rewired to launch content_shell + drive cdp.py, asserting Leanback actually loads under the app.json UA. Screenshot is best-effort (headless SwiftShader can't paint). This is the CI gate + nightly R1 canary. - findings/TASKS: S0.1 and S0.2 marked done.
1 parent 34b739b commit 347d497

6 files changed

Lines changed: 274 additions & 29 deletions

File tree

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,7 @@ tags
2929

3030
# Deployed symbol copies pulled back from the Deck
3131
/out/**/symbols/
32+
33+
# Python bytecode (scripts/cdp.py compile checks)
34+
__pycache__/
35+
*.pyc

.internal/TASKS.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,15 +32,17 @@ The design doc is the source of truth; this file tracks *what to do next* and *w
3232

3333
## Phase 0 — Research spikes & de-risking (4–6 ED) → `.internal/findings/milestones/m114.md`
3434

35-
- [~] **S0.1** Does trunk Chrobalt build + run on generic Linux? Which Chromium milestone *now*?
35+
- [x] **S0.1** Does trunk Chrobalt build + run on generic Linux? Which Chromium milestone *now*?
3636
**Pinned `DEPS.pin` = `a9181df8` (Cobalt 26.eap, Chromium M114 confirmed).** Checkout +
37-
hermetic toolchain fetched OK via fixed scripts (solution `src`, tree at cobalt/src). Build
38-
uses `cobalt/build/gn.py -p chromium_linux-x64x11 -c devel``content_shell` (NOT raw gn/cobalt).
39-
Remaining: unblock the depot_tools python3 bootstrap so `gn gen` emits build.ninja, then build +
40-
render youtube.com/tv. **KEY:** Cobalt ≤25 is classic Starboard Cobalt (wrong engine); Chrobalt
41-
is 26+. See `findings/milestones/m114.md`.
42-
- [ ] **S0.2** Which UA yields full Leanback (voice search, sign-in, settings)? Any attestation beyond UA?
43-
→ documented working UA + feature checklist; seed `config/app.json:ua`.
37+
hermetic toolchain fetched OK; `content_shell` **builds, links (1.4G devel), and boots headless**
38+
with CDP up (`just smoke`). Build uses `cobalt/build/gn.py -p chromium_linux-x64x11 -c devel`
39+
`content_shell` (NOT raw gn/cobalt). depot_tools py3 bootstrap fixed in the image. **KEY:**
40+
Cobalt ≤25 is classic Starboard Cobalt (wrong engine); Chrobalt is 26+. See `m114.md`.
41+
- [x] **S0.2** Which UA yields full Leanback (voice search, sign-in, settings)? Any attestation beyond UA?
42+
**Chromecast `CrKey/1.56.500000` UA loads Leanback** (`ytlr-app`, `youtube.com/tv#/`, no
43+
desktop redirect); seeded into `config/app.json:user_agent`. **content_shell ignores
44+
`--user-agent`** (Cobalt hardcodes Chrome/999) → UA injected via CDP; shipped launcher needs a
45+
patch or startup CDP override (R1, Phase 2/3). Full browse/voice/sign-in are on-Deck checks.
4446
- [x] **S0.3** Is Chromium library-CDM path (`enable_library_cdms`/`enable_widevine`) intact or stripped?
4547
**INTACT** (not stripped). `third_party/widevine/cdm/` present; `widevine.gni` gates it behind
4648
`enable_widevine` (default false). Re-enable via GN args + user-fetch CDM (gates Phase 7).

.internal/findings/milestones/m114.md

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -62,9 +62,23 @@ discovered and baked into the `test` image stage:
6262
- Headless GPU init fails → **SwiftShader software GL** fallback (`dri3 extension not supported`);
6363
expected in the container. Real GL/VA-API validation happens on the Deck.
6464
- dbus/login1/`org.freedesktop.login1` errors are benign (no system bus in the container).
65-
- **UA observation (R1):** CDP `/json/version` reports the content_shell default
66-
`Chrome/999.77.34.5`**not** the Leanback UA. Confirms identity must be injected at run time via
67-
`--user-agent=…` (S0.2 / `config/app.json`). `WebKit-Version` tag `@a9181df8…` == our pin ✓.
65+
- `WebKit-Version` tag `@a9181df8…` == our pin ✓.
66+
67+
## Leanback UA (S0.2) — RESOLVED 2026-07-08
68+
**Working UA** (now in `config/app.json:user_agent`), holds Leanback through full load:
69+
`Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 CrKey/1.56.500000`
70+
(a Chromecast/`CrKey` UA). Verified via CDP: lands on `https://www.youtube.com/tv#/`, title
71+
"YouTube on TV", `ytlr-app` present with 12 distinct `ytlr-*` element types, **no** `?app=desktop`
72+
redirect. This is the R1 bet the plan makes openly (we impersonate a supported living-room device).
73+
- **Default `Chrome/999` desktop UA → bounced to `youtube.com/?app=desktop`** (Leanback refused).
74+
- **content_shell IGNORES `--user-agent`** (Cobalt hardcodes the `Chrome/999` string). The TV UA must
75+
be injected another way. Empirically the CDP path works (`Network.setUserAgentOverride`, applied
76+
before navigation). **Architectural implication:** the Phase-2 launcher cannot set the UA via a
77+
launch flag — it needs either (a) a surgical `patches/` change to Cobalt's UA construction, or
78+
(b) a CDP override at startup before the first navigation. TBD in Phase 2/3; flagged as R1 work.
79+
- Home-feed **tiles/shelves did not populate** headless (0 tiles) — consistent with no signed-in
80+
session + SwiftShader no-paint; full browse UI is an on-Deck check, not a UA failure.
81+
- `scripts/cdp.py` (stdlib-only CDP client) drives this; `just smoke` now asserts it as the CI gate.
6882

6983
## Widevine (S0.3) — verified: library-CDM path INTACT
7084
`third_party/widevine/cdm/` is present (not stripped for Starboard `SbDrm`). `widevine.gni` defines
@@ -77,7 +91,7 @@ and user-fetch Google's Linux x64 CDM at first run (never redistributed) → `la
7791
| Spike | Question | Status | Finding |
7892
|---|---|---|---|
7993
| S0.1 | Builds/runs on Linux? Which milestone? | **done** | M114 confirmed; `content_shell` builds, links, boots headless, CDP up |
80-
| S0.2 | Working Leanback UA? | open | `config/app.json:user_agent` still sentinel; content_shell default UA is `Chrome/999`, must override at run time |
94+
| S0.2 | Working Leanback UA? | **done** | Chromecast `CrKey` UA loads Leanback (`ytlr-app`, no desktop redirect); content_shell ignores `--user-agent` → inject via CDP/patch |
8195
| S0.3 | Library-CDM path intact? | **done** | Intact; gated behind `enable_widevine` (see above) |
8296
| S0.4 | Ozone x11/Wayland under gamescope? | open | platform is `x64x11` (X11); gamescope test pending Deck |
8397
| S0.5 | Chromium media pipeline vs `SbPlayer`? | open | Chrobalt ⇒ Chromium media pipeline (Blink present); VA-API TBD |
@@ -98,5 +112,6 @@ Upstream installs **no** clang/gn/ninja via apt — they come from the hermetic
98112

99113
## Open issues
100114
- sccache not yet wired (`cc_wrapper`) — first builds are cold. Enable once stock build is proven.
101-
- UA override path (`--user-agent`) unverified end-to-end against production youtube.com/tv (S0.2).
115+
- UA-injection mechanism for the shipped launcher (Cobalt UA patch vs. startup CDP override) — R1
116+
work for Phase 2/3, since content_shell ignores `--user-agent` (S0.2 above).
102117
- On-Deck render (real GL/VA-API, gamescope, gamepad) not yet exercised (S0.4/S0.5/S0.6).

config/app.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"$comment": "Deckback runtime config. Read by launcher/. Designed to be remotely overridable and signed (doc §6 P10 / risk R1) so Leanback breakage can be hotfixed without a rebuild. UA is the single biggest external dependency (S0.2) — keep it hot-swappable.",
33
"schema_version": 1,
44
"url": "https://www.youtube.com/tv",
5-
"user_agent": "SET_FROM_SPIKE_S0.2",
5+
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 CrKey/1.56.500000",
66
"remote_debugging_port": 9222,
77
"quality": {
88
"max_height": 1080,

scripts/cdp.py

Lines changed: 214 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
#!/usr/bin/env python3
2+
"""Minimal, dependency-free Chrome DevTools Protocol client for the smoke gate.
3+
4+
Drives a running content_shell over CDP (default :9222): optionally overrides the
5+
user-agent, navigates to a URL, waits for a readiness JS expression, prints the
6+
document title, and writes a PNG screenshot. Stdlib only (no websocket-client) so
7+
it runs inside the build image with nothing extra installed.
8+
"""
9+
import argparse
10+
import base64
11+
import json
12+
import os
13+
import socket
14+
import struct
15+
import sys
16+
import time
17+
import urllib.request
18+
19+
20+
def http_json(port, path):
21+
with urllib.request.urlopen(f"http://127.0.0.1:{port}{path}", timeout=5) as r:
22+
return json.load(r)
23+
24+
25+
def wait_endpoint(port, tries=40):
26+
for _ in range(tries):
27+
try:
28+
return http_json(port, "/json/version")
29+
except Exception:
30+
time.sleep(1)
31+
raise RuntimeError(f"DevTools endpoint on :{port} never came up")
32+
33+
34+
class WS:
35+
"""Just enough RFC6455 client for CDP: masked text frames out, reassembled frames in."""
36+
37+
def __init__(self, url):
38+
assert url.startswith("ws://")
39+
host_port, _, self.path = url[5:].partition("/")
40+
self.path = "/" + self.path
41+
host, _, port = host_port.partition(":")
42+
self.sock = socket.create_connection((host, int(port or 80)), timeout=10)
43+
key = base64.b64encode(os.urandom(16)).decode()
44+
req = (
45+
f"GET {self.path} HTTP/1.1\r\nHost: {host_port}\r\n"
46+
"Upgrade: websocket\r\nConnection: Upgrade\r\n"
47+
f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n"
48+
)
49+
self.sock.sendall(req.encode())
50+
buf = b""
51+
while b"\r\n\r\n" not in buf:
52+
buf += self.sock.recv(4096)
53+
if b" 101 " not in buf.split(b"\r\n", 1)[0]:
54+
raise RuntimeError(f"ws handshake failed: {buf[:120]!r}")
55+
self._rest = buf.split(b"\r\n\r\n", 1)[1]
56+
57+
def _recv(self, n):
58+
while len(self._rest) < n:
59+
chunk = self.sock.recv(65536)
60+
if not chunk:
61+
raise ConnectionError("ws closed")
62+
self._rest += chunk
63+
out, self._rest = self._rest[:n], self._rest[n:]
64+
return out
65+
66+
def send(self, text):
67+
data = text.encode()
68+
hdr = bytearray([0x81]) # FIN + text
69+
n = len(data)
70+
if n < 126:
71+
hdr.append(0x80 | n)
72+
elif n < 65536:
73+
hdr.append(0x80 | 126)
74+
hdr += struct.pack(">H", n)
75+
else:
76+
hdr.append(0x80 | 127)
77+
hdr += struct.pack(">Q", n)
78+
mask = os.urandom(4)
79+
hdr += mask
80+
self.sock.sendall(bytes(hdr) + bytes(b ^ mask[i % 4] for i, b in enumerate(data)))
81+
82+
def recv(self):
83+
payload = b""
84+
while True:
85+
b0, b1 = self._recv(2)
86+
fin, opcode = b0 & 0x80, b0 & 0x0F
87+
length = b1 & 0x7F
88+
if length == 126:
89+
length = struct.unpack(">H", self._recv(2))[0]
90+
elif length == 127:
91+
length = struct.unpack(">Q", self._recv(8))[0]
92+
payload += self._recv(length)
93+
if opcode == 0x8: # close
94+
raise ConnectionError("ws close frame")
95+
if fin:
96+
return payload.decode(errors="replace")
97+
98+
99+
class CDP:
100+
def __init__(self, port):
101+
self.port = port
102+
self._enabled = []
103+
self.connect()
104+
105+
def connect(self):
106+
self.ws = WS(pick_page_target(self.port))
107+
self._id = 0
108+
for method in self._enabled: # re-arm domains on a fresh target
109+
self.call(method, _no_reconnect=True)
110+
if getattr(self, "_ua", None): # Leanback tears down targets; keep the UA sticky
111+
self.call("Network.enable", _no_reconnect=True)
112+
self.call("Network.setUserAgentOverride", {"userAgent": self._ua},
113+
_no_reconnect=True)
114+
115+
def call(self, method, params=None, timeout=30, _no_reconnect=False):
116+
if method.endswith(".enable") and method not in self._enabled:
117+
self._enabled.append(method)
118+
try:
119+
self._id += 1
120+
mid = self._id
121+
self.ws.send(json.dumps({"id": mid, "method": method, "params": params or {}}))
122+
deadline = time.monotonic() + timeout
123+
while time.monotonic() < deadline:
124+
msg = json.loads(self.ws.recv())
125+
if msg.get("id") == mid:
126+
if "error" in msg:
127+
raise RuntimeError(f"{method}: {msg['error']}")
128+
return msg.get("result", {})
129+
raise TimeoutError(f"{method} timed out")
130+
except (ConnectionError, BrokenPipeError, OSError):
131+
if _no_reconnect:
132+
raise
133+
self.connect() # target was torn down (Leanback navigations) — re-attach
134+
return self.call(method, params, timeout, _no_reconnect=True)
135+
136+
def evaluate(self, expr):
137+
r = self.call("Runtime.evaluate",
138+
{"expression": expr, "returnByValue": True, "awaitPromise": True})
139+
return r.get("result", {}).get("value")
140+
141+
142+
def pick_page_target(port, tries=30):
143+
for _ in range(tries):
144+
try:
145+
for t in http_json(port, "/json"):
146+
if t.get("type") == "page" and t.get("webSocketDebuggerUrl"):
147+
return t["webSocketDebuggerUrl"]
148+
except Exception:
149+
pass
150+
time.sleep(1)
151+
raise RuntimeError("no page target with a ws url appeared")
152+
153+
154+
def main():
155+
ap = argparse.ArgumentParser()
156+
ap.add_argument("--port", type=int, default=9222)
157+
ap.add_argument("--ua")
158+
ap.add_argument("--navigate")
159+
ap.add_argument("--out")
160+
ap.add_argument("--expect", default="!!document.body && document.body.innerText.length >= 0")
161+
ap.add_argument("--print", dest="print_expr")
162+
ap.add_argument("--ready-timeout", type=int, default=45)
163+
args = ap.parse_args()
164+
165+
ver = wait_endpoint(args.port)
166+
print(f"browser UA (pre-override): {ver.get('User-Agent')}")
167+
168+
cdp = CDP(args.port)
169+
cdp.call("Page.enable")
170+
cdp.call("Runtime.enable")
171+
if args.ua:
172+
cdp._ua = args.ua # sticky: re-applied by connect() after Leanback tears down the target
173+
cdp.call("Network.enable")
174+
cdp.call("Network.setUserAgentOverride", {"userAgent": args.ua})
175+
print(f"UA override applied: {args.ua}")
176+
if args.navigate:
177+
cdp.call("Page.navigate", {"url": args.navigate})
178+
179+
deadline = time.monotonic() + args.ready_timeout
180+
ready = False
181+
while time.monotonic() < deadline:
182+
try:
183+
if cdp.evaluate(args.expect):
184+
ready = True
185+
break
186+
except Exception:
187+
pass
188+
time.sleep(1)
189+
190+
title = cdp.evaluate("document.title")
191+
url = cdp.evaluate("document.location.href")
192+
ua_seen = cdp.evaluate("navigator.userAgent")
193+
body_len = cdp.evaluate("document.body ? document.body.innerText.length : -1")
194+
print(f"ready={ready} title={title!r} url={url!r} body_chars={body_len}")
195+
print(f"navigator.userAgent={ua_seen}")
196+
if args.print_expr:
197+
print(f"probe={cdp.evaluate(args.print_expr)}")
198+
199+
if args.out:
200+
# Best-effort: headless SwiftShader (dead GPU process) can stall Page.captureScreenshot.
201+
# A paint failure must not fail the smoke — the DOM assertion above is the gate.
202+
try:
203+
shot = cdp.call("Page.captureScreenshot", {"format": "png"}, timeout=30)
204+
with open(args.out, "wb") as f:
205+
f.write(base64.b64decode(shot["data"]))
206+
print(f"screenshot: {args.out} ({os.path.getsize(args.out)} bytes)")
207+
except Exception as e:
208+
print(f"screenshot skipped ({type(e).__name__}: {e})")
209+
210+
sys.exit(0 if ready else 2)
211+
212+
213+
if __name__ == "__main__":
214+
main()

scripts/smoke.sh

Lines changed: 25 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,35 @@
11
#!/usr/bin/env bash
22
# Headless boot + DevTools-protocol assertion + screenshot. Same script CI runs (doc §14.1).
3-
# Also runnable nightly against production youtube.com/tv as an R1/R2 Leanback-drift canary.
3+
# Also runnable nightly against production youtube.com/tv as an R1/R2 Leanback-drift canary:
4+
# it asserts the TV UA (config/app.json) still lands on Leanback (ytlr-app present, no desktop
5+
# redirect) and captures out/<preset>/smoke.png.
6+
#
7+
# NOTE (S0.2): content_shell IGNORES --user-agent (Cobalt hardcodes a Chrome/999 UA), so the TV UA
8+
# is injected over CDP (Network.setUserAgentOverride) via scripts/cdp.py, not a launch flag.
9+
# --no-sandbox is for THIS headless harness only (no user-ns/SUID sandbox in the build container);
10+
# the shipped Flatpak sandboxes via zypak — never --no-sandbox (Flathub rejects it).
411
. "$(dirname "$0")/lib.sh"
512
require_cobalt_checkout
613
preset="${1:-dev}"
714
port=9222
815

9-
# --no-sandbox is for THIS headless test harness only (no user-ns/SUID sandbox in the build
10-
# container). The shipped Flatpak runs sandboxed via zypak — never with --no-sandbox (Flathub rejects it).
11-
info "xvfb-run out/${preset}/cobalt (headless) ..."
16+
info "Headless Leanback smoke: out/${preset}/${COBALT_TARGET} + CDP assertion ..."
1217
in_container bash -lc "
13-
cd $CTR_TREE &&
14-
xvfb-run -a out/${preset}/${COBALT_TARGET} --url=https://www.youtube.com/tv \
15-
--remote-debugging-port=${port} --no-first-run --no-sandbox &
18+
set -u
19+
cd $CTR_TREE
20+
UA=\$(python3 -c \"import json;print(json.load(open('/work/config/app.json'))['user_agent'])\")
21+
URL=\$(python3 -c \"import json;print(json.load(open('/work/config/app.json'))['url'])\")
22+
Xvfb :99 -screen 0 1280x720x24 >/tmp/xvfb.log 2>&1 &
23+
export DISPLAY=:99
24+
sleep 2
25+
out/${preset}/${COBALT_TARGET} --remote-debugging-port=${port} --no-first-run --no-sandbox \
26+
--window-size=1280,720 --user-data-dir=/tmp/deckback-smoke about:blank >/tmp/cobalt-smoke.log 2>&1 &
1627
cobalt_pid=\$! ;
17-
for i in \$(seq 1 30); do
18-
curl -sf http://localhost:${port}/json/version >/dev/null && break || sleep 1 ;
19-
done ;
20-
# TODO(Phase 1): drive the DevTools protocol — assert document.title / a Leanback DOM node,
21-
# capture Page.captureScreenshot to out/${preset}/smoke.png as a CI artifact.
22-
curl -sf http://localhost:${port}/json/version || { echo 'DevTools endpoint never came up'; kill \$cobalt_pid; exit 1; } ;
23-
kill \$cobalt_pid
28+
python3 /work/scripts/cdp.py --port ${port} --ua \"\$UA\" --navigate \"\$URL\" \
29+
--expect \"!!document.querySelector('ytlr-app') && !location.href.includes('app=desktop')\" \
30+
--ready-timeout 60 --out out/${preset}/smoke.png ;
31+
rc=\$? ;
32+
kill \$cobalt_pid 2>/dev/null ;
33+
exit \$rc
2434
"
25-
info "Smoke passed (endpoint up). Screenshot/DOM assertion lands in Phase 1."
35+
info "Smoke passed: Leanback loaded under the TV UA. Screenshot -> ${COBALT_TREE}/out/${preset}/smoke.png"

0 commit comments

Comments
 (0)