test: real-hardware conformance corpus + opt-in hardware lane

tests/conformance/README.md

@@ -0,0 +1,28 @@
+# Conformance corpus
+
+Recorded host↔device sessions captured from **real Marlin hardware**, replayed
+hardware-free by `tests/test_conformance.py` through `FakeTransport.from_trace`.
+Each line is `> tx` (host→device) or `< rx` (device→host) — the format
+`TracingTransport` emits, so a captured session drops straight in here.
+
+These pin the modeled behavior against genuine wire output: a regression in
+parsing, M115 negotiation, or streaming fails the replay with no board attached.
+
+## Fixtures
+
+| File | Source | Exercises |
+|------|--------|-----------|
+| `connect-ramps-2.1.x.trace` | RAMPS 1.4, Marlin `bugfix-2.1.x` (Jan 2026), 250000 baud, no 12V | connect: `start` banner → M115 negotiation → real capability set (`EMERGENCY_PARSER:0`, `EEPROM:0`, `ARCS:1`, …) |
+| `stream-ramps-2.1.x.trace` | same board | connect + a framed stream of queries / state-sets / tiny relative moves, each `ok` |
+
+## Capturing more
+
+Run the hardware lane against a board (motor/heater power optional):
+
+```sh
+uv run pytest tests/hardware --port /dev/ttyACM0
+```
+
+To record a fresh trace, wrap the transport in `TracingTransport` with a file
+sink (see `working/bringup.py`) and curate the relevant slice into a `.trace`
+file here, then add assertions in `test_conformance.py`.

tests/conformance/connect-ramps-2.1.x.trace

@@ -0,0 +1,71 @@
+< start
+< Marlin bugfix-2.1.x
+< echo: Last Updated: 2026-01-21 | Author: (MarlinFirmware)
+< echo: Compiled: Jan 21 2026
+< echo: Free Memory: 3590  PlannerBufferBytes: 1328
+< echo:Hardcoded Default Settings Loaded
+< echo:; Linear Units:
+< echo:  G21 ; (mm)
+< echo:; Auto Temp Control:
+< echo:  M104 S210 B250 F0.10
+< echo:; Temperature Units:
+< echo:  M149 C ; Units in Celsius
+< echo:; Filament settings (Disabled):
+< echo:  M200 S0 D1.75
+< echo:; Steps per unit:
+< echo:  M92 X80.00 Y80.00 Z400.00 E500.00
+< echo:; Max feedrates (units/s):
+< echo:  M203 X300.00 Y300.00 Z5.00 E25.00
+< echo:; Max Acceleration (units/s2):
+< echo:  M201 X3000.00 Y3000.00 Z100.00 E10000.00
+< echo:; Acceleration (units/s2) (P<print-accel> R<retract-accel> T<travel-accel>):
+< echo:  M204 P3000.00 R3000.00 T3000.00
+< echo:; Advanced (B<min_segment_time_us> S<min_feedrate> T<min_travel_feedrate> J<junc_dev>):
+< echo:  M205 B20000.00 S0.00 T0.00 J0.01
+< echo:; Home offset:
+< echo:  M206 X0.00 Y0.00 Z0.00
+< echo:; Material heatup parameters:
+< echo:  M145 S0 H180.00 F0
+< echo:  M145 S1 H240.00 F0
+< echo:; Hotend PID:
+< echo:  M301 P22.20 I1.08 D114.00
+> M115
+< FIRMWARE_NAME:Marlin bugfix-2.1.x (Jan 21 2026 20:48:29) SOURCE_CODE_URL:github.com/MarlinFirmware/Marlin PROTOCOL_VERSION:1.0 MACHINE_TYPE:3D Printer KINEMATICS:Cartesian EXTRUDER_COUNT:1 UUID:cede2a2f-41a2-4748-9b12-c55c62f367ff
+< Cap:SERIAL_XON_XOFF:0
+< Cap:BINARY_FILE_TRANSFER:0
+< Cap:EEPROM:0
+< Cap:VOLUMETRIC:1
+< Cap:AUTOREPORT_POS:0
+< Cap:AUTOREPORT_TEMP:1
+< Cap:PROGRESS:0
+< Cap:PRINT_JOB:1
+< Cap:AUTOLEVEL:0
+< Cap:RUNOUT:0
+< Cap:Z_PROBE:0
+< Cap:LEVELING_DATA:0
+< Cap:BUILD_PERCENT:0
+< Cap:SOFTWARE_POWER:0
+< Cap:TOGGLE_LIGHTS:0
+< Cap:CASE_LIGHT_BRIGHTNESS:0
+< Cap:EMERGENCY_PARSER:0
+< Cap:HOST_ACTION_COMMANDS:0
+< Cap:PROMPT_SUPPORT:0
+< Cap:SDCARD:1
+< Cap:MULTI_VOLUME:0
+< Cap:REPEAT:0
+< Cap:SD_WRITE:1
+< Cap:AUTOREPORT_SD_STATUS:0
+< Cap:LONG_FILENAME:0
+< Cap:LFN_WRITE:0
+< Cap:CUSTOM_FIRMWARE_UPLOAD:0
+< Cap:EXTENDED_M20:0
+< Cap:THERMAL_PROTECTION:1
+< Cap:MOTION_MODES:0
+< Cap:ARCS:1
+< Cap:BABYSTEPPING:0
+< Cap:EP_BABYSTEP:0
+< Cap:CHAMBER_TEMPERATURE:0
+< Cap:COOLER_TEMPERATURE:0
+< Cap:MEATPACK:0
+< Cap:CONFIG_EXPORT:0
+< ok

tests/conformance/stream-ramps-2.1.x.trace

@@ -0,0 +1,65 @@
+< start
+> M115
+< FIRMWARE_NAME:Marlin bugfix-2.1.x (Jan 21 2026 20:48:29) SOURCE_CODE_URL:github.com/MarlinFirmware/Marlin PROTOCOL_VERSION:1.0 MACHINE_TYPE:3D Printer KINEMATICS:Cartesian EXTRUDER_COUNT:1 UUID:cede2a2f-41a2-4748-9b12-c55c62f367ff
+< Cap:SERIAL_XON_XOFF:0
+< Cap:BINARY_FILE_TRANSFER:0
+< Cap:EEPROM:0
+< Cap:VOLUMETRIC:1
+< Cap:AUTOREPORT_POS:0
+< Cap:AUTOREPORT_TEMP:1
+< Cap:PROGRESS:0
+< Cap:PRINT_JOB:1
+< Cap:AUTOLEVEL:0
+< Cap:RUNOUT:0
+< Cap:Z_PROBE:0
+< Cap:LEVELING_DATA:0
+< Cap:BUILD_PERCENT:0
+< Cap:SOFTWARE_POWER:0
+< Cap:TOGGLE_LIGHTS:0
+< Cap:CASE_LIGHT_BRIGHTNESS:0
+< Cap:EMERGENCY_PARSER:0
+< Cap:HOST_ACTION_COMMANDS:0
+< Cap:PROMPT_SUPPORT:0
+< Cap:SDCARD:1
+< Cap:MULTI_VOLUME:0
+< Cap:REPEAT:0
+< Cap:SD_WRITE:1
+< Cap:AUTOREPORT_SD_STATUS:0
+< Cap:LONG_FILENAME:0
+< Cap:LFN_WRITE:0
+< Cap:CUSTOM_FIRMWARE_UPLOAD:0
+< Cap:EXTENDED_M20:0
+< Cap:THERMAL_PROTECTION:1
+< Cap:MOTION_MODES:0
+< Cap:ARCS:1
+< Cap:BABYSTEPPING:0
+< Cap:EP_BABYSTEP:0
+< Cap:CHAMBER_TEMPERATURE:0
+< Cap:COOLER_TEMPERATURE:0
+< Cap:MEATPACK:0
+< Cap:CONFIG_EXPORT:0
+< ok
+> N4 G90*20
+< ok
+> N5 M82*28
+< ok
+> N6 G21*28
+< ok
+> N7 M114*32
+< X:0.00 Y:0.00 Z:0.00 E:0.00 Count X:0 Y:0 Z:0
+< ok
+> N8 G4 P20*87
+< ok
+> N9 G91*24
+< ok
+> N10 G1 X0.2 F300*24
+< ok
+> N11 G1 X-0.2 F300*52
+< ok
+> N12 G90*35
+< ok
+> N13 M400*21
+< ok
+> N14 M114*18
+< X:0.00 Y:0.00 Z:0.00 E:0.00 Count X:0 Y:0 Z:0
+< ok

tests/conftest.py

@@ -0,0 +1,55 @@
+"""Shared pytest config: the opt-in hardware-test gate.
+
+The default suite is 100% hardware-free (FakeTransport / recorded-trace replay).
+Tests marked ``@pytest.mark.hardware`` talk to a real Marlin controller and are
+**skipped unless** a port is supplied:
+
+    uv run pytest --port /dev/ttyACM0          # run everything incl. hardware
+    uv run pytest --port /dev/ttyACM0 --baud 250000
+
+With no ``--port`` (the CI default) the hardware tests are collected but skipped,
+so they never gate a hardware-free run.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+
+def pytest_addoption(parser: pytest.Parser) -> None:
+    parser.addoption(
+        "--port",
+        action="store",
+        default=None,
+        help="Serial port of a real Marlin controller; enables @pytest.mark.hardware tests",
+    )
+    parser.addoption(
+        "--baud", action="store", default="250000", help="Baud rate for hardware tests"
+    )
+
+
+def pytest_configure(config: pytest.Config) -> None:
+    config.addinivalue_line(
+        "markers", "hardware: requires a real Marlin controller; runs only with --port"
+    )
+
+
+def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
+    if config.getoption("--port"):
+        return  # a port was supplied — let the hardware tests run
+    skip = pytest.mark.skip(reason="hardware test — pass --port /dev/ttyACMx to run")
+    for item in items:
+        if "hardware" in item.keywords:
+            item.add_marker(skip)
+
+
+@pytest.fixture
+def port(request: pytest.FixtureRequest) -> str:
+    value = request.config.getoption("--port")
+    assert value is not None, "hardware tests require --port"  # gated by collection skip
+    return str(value)
+
+
+@pytest.fixture
+def baud(request: pytest.FixtureRequest) -> int:
+    return int(request.config.getoption("--baud") or 250000)

tests/hardware/test_hardware.py

@@ -0,0 +1,69 @@
+"""Hardware lane: live validation against a real Marlin controller.
+
+Skipped unless ``--port`` is given (see ``tests/conftest.py``). Safe to run with
+motor/heater power OFF — these exercise the protocol/streaming/reliability path
+at logic level (queries, framed sends, tiny relative moves, an M112 + recovery);
+nothing depends on actual motion or heating.
+
+    uv run pytest tests/hardware --port /dev/ttyACM0
+
+Re-validates what the conformance corpus replays, and is how new traces are
+captured (wrap the transport in TracingTransport with a file sink — see
+working/bringup.py).
+"""
+
+from __future__ import annotations
+
+from collections.abc import Iterator
+from types import SimpleNamespace
+
+import pytest
+
+from marlin_host import HaltError, MarlinHost, SerialTransport
+
+pytestmark = pytest.mark.hardware
+
+
+@pytest.fixture
+def rig(port: str, baud: int) -> Iterator[SimpleNamespace]:
+    transport = SerialTransport(port, baud)  # reset_on_open -> DTR pulse resets the board
+    host = MarlinHost(transport, reliable=True, startup_timeout=4.0)
+    host.connect()
+    try:
+        yield SimpleNamespace(host=host, transport=transport)
+    finally:
+        # Leave the board in a clean, non-halted state for the next test.
+        if host.is_halted:
+            transport.reset(flush=True)
+        host.close()
+
+
+def test_connect_and_negotiate(rig: SimpleNamespace) -> None:
+    assert rig.host.is_connected
+    assert rig.host.profile is not None
+    assert "Marlin" in rig.host.profile.firmware
+
+
+def test_queries(rig: SimpleNamespace) -> None:
+    assert rig.host.query("M114")  # at least the position report line
+    assert rig.host.send("M105").is_ack  # temperature rides on the `ok` line
+
+
+def test_reliable_stream(rig: SimpleNamespace) -> None:
+    program = ["G91", "G1 X0.2 F300", "G1 X-0.2 F300", "G90", "M400"]
+    progress = list(rig.host.stream(program))
+    assert len(progress) == len(program)
+    assert all(p.response.is_ack for p in progress)
+
+
+def test_emergency_stop_then_recover(rig: SimpleNamespace) -> None:
+    rig.host.emergency_stop()
+    assert rig.host.is_halted
+    with pytest.raises(HaltError):
+        rig.host.send("M114")
+
+    # Recovery path: flush the stale halt output, reset, reconnect (see #21).
+    rig.transport.reset(flush=True)
+    rig.host.connect()
+    assert not rig.host.is_halted
+    assert rig.host.profile is not None

tests/test_conformance.py

@@ -0,0 +1,64 @@
+"""Conformance corpus: replay real recorded sessions, hardware-free.
+
+The ``.trace`` files in ``conformance/`` are captured from real Marlin hardware
+(see that directory's README). Replaying them through ``FakeTransport.from_trace``
+pins the host's modeled behavior against genuine wire output — so a regression in
+parsing/negotiation/streaming is caught without a board attached. This is the
+"replay" half of the capture→replay loop; the live capture lives in the
+``@pytest.mark.hardware`` lane.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from marlin_host import FakeTransport, MarlinHost
+
+CORPUS = Path(__file__).parent / "conformance"
+
+
+def _load(name: str) -> list[str]:
+    return (CORPUS / name).read_text().splitlines()
+
+
+def test_connect_negotiates_the_real_ramps_profile() -> None:
+    host = MarlinHost(FakeTransport.from_trace(_load("connect-ramps-2.1.x.trace")))
+    host.connect()
+
+    profile = host.profile
+    assert profile is not None
+    assert "Marlin bugfix-2.1.x" in profile.firmware
+    # The genuine dialect of this RAMPS 1.4 build:
+    assert profile.has("ARCS")
+    assert profile.has("THERMAL_PROTECTION")
+    assert not profile.has("EMERGENCY_PARSER")  # M112 is parsed in order, not immediate
+    assert not profile.has("EEPROM")  # hardcoded defaults
+    assert profile.advanced_ok is False  # bare `ok`, no `P..B..`
+    assert profile.emergency_stop_immediate is False
+
+
+def test_stream_replays_the_real_session() -> None:
+    host = MarlinHost(FakeTransport.from_trace(_load("stream-ramps-2.1.x.trace")), reliable=True)
+    host.connect()
+
+    # Command text is cosmetic on replay (responses are positional), but the count
+    # and shape must match the recorded groups: 11 commands, two of them M114.
+    program = [
+        "G90",
+        "M82",
+        "G21",
+        "M114",
+        "G4 P20",
+        "G91",
+        "G1 X0.2 F300",
+        "G1 X-0.2 F300",
+        "G90",
+        "M400",
+        "M114",
+    ]
+    progress = list(host.stream(program))
+
+    assert len(progress) == len(program)
+    assert all(p.response.is_ack for p in progress)
+    assert progress[-1].commands_sent == len(program)
+    assert progress[-1].total_commands == len(program)