feat(compat): add Runner.SkipDataPath for credentialless-CI exporters

The three §4 data-path subtests (JSONIsArray, CSVHasHeader,
DefaultIsMarkdown) invoke the binary with a real --format value and
expect a clean exit. For exporters whose data path requires
credentials that the CI environment does not have — crono's env-var
auth, liftoff's stored OAuth token at
~/.config/liftoff-export/auth.json, withings' OAuth refresh token —
a vanilla `--format json` invocation under compat's default empty
env exits non-zero with "not logged in" before the JSON-array check
fires. Today the only escape route is SupportedFormats: []string{},
which semantically claims "we implement no codecs" — misleading,
since liftoff/crono do implement markdown+json, just not from a
credentialless CI.

SkipDataPath is the explicit escape hatch the QUA-15 PR review
specifically proposed: when true, the three data-path subtests skip
with reason `data-path subtests disabled via Runner.SkipDataPath`,
while the parse-level subtests (HelpDocumentsFormatFlag,
UnknownFormatFails, FlagValidationIsHermetic) still run because
they only exercise flag parsing and expect non-zero exit on the bad
value. Composable with SupportedFormats — nil SupportedFormats +
SkipDataPath: true is the typical liftoff/crono shape today.

Flipping the bool back to false later — once auth is mockable in CI
or secrets are provisioned — promotes the codec rows from human-
attested to machine-attested per the CONTRACT.md Status table flip
plan, without losing the parse-level attestation we get for free
today.

Self-test (TestRunContract_SkipDataPath_GatesDataSubtests) is
adversarial: builds the stub with STUBCLI_FORMATS=__never__ so
every --format value is rejected, then runs the full bundle with
SkipDataPath: true. Parse-level subtests pass against the rejection
side; data-path subtests skip via the guard. Delete the guard in
jsonIsArray / csvHasHeader / defaultIsMarkdown and the test fails
because the data-path subtests run against a stub that rejects every
codec.

Unblocks QUA-16 (liftoff onboarding as second consumer of
compat/formats) without forcing the misleading
SupportedFormats: []string{} workaround.

Co-Authored-By: Paperclip <[email protected]>

Author: LeadGoEngineer

compat/README.md

@@ -102,7 +102,7 @@ When `compat.Runner.Subcommands` is set, every row above runs once per declared
 | `CSVHasHeader` | §4 | `--format csv` exits 0 and emits at least one non-empty line on stdout (the header row, present even on an empty result). |
 | `DefaultIsMarkdown` | §4 | No `--format` flag produces byte-identical stdout to `--format markdown`. |
 
-`JSONIsArray`, `CSVHasHeader`, and `DefaultIsMarkdown` invoke the data path with no extra args beyond `--format`. Integrators whose data path needs credentials or other env to succeed pass them via `compat.Runner.Env`. As with `dates`, `subcommand=NAME/...` subtest groups fire when `Subcommands` is set.
+`JSONIsArray`, `CSVHasHeader`, and `DefaultIsMarkdown` invoke the data path with no extra args beyond `--format`. Integrators whose data path needs credentials or other env to succeed pass them via `compat.Runner.Env`; integrators whose data path is not runnable in CI at all today set `compat.Runner.SkipDataPath: true` (see below). As with `dates`, `subcommand=NAME/...` subtest groups fire when `Subcommands` is set.
 
 ### Partial-codec exporters: `Runner.SupportedFormats`
 
@@ -123,6 +123,21 @@ Semantics:
 
 **Per-CLI flip plan.** When a consumer wires the bundle into its CI with `SupportedFormats` matching its actual surface, the CONTRACT.md Status table's `--format` row for that CLI flips to **machine** for the codecs it declares. Codecs the CLI does not implement remain human-attested per-cell until the writer lands. Today the Status table has one cell per (CLI, codec) so the flip is per-cell, not per-row.
 
+### Credentialless CI: `Runner.SkipDataPath`
+
+The bundle's three data-path subtests (`JSONIsArray`, `CSVHasHeader`, `DefaultIsMarkdown`) invoke the CLI with a real `--format` value and expect a clean exit — which means the data path has to be runnable in the CI environment. For crono (env-var auth), liftoff (stored OAuth token), and withings (OAuth refresh token), that is not true today: a vanilla `--format json` invocation under `compat`'s default empty env exits non-zero with "not logged in" before the JSON-array check fires.
+
+`compat.Runner.SkipDataPath` is the explicit escape hatch: when true, the data-path subtests skip with reason `data-path subtests disabled via Runner.SkipDataPath`. The parse-level subtests still run because they assert on flag parsing, not on a working data path. This lets an exporter wire the bundle in for parse-level attestation today, then flip the bool back to false later (once auth is mockable in CI or secrets are provisioned) to promote its codec rows from human to machine.
+
+```go
+formats.RunContract(t, compat.Runner{
+    Binary:       os.Getenv("EXPORT_CLI_BIN"),
+    SkipDataPath: true, // CI does not have upstream credentials
+})
+```
+
+`SkipDataPath` and `SupportedFormats` compose: `nil` SupportedFormats + `SkipDataPath: true` is the typical liftoff/crono shape today (declare the full surface, but skip the data-path subtests until creds are available). Use `SupportedFormats: []string{...}` instead of `SkipDataPath: true` when the CLI structurally lacks a codec — those are two different gaps and should not be confused.
+
 ## What it does NOT cover yet
 
 The actual local-midnight semantics of `--since 2026-04-15` (the harmonization that just landed across crono/liftoff/withings) is still **human-attested** in the status table. Asserting it black-box requires either:

compat/compat.go

@@ -78,6 +78,37 @@ type Runner struct {
 	// declare "all of §4".
 	SupportedFormats []string
 
+	// SkipDataPath, when true, causes section bundles to skip every
+	// subtest that requires the CLI's data path to succeed in CI —
+	// i.e. the subtests that invoke the binary with a real --format
+	// value and expect a clean exit. The parse-level subtests
+	// (HelpDocumentsFormatFlag, UnknownFormatFails,
+	// FlagValidationIsHermetic) still run because they only exercise
+	// flag parsing and explicitly expect non-zero exit on the bad
+	// value.
+	//
+	// This is the escape hatch for exporters whose data path requires
+	// credentials that the CI environment does not have. Crono uses
+	// env-var auth (CRONOMETER_USERNAME/PASSWORD); liftoff uses a
+	// stored OAuth token at ~/.config/liftoff-export/auth.json;
+	// withings uses an OAuth refresh token. None of these are present
+	// in `compat`'s default empty-env invocation, so an unconditional
+	// `--format json` run exits non-zero with "not logged in" before
+	// the JSON-array check can run. Setting SkipDataPath: true lets
+	// the exporter wire the bundle in for parse-level attestation
+	// without a misleading SupportedFormats: []string{} workaround
+	// that implies "we implement no codecs".
+	//
+	// Flipping it back to false later — once auth is mockable in CI
+	// or secrets are provisioned — promotes the codec rows from
+	// human-attested to machine-attested per the CONTRACT.md Status
+	// table flip plan.
+	//
+	// Composable with SupportedFormats: when both are set, the
+	// codec-specific gate runs first, so SupportedFormats: nil +
+	// SkipDataPath: true is the typical liftoff/crono shape today.
+	SkipDataPath bool
+
 	// subcommand, when non-empty, is prepended to args on every Run
 	// call. Set via WithSubcommand; section bundles use it to dispatch
 	// per-subcommand. Callers do not need to set it directly — set

compat/formats/formats.go

@@ -181,9 +181,13 @@ func flagValidationIsHermetic(t *testing.T, r compat.Runner) {
 // covered implicitly so long as the integrator's data path returns
 // successfully.
 //
-// Skipped (not failed) if "json" is not in Runner.SupportedFormats.
+// Skipped (not failed) if "json" is not in Runner.SupportedFormats,
+// or if Runner.SkipDataPath is true.
 func jsonIsArray(t *testing.T, r compat.Runner) {
 	t.Helper()
+	if r.SkipDataPath {
+		t.Skipf("data-path subtests disabled via Runner.SkipDataPath")
+	}
 	if !r.SupportsFormat("json") {
 		t.Skipf("--format json not declared in Runner.SupportedFormats")
 	}
@@ -207,12 +211,16 @@ func jsonIsArray(t *testing.T, r compat.Runner) {
 // success with "no rows" for CSV — the header row is still required,
 // so even a zero-row CSV must have one line.
 //
-// Skipped (not failed) if "csv" is not in Runner.SupportedFormats.
+// Skipped (not failed) if "csv" is not in Runner.SupportedFormats,
+// or if Runner.SkipDataPath is true.
 // crono-export-cli and liftoff-export-cli are partial-codec exporters
 // today; the bundle becomes adoptable for them by declaring
 // SupportedFormats: []string{"markdown", "json"}.
 func csvHasHeader(t *testing.T, r compat.Runner) {
 	t.Helper()
+	if r.SkipDataPath {
+		t.Skipf("data-path subtests disabled via Runner.SkipDataPath")
+	}
 	if !r.SupportsFormat("csv") {
 		t.Skipf("--format csv not declared in Runner.SupportedFormats")
 	}
@@ -232,11 +240,14 @@ func csvHasHeader(t *testing.T, r compat.Runner) {
 // available without parsing markdown.
 //
 // Skipped (not failed) if "markdown" is not in
-// Runner.SupportedFormats. A CLI that does not declare markdown
-// cannot be expected to default to it, and forcing the equality check
-// would just measure noise.
+// Runner.SupportedFormats, or if Runner.SkipDataPath is true. A CLI
+// that does not declare markdown cannot be expected to default to it,
+// and forcing the equality check would just measure noise.
 func defaultIsMarkdown(t *testing.T, r compat.Runner) {
 	t.Helper()
+	if r.SkipDataPath {
+		t.Skipf("data-path subtests disabled via Runner.SkipDataPath")
+	}
 	if !r.SupportsFormat("markdown") {
 		t.Skipf("--format markdown not declared in Runner.SupportedFormats")
 	}

compat/formats/formats_test.go

@@ -64,6 +64,36 @@ func TestRunContract_PartialCodec_SkipsCSV(t *testing.T) {
 	formats.RunContract(t, r)
 }
 
+// TestRunContract_SkipDataPath_GatesDataSubtests pins down the
+// Runner.SkipDataPath affordance: when set, the data-path subtests
+// (JSONIsArray / CSVHasHeader / DefaultIsMarkdown) skip even though
+// the codec is otherwise "supported". This is what lets liftoff /
+// crono / withings wire the bundle in for parse-level attestation
+// before their data paths are runnable in CI without credentials.
+//
+// The proof is adversarial. We build the stub with
+// STUBCLI_FORMATS=__never__ so every --format value (markdown, json,
+// csv, AND the unknown sentinel) is rejected with a non-zero exit.
+// The parse-level subtests still pass because they assert on the
+// rejection side. With SkipDataPath: true, the bundle is green.
+// Remove the SkipDataPath guard in jsonIsArray / csvHasHeader /
+// defaultIsMarkdown and this test fails because the data-path
+// subtests now run against a stub that rejects every codec.
+//
+// Critically, SupportedFormats is left nil so this exercises the
+// SkipDataPath gate independently of the codec-membership gate. The
+// pair compose: nil + SkipDataPath: true is the liftoff/crono shape
+// today.
+func TestRunContract_SkipDataPath_GatesDataSubtests(t *testing.T) {
+	bin := buildStub(t)
+	r := compat.Runner{
+		Binary:       bin,
+		Env:          []string{"STUBCLI_FORMATS=__never__"},
+		SkipDataPath: true,
+	}
+	formats.RunContract(t, r)
+}
+
 // TestSupportsFormat documents the nil-vs-empty-vs-subset behavior
 // of compat.Runner.SupportsFormat at the package level. The formats
 // bundle's skip guards rely on these semantics being stable.