Skip to content

Commit d54ae6a

Browse files
LeadGoEngineerPaperclip-Paperclip
andcommitted
test(compat): add STUBCLI_NEEDS_TOKEN stub hook + token-style SkipDataPath self-test
The merged `Runner.SkipDataPath` self-test (`TestRunContract_SkipDataPath_GatesDataSubtests`) is adversarial in the broadest possible way: `STUBCLI_FORMATS=__never__` makes the stub reject every `--format` value at parse time. That proves the `SkipDataPath` guard is load-bearing, but against a failure mode (parse-rejection) that no real exporter actually exhibits. `STUBCLI_NEEDS_TOKEN=1` adds the tighter adversarial model called out in DT's review of #14: parse succeeds, the data call fails non-zero with `error: not logged in` on stderr. That is the shape real credentialed exporters (crono / liftoff / withings) hit in CI when no secrets are provisioned — the exact scenario `SkipDataPath` exists to escape. The new self-test (`TestRunContract_SkipDataPath_WithNotLoggedIn`) runs the full formats bundle with `STUBCLI_NEEDS_TOKEN=1` and `SkipDataPath: true`. Parse-level subtests pass because they assert on the rejection side of unknown flags, which never reach the token check. Data-path subtests skip via the guard. Neutering the `SkipDataPath` guards in jsonIsArray / csvHasHeader / defaultIsMarkdown makes the test fail with `--format <codec> exited 1; stderr="error: not logged in\n"` — a distinct failure shape from the `__never__` variant's `invalid value for --format`. The two adversarial models therefore cover different regressions of the same guard. No contract or `Runner` surface changes. SkipDataPath shipped in #14 and is settled. Refs: - Closed PR #15 — origin of STUBCLI_NEEDS_TOKEN - Merged PR #14 — Runner.SkipDataPath - QUA-34 Co-Authored-By: Paperclip <[email protected]>
1 parent 1cf4a39 commit d54ae6a

2 files changed

Lines changed: 56 additions & 0 deletions

File tree

compat/formats/formats_test.go

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,36 @@ func TestRunContract_SkipDataPath_GatesDataSubtests(t *testing.T) {
9494
formats.RunContract(t, r)
9595
}
9696

97+
// TestRunContract_SkipDataPath_WithNotLoggedIn is the token-style
98+
// adversarial complement to TestRunContract_SkipDataPath_GatesDataSubtests.
99+
//
100+
// The __never__ variant proves the SkipDataPath guard is load-bearing
101+
// against a stub that rejects every --format value at parse time. This
102+
// variant proves it is load-bearing against the failure mode real
103+
// credentialed exporters (crono / liftoff / withings) actually hit in
104+
// CI: --format parses cleanly, but the data call fails non-zero with
105+
// a "not logged in"-style error because no token is available.
106+
//
107+
// STUBCLI_NEEDS_TOKEN=1 produces exactly that shape. Parse-level
108+
// subtests still pass (--help has no token check; --format
109+
// obviously-not-a-format is rejected at parse, before the token
110+
// check). Data-path subtests skip via SkipDataPath.
111+
//
112+
// Delete the SkipDataPath guard in jsonIsArray / csvHasHeader /
113+
// defaultIsMarkdown and this test fails with `--format <codec> exited
114+
// 1; stderr=…not logged in…` — a distinct failure shape from the
115+
// __never__ variant's `…invalid value for --format…`, so the two
116+
// adversarial models cover different regressions of the same guard.
117+
func TestRunContract_SkipDataPath_WithNotLoggedIn(t *testing.T) {
118+
bin := buildStub(t)
119+
r := compat.Runner{
120+
Binary: bin,
121+
Env: []string{"STUBCLI_NEEDS_TOKEN=1"},
122+
SkipDataPath: true,
123+
}
124+
formats.RunContract(t, r)
125+
}
126+
97127
// TestSupportsFormat documents the nil-vs-empty-vs-subset behavior
98128
// of compat.Runner.SupportsFormat at the package level. The formats
99129
// bundle's skip guards rely on these semantics being stable.

compat/internal/stubcli/main.go

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,19 @@
3939
// rather than running it against a stub that would reject the call.
4040
// When unset, all three §4 codecs are accepted.
4141
//
42+
// STUBCLI_NEEDS_TOKEN=1, when set, makes the stub simulate a
43+
// credentialless-CI failure: `--help` and `--format <unknown>` still
44+
// behave normally (the hook does not fire during parse), but a
45+
// successful parse of `--format <known codec>` does NOT emit data —
46+
// instead the stub exits non-zero with `error: not logged in` on
47+
// stderr and an empty stdout. This is the failure mode real
48+
// credentialed exporters (crono / liftoff / withings) hit in CI:
49+
// flag parsing succeeds, the data call fails because no token is
50+
// available. The compat/formats self-test uses this to prove that
51+
// Runner.SkipDataPath is load-bearing against the token-style
52+
// adversarial model — not just the broader STUBCLI_FORMATS=__never__
53+
// reject-everything model.
54+
//
4255
// stubcli never makes a network request.
4356
package main
4457

@@ -160,6 +173,19 @@ func parseAndEmit(progName string, args []string) {
160173
os.Exit(2)
161174
}
162175

176+
// STUBCLI_NEEDS_TOKEN=1 simulates a credentialless-CI failure:
177+
// flags parsed cleanly, but the data call fails because no token
178+
// is available. Mirrors the shape real credentialed exporters
179+
// (crono / liftoff / withings) hit when run from a CI environment
180+
// without secrets. Exit code 1 (data-call failure) is distinct
181+
// from the 2 used for parse failures above, so a test that
182+
// expected a parse-rejection but accidentally took the data path
183+
// sees a different shape.
184+
if os.Getenv("STUBCLI_NEEDS_TOKEN") == "1" {
185+
fmt.Fprintln(os.Stderr, "error: not logged in")
186+
os.Exit(1)
187+
}
188+
163189
// The stub has no upstream, so it always reports the empty data
164190
// set. Per CONTRACT §4: `[]` for json, header-only for csv,
165191
// nothing for markdown.

0 commit comments

Comments
 (0)