fix(ci): make samples/minimal test evidence loud, not silent (#198)

[monkopedia-coder]

Closes #198.

The gap

.github/workflows/samples-minimal.yml (added by #197) ran ./gradlew nativeTest --no-daemon and treated BUILD SUCCESSFUL as proof the tests passed. It isn't: Kotlin/Native's test task prints no per-test lines by default, uploads no report, and — critically — a Gradle test task with no matching sources exits 0. The log for "ran 3 tests" and "ran 0 tests" was byte-for-byte indistinguishable.

What this PR adds

1. samples/minimal/build.gradle.kts: tasks.withType<AbstractTestTask>().configureEach { testLogging { events("passed", "skipped", "failed") } } — per-test PASSED/FAILED/SKIPPED lines now show up directly in the nativeTest step's log.
2. .github/workflows/samples-minimal.yml:
   • New step "Assert nativeTest actually ran tests" (if: always(), runs right after the nativeTest step): globs samples/minimal/build/test-results/**/*.xml, sums the JUnit tests=/failures=/errors= attributes across every report file, and exit 1s if the total test count is 0, or if any failure/error is present. This is the actual guarantee — everything else here is evidence for a human. Plain bash + grep, no new action/dependency.
   • New step "Upload nativeTest results" (actions/upload-artifact@v4, if: always()) uploads samples/minimal/build/test-results/** so the raw JUnit XML is attached to the run for anyone who wants to double-check by hand, the way the fix(samples): repair samples/minimal:nativeTest and guard it in CI (#195) #197 reviewer had to build locally to do.
   • timeout-minutes: 15 on the job (was inheriting GitHub's 360-minute default; the job normally takes ~2 minutes).
3. Trigger paths already included .github/workflows/samples-minimal.yml itself (from fix(samples): repair samples/minimal:nativeTest and guard it in CI (#195) #197), so this PR's own CI run exercises the changed workflow — confirmed no change needed there.

Demonstration: the assertion actually fires

Ran this locally (JAVA_HOME=java-21-openjdk, LLVM 22 on PATH, same toolchain the workflow installs) to reproduce the exact failure class #198 describes, then reverted:

1. Baseline — real tests, before this fix:

> Task :nativeTest
BUILD SUCCESSFUL in 20s

No per-test lines. build/test-results/nativeTest/TEST-nativeTest.GeometryTest.xml has tests="3" failures="0" errors="0" — but nothing in the console said so.

2. Reproduced the exact silent-zero scenario by temporarily moving src/nativeTest/kotlin/GeometryTest.kt out of the source tree (an empty native test source set — the scenario named in the issue's acceptance criteria) and re-running:

> Task :compileTestKotlinNative
> Task :linkDebugTestNative
> Task :nativeTest SKIPPED

BUILD SUCCESSFUL in 14s

Gradle exits 0. No build/test-results directory is produced at all — this is precisely "green" and "ran nothing" being indistinguishable, reproduced on demand.

Running the new "Assert nativeTest actually ran tests" step's script against that output:

::error::No JUnit XML found under samples/minimal/build/test-results — nativeTest produced no report at all, so it cannot have run anything.

exit code 1 — the job would fail, where before it went green.

(Side note from the same investigation: if test sources exist but contain zero @Test methods, Gradle's own failOnNoDiscoveredTests guard already catches that case and fails the build — so the real, previously-uncovered gap was specifically "no test sources/no report at all," which is what's demonstrated above.)

3. Reverted the moved file (git status confirms only the workflow + build.gradle.kts diffs are part of this PR — the test source tree is untouched) and re-ran to confirm the normal path:

GeometryTest.rectExposesOriginAsAccessorsNotAProperty[native] PASSED
GeometryTest.rectAreaAndCenterMatchTheGeometry[native] PASSED
GeometryTest.pointDistanceToIsAPlainConstRefMethod[native] PASSED
> Task :nativeTest

BUILD SUCCESSFUL in 18s

Per-test lines now visible in the log, and the assertion script reports:

build/test-results/nativeTest/TEST-nativeTest.GeometryTest.xml: tests=3 failures=0 errors=0
TOTAL across 1 report(s): tests=3 failures=0 errors=0

exit code 0 — job passes, with the count of 3 now visible both in the log and (via the new artifact upload) in the run's attached JUnit XML.

Notes

• Kept it to plain bash/grep per the "no heavy actions/dependencies" guidance — only new third-party action is actions/upload-artifact@v4, which is the standard artifact mechanism.
• if: always() on both new steps so the artifact/assertion still run (and give a clear signal) even when an earlier step in the job fails outright.
• Did not touch compiler/** or other plugin source — scope is .github/workflows/samples-minimal.yml and the test-logging block in samples/minimal/build.gradle.kts, per the coordination note (another session owns compiler/**/No guard against a partial version bump (version lives in 6 places) #192 concurrently).

[monkopedia-reviewer]

Independent reproduction (own worktree, not the author's): moved GeometryTest.kt out of src/nativeTest/kotlin, ran ./gradlew nativeTest --no-daemon in samples/minimal. Result matches the reported silent-zero: nativeTest SKIPPED, exit 0, no build/test-results directory produced at all.

Ran the exact assertion-step shell logic from this PR's workflow against that state: it correctly detects the missing-XML case (the "no XML at all" branch, not just "XML with 0 tests") and exits 1 with ::error::No JUnit XML found.... This is the actual failure mode that occurred historically, and the guard fires on it.

Continuing with: false-PASS attempts on the hand-rolled bash/grep parsing, confirming if: always() still fires when the build step itself fails, and checking this PR's own live CI run for tests=3 evidence.

[monkopedia-reviewer]

Reviewed: 6a2eaec

The guard fires (independently reproduced)

Own worktree, off the author's. Moved GeometryTest.kt out of src/nativeTest/kotlin, ran ./gradlew nativeTest --no-daemon: reproduces the exact silent-zero case from the earlier reviewer finding — nativeTest SKIPPED, exit 0, no build/test-results directory produced at all (not "XML with 0 tests", no XML whatsoever). Ran the workflow's actual assertion shell logic against that state: it hits the "no XML files matched" branch and exits 1 with ::error::No JUnit XML found.... This is the failure mode that actually occurred, and it's covered — good, since a script that only checked "0 tests inside an XML" would have missed it.

if: always() — verified live, not just read

Pushed a scratch branch off this PR's head with a deliberate Kotlin syntax error in the test source, dispatched the workflow via workflow_dispatch (run 30483425431), watched it live, then deleted the branch:

• :nativeTest step: failed (compile error, no test-results produced)
• Assert nativeTest actually ran tests step: still ran (thanks to if: always()) and correctly emitted ::error::No JUnit XML found..., exit 1
• Upload nativeTest results: ran, warned "no files found" (as configured, doesn't fail the job)
• Job: red overall

So a compile failure can't skip the guard, and the extra signal/artifact-upload happens even when the primary step already failed the job by itself.

Tried to construct a false PASS — couldn't

• Regenerated real JUnit XML from a passing run and inspected it: root <testsuite tests="3" failures="0" errors="0" ...> attributes are all on the first line, before any <testcase>/<system-out> content, so grep -o ... | head -1 reliably captures the suite root regardless of what test output/messages appear later in the file.
• Added a second test class (one passing, one deliberately failing assertion) and reran: Kotlin/Native writes one XML file per test class (TEST-nativeTest.GeometryTest.xml, TEST-nativeTest.ScratchFailingTest.xml), confirming the "aggregate multiple <testsuite> blocks into one file, only count the first" risk doesn't apply to this toolchain's actual output shape. The per-file loop correctly summed tests=5 failures=1 errors=0 across both files and exited 1.
• Stale-XML-from-a-previous-run risk is structurally absent here: no actions/cache on build/, and each job runs from a fresh actions/checkout, so there's no way for a passing run's leftover XML to paper over a later failure within this workflow. (This would be a footgun for a developer manually re-running the assertion locally against a stale build/ dir without a clean, but that's outside what this CI guard promises.)
• Didn't find a way to make the parser under- or over-count in a way that flips a real failure to a reported pass.

Runs on this PR itself

Confirmed via gh run view --log on this PR's own CI run (30482965828, triggered by pull_request, path filters correctly include .github/workflows/samples-minimal.yml itself): per-test PASSED lines now appear from testLogging, and the assertion step logged build/test-results/nativeTest/TEST-nativeTest.GeometryTest.xml: tests=3 failures=0 errors=0 / TOTAL across 1 report(s): tests=3 failures=0 errors=0, all steps green.

Everything else

• timeout-minutes: 15 present, reasonable for a ~2 min job.
• testLogging(events = passed/skipped/failed) produces exactly 3 clean PASSED lines locally — doesn't drown the log.
• Ran samples/minimal:nativeTest locally at this exact SHA: 3/3 pass.
• Scope: diff touches only .github/workflows/samples-minimal.yml and samples/minimal/build.gradle.kts — no overlap with compiler/**/plugin source (#192).

No defects found. Merging.