Flaky smoke: self-serve-join-auth null-deref under load (eventual-consistency race)

[davidfarah2003]

Summary

pnpm smoke:ci intermittently fails (~1 in 6, and reliably under load) on smoke:self-serve-join:auth with an unguarded null-deref. This is a pre-existing flake — acknowledged in the smoke's own comments — that surfaces when the box is under load (many leftover nats-server processes). Not caused by any recent change; the feat/per-user-auth manager-least-privilege work does not touch this smoke or the readMember path.

Symptom

packages/core/smoke/self-serve-join-auth.smoke.ts, the manager-present (Plane-3 backstop) scenario:

  ! alice: timeout            # expected — this scenario tests the timeout path
  ✗ FAIL: alice's boot 'ops' membership is present (self-joined at connect) []   # line 284
  ✗ scenario threw: Cannot read properties of undefined (reading 'record')       # line 296

→ SELF-SERVE-JOIN SMOKE FAILED ❌ (14 passed, 2 failed)

Root cause

Eventual-consistency race. Alice self-joins ops in Phase 1 (no daemon yet); the daemon reconciles that boot membership only after it comes up in Phase 2.

• Line 280–282 polls channelMembers("ops") for up to 160×50ms (8s). Under load this window can lag, so line 284 fails with [].
• Line 296 then does const opsRec = (await readMember(kv, "ops", aId.id))!.record;. If the membership is still racing, readMember returns null and the !.record deref throws.

The smoke's own comments (lines 276–278, 286) already flag this as a known eventual-consistency flake class ("a slow round-trip (CI/Windows) lagged it and flaked this check").

Frequency

~1/6 in isolation (pnpm smoke:self-serve-join:auth ×6 → 5 pass / 1 fail), and reliably in a full smoke:ci run when the box is under load (20+ leftover nats-server procs from concurrent test sessions). Clean-box single runs are green.

Suggested fix (test-hardening only, no product change)

1. Line 296 — poll readMember until non-null (reuse the eventual-consistency poll pattern already at lines 280–282), then guard the null. If still null after the window, skip the rest of the scenario with a clear skipped: eventual-consistency lag line instead of crashing on an unguarded deref.
2. Line 284 — treat a missed poll as a soft/flake-skip under load (or re-poll with backoff) rather than a hard failure.

Discovered by

The tester team during feat/per-user-auth PR 1 (manager least-privilege) end-to-end validation. Filed separately so it doesn't bloat that PR.