You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem:
pkg/prometheus/client.go hardcodes the demo Prometheus as a silent fallback when PROMETHEUS_URL is unset. A misconfigured run compares the local OpenCost API against demo metrics, producing false failures, or worse, false passes against the wrong cluster.
This caused false failures the team hit while running the suite locally.
Nothing in the current tickets covers it: #74's smoke tests verify each endpoint is reachable, but not that the two endpoints observe the same cluster.
Proposed test:
Fail fast if either env var is unset (refuse the silent fallback).
Query both endpoints for the same fixed, completed historical window — kube_pod_container_status_running from Prometheus, /allocation aggregated by pod — using a shared end timestamp.
Filter synthetic entries (idle, *-unmounted-pvcs), compare pod-name sets, require overlap above a threshold (~50%; same-cluster is ~90%+, different-cluster is ~0%, so churn can't cross the boundary).
t.Fatal on mismatch, printing both URLs and sample pods from each side — if environments are incoherent, all downstream ground-truth results are meaningless.
Fresh clusters with no scrape history fall back to a short recent window. Shares window-alignment machinery with Stabilize pod-count test: shared timestamp, churn tolerance, retries #94. Runs before the rest of the suite as a precondition guard.
Problem:
pkg/prometheus/client.go hardcodes the demo Prometheus as a silent fallback when PROMETHEUS_URL is unset. A misconfigured run compares the local OpenCost API against demo metrics, producing false failures, or worse, false passes against the wrong cluster.
This caused false failures the team hit while running the suite locally.
Nothing in the current tickets covers it: #74's smoke tests verify each endpoint is reachable, but not that the two endpoints observe the same cluster.
Proposed test:
Fresh clusters with no scrape history fall back to a short recent window. Shares window-alignment machinery with Stabilize pod-count test: shared timestamp, churn tolerance, retries #94. Runs before the rest of the suite as a precondition guard.