perf-nightly #57

Workflow file for this run

.github/workflows/perf-nightly.yml at d884435

	name: perf-nightly

	# TF-8 — nightly LUBM-10 performance regression gate.
	#
	# Builds the pgRDF extension from source, builds the UBA generator
	# container, generates LUBM-10 into the pgrdf-lubm-data volume, runs
	# the TF-10 perf runner, and compares against the committed
	# tests/perf/lubm/baseline.lubm-10.json (TF-9). Fails if correctness
	# drifts OR if timing exceeds the per-fixture
	# comparison_tolerance.elapsed_ms_pct (default ±50%).
	#
	# Scope locked to LUBM-10 localhost per [[lubm-localhost-only]]: no
	# LUBM-100, no Azure, no hosted-runner-class expansion. The cron
	# fires nightly on the standard `ubuntu-latest` runner; runtime is
	# bounded (build ~5 min, ingest ~30s, query negligible).

	on:
	schedule:
	# 03:00 UTC daily. Pick a quiet hour so the artifact builds don't
	# contend with the human-cycle ci.yml runs that happen during the
	# day. Adjust if telemetry shows actions-runner queue pressure at
	# this hour.
	- cron: "0 3 * * *"
	# On-demand: lets a maintainer fire the gate after a perf-suspect
	# change without waiting for the cron tick.
	workflow_dispatch: {}

	# Read-only — nothing this workflow does writes back to the repo.
	# Regression is surfaced as workflow-failure + the action logs; the
	# baseline.lubm-10.json file is updated by a human commit after
	# investigation, never by this workflow auto-blessing drift.
	permissions:
	contents: read

	env:
	CARGO_TERM_COLOR: always
	PGRX_VERSION: "0.19.1"

	jobs:
	lubm-10:
	name: LUBM-10 dev-gate (TF-7 + TF-8)
	runs-on: ubuntu-22.04
	# Hard ceiling so a hung generator/runner doesn't burn an
	# actions-runner indefinitely. Typical end-to-end is ~6-8 min.
	timeout-minutes: 25
	env:
	PGRDF_RUNTIME: docker
	PGRDF_BUILD_RUNTIME: docker
	steps:
	- uses: actions/checkout@v4

	- uses: dtolnay/rust-toolchain@stable

	- uses: ./.github/actions/setup-pgrx
	with:
	pg: "17"

	# Build the extension artifacts into compose/extensions/ — same
	# path the regression job in ci.yml uses; the TF-10 runner
	# bind-mounts these into the postgres sidecar.
	- name: Set up Docker Buildx
	uses: docker/setup-buildx-action@v3

	# The Justfile recipes below need `just` itself. ubuntu-22.04
	# doesn't ship it; taiki-e/install-action pulls a precompiled
	# binary in <1s. Caught on the 04:22 UTC 2026-05-30 cron run
	# (`just: command not found`, exit 127) — the workflow was
	# added in v0.5.20 without this step.
	- name: Install just
	uses: taiki-e/install-action@v2
	with:
	tool: just

	- name: Build extension artifacts
	run: just build-ext

	# Build the UBA generator + generate BOTH LUBM-1 and LUBM-10
	# into the pgrdf-lubm-data named volume. LUBM-1 (~103k triples,
	# ~5s ingest on the CI runner) is the smaller-tier reference
	# for Track A's ingest-path spikes; LUBM-10 is the dev-gate.
	# Generating both costs ~10s.
	- name: Build LUBM generator
	run: just lubm-build

	- name: Generate LUBM-1
	run: just lubm-gen 1

	- name: Generate LUBM-10
	run: just lubm-gen 10

	# Run the runner + compare against the committed baseline for
	# each tier. `--no-timing-gate`: correctness fields stay strict
	# (Q14 count, dict_lookups, triple count must match exactly);
	# timing diffs print as `TIMING:` info but don't fail the job.
	# Rationale: GH Actions runner is ~2.5× slower than the
	# localhost machine where the baselines were captured, so a
	# localhost-tight tolerance is unreachable here. The CI run is
	# then a trend snapshot — numbers visible in workflow logs +
	# uploaded artifacts — and a correctness gate — Q14 count
	# diverging means a real bug.
	#
	# `target/perf-report.json` is overwritten between tiers, so
	# capture each tier's report to a distinct path for the artifact.
	- name: Run LUBM-1 perf snapshot
	run: \|
	set -euo pipefail
	mkdir -p target
	OUTFILE=target/perf-report.lubm-1.json \
	JSON_SCHEMA_VALIDATE=0 \
	bash tests/perf/lubm/run-lubm.sh 1
	python3 tests/perf/lubm/compare-to-baseline.py \
	--actual target/perf-report.lubm-1.json \
	--baseline tests/perf/lubm/baseline.lubm-1.json \
	--no-timing-gate

	- name: Run LUBM-10 perf snapshot
	run: \|
	set -euo pipefail
	OUTFILE=target/perf-report.lubm-10.json \
	JSON_SCHEMA_VALIDATE=0 \
	bash tests/perf/lubm/run-lubm.sh 10
	python3 tests/perf/lubm/compare-to-baseline.py \
	--actual target/perf-report.lubm-10.json \
	--baseline tests/perf/lubm/baseline.lubm-10.json \
	--no-timing-gate

	# ── Benchmark harness: ingest + RDFS + OWL-RL + Q14 ─────────
	#
	# Persistent per-run history accumulates in
	# `target/perf-history/runs.jsonl` (gitignored) + a
	# self-contained HTML report at
	# `target/perf-history/index.html`. CI uploads BOTH the
	# JSONL and the HTML as artifacts so a reviewer can grab
	# the HTML, open it locally, and see the chart of all runs
	# captured in that JSONL.
	#
	# The HTML is generated from the SINGLE jsonl line this
	# run appends — CI doesn't see prior runs because the
	# workspace is fresh each invocation. To see drift across
	# CI runs, a follow-up could either (a) commit the JSONL
	# to a tracked branch, or (b) accumulate via a cache
	# action / external store. Out of scope for the MVP; the
	# local dev path (`just benchmark`) accumulates locally.
	- name: Run benchmark harness (LUBM-10 + RDFS + OWL-RL + Q14)
	run: \|
	set -euo pipefail
	bash tests/perf/benchmark-runner.sh 10

	# Always upload — the trend value is in collecting numbers
	# across nightly runs, not just failure triage. 90-day
	# retention default; bump if we ever want longer history.
	- name: Upload perf snapshots
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: perf-snapshots-${{ github.run_id }}
	path: target/perf-report.lubm-*.json
	if-no-files-found: warn
	retention-days: 90

	- name: Upload benchmark history + HTML report
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: perf-history-${{ github.run_id }}
	path: target/perf-history/
	if-no-files-found: warn
	retention-days: 90

	# Surface the latest numbers in the workflow's Job Summary
	# so a reviewer doesn't have to download the artifact to
	# see what changed. GITHUB_STEP_SUMMARY accepts markdown.
	- name: Job summary
	if: always()
	run: \|
	set -euo pipefail
	if [ -f target/perf-history/last-run.json ]; then
	python3 - <<'PY' >> "${GITHUB_STEP_SUMMARY}"
	import json, pathlib
	d = json.loads(pathlib.Path("target/perf-history/last-run.json").read_text())
	ing = d.get("ingest", {}) or {}
	mat = d.get("materialize", {}) or {}
	rdfs = (mat.get("rdfs") or {})
	owl = (mat.get("owl_rl") or {})
	q14 = d.get("q14", {}) or {}
	def fm(x):
	if x is None: return "—"
	try: return f"{float(x):.1f}"
	except Exception: return str(x)
	def fi(x):
	if x is None: return "—"
	try: return f"{int(x):,}"
	except Exception: return str(x)
	print(f"## pgRDF benchmark · {d.get('ts','')}\n")
	print(f"- pgRDF: `{d.get('pgrdf_version','?')}` ({d.get('git_sha','')[:8]})")
	print(f"- LUBM size: {d.get('lubm_size','?')} ({fi(d.get('triples'))} triples)")
	print(f"- Ingest: {fm(ing.get('elapsed_ms'))} ms (parse {fm(ing.get('parse_ms'))} · dict {fm(ing.get('dict_ms'))} · insert {fm(ing.get('insert_ms'))})")
	print(f"- Materialize RDFS: {fm(rdfs.get('elapsed_ms'))} ms · {fi(rdfs.get('triples_inferred'))} inferred")
	print(f"- Materialize OWL-RL: {fm(owl.get('elapsed_ms'))} ms · {fi(owl.get('triples_inferred'))} inferred")
	print(f"- Q14 (median of 3 warm): {fm(q14.get('elapsed_ms_median'))} ms · {fi(q14.get('result_count'))} rows")
	print("\nFull HTML report: download the `perf-history-<run_id>` artifact below + open `index.html`.")
	PY
	fi

Navigation Menu

perf-nightly #57

Workflow file

perf-nightly #57

Uh oh!

Workflow file for this run