Skip to content

Gate releases on vendored template integrity - #514

Draft
jdchawla29 wants to merge 1 commit into
jaideep/templates-cifrom
jaideep/templates-integrity
Draft

Gate releases on vendored template integrity#514
jdchawla29 wants to merge 1 commit into
jaideep/templates-cifrom
jaideep/templates-integrity

Conversation

@jdchawla29

@jdchawla29 jdchawla29 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #513.

Adds .github/workflows/templates-integrity.yml, a reusable workflow that for each environments/ template: installs its pinned deps, overrides hud with either the repo checkout (nightly, manual dispatch) or the built wheel (source: wheel), runs the template's test suite, and builds its Dockerfile.hud image.

release.yml now runs it as a template-gate job with source: wheel and deploy depends on it — a wheel that breaks a canonical template, or a template image that no longer builds, cannot be published. The nightly schedule surfaces the same drift between releases instead of on release day.

Notes:

  • The coding template's suite already asserts oracle/null grading (golden=1.0, baseline=0.0) through a real local rollout, so the gate covers the serve/connect/run/grade path against the exact release artifact.
  • The Docker builds catch image rot; an in-container cua oracle run (scripted desktop actions over rfb) is a follow-up once the runner story for it is settled.
  • Template Dockerfiles install hud from PyPI at image build time, so the gate's docker step exercises the previous release inside the image; the wheel-override applies to the test suite. Aligning the image install with the release candidate is part of the later hud init vendoring work.

Note

Low Risk
Changes are limited to GitHub Actions orchestration; no runtime SDK or application code is modified.

Overview
Release publishing is gated on vendored environments/ templates: release.yml adds a template-gate job that calls the new workflow with source: wheel, and deploy now needs it so PyPI publish only runs after the gate passes.

The new templates-integrity.yml workflow runs on a nightly schedule, manual dispatch, and as a reusable workflow_call. For each matrix template (coding, cua), it syncs pinned deps, installs hud from the checkout (editable) or from dist/hud-*.whl when source is wheel, runs pytest, and builds Dockerfile.hud.

Nightly runs surface SDK–template drift between releases; on release, the gate ensures a wheel that breaks canonical template tests or image builds cannot ship.

Reviewed by Cursor Bugbot for commit 713acdf. Bugbot is set up for automated code reviews on this repo. Configure here.

HUD-2276

New reusable templates-integrity workflow: install each environments/
template with hud overridden by the checkout (nightly/manual) or the
built wheel (release gate), run its test suite, and build its Docker
image. release.yml publishes only after the gate passes against the
exact wheel being released; the nightly schedule catches drift between
releases.
@jdchawla29
jdchawla29 marked this pull request as draft July 27, 2026 03:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant