TOOL-30021 Enable sdb upstream tracking + bootstrap upstreams branch on first sync#393
Merged
Merged
Conversation
…ssing fetch_repo_from_git previously failed hard if a package's upstreams/$DEFAULT_GIT_BRANCH branch did not yet exist on origin, blocking the very first upstream-sync run for any newly onboarded upstream-tracked package. The package-onboarding flow expected this branch to be pre-seeded manually before automation could run, but the branch protection (push restricted to delphix-devops-bot) makes that manual step inaccessible to the spec authors typically onboarding the package. Detect the missing-branch case and bootstrap upstream-HEAD directly from UPSTREAM_GIT_URL/UPSTREAM_GIT_BRANCH, marking $WORKDIR/upstream-updated so sync-with-upstream.sh pushes upstream-HEAD back to origin -- git push creates the upstreams/<branch> branch on origin in the process. Requires both UPSTREAM_GIT_URL and UPSTREAM_GIT_BRANCH to be set; falls back to the previous die with a clearer message otherwise. To make the bootstrap fallback possible, git_fetch_helper no longer dies internally on failure -- the four existing logmust callers still fail-hard via logmust's own error handling, while the new fallback caller can branch on the exit code. Follow-up to delphix/cd-aidlc spec 0003 Phase 2, surfaced by the first run of check-updates against the new sdb upstream-tracking recipe. Co-Authored-By: Claude Opus 4.7 <[email protected]>
Wire packages/sdb/config.sh up to the upstream-tracking automation introduced by the prior commit, completing Phase 2 of delphix/cd-aidlc spec 0003 (Adopt sdimitro/sdb as upstream; repackage via dh-virtualenv). The build() override derives PACKAGE_VERSION from setuptools_scm so the .deb's metadata version follows upstream sdb tags automatically; pyproject.toml's local_scheme="no-local-version" keeps the value debian-version-safe (no embedded "-"), letting linux-pkg's set_changelog append the standard "-1delphix.<ts>" revision suffix without further massaging. Phase 1 (delphix/sdb#360, merged 2026-05-06) landed the bootstrap merge from sdimitro/develop plus the dh-virtualenv overlay, including python3-setuptools-scm in Build-Depends so the build() override here works with no companion delphix/sdb change. Co-Authored-By: Claude Opus 4.7 <[email protected]>
1b64822 to
9bb44b7
Compare
david-mendez1
approved these changes
May 26, 2026
nealquigley
approved these changes
May 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Spec 0003 Phase 2 onboards
sdbas an upstream-tracked package againstsdimitro/sdb. Phase 1 (delphix/sdb#360) merged 2026-05-06 and landed the bootstrap merge fromsdimitro/developplus thedh-virtualenvoverlay. The linux-pkg side is wiringupdate_upstream()up againstsdimitro/sdb'sdevelop, and havingbuild()derivePACKAGE_VERSIONfromsetuptools_scmso the.deb's metadata version follows upstream sdb tags automatically.Problem
The first check-updates run against the new sdb recipe fails before reaching
update_upstreamat all, e.g.fetch_repo_from_gitrequiresupstreams/$DEFAULT_GIT_BRANCHto already exist on the package repo's origin before the update_upstream stage even runs;update_upstream_from_gitonly fast-forwards an existing localupstream-HEAD, it doesn't create it. The linux-pkg README's onboarding section (around line 740) says you're supposed to push that branch by hand at repo-creation time, but the branch protection ondelphix/sdbrestricts pushes toupstreams/*todelphix-devops-botonly, so a human spec author can't do it themselves; the bot only pushes during the normal sync flow, which can't run because the branch doesn't exist. Chicken-and-egg.The same gap applies to every future upstream-tracked package that gets onboarded.
Solution
The solution is to bootstrap
upstream-HEADfromUPSTREAM_GIT_URL/UPSTREAM_GIT_BRANCHdirectly when the package repo'supstreams/<branch>is missing, andtouch $WORKDIR/upstream-updatedsosync-with-upstream.shpushes the bootstrapped ref back via the bot creds it already uses at the push step.git pushof a non-existent remote branch creates it, so the first sync run becomes the momentupstreams/<branch>is born on origin; subsequent runs are no different from the existing fast-forward path.To make the conditional fetch possible,
git_fetch_helper's internal|| dieis dropped. The four existing callers all uselogmust, which handles failure identically, so the new fallback caller can branch on the exit code without losing the fail-hard behavior elsewhere.Two files change: the bootstrap fallback in
lib/common.sh, and the sdb recipe atpackages/sdb/config.sh, which surfaces the gap and is needed to verify the fallback end-to-end against a real upstream-tracked recipe.Testing Done
Jenkins check-updates #1148 on this branch with
PACKAGES=sdb,UPDATE_PACKAGES=false. The bootstrap path hits,upstream-HEADpopulates fromsdimitro/sdbdevelop, and "Package sdb has updates available" reports cleanly, e.g.For contrast, #1147 at the prior branch tip confirms the original failure mode.
Post-merge verification (out of scope here):
update_upstream()definition and auto-generatesupdate-package/sdb.upstreams/developtodelphix/sdband merges upstream intodevelop. That also satisfies the long-deferred Phase 1 AC Add unit tests #2 as a side-effect.build-package/sdb/pre-pushto verify the.deb'sVersion:field derives fromsetuptools_scm(Phase 2 AC Add package definition for make-jpkg #11).Spec context: delphix/cd-aidlc#62. JIRA: TOOL-30021.