Conversation
…PG sockets reconcileGPGState runs gpgconf --kill gpg-agent, then removes every socket left in GNUPGHOME on the assumption that anything still there belongs to a dead daemon. That assumption breaks with keyboxd: it is a separate daemon from gpg-agent (introduced for the public keybox), so killing gpg-agent alone leaves it running while its own socket gets deleted right out from under it. The next git commit -S has to reconnect to keyboxd, and when it loses that race it fails with: gpg: can't connect to the keyboxd: IPC connect call failed gpg: error opening key DB: No Keybox daemon running gpg: signing failed: Input/output error fatal: failed to write commit object Since the failure depends on exact timing, most promotions succeed and this shows up as an intermittent signing failure that clears up on retry - which is what made it hard to pin down. Fix: gpgconf --kill all, which covers every component that might be running (gpg-agent, dirmngr, scdaemon, keyboxd) instead of naming one. Verified by adapting the repro loop from the issue: with the old gpgconf --kill gpg-agent, keyboxd was still alive after the socket sweep on every iteration (0/40, then 0/300 - the crash itself is timing dependent, but the unsafe state it depends on was 100% reproducible). With gpgconf --kill all, keyboxd was gone every time, 40/40. Fixes akuity#7133 Signed-off-by: Ivo Marino <[email protected]>
✅ Deploy Preview for docs-kargo-io ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
kargo-governance-bot
Bot
added
kind/bug
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7133.
reconcileGPGStaterunsgpgconf --kill gpg-agent, then walks GNUPGHOME and removes every leftover UNIX socket on the assumption that anything still there belongs to a daemon that just got killed. That assumption doesn't hold forkeyboxd: it's a separate process fromgpg-agent(added for the public keybox in modern GnuPG), so killinggpg-agentalone leaveskeyboxdrunning while its socket gets pulled out from under it a few lines later.The next
git commit -Sneeds to reconnect tokeyboxd, and when the timing doesn't work out, it fails with:Since it's a timing race, most promotions go through fine and a retry usually succeeds - which is exactly what made this hard to track down (the issue's own writeup, including a full repro script, does a great job of narrowing it to this exact mechanism).
Fix:
gpgconf --kill allinstead of naminggpg-agentspecifically - covers whatever's actually running (gpg-agent,dirmngr,scdaemon,keyboxd) rather than one component by name.Testing: adapted the repro loop from the issue to check the actual precondition directly (is
keyboxdstill alive right after the socket sweep). With the old--kill gpg-agent, it was alive every single time across two separate runs (40 and 300 iterations) - the crash itself didn't trigger in either run (it's genuinely timing-dependent, matching the report), but the unsafe state it depends on reproduced 100% of the time. With--kill all,keyboxdwas gone every time, 40/40. Also ran the existingpkg/controller/gittest suite, unaffected.Didn't add a new automated test for this specific one, since exercising it meaningfully needs a real
gpg/gpgconf/keyboxdon the runner and I wasn't sure that's a safe assumption for CI - happy to add one if there's an existing pattern for that in this repo I should follow.