De-flake testIdpUpdate: atomic issuer deletion in Cassandra batch#5316
Open
blackheaven wants to merge 3 commits into
Open
De-flake testIdpUpdate: atomic issuer deletion in Cassandra batch#5316blackheaven wants to merge 3 commits into
blackheaven wants to merge 3 commits into
Conversation
The flaky testIdpUpdate test failed intermittently with HTTP 400 where 200 was expected. Root cause: the IdP update in idpUpdateXML performed the issuer index deletion as a separate Cassandra write after the config insertion batch. When an issuer was reused across updates (e.g., A -> B -> A), the tombstone from the first deletion could shadow the later insert due to clock skew between Cassandra coordinators, causing the finalize-login endpoint to fail its issuer-based IdP lookup. Server fix: move old-issuer deletion into the same logged batch as insertIdPConfig, so insert and delete share the same batch timestamp. Test fix: add status checks on updateIdp calls and wrap post-update logins with 'eventually' for resilience against residual consistency delays.
zebot
added
the
ok-to-test
akshaymankar
reviewed
Jul 7, 2026
Comment on lines
+111
to
+119
| -- Delete old issuer mappings atomically with the new config insertion to | ||
| -- prevent Cassandra tombstone shadowing when an issuer is reused across | ||
| -- multiple updates (e.g., updating from issuer A to B and back to A). | ||
| forM_ oldIssuers $ \oldIssuer -> | ||
| case thisVersion of | ||
| WireIdPAPIV2 -> addPrepQuery delOldIssuerV2 (oldIssuer, tid) | ||
| WireIdPAPIV1 -> do | ||
| addPrepQuery delOldIssuerV1 (Identity oldIssuer) | ||
| addPrepQuery delOldIssuerV1' (Identity oldIssuer) |
Member
There was a problem hiding this comment.
Not all callers of this function were deleting old configs, maybe we should parameterize this.
Address PR review feedback: not all callers of insertConfig were previously deleting old issuer mappings. Moving the deletion unconditionally into the batch changed behavior for idpCreate and storeIdPConfig. Introduce InsertMode (InsertOnly | InsertWithIssuerCleanup) so only idpUpdateXML opts into atomic issuer cleanup, preserving the original behavior of all other callers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
changelog.d