Add tripwire tests for the RTAS policy merge - #659
Open
mkuchenbecker wants to merge 6 commits into
Open
Conversation
RTAS re-materialized the policies table property only from the request, so a replace that omitted policies silently wiped retention, sharing, PII column tags, replication, and history. Policies are table metadata and must survive a replace: merge the existing table's policies with the request's, each plane the request provides winning and every omitted plane carried forward. Co-authored-by: Copilot <[email protected]>
Capture the merge contract: each plane merged, omitted planes carried forward, provided planes win (partial merge), planes can be added via RTAS, columnTags empty-vs-nonempty, and sharingEnabled takes the request value. Adds a unit test over mergePolicies and an e2e test that RTAS applies a requested policy. Co-authored-by: Copilot <[email protected]>
Replace the mergePolicies unit test with black-box tests that drive real Spark SQL against a real embedded OpenHouse server (OpenHouseSparkITest): set policies via ALTER TABLE ... SET POLICY, run CREATE OR REPLACE TABLE ... AS SELECT, and assert via SHOW TBLPROPERTIES that retention, sharing, PII column tags, and history survive the replace. Also fix RTASTest.testRTAS, which asserted the old buggy behavior (policies wiped on RTAS); it now asserts the history policy is preserved. Co-authored-by: Copilot <[email protected]>
Guard mergePolicies against a new policy plane being added to Policies without teaching the RTAS merge to carry it forward. A behavioral test populates every object plane on the existing table, merges a request that omits them, and reflectively asserts each survives; a structural test pins the exact set of Policies fields so any add/rename/remove (including a primitive) trips it. Co-authored-by: Copilot <[email protected]>
mkuchenbecker
changed the base branch from
main
to
mkuchenbecker/r2-rtas-policies-merge
July 24, 2026 16:40
mkuchenbecker
marked this pull request as ready for review
August 4, 2026 16:26
mkuchenbecker
marked this pull request as draft
August 4, 2026 16:27
mkuchenbecker
marked this pull request as ready for review
August 4, 2026 16:27
mkuchenbecker
changed the base branch from
mkuchenbecker/r2-rtas-policies-merge
to
main
August 4, 2026 16:28
…licy-tripwire Co-authored-by: Copilot <[email protected]>
Contributor
Author
|
@kamanavishnu PTAL |
kamanavishnu
approved these changes
Aug 4, 2026
| "lockState")); | ||
|
|
||
| Set<String> actual = | ||
| java.util.Arrays.stream(Policies.class.getDeclaredFields()) |
Collaborator
There was a problem hiding this comment.
minor nit: Add import instead of fully qualified package name ?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #652 (base branch
mkuchenbecker/r2-rtas-policies-merge). Adds tripwire tests that guardthe RTAS policy merge against a future policy plane being added to
Policieswithout updatingmergePoliciesto carry it forward (which would reintroduce the silent policy-drop #652 fixes).omits them, and reflectively assert each plane survives — a new plane that
mergePoliciesforgetscomes back
nulland fails with a pointed message.Policiesfield names, so any add / remove / rename(including a primitive the behavioral test can't observe) trips the test and forces a conscious
merge update.
Testing Done
RtasPolicyMergeTripwireTest(both tests) — pass.mergePoliciesfails the behavioral test with"RTAS policy merge dropped Policies.". Spotless clean.