[default values] Implement ReadBridge.apply without an Iceberg fork - #675
Closed
cbb330 wants to merge 9 commits into
Closed
[default values] Implement ReadBridge.apply without an Iceberg fork#675cbb330 wants to merge 9 commits into
cbb330 wants to merge 9 commits into
Conversation
8 tasks
cbb330
force-pushed
the
chbush/read-bridge-apply
branch
from
August 12, 2026 02:24
c84734e to
4f2f134
Compare
17 tasks
cbb330
force-pushed
the
chbush/read-bridge-apply
branch
5 times, most recently
from
August 12, 2026 05:33
a62d4e6 to
3266fa6
Compare
cbb330
marked this pull request as ready for review
August 12, 2026 16:10
cbb330
force-pushed
the
chbush/read-bridge-apply
branch
from
August 12, 2026 16:51
3266fa6 to
b3f7c4c
Compare
Co-authored-by: Cursor <[email protected]>
Keep the deployment-specific ColumnDefaultsSource optional and data-only; OpenHouse owns the read-bridge.column-default feature id, self-serve enabled property, and fail-open toggle lookup before asking for defaults.
Correct the self-service property name and drop the contradictory "cluster kill switch" claim. Add coverage for a real source that returns no defaults, and assert table-property opt-in never calls the server toggle. Testing Done: - :services:tables:test --tests '*ReadBridgeConfigResolverTest'
BaseTableFeatureToggle looks up (databaseId, tableId, featureId) exactly; claiming a * / * fleet kill switch was inaccurate. Also note HTS is only hit when the self-service property is absent.
Replace essay javadoc with brief ownership, contract, ObjectProvider, and fail-open notes so the PR description carries the design narrative.
Co-authored-by: Cursor <[email protected]>
Read-bridge overlays must not persist: a later write would stamp initial-default onto disk and survive ramp-off. Restore default slots for field-ids that existed at load; keep writer defaults on new ids so V2 schema evolution still works.
6 tasks
Overlay server-stamped column defaults onto every schema-id at metadata load using NestedField.initialDefault and a public TableMetadataParser JSON rebuild, keeping decode on the from()/apply() split from #668.
cbb330
force-pushed
the
chbush/read-bridge-apply
branch
from
August 12, 2026 22:59
b3f7c4c to
b85fd39
Compare
Collaborator
Author
|
Restacking onto #678 (sanitize-on-write). GitHub would not change this PR's base because it is part of a stack, so apply moves to a new PR with the correct base. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Depends on #674 (policy), stacked on #668 → #645.
Finish the client half of the column-default read bridge: turn server-stamped
configinto IcebergNestedField.initialDefaultat metadata load — without forking Iceberg.ReadBridge.from(config)indoRefresh/ before IOopenhouse.read-bridge.column-default.<fieldId>(kept from #668, off the retry path)ReadBridge.apply(raw)inloadMetadataHow apply works
raw.schemas(), rebuild fields withNestedField.builder+withInitialDefaultwhen a stamped field-id is present and binds.TableMetadatavia publicTableMetadataParser.toJson/fromJson(patchschemas/schema-id; format-v1 also patchesschema). NowithSchemaOverlay, no Iceberg fork.Failure policy (mirrors the server encoder):
rawunchangedIllegalStateException)openhouse.read-bridge.*keySemantics notes:
Changes
ReadBridge.applyactually overlaysinitialDefaultreplaceSchemasJSON rebuild helper (public Iceberg APIs only)ReadBridgeTest: single/multi-schema overlay, nested fields, gaps, fail-loud bind, inert pathWhy not fork Iceberg
LinkedIn Iceberg already exposes
NestedField.initialDefault/withInitialDefault. The missing piece is replacing schemas on an existingTableMetadatainstance. A JSON round-trip throughTableMetadataParseris enough for the client overlay and keeps OH off an Iceberg fork. If Iceberg later ships a real overlay helper, swapreplaceSchemasonly.Testing Done
./gradlew :integrations:java:iceberg-1.2:openhouse-java-itest:test --tests '*ReadBridgeTest'Iceberg 1.5 / Spark 3.5 java runtime shares these sources via Gradle
sourceSets(same apply path).Additional Information
Stack
ReadBridge.applyDepends on #678. Overlay + sanitize round-trip is covered by
sanitizeAfterApplyRestoresOnDiskSchema.Rollback Plan
Revert this PR. Clients keep decoding config (#668) but stop overlaying
initialDefault; readers behave as today (NULL for missing columns). Server stamp (#645/#674) can remain inert for old clients.