Skip to content
Merged
16 changes: 4 additions & 12 deletions modules/get-started/pages/release-notes/redpanda.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,11 @@ This topic includes new content added in version {page-component-version}. For a
* xref:cloud-data-platform:get-started:whats-new-cloud.adoc[]
* xref:cloud-data-platform:get-started:cloud-overview.adoc#redpanda-cloud-vs-self-managed-feature-compatibility[Redpanda Cloud vs Self-Managed feature compatibility]

== Iceberg: Decoded message keys and headers
== Iceberg translation and health monitoring

You can now independently control how Redpanda translates a record's key, value, and headers into the Iceberg table using the `redpanda.iceberg.mode` topic property. A new section-based syntax lets you decode both keys and values using a schema registered in the Schema Registry, store keys or values as UTF-8 strings, and decode header values from raw bytes to strings.

The `key_value`, `value_schema_id_prefix`, and `value_schema_latest` modes continue to work and serve as shorthand for common combinations of these sections.

See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Specify Iceberg Schema] for the syntax, options, and examples.

== Iceberg translation with Schema Registry contexts

Iceberg-enabled topics can now resolve schemas from a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] instead of always using the default context. Set the new `redpanda.schema.registry.context` topic property to bind a topic's Iceberg translation to a context. Binding to a context is useful for multi-tenant or per-environment schema isolation.

See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context] for configuration examples and considerations for changing a topic's context.
* *Decoded message keys and headers*: Independently control how Redpanda translates a record's key, value, and headers into the Iceberg table with a new section-based syntax for the `redpanda.iceberg.mode` topic property. You can decode keys and values using a schema registered in the Schema Registry, store keys or values as UTF-8 strings, or decode header values from raw bytes to strings. The `key_value`, `value_schema_id_prefix`, and `value_schema_latest` modes continue to work as shorthand for common combinations. See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Specify Iceberg Schema].
* *Schema Registry context for translation*: Bind a topic's Iceberg translation to a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] using the new `redpanda.schema.registry.context` topic property, instead of always using the default context. This is useful for multi-tenant or per-environment schema isolation. See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context].
* *Health monitoring*: Monitor the health of Iceberg topics with a new Admin API endpoint, including connectivity to the external REST catalog and per-partition commit lag. Use it to test the catalog connection and find the partitions with the highest commit lag. See xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Monitor Iceberg health].

== Cluster health metrics

Expand Down
8 changes: 8 additions & 0 deletions modules/manage/pages/iceberg/about-iceberg-topics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,14 @@ Querying the Iceberg table for `demo-topic` includes the new column `ts`:
+---------+--------------+--------------------------+
----

ifndef::env-cloud[]
== Monitor Iceberg health

To test connectivity with your external Iceberg REST catalog and check the per-partition commit lag for each Iceberg topic, use the read-only `GetIcebergStatus` Admin API endpoint (Redpanda version 26.2 and later). Commit lag is the number of records that Redpanda has written to Parquet but not yet committed to the catalog.

This endpoint requires superuser privileges if authentication is enabled on your cluster. See the link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-icebergservice-geticebergstatus[Admin API reference] for the full request and response schema.
endif::[]

== Next steps

* xref:manage:iceberg/use-iceberg-catalogs.adoc[]
Expand Down
4 changes: 4 additions & 0 deletions modules/manage/pages/iceberg/iceberg-troubleshooting.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -118,4 +118,8 @@ The following xref:reference:public-metrics-reference.adoc#iceberg-metrics[Icebe
* xref:reference:public-metrics-reference.adoc#redpanda_iceberg_translation_invalid_records[`redpanda_iceberg_translation_invalid_records`]: Number of invalid records encountered during translation, labeled by cause. See <<drop-invalid-records>> to configure how Redpanda handles these records.
* xref:reference:public-metrics-reference.adoc#redpanda_iceberg_rest_client_num_commit_table_update_requests_failed[`redpanda_iceberg_rest_client_num_commit_table_update_requests_failed`]: Failed table commit requests to the REST catalog. Applies only when using a REST catalog (`iceberg_catalog_type: rest`). Persistent failures indicate catalog connectivity or permission issues.

ifndef::env-cloud[]
For an on-demand check of external catalog connectivity and per-partition commit lag, use the `GetIcebergStatus` Admin API endpoint. Commit lag is the number of records written to Parquet but not yet committed to the catalog; the endpoint does not report translation lag (how far Iceberg translation trails the Kafka log). See xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Monitor Iceberg health].
endif::[]

// end::single-source[]
8 changes: 4 additions & 4 deletions modules/manage/pages/use-admin-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,11 @@ The new endpoints differ from the legacy endpoints in the following ways:
* URL paths use the fully-qualified names of the ConnectRPC services.
* ConnectRPC endpoints accept only POST requests.

Use ConnectRPC endpoints with features introduced in v25.3 such as:
Use ConnectRPC endpoints with features such as:

// TODO: Add links to docs when they are merged
* Shadowing
* Connected client monitoring
* xref:manage:disaster-recovery/shadowing/index.adoc[Shadowing]
* xref:manage:cluster-maintenance/manage-throughput.adoc#view-connected-client-details[Connected client monitoring]
* xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Iceberg health monitoring]

For a full list of available endpoints, see the link:/api/doc/admin/v2/[Admin API Reference]. Select "v2" in the version selector to view the ConnectRPC endpoints.

Expand Down
Loading