diff --git a/modules/get-started/pages/release-notes/redpanda.adoc b/modules/get-started/pages/release-notes/redpanda.adoc index 72d17398d2..57d00d7aa6 100644 --- a/modules/get-started/pages/release-notes/redpanda.adoc +++ b/modules/get-started/pages/release-notes/redpanda.adoc @@ -7,19 +7,11 @@ This topic includes new content added in version {page-component-version}. For a * xref:cloud-data-platform:get-started:whats-new-cloud.adoc[] * xref:cloud-data-platform:get-started:cloud-overview.adoc#redpanda-cloud-vs-self-managed-feature-compatibility[Redpanda Cloud vs Self-Managed feature compatibility] -== Iceberg: Decoded message keys and headers +== Iceberg translation and health monitoring -You can now independently control how Redpanda translates a record's key, value, and headers into the Iceberg table using the `redpanda.iceberg.mode` topic property. A new section-based syntax lets you decode both keys and values using a schema registered in the Schema Registry, store keys or values as UTF-8 strings, and decode header values from raw bytes to strings. - -The `key_value`, `value_schema_id_prefix`, and `value_schema_latest` modes continue to work and serve as shorthand for common combinations of these sections. - -See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Specify Iceberg Schema] for the syntax, options, and examples. - -== Iceberg translation with Schema Registry contexts - -Iceberg-enabled topics can now resolve schemas from a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] instead of always using the default context. Set the new `redpanda.schema.registry.context` topic property to bind a topic's Iceberg translation to a context. Binding to a context is useful for multi-tenant or per-environment schema isolation. - -See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context] for configuration examples and considerations for changing a topic's context. +* *Decoded message keys and headers*: Independently control how Redpanda translates a record's key, value, and headers into the Iceberg table with a new section-based syntax for the `redpanda.iceberg.mode` topic property. You can decode keys and values using a schema registered in the Schema Registry, store keys or values as UTF-8 strings, or decode header values from raw bytes to strings. The `key_value`, `value_schema_id_prefix`, and `value_schema_latest` modes continue to work as shorthand for common combinations. See xref:manage:iceberg/specify-iceberg-schema.adoc#configure-key-value-and-header-translation[Specify Iceberg Schema]. +* *Schema Registry context for translation*: Bind a topic's Iceberg translation to a specific xref:manage:schema-reg/schema-reg-contexts.adoc[Schema Registry context] using the new `redpanda.schema.registry.context` topic property, instead of always using the default context. This is useful for multi-tenant or per-environment schema isolation. See xref:manage:iceberg/specify-iceberg-schema.adoc#resolve-schemas-within-a-context[Resolve schemas within a Schema Registry context]. +* *Health monitoring*: Monitor the health of Iceberg topics with a new Admin API endpoint, including connectivity to the external REST catalog and per-partition commit lag. Use it to test the catalog connection and find the partitions with the highest commit lag. See xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Monitor Iceberg health]. == Cluster health metrics diff --git a/modules/manage/pages/iceberg/about-iceberg-topics.adoc b/modules/manage/pages/iceberg/about-iceberg-topics.adoc index 22c1c5afab..103b45e537 100644 --- a/modules/manage/pages/iceberg/about-iceberg-topics.adoc +++ b/modules/manage/pages/iceberg/about-iceberg-topics.adoc @@ -316,6 +316,14 @@ Querying the Iceberg table for `demo-topic` includes the new column `ts`: +---------+--------------+--------------------------+ ---- +ifndef::env-cloud[] +== Monitor Iceberg health + +To test connectivity with your external Iceberg REST catalog and check the per-partition commit lag for each Iceberg topic, use the read-only `GetIcebergStatus` Admin API endpoint (Redpanda version 26.2 and later). Commit lag is the number of records that Redpanda has written to Parquet but not yet committed to the catalog. + +This endpoint requires superuser privileges if authentication is enabled on your cluster. See the link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-icebergservice-geticebergstatus[Admin API reference] for the full request and response schema. +endif::[] + == Next steps * xref:manage:iceberg/use-iceberg-catalogs.adoc[] diff --git a/modules/manage/pages/iceberg/iceberg-troubleshooting.adoc b/modules/manage/pages/iceberg/iceberg-troubleshooting.adoc index 31234de327..c6681e6259 100644 --- a/modules/manage/pages/iceberg/iceberg-troubleshooting.adoc +++ b/modules/manage/pages/iceberg/iceberg-troubleshooting.adoc @@ -118,4 +118,8 @@ The following xref:reference:public-metrics-reference.adoc#iceberg-metrics[Icebe * xref:reference:public-metrics-reference.adoc#redpanda_iceberg_translation_invalid_records[`redpanda_iceberg_translation_invalid_records`]: Number of invalid records encountered during translation, labeled by cause. See <> to configure how Redpanda handles these records. * xref:reference:public-metrics-reference.adoc#redpanda_iceberg_rest_client_num_commit_table_update_requests_failed[`redpanda_iceberg_rest_client_num_commit_table_update_requests_failed`]: Failed table commit requests to the REST catalog. Applies only when using a REST catalog (`iceberg_catalog_type: rest`). Persistent failures indicate catalog connectivity or permission issues. +ifndef::env-cloud[] +For an on-demand check of external catalog connectivity and per-partition commit lag, use the `GetIcebergStatus` Admin API endpoint. Commit lag is the number of records written to Parquet but not yet committed to the catalog; the endpoint does not report translation lag (how far Iceberg translation trails the Kafka log). See xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Monitor Iceberg health]. +endif::[] + // end::single-source[] diff --git a/modules/manage/pages/use-admin-api.adoc b/modules/manage/pages/use-admin-api.adoc index a3b6717faf..15eb523481 100644 --- a/modules/manage/pages/use-admin-api.adoc +++ b/modules/manage/pages/use-admin-api.adoc @@ -81,11 +81,11 @@ The new endpoints differ from the legacy endpoints in the following ways: * URL paths use the fully-qualified names of the ConnectRPC services. * ConnectRPC endpoints accept only POST requests. -Use ConnectRPC endpoints with features introduced in v25.3 such as: +Use ConnectRPC endpoints with features such as: -// TODO: Add links to docs when they are merged -* Shadowing -* Connected client monitoring +* xref:manage:disaster-recovery/shadowing/index.adoc[Shadowing] +* xref:manage:cluster-maintenance/manage-throughput.adoc#view-connected-client-details[Connected client monitoring] +* xref:manage:iceberg/about-iceberg-topics.adoc#monitor-iceberg-health[Iceberg health monitoring] For a full list of available endpoints, see the link:/api/doc/admin/v2/[Admin API Reference]. Select "v2" in the version selector to view the ConnectRPC endpoints.