Merge pull request #313638 from sethmanheim/relfixes

Fixes for data flow release 2603

Author: JillGrant615

articles/iot-operations/connect-to-cloud/concept-dataflow-graphs-expressions.md

@@ -15,7 +15,7 @@ ai-usage: ai-assisted
 
 [!INCLUDE [kubernetes-management-preview-note](../includes/kubernetes-management-preview-note.md)]
 
-<!-- This reference applies to both [data flows](overview-dataflow.md) and [data flow graphs](concept-dataflow-graphs.md). Both use the same expression language for map, filter, and enrichment transforms. Data flow graphs also support branch and window (accumulate) transforms, which are noted where applicable. -->
+This reference applies to both [data flows](overview-dataflow.md) and [data flow graphs](concept-dataflow-graphs.md). Both use the same expression language for map, filter, and enrichment transforms. Data flow graphs also support branch and window (accumulate) transforms, which are noted where applicable.
 
 ## Positional variables
 
@@ -149,7 +149,7 @@ These functions are available only in accumulation rules within window transform
 | `first($n)` | First value in the window | Error |
 | `last($n)` | Last value in the window | Error |
 
-<!-- For details on using aggregation functions, see [Aggregate data over time](howto-dataflow-graphs-window.md). -->
+For details on using aggregation functions, see [Aggregate data over time](howto-dataflow-graphs-window.md).
 
 ## Conditional logic
 
@@ -262,8 +262,8 @@ JSON objects and arrays are preserved as-is when fields are copied without an ex
 ## Related content
 
 - [Map data by using data flows](concept-dataflow-mapping.md)
-<!-- - [Filter data in a data flow](howto-dataflow-filter.md)
+- [Filter data in a data flow](howto-dataflow-filter.md)
 - [Transform data with map in data flow graphs](howto-dataflow-graphs-map.md)
 - [Filter and route data in data flow graphs](howto-dataflow-graphs-filter-route.md)
 - [Aggregate data over time](howto-dataflow-graphs-window.md)
-- [Enrich with external data](howto-dataflow-graphs-enrich.md) -->
+- [Enrich with external data](howto-dataflow-graphs-enrich.md)

articles/iot-operations/connect-to-cloud/concept-dataflow-graphs-schema.md

@@ -89,7 +89,7 @@ Data flow graph destinations support an `outputSchemaSettings` field to control
 }
 ```
 
-<!-- When using storage destinations (ADLS, Fabric, ADX), the output schema and serialization format are configured on the destination endpoint. See [Configure a data flow destination](howto-configure-dataflow-destination.md#serialize-the-output-with-a-schema). -->
+When using storage destinations (ADLS, Fabric, ADX), the output schema and serialization format are configured on the destination endpoint. See [Configure a data flow destination](howto-configure-dataflow-destination.md#serialize-the-output-with-a-schema).
 
 # [Kubernetes (preview)](#tab/kubernetes)
 
@@ -101,7 +101,7 @@ Data flow graph destinations support an `outputSchemaSettings` field to control
     dataDestination: my-container
 ```
 
-<!-- When using storage destinations (ADLS, Fabric, ADX), the output schema and serialization format are configured on the destination endpoint. See [Configure a data flow destination](howto-configure-dataflow-destination.md#serialize-the-output-with-a-schema). -->
+When using storage destinations (ADLS, Fabric, ADX), the output schema and serialization format are configured on the destination endpoint. See [Configure a data flow destination](howto-configure-dataflow-destination.md#serialize-the-output-with-a-schema).
 
 ---
 
@@ -120,5 +120,5 @@ Data flow graph destinations support an `outputSchemaSettings` field to control
 ## Related content
 
 - [Understand message schemas](concept-schema-registry.md)
-<!-- - [Data flow graphs overview](concept-dataflow-graphs.md)
-- [Configure a data flow destination](howto-configure-dataflow-destination.md) -->
+- [Data flow graphs overview](concept-dataflow-graphs.md)
+- [Configure a data flow destination](howto-configure-dataflow-destination.md)

articles/iot-operations/connect-to-cloud/concept-dataflow-graphs.md

@@ -41,15 +41,15 @@ For new projects that use supported endpoint types, we recommend data flow graph
 
 Each transform is a pre-built processing step that you configure with rules and chain with other transforms inside a `DataflowGraph` resource.
 
-<!-- | Transform | What it does | Learn more |
+| Transform | What it does | Learn more |
 |-----------|-------------|------------|
 | **Map** | Rename, restructure, compute, and copy fields | [Transform data with map](howto-dataflow-graphs-map.md) |
 | **Filter** | Drop messages that match a condition | [Filter and route data](howto-dataflow-graphs-filter-route.md) |
 | **Branch** | Route each message to a `true` or `false` path based on a condition | [Filter and route data](howto-dataflow-graphs-filter-route.md#branch-transform) |
 | **Concat** | Merge two or more paths back into one | [Filter and route data](howto-dataflow-graphs-filter-route.md#merge-paths-with-concat) |
-| **Window** | Collect messages over a time interval, then aggregate | [Aggregate data over time](howto-dataflow-graphs-window.md) | -->
+| **Window** | Collect messages over a time interval, then aggregate | [Aggregate data over time](howto-dataflow-graphs-window.md) |
 
-<!-- All transforms share an [expression language](concept-dataflow-graphs-expressions.md) for operators, functions, and field references. You can also [enrich](howto-dataflow-graphs-enrich.md) messages with external data from a state store in map, filter, and branch transforms. -->
+All transforms share an [expression language](concept-dataflow-graphs-expressions.md) for operators, functions, and field references. You can also [enrich](howto-dataflow-graphs-enrich.md) messages with external data from a state store in map, filter, and branch transforms.
 
 ## How transforms compose
 
@@ -69,7 +69,7 @@ Here's a complete example that reads temperature data from an MQTT topic, conver
 
 # [Operations experience](#tab/portal)
 
-<!-- ![Screenshot of the operations experience showing a data flow graph example with source, transform, and destination.](media/concept-dataflow-graphs/dataflow-graph-example.png) -->
+![Screenshot of the operations experience showing a data flow graph example with source, transform, and destination.](media/concept-dataflow-graphs/dataflow-graph-example.png)
 
 In the Operations experience:
 
@@ -182,7 +182,7 @@ spec:
 
 The pipeline defines three elements: a source, a transform (indicated by `nodeType: Graph`), and a destination. The connections describe how data flows between them. The transform's `configuration` passes rules as a JSON string under the `rules` key.
 
-<!-- In the how-to articles that follow, examples focus on the transform rules themselves. For a step-by-step guide to creating a data flow graph, see [Create a data flow graph](howto-create-dataflow-graph.md). -->
+In the how-to articles that follow, examples focus on the transform rules themselves. For a step-by-step guide to creating a data flow graph, see [Create a data flow graph](howto-create-dataflow-graph.md).
 
 ## Built-in transforms vs. WASM transforms
 
@@ -223,7 +223,7 @@ To use data flow graphs, you need:
 
 ## Next steps
 
-<!-- - [Data flows vs. data flow graphs](overview-dataflow-comparison.md)
+- [Data flows vs. data flow graphs](overview-dataflow-comparison.md)
 - [Create a data flow graph](howto-create-dataflow-graph.md)
 - [Transform data with map](howto-dataflow-graphs-map.md)
 - [Filter and route data](howto-dataflow-graphs-filter-route.md)
@@ -232,4 +232,4 @@ To use data flow graphs, you need:
 - [Expressions reference](concept-dataflow-graphs-expressions.md)
 - [Route messages to different topics](howto-dataflow-graphs-topic-routing.md)
 - [Expressions reference](concept-dataflow-graphs-expressions.md)
-- [Use WASM transforms in data flow graphs](howto-dataflow-graph-wasm.md) -->
+- [Use WASM transforms in data flow graphs](howto-dataflow-graph-wasm.md)

articles/iot-operations/connect-to-cloud/howto-configure-dataflow-destination.md

@@ -230,7 +230,7 @@ In data flow graphs, you can set the destination topic dynamically based on mess
 
 This approach is more flexible than source topic routing because it lets you set the destination topic based on any field or computed value in the message, not just the source topic structure.
 
-<!-- For more information and complete examples, see [Route messages to different topics](howto-dataflow-graphs-topic-routing.md). -->
+For more information and complete examples, see [Route messages to different topics](howto-dataflow-graphs-topic-routing.md).
 
 ## Serialize the output with a schema
 
@@ -247,7 +247,7 @@ Specify the schema and serialization format in the data flow endpoint details. T
 
 # [Azure CLI](#tab/cli)
 
-<!-- After you [upload a schema to the schema registry](concept-schema-registry.md#upload-a-schema), reference it in the data flow configuration. -->
+After you [upload a schema to the schema registry](concept-schema-registry.md#upload-schema-with-the-cli), reference it in the data flow configuration.
 
 ```json
 {
@@ -260,7 +260,7 @@ Specify the schema and serialization format in the data flow endpoint details. T
 
 # [Bicep](#tab/bicep)
 
-<!-- After you [upload a schema to the schema registry](concept-schema-registry.md#upload-a-schema), reference it in the data flow configuration. -->
+After you [upload a schema to the schema registry](concept-schema-registry.md#upload-schema-with-the-cli), reference it in the data flow configuration.
 
 ```bicep
 builtInTransformationSettings: {
@@ -271,7 +271,7 @@ builtInTransformationSettings: {
 
 # [Kubernetes (preview)](#tab/kubernetes)
 
-<!-- After you [upload a schema to the schema registry](concept-schema-registry.md#upload-a-schema), reference it in the data flow configuration. -->
+After you [upload a schema to the schema registry](concept-schema-registry.md#upload-schema-with-the-cli), reference it in the data flow configuration.
 
 ```yaml
 builtInTransformationSettings:
@@ -285,7 +285,7 @@ For more information about schema registry, see [Understand message schemas](con
 
 ## Next steps
 
-<!-- - [Configure a data flow source](howto-configure-dataflow-source.md)
+- [Configure a data flow source](howto-configure-dataflow-source.md)
 - [Create a data flow](howto-create-dataflow.md)
 - [Data flow graphs overview](concept-dataflow-graphs.md)
-- [Route messages to different topics with data flow graphs](howto-dataflow-graphs-topic-routing.md) -->
+- [Route messages to different topics with data flow graphs](howto-dataflow-graphs-topic-routing.md)

articles/iot-operations/connect-to-cloud/howto-configure-dataflow-source.md

@@ -18,7 +18,7 @@ ai-usage: ai-assisted
 
 The source is where data enters a data flow or data flow graph. You configure the source by specifying an endpoint reference and a list of data sources (topics) for that endpoint.
 
-<!-- This page applies to both [data flows](overview-dataflow.md) and [data flow graphs](concept-dataflow-graphs.md). For data flows, the source is an operation in the `Dataflow` resource. For data flow graphs, the source is a `Source` node in the `DataflowGraph` resource. -->
+This page applies to both [data flows](overview-dataflow.md) and [data flow graphs](concept-dataflow-graphs.md). For data flows, the source is an operation in the `Dataflow` resource. For data flow graphs, the source is a `Source` node in the `DataflowGraph` resource.
 
 > [!IMPORTANT]
 > Data flows support MQTT and Kafka source endpoints. Data flow graphs support MQTT, Kafka, and OpenTelemetry source endpoints. Each data flow must have the Azure IoT Operations local MQTT broker default endpoint as either the source or destination. For more information, see [Data flows must use local MQTT broker endpoint](./howto-configure-dataflow-endpoint.md#data-flows-must-use-local-mqtt-broker-endpoint).
@@ -436,6 +436,6 @@ For more information, see [Understand message schemas](concept-schema-registry.m
 
 ## Next steps
 
-<!-- - [Configure a data flow destination](howto-configure-dataflow-destination.md)
+- [Configure a data flow destination](howto-configure-dataflow-destination.md)
 - [Create a data flow](howto-create-dataflow.md)
-- [Data flow graphs overview](concept-dataflow-graphs.md) -->
+- [Data flow graphs overview](concept-dataflow-graphs.md)

articles/iot-operations/connect-to-cloud/howto-configure-disk-persistence.md

@@ -147,5 +147,5 @@ spec:
 ## Related content
 
 - [Create a data flow](howto-create-dataflow.md)
-<!-- - [Data flow graphs overview](concept-dataflow-graphs.md) -->
+- [Data flow graphs overview](concept-dataflow-graphs.md)
 - [Configure MQTT broker persistence](../manage-mqtt-broker/howto-broker-persistence.md)

articles/iot-operations/connect-to-cloud/overview-dataflow-comparison.md

@@ -24,7 +24,7 @@ Data flows are generally available and support all endpoint types.
 
 ## What are data flow graphs?
 
-<!-- A [data flow graph](concept-dataflow-graphs.md) is a composable, graph-based pipeline that connects multiple transforms in any topology you define. You can chain, branch, and merge processing steps. Each transform is a pre-built processing unit (map, filter, branch, window, enrichment) that you configure with rules. You configure a data flow graph by creating a `DataflowGraph` custom resource. -->
+A [data flow graph](concept-dataflow-graphs.md) is a composable, graph-based pipeline that connects multiple transforms in any topology you define. You can chain, branch, and merge processing steps. Each transform is a pre-built processing unit (map, filter, branch, window, enrichment) that you configure with rules. You configure a data flow graph by creating a `DataflowGraph` custom resource.
 
 Data flow graphs are in preview and support MQTT, Kafka, and OpenTelemetry endpoints.
 
@@ -49,9 +49,9 @@ Data flows and data flow graphs share:
 
 - **Endpoints**: Both use `DataflowEndpoint` resources to connect to MQTT brokers, Kafka, storage, and other services. See [Prepare endpoints](howto-configure-dataflow-endpoint.md).
 - **Profiles**: Both reference a `DataflowProfile` for scaling and instance configuration. See [Manage data flow profiles](howto-configure-dataflow-profile.md).
-<!-- - **Source configuration**: Both support the same source endpoint options (default broker, asset, custom endpoint) and data source (topic) configuration. See [Configure a source](howto-configure-dataflow-source.md).
+- **Source configuration**: Both support the same source endpoint options (default broker, asset, custom endpoint) and data source (topic) configuration. See [Configure a source](howto-configure-dataflow-source.md).
 - **Destination configuration**: Both support static and dynamic destination topics. See [Configure a destination](howto-configure-dataflow-destination.md).
-- **Disk persistence**: Both support `requestDiskPersistence` to survive restarts. See [Configure disk persistence](howto-configure-disk-persistence.md). -->
+- **Disk persistence**: Both support `requestDiskPersistence` to survive restarts. See [Configure disk persistence](howto-configure-disk-persistence.md).
 
 ## When to use which
 
@@ -73,6 +73,6 @@ For new projects that use supported endpoint types, we recommend data flow graph
 ## Related content
 
 - [Data flows overview](overview-dataflow.md)
-<!-- - [Data flow graphs overview](concept-dataflow-graphs.md)
+- [Data flow graphs overview](concept-dataflow-graphs.md)
 - [Create a data flow](howto-create-dataflow.md)
-- [Create a data flow graph](howto-create-dataflow-graph.md) -->
+- [Create a data flow graph](howto-create-dataflow-graph.md)

articles/iot-operations/toc.yml

@@ -195,9 +195,9 @@ items:
               - name: Local storage or ACSA
                 href: connect-to-cloud/howto-configure-local-storage-endpoint.md
                 displayName: data flow, endpoint, northbound, Azure Container Storage enabled by Azure Arc
-          # - name: Source
-          #   href: connect-to-cloud/howto-configure-dataflow-source.md
-          #   displayName: data flow, source, MQTT, Kafka, topic, asset, broker, shared subscription
+          - name: Source
+            href: connect-to-cloud/howto-configure-dataflow-source.md
+            displayName: data flow, source, MQTT, Kafka, topic, asset, broker, shared subscription
           - name: Data flows
             items:
               - name: Overview
@@ -217,39 +217,39 @@ items:
                 displayName: data flow, serialize, deserialize, schema registry
           - name: Data flow graphs
             items:
-              # - name: Overview
-              #   href: connect-to-cloud/concept-dataflow-graphs.md
-              #   displayName: data flow, graph, transform, map, filter, branch, window, compose, pipeline
-              # - name: Create a data flow graph
-              #   href: connect-to-cloud/howto-create-dataflow-graph.md
-              #   displayName: data flow, graph, create, deploy, pipeline, source, destination
-              # - name: Transform data with map
-              #   href: connect-to-cloud/howto-dataflow-graphs-map.md
-              #   displayName: data flow, graph, map, rename, restructure, wildcard, copy, compute
-              # - name: Filter and route data
-              #   href: connect-to-cloud/howto-dataflow-graphs-filter-route.md
-              #   displayName: data flow, graph, filter, drop, branch, concat, merge, route, condition
-              # - name: Aggregate data over time
-              #   href: connect-to-cloud/howto-dataflow-graphs-window.md
-              #   displayName: data flow, graph, window, aggregate, average, sum, min, max, tumbling
-              # - name: Enrich with external data
-              #   href: connect-to-cloud/howto-dataflow-graphs-enrich.md
-              #   displayName: data flow, graph, enrich, dataset, state store, context, lookup
-              # - name: Route messages to different topics
-              #   href: connect-to-cloud/howto-dataflow-graphs-topic-routing.md
-              #   displayName: data flow, graph, topic, routing, metadata, dynamic
+              - name: Overview
+                href: connect-to-cloud/concept-dataflow-graphs.md
+                displayName: data flow, graph, transform, map, filter, branch, window, compose, pipeline
+              - name: Create a data flow graph
+                href: connect-to-cloud/howto-create-dataflow-graph.md
+                displayName: data flow, graph, create, deploy, pipeline, source, destination
+              - name: Transform data with map
+                href: connect-to-cloud/howto-dataflow-graphs-map.md
+                displayName: data flow, graph, map, rename, restructure, wildcard, copy, compute
+              - name: Filter and route data
+                href: connect-to-cloud/howto-dataflow-graphs-filter-route.md
+                displayName: data flow, graph, filter, drop, branch, concat, merge, route, condition
+              - name: Aggregate data over time
+                href: connect-to-cloud/howto-dataflow-graphs-window.md
+                displayName: data flow, graph, window, aggregate, average, sum, min, max, tumbling
+              - name: Enrich with external data
+                href: connect-to-cloud/howto-dataflow-graphs-enrich.md
+                displayName: data flow, graph, enrich, dataset, state store, context, lookup
+              - name: Route messages to different topics
+                href: connect-to-cloud/howto-dataflow-graphs-topic-routing.md
+                displayName: data flow, graph, topic, routing, metadata, dynamic
               - name: Use WASM transforms
                 href: connect-to-cloud/howto-dataflow-graph-wasm.md
                 displayName: data flow, WASM, WebAssembly, graph, transform, custom, developer, deploy
               - name: Schemas
                 href: connect-to-cloud/concept-dataflow-graphs-schema.md
                 displayName: data flow, graph, schema, nodeConnections, serialization
-          # - name: Destination
-          #   href: connect-to-cloud/howto-configure-dataflow-destination.md
-          #   displayName: data flow, destination, MQTT, Kafka, topic, container, table, dynamic
-          # - name: Disk persistence
-          #   href: connect-to-cloud/howto-configure-disk-persistence.md
-          #   displayName: data flow, persistence, disk, restart, state
+          - name: Destination
+            href: connect-to-cloud/howto-configure-dataflow-destination.md
+            displayName: data flow, destination, MQTT, Kafka, topic, container, table, dynamic
+          - name: Disk persistence
+            href: connect-to-cloud/howto-configure-disk-persistence.md
+            displayName: data flow, persistence, disk, restart, state
           - name: Expressions
             href: connect-to-cloud/concept-dataflow-graphs-expressions.md
             displayName: data flow, expression, function, operator, metadata, variable, conversion