Skip to content

Commit 1a6c4db

Browse files
Address content team feedback: convert JPG to PNG, update tutorial and private connectivity docs
1 parent 622790e commit 1a6c4db

9 files changed

Lines changed: 32 additions & 25 deletions

articles/iot-operations/connect-to-cloud/howto-configure-dataflow-endpoint.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,9 @@ Similar to the MQTT example, you can create multiple data flows that use the sam
268268
269269
## Next steps
270270
271+
> [!TIP]
272+
> To route dataflow traffic to cloud destinations through Private Link instead of public endpoints, see [Configure dataflow destinations with private endpoints](../manage-layered-network/howto-private-connectivity.md#configure-dataflow-destinations-with-private-endpoints).
273+
271274
Create a data flow endpoint:
272275
273276
- [MQTT or Event Grid](howto-configure-mqtt-endpoint.md)

articles/iot-operations/end-to-end-tutorials/tutorial-layered-network-private-connectivity.md

Lines changed: 22 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,14 @@ ms.date: 03/24/2026
88

99
#CustomerIntent: As an operator in an industrial environment with Purdue-style network segmentation, I want to deploy Azure IoT Operations with private Azure connectivity so that no endpoints are exposed to the public internet.
1010
ms.service: azure-iot-operations
11+
ms.subservice: layered-network-management
1112
---
1213

1314
# Tutorial: Deploy Azure IoT Operations in a layered network with private connectivity
1415

15-
This tutorial describes how to deploy Azure IoT Operations in a physically layered network topology with explicit proxy routing and private connectivity to Azure services via ExpressRoute. This deployment uses Private Link to reach services like Event Grid, and exposes no public endpoints at any network layer.
16+
This tutorial describes how to deploy Azure IoT Operations in a physically layered network topology with explicit proxy routing and private connectivity to Azure services through ExpressRoute. This deployment uses Private Link to reach services like Event Grid, and exposes no public endpoints at any network layer.
1617

17-
This scenario was validated using physical machines in a Purdue/ISA-95 segmented network spanning Levels 2 through 4. The Azure Firewall Explicit Proxy is deployed in an Azure VNet, with connectivity provided via ExpressRoute.
18+
This scenario was validated by using physical machines in a Purdue/ISA-95 segmented network spanning Levels 2 through 4. The Azure Firewall Explicit Proxy is deployed in an Azure VNet, with connectivity provided through ExpressRoute.
1819

1920
In this article, you:
2021

@@ -34,20 +35,20 @@ In this article, you:
3435
- Sufficient permissions to create Private Endpoints, Private DNS Zones, and role assignments (typically **Owner** or **Contributor** + **User Access Administrator**). This tutorial uses custom roles defined in the [Appendix](#appendix).
3536
- A [Kubernetes cluster](/azure/iot-operations/deploy-iot-ops/howto-prepare-cluster) deployed at each network layer (Level 2, Level 3, and Level 4), with devices or VMs assigned static IPs.
3637
- An existing network segmentation between layers (for example, firewalls allowing only L2 ↔ L3 ↔ L4 communication) with DNS resolution across layers using CoreDNS.
37-
- [Azure Private Endpoints](/azure/private-link/private-endpoint-overview) for Event Grid, Storage (for Schema Registry), and Key Vault, assigned private IPs and accessible via [ExpressRoute](/azure/expressroute/expressroute-introduction) or equivalent private routing.
38+
- [Azure Private Endpoints](/azure/private-link/private-endpoint-overview) for Event Grid, Storage (for Schema Registry), and Key Vault, assigned private IPs and accessible through [ExpressRoute](/azure/expressroute/expressroute-introduction) or equivalent private routing.
3839
- [Azure Firewall Explicit Proxy](/azure/firewall/explicit-proxy) at Level 4 (ports 8080/8443), reachable from Level 4 over ExpressRoute. All outbound HTTP/HTTPS traffic from Level 4 flows through this proxy.
3940
- [Azure CLI](/cli/azure/install-azure-cli), [kubectl](https://kubernetes.io/docs/tasks/tools/), and [Docker](https://docs.docker.com/get-docker/) installed on the machine you use to deploy resources and manage Kubernetes clusters.
4041
- (Optional) Familiarity with the [Purdue Model](https://www.isa.org/isa95/), which defines levels of industrial control systems and is commonly used in manufacturing environments.
4142

4243
> [!NOTE]
43-
> In the validated telemetry flow, only HTTPS (port 8443) was used. In customer environments, Level 4 may route through your own enterprise proxy instead.
44+
> In the validated telemetry flow, only HTTPS (port 8443) is used. In customer environments, Level 4 might route through your own enterprise proxy instead.
4445
4546
## Architecture summary
4647

4748
This deployment aligns with the Purdue Model, implementing a physically segmented, multi-level architecture spanning Levels 2 through 4.
4849

4950
Each level is separated by network firewalls that restrict communication to adjacent layers only (for example, L2 ↔ L3 ↔ L4), ensuring tight segmentation.
50-
Outbound traffic to Azure is routed through an explicit proxy and Private Link (optionally via Arc Gateway), ensuring no internet-exposed endpoints are used at any layer.
51+
Outbound traffic to Azure is routed through an explicit proxy and Private Link (optionally through Arc Gateway), ensuring no internet-exposed endpoints are used at any layer.
5152

5253
:::image type="content" source="media/layered-network-private-connectivity-architecture.png" alt-text="Diagram showing Layered Network Guidance for Azure IoT Operations in segmented industrial-style network environments, with a Purdue model pyramid spanning Levels 2 through 5 on the left and an Azure Arc architecture on the right showing CoreDNS, Envoy, and Azure IoT Operations deployed across Levels 3 and 4.":::
5354

@@ -57,7 +58,7 @@ Outbound traffic to Azure is routed through an explicit proxy and Private Link (
5758
| ----- | ---------- | ------- |
5859
| L2 | CoreDNS, Azure IoT Operations Dataflows, Azure IoT Operations MQTT Broker | Ingests telemetry from OPC UA sources, applies initial enrichment, and forwards data upward |
5960
| L3 | CoreDNS, Envoy Proxy, Azure IoT Operations Dataflows, Azure IoT Operations MQTT Broker | Aggregates and transforms data, resolves DNS to reach L4, and securely forwards telemetry |
60-
| L4 | Envoy Proxy | Forwards enriched telemetry to Event Grid via Azure Firewall Explicit Proxy and Private Endpoint over ExpressRoute |
61+
| L4 | Envoy Proxy | Forwards enriched telemetry to Event Grid through Azure Firewall Explicit Proxy and Private Endpoint over ExpressRoute |
6162

6263
## Prepare your layered network environment
6364

@@ -116,7 +117,7 @@ sudo netplan apply
116117
| ----- | ------- | ---------------- | ---------- | ----- |
117118
| L2 | OPC UA simulator, Azure IoT Operations (MQTT Broker, Dataflows), Arc, CoreDNS | p3tiny-01 | 172.22.232.X | Arc-enabled, Azure IoT Operations deployed |
118119
| L3 | Azure IoT Operations (MQTT Broker, Dataflows), CoreDNS, Arc | p3tiny-02 | 172.22.232.Y | Arc-enabled, Azure IoT Operations deployed |
119-
| L4 | Envoy Proxy (egress only, outbound access) | p3tiny-03 | 172.22.232.Z | Not Arc-enabled, handles egress via Azure Firewall Explicit Proxy over ExpressRoute |
120+
| L4 | Envoy Proxy (egress only, outbound access) | p3tiny-03 | 172.22.232.Z | Not Arc-enabled, handles egress through Azure Firewall Explicit Proxy over ExpressRoute |
120121

121122
### Step 3: Enforce network isolation between layers
122123

@@ -128,7 +129,7 @@ Use firewalls or host-level policies (for example, [UFW](https://help.ubuntu.com
128129
| L3 ↔ L4 | Allow |
129130
| L2 ↔ L4 | Block |
130131
| L2/L3 → Internet | Block |
131-
| L4 → Azure | Allow via Azure Firewall Explicit Proxy over ExpressRoute |
132+
| L4 → Azure | Allow through Azure Firewall Explicit Proxy over ExpressRoute |
132133

133134
Example UFW rules for the L2 host (allow L3 only, deny everything else):
134135

@@ -146,7 +147,7 @@ Repeat with appropriate rules on each host. See [UFW documentation](https://help
146147

147148
### Step 4: Route Azure-bound traffic through L4 only
148149

149-
Only the Level 4 node may initiate outbound traffic, forwarding it to the [Azure Firewall Explicit Proxy](/azure/firewall/explicit-proxy) over ExpressRoute, which then routes it to Azure services via Private Link.
150+
Only the Level 4 node might initiate outbound traffic, forwarding it to the [Azure Firewall Explicit Proxy](/azure/firewall/explicit-proxy) over ExpressRoute, which then routes it to Azure services through Private Link.
150151

151152
1. Deploy Envoy Proxy on the L4 machine. For sample configurations, see [Configure infrastructure](https://github.com/Azure-Samples/explore-iot-operations/blob/main/samples/layered-networking/configure-infrastructure.md).
152153
1. Forward HTTPS traffic to the Azure Firewall Explicit Proxy:
@@ -261,7 +262,7 @@ For the full list of private DNS zone names, see [Azure Private DNS Zone values]
261262
Deploy [CoreDNS](https://coredns.io/) on L2 and L3 to forward private Azure domain queries to `168.63.129.16`, Azure's internal DNS resolver for Private Endpoint domains. For deployment instructions, see [Configure infrastructure](https://github.com/Azure-Samples/explore-iot-operations/blob/main/samples/layered-networking/configure-infrastructure.md).
262263

263264
> [!NOTE]
264-
> `168.63.129.16` is Azure's internal wire server DNS and is only reachable from inside an Azure VNet. In this lab, L4 is a VNet-hosted VM, so CoreDNS queries from L2/L3 are forwarded through Envoy at L3 and L4, ultimately reaching `168.63.129.16` via the L4 VNet.
265+
> `168.63.129.16` is Azure's internal wire server DNS and is only reachable from inside an Azure VNet. In this lab, L4 is a VNet-hosted VM, so CoreDNS queries from L2/L3 are forwarded through Envoy at L3 and L4, ultimately reaching `168.63.129.16` through the L4 VNet.
265266
266267
#### On-premises deployments
267268

@@ -338,7 +339,7 @@ nslookup <eventgrid-namespace>.privatelink.eventgrid.azure.net
338339
nslookup <account>.privatelink.blob.core.windows.net
339340
```
340341

341-
Confirm traffic flows via Envoy and Private Link, not the public internet:
342+
Confirm traffic flows through Envoy and Private Link, not the public internet:
342343

343344
```bash
344345
curl -v https://<eventgrid-namespace>.ts.eventgrid.azure.net
@@ -350,7 +351,7 @@ Verify that:
350351
- Traffic flows through Envoy and Private Link, not public endpoints.
351352

352353
> [!NOTE]
353-
> Arc requires working DNS resolution (via CoreDNS) to complete onboarding.
354+
> Arc requires working DNS resolution (through CoreDNS) to complete onboarding.
354355
355356
## Arc-enable clusters with Arc Gateway (optional)
356357

@@ -443,8 +444,8 @@ Assign these roles to the Azure IoT Operations system-assigned managed identity
443444
| Identity | Role | Scope | Notes |
444445
| -------- | ---- | ----- | ----- |
445446
| L2/L3 Azure IoT Operations system-assigned managed identity | Storage Blob Contributor | Storage account containing schema files | For Schema Registry |
446-
| L3 Azure IoT Operations system-assigned managed identity | EventGrid TopicSpaces Publisher | Event Grid namespace | L3 publishes to Event Grid via L4 Envoy pass-through |
447-
| L3 Azure IoT Operations system-assigned managed identity | EventGrid TopicSpaces Subscriber | Event Grid namespace | L3 subscribes to Event Grid via L4 Envoy pass-through |
447+
| L3 Azure IoT Operations system-assigned managed identity | EventGrid TopicSpaces Publisher | Event Grid namespace | L3 publishes to Event Grid through L4 Envoy pass-through |
448+
| L3 Azure IoT Operations system-assigned managed identity | EventGrid TopicSpaces Subscriber | Event Grid namespace | L3 subscribes to Event Grid through L4 Envoy pass-through |
448449

449450
Assign each role using Azure CLI:
450451

@@ -518,12 +519,12 @@ The end-to-end telemetry flow follows this path:
518519
1. **DNS resolution (L3):** Azure IoT Operations Dataflows and CoreDNS at L3 resolve private service names to reach L4.
519520
1. **Proxy forwarding (L3 to L4):** Envoy Proxy on L3 forwards MQTT traffic to Envoy Proxy on L4.
520521
1. **Egress (L4):** Envoy Proxy on L4 sends traffic to the Azure Firewall Explicit Proxy on port 8443 over ExpressRoute.
521-
1. **Private routing:** The proxy routes requests to Azure services via Private Endpoints.
522+
1. **Private routing:** The proxy routes requests to Azure services through Private Endpoints.
522523
1. **Cloud integration:** Services such as Event Grid Topic Spaces, Azure Storage, and Azure Key Vault are accessed privately using Azure Private Link. Public network access is disabled for all Azure services in the deployment.
523524

524525
### Event Grid topic spaces
525526

526-
Data is published to Event Grid via MQTT over WebSocket (`/mqtt` path suffix). L3's Azure IoT Operations Dataflow authenticates using its system-assigned managed identity (which has EventGrid TopicSpaces Publisher and Subscriber roles) and sends traffic through Envoy Proxy on L3, which forwards to Envoy Proxy on L4. Outbound traffic from Level 4 is routed through the Azure Firewall Explicit Proxy (port 8443), which then forwards to the Event Grid private endpoint over ExpressRoute. Level 4 acts as a pure pass-through and doesn't authenticate.
527+
Data is published to Event Grid through MQTT over WebSocket (`/mqtt` path suffix). L3's Azure IoT Operations Dataflow authenticates using its system-assigned managed identity (which has EventGrid TopicSpaces Publisher and Subscriber roles) and sends traffic through Envoy Proxy on L3, which forwards to Envoy Proxy on L4. Outbound traffic from Level 4 is routed through the Azure Firewall Explicit Proxy (port 8443), which then forwards to the Event Grid private endpoint over ExpressRoute. Level 4 acts as a pure pass-through and doesn't authenticate.
527528

528529
Telemetry is validated against schemas defined in Blob Storage, enforced by the Azure IoT Operations Dataflows running at L2 and L3.
529530

@@ -636,9 +637,10 @@ If any query returns a public IP, check your CoreDNS forwarding rules and Privat
636637

637638
## Known limitations
638639

639-
- **Platform validation:** This scenario was validated on K3s running on Ubuntu Server 24.04 only. Other Kubernetes distributions or operating systems haven't been validated.
640-
- **Proxy support:** Only Azure Firewall Explicit Proxy was validated. Third-party proxies (for example, Palo Alto) or transparent proxies aren't supported in this validated scenario.
641-
- **Schema Registry public access:** Schema Registry may require public access enabled at creation time. After creation, you can disable public access. Use the `--skip-ra` flag when creating the Schema Registry to avoid requiring Owner-level permissions.
640+
For common limitations related to platform validation, proxy support, and Schema Registry, see [Known limitations in Deploy Azure IoT Operations with private connectivity](../manage-layered-network/howto-private-connectivity.md#known-limitations).
641+
642+
The following limitations are specific to the layered network tutorial:
643+
642644
- **Level 1:** The L1 device layer is unused in this deployment flow.
643645
- **Level 4 Arc:** Level 4 is not Arc-enabled; only Envoy Proxy is deployed at this layer.
644646
- **Sovereign clouds:** This scenario was validated in Azure public cloud only. Sovereign cloud environments (for example, Azure Government, Azure China 21Vianet) use different endpoints and Private DNS Zone names and haven't been validated.
@@ -720,7 +722,7 @@ Azure IoT Operations Instances:
720722
```
721723

722724
> [!NOTE]
723-
> In environments using Azure Policy automation, these manual role definitions may not be required for OT teams. Policies can pre-assign Contributor or Storage Blob Data Contributor roles as needed.
725+
> In environments using Azure Policy automation, these manual role definitions might not be required for OT teams. Policies can pre-assign Contributor or Storage Blob Data Contributor roles as needed.
724726
725727
## Related content
726728

articles/iot-operations/manage-layered-network/howto-private-connectivity.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ These scenarios apply to environments with a single Arc-enabled Kubernetes clust
4545

4646
If you don't already have an Arc Gateway resource, create one. You need the gateway resource ID when you connect the cluster in the next section. For creation steps, see [Create the Arc Gateway resource](/azure/azure-arc/kubernetes/arc-gateway-simplify-networking#create-the-arc-gateway-resource).
4747

48-
:::image type="content" source="media/howto-private-connectivity/arc-gateway-portal.jpg" alt-text="Screenshot of the Azure portal showing an Arc Gateway resource with its Gateway URL and resource properties.":::
48+
:::image type="content" source="media/howto-private-connectivity/arc-gateway-portal.png" alt-text="Screenshot of the Azure portal showing an Arc Gateway resource with its Gateway URL and resource properties.":::
4949

5050
> [!NOTE]
5151
> A maximum of five Arc Gateway resources are supported per subscription.
@@ -325,7 +325,7 @@ az connectedk8s connect \
325325
This command configures all Arc traffic to route through the Azure Firewall Explicit Proxy and the Arc Gateway, consolidating ~200+ endpoints to ~9 allowed FQDNs with no public internet exposure.
326326

327327
> [!IMPORTANT]
328-
> Arc agent traffic, including extension installs and container image pulls from MCR (`mcr.microsoft.com`), routes through the proxy automatically because `az connectedk8s connect` injects the proxy environment variables into the Arc agent pods. However, if your container runtime (containerd or CRI-O) pulls images outside of the Arc agent (for example, during node-level kubelet pulls), you may also need to configure proxy settings at the node level. On Ubuntu with systemd, create `/etc/systemd/system/containerd.service.d/http-proxy.conf` with your proxy values, then run `sudo systemctl daemon-reload && sudo systemctl restart containerd`.
328+
> Arc agent traffic, including extension installs and container image pulls from MCR (`mcr.microsoft.com`), routes through the proxy automatically because `az connectedk8s connect` injects the proxy environment variables into the Arc agent pods. However, if your container runtime (containerd or CRI-O) pulls images outside of the Arc agent (for example, during node-level kubelet pulls), you might also need to configure proxy settings at the node level. On Ubuntu with systemd, create `/etc/systemd/system/containerd.service.d/http-proxy.conf` with your proxy values, then run `sudo systemctl daemon-reload && sudo systemctl restart containerd`.
329329
330330
> [!TIP]
331331
> **For existing Arc-enabled clusters:** If your cluster is already connected to Azure Arc, use `az connectedk8s update` instead of `az connectedk8s connect`:
@@ -685,7 +685,7 @@ Then check the dataflow is working:
685685
1. Navigate to your Event Grid namespace in the Azure portal.
686686
1. Check **Metrics** for incoming MQTT messages.
687687

688-
:::image type="content" source="media/howto-private-connectivity/event-grid-messaging-metrics.jpg" alt-text="Screenshot of Event Grid namespace metrics showing successful MQTT published messages.":::
688+
:::image type="content" source="media/howto-private-connectivity/event-grid-messaging-metrics.png" alt-text="Screenshot of Event Grid namespace metrics showing successful MQTT published messages.":::
689689

690690
1. Verify the dataflow pod logs show successful message delivery:
691691

@@ -700,7 +700,7 @@ After disabling public access on any Azure resource, verify Azure IoT Operations
700700
## Known limitations
701701

702702
- **Platform validation:** The private connectivity patterns described here are based on validated K3s on Ubuntu Server 24.04 scenarios. Other Kubernetes distributions or operating systems haven't been independently validated.
703-
- **Schema Registry creation:** Schema Registry may require public access enabled at creation time. After creation, you can disable public access and rely on Private Endpoints plus trusted service bypass. Use the `--skip-ra` flag when creating the Schema Registry to avoid requiring Owner-level permissions.
703+
- **Schema Registry creation:** Schema Registry might require public access enabled at creation time. After creation, you can disable public access and rely on Private Endpoints plus trusted service bypass. Use the `--skip-ra` flag when creating the Schema Registry to avoid requiring Owner-level permissions.
704704
- **TLS inspection:** Arc Gateway doesn't support TLS termination or inspection. If your firewall performs TLS inspection, you must exclude the Arc Gateway endpoint from inspection. See [Arc Gateway and TLS inspection](/azure/azure-arc/kubernetes/arc-gateway-simplify-networking#azure-arc-gateway-and-tls-inspection).
705705
- **Arc Gateway limits:** A maximum of five Arc Gateway resources are supported per subscription.
706706
- **Explicit Proxy:** Only Azure Firewall Explicit Proxy has been validated. Third-party proxies (for example, Palo Alto) or transparent proxies aren't supported in validated scenarios. Azure IoT Operations doesn't support proxy servers that require a trusted certificate.
321 KB
Loading
200 KB
Loading

0 commit comments

Comments
 (0)