Skip to content

Commit 989a530

Browse files
authored
Merge pull request #54252 from weslbo/refresh-update
Improve clarity and accuracy of Azure Stream Analytics module
2 parents 03a1d45 + 27d5ee9 commit 989a530

16 files changed

Lines changed: 123 additions & 96 deletions

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/1-introduction.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Introduction
44
metadata:
55
title: Introduction
66
description: "Introduction"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/2-stream-ingestion-scenarios.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Stream ingestion scenarios
44
metadata:
55
title: Stream Ingestion Scenarios
66
description: "Stream ingestion scenarios"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/3-configure-inputs-outputs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Configure inputs and outputs
44
metadata:
55
title: Configure Inputs and Outputs
66
description: "Configure inputs and outputs"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/4-define-query.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Define a query to select, filter, and aggregate data
44
metadata:
55
title: Define a Query to Select, Filter, and Aggregate Data
66
description: "Define a query to select, filter, and aggregate data"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/5-run-job-ingest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Run a job to ingest data
44
metadata:
55
title: Run a Job to Ingest Data
66
description: "Run a job to ingest data"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/6-exercise-ingest-streaming-data.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Exercise - Ingest streaming data into Azure Synapse Analytics
44
metadata:
55
title: Exercise - Ingest Streaming Data Into Azure Synapse Analytics
66
description: "Exercise - Ingest streaming data into Azure Synapse Analytics"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/7-knowledge-check.yml

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,44 @@ title: Module assessment
44
metadata:
55
title: Module Assessment
66
description: "Knowledge check"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit
1111
module_assessment: true
1212
durationInMinutes: 3
1313
quiz:
1414
questions:
15-
- content: "Which type of output should you use to ingest the results of an Azure Stream Analytics job into a dedicated SQL pool table in Azure Synapse Analytics?"
15+
- content: "Which Azure Stream Analytics window type groups events based on periods of inactivity between consecutive events?"
1616
choices:
17-
- content: "Azure Synapse Analytics"
18-
isCorrect: true
19-
explanation: "Correct. An Azure Synapse Analytics output writes data to a table in an Azure Synapse Analytics dedicated SQL pool."
20-
- content: "Blob storage/ADLS Gen2"
17+
- content: "Tumbling"
2118
isCorrect: false
22-
explanation: "Incorrect. A Blob storage/ADLS Gen2 output does not write data to a relational table in a dedicated SQL pool."
23-
- content: "Azure Event Hubs"
19+
explanation: "Incorrect. A tumbling window groups events into fixed-size, nonoverlapping intervals regardless of gaps between events."
20+
- content: "Session"
21+
isCorrect: true
22+
explanation: "Correct. A session window groups events that arrive within a configurable timeout of each other, creating variable-length windows bounded by inactivity gaps."
23+
- content: "Snapshot"
2424
isCorrect: false
25-
explanation: "Incorrect. An Azure Event Hubs output does not write data to a relational table in a dedicated SQL pool."
26-
- content: "Which type of output should be used to ingest the results of an Azure Stream Analytics job into files in a data lake for analysis in Azure Synapse Analytics?"
25+
explanation: "Incorrect. A snapshot window groups events that share the same timestamp using System.Timestamp()."
26+
- content: "You need to continuously write processed stream events to files in a data lake for later batch analytics. Which output type should you configure?"
2727
choices:
28-
- content: "Azure Synapse Analytics"
28+
- content: "Azure SQL Database"
2929
isCorrect: false
30-
explanation: "Incorrect. An Azure Synapse Analytics output does not write data to files in a data lake."
30+
explanation: "Incorrect. An Azure SQL Database output writes to a relational table, not to files in a data lake."
3131
- content: "Blob storage/ADLS Gen2"
3232
isCorrect: true
33-
explanation: "Correct. A Blob storage/ADLS Gen2 output writes data to files in a data lake."
33+
explanation: "Correct. A Blob storage/ADLS Gen2 output writes data to files in Azure Data Lake Storage Gen2, which is suitable for batch analytics workloads."
34+
- content: "Power BI"
35+
isCorrect: false
36+
explanation: "Incorrect. A Power BI output writes to a streaming dataset for near real-time visualization, not to file-based storage."
37+
- content: "You want to forward enriched events from a Stream Analytics job to a downstream application via a message hub. Which output type should you use?"
38+
choices:
3439
- content: "Azure Event Hubs"
40+
isCorrect: true
41+
explanation: "Correct. An Azure Event Hubs output forwards events to an event hub, enabling downstream consumers such as other jobs, functions, or applications to receive the enriched stream."
42+
- content: "Azure SQL Database"
43+
isCorrect: false
44+
explanation: "Incorrect. An Azure SQL Database output writes to a relational table, not to a message hub."
45+
- content: "Blob storage/ADLS Gen2"
3546
isCorrect: false
36-
explanation: "Incorrect. An Azure Event Hubs output does not write data to files in a data lake."
47+
explanation: "Incorrect. A Blob storage/ADLS Gen2 output writes to files in a data lake, not to a message hub for real-time event forwarding."

learn-pr/wwl-data-ai/ingest-streaming-data-use-azure-stream-analytics-synapse/8-summary.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ title: Summary
44
metadata:
55
title: Summary
66
description: "Summary"
7-
ms.date: 08/21/2025
7+
ms.date: 04/15/2026
88
author: weslbo
99
ms.author: wedebols
1010
ms.topic: unit
Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11

2-
Suppose a retail company captures real-time sales transaction data from an e-commerce website, and wants to analyze this data along with more static data related to products, customers, and employees. A common way to approach this problem is to ingest the stream of real-time data into a data lake or data warehouse, where it can be queried together with data that is loaded using batch processing techniques.
2+
Suppose a manufacturing company captures real-time telemetry data from factory floor sensors, and wants to monitor equipment performance, detect anomalies, and archive event data for long-term analysis. A common approach is to use a stream processing engine to continuously filter and aggregate the flow of sensor events, and route the results to one or more destinations—such as a data lake for storage, a relational database for operational reporting, or a message hub for downstream alerting systems.
33

4-
Microsoft Azure Synapse Analytics provides a comprehensive enterprise data analytics platform, into which real-time data captured in Azure Event Hubs or Azure IoT Hub, and processed by Azure Stream Analytics can be loaded.
4+
Azure Stream Analytics is a fully managed, cloud-based stream processing service that enables you to build real-time analytics pipelines. It connects to streaming data sources such as Azure Event Hubs, Azure IoT Hub, and Azure Data Lake Storage, processes data using a SQL-based query language, and writes results to a wide range of output destinations.
55

6-
![A diagram of a data stream in Azure Event Hubs being queried by Azure Stream Analytics and loaded into Azure Synapse Analytics.](../media/stream-ingestion.png)
6+
![A diagram of a data stream in Azure Event Hubs being queried by Azure Stream Analytics and loaded into multiple output destinations.](../media/stream-ingestion.png)
77

8-
A typical pattern for real-time data ingestion in Azure consists of the following sequence of service integrations:
8+
A typical pattern for real-time data processing in Azure consists of the following sequence:
99

1010
1. A real-time source of data is captured in an event ingestor, such as Azure Event Hubs or Azure IoT Hub.
11-
2. The captured data is perpetually filtered and aggregated by an Azure Stream Analytics query.
12-
3. The results of the query are loaded into a data lake or data warehouse in Azure Synapse Analytics for subsequent analysis.
11+
2. The captured data is perpetually filtered, aggregated, or enriched by an Azure Stream Analytics query.
12+
3. The results of the query are written to one or more output destinations—such as a data lake, a relational database, another event hub, or a real-time dashboard.
1313

14-
In this module, you'll explore multiple ways in which you can use Azure Stream Analytics to ingest real-time data into Azure Synapse Analytics.
14+
In this module, you'll learn how to configure Azure Stream Analytics jobs to process streaming data and route the results to a variety of output destinations.
Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,22 @@
11

2-
Azure Synapse Analytics provides multiple ways to analyze large volumes of data. Two of the most common approaches to large-scale data analytics are:
2+
Azure Stream Analytics can route the results of stream processing to multiple types of output destinations, depending on whether you need to store, analyze, forward, or visualize the data.
33

4-
- **Data warehouses** - relational databases, optimized for distributed storage and query processing. Data is stored in tables and queried using SQL.
5-
- **Data lakes** - distributed file storage in which data is stored as files that can be processed and queried using multiple runtimes, including Apache Spark and SQL.
4+
## Data lake storage
65

7-
## Data warehouses in Azure Synapse Analytics
6+
A common use case is to write stream processing results to a data lake hosted in Azure Data Lake Storage Gen2. Data stored in a data lake can later be processed and queried using batch analytics tools such as Apache Spark or serverless SQL engines. This approach is well suited to scenarios where you want to retain raw or lightly processed event data for historical analysis, compliance, or machine learning workloads.
87

9-
Azure Synapse Analytics provides dedicated SQL pools that you can use to implement enterprise-scale relational data warehouses. Dedicated SQL pools are based on a *massively parallel processing* (MPP) instance of the Microsoft SQL Server relational database engine in which data is stored and queried in tables.
8+
![A diagram of a stream of data being ingested into an Azure Storage data lake.](../media/data-lake.png)
109

11-
To ingest real-time data into a relational data warehouse, your Azure Stream Analytics query must write its results to an output that references the table into which you want to load the data.
10+
## Relational database storage
1211

13-
![A diagram of a stream of data being ingested into a dedicated SQL pool in Azure Synapse Analytics.](../media/data-warehouse.png)
12+
When streaming results need to be available to applications or reporting tools that rely on relational data, you can write the output of a Stream Analytics job to a table in Azure SQL Database or Azure Synapse Analytics dedicated SQL pool. This approach enables dashboards and reports to query the most recently ingested data using standard SQL.
1413

15-
## Data lakes in Azure Synapse Analytics
14+
![A diagram of a stream of data being ingested into a relational database.](../media/data-warehouse.png)
1615

17-
An Azure Synapse Analytics workspace typically includes at least one storage service that is used as a data lake. Most commonly, the data lake is hosted in an Azure Storage account using a container configured to support Azure Data Lake Storage Gen2. Files in the data lake are organized hierarchically in directories (folders), and can be stored in multiple file formats, including delimited text (such as comma-separated values, or CSV), Parquet, and JSON.
16+
## Real-time dashboards
1817

19-
When ingesting real-time data into a data lake, your Azure Stream Analytics query must write its results to an output that references the location in the Azure Data Lake Gen2 storage container where you want to save the data files. Data analysts, engineers, and scientists can then process and query the files in the data lake by running code in an Apache Spark pool, or by running SQL queries using a serverless SQL pool.
18+
For scenarios that require live visualization of streaming metrics—such as monitoring sensor readings or tracking website activity in real time—Azure Stream Analytics can write output directly to a Power BI streaming dataset. Power BI then renders the data in near real time without requiring a scheduled data refresh.
2019

21-
![A diagram of a stream of data being ingested into an Azure Storage data lake and queried in Azure Synapse Analytics.](../media/data-lake.png)
20+
## Event forwarding
21+
22+
Azure Stream Analytics can also write filtered or enriched events to another Azure Event Hubs instance. This pattern is used to build multi-stage streaming pipelines, where one Stream Analytics job performs initial filtering or enrichment and forwards the results to a downstream consumer such as another job, an Azure Function, or a custom application.

0 commit comments

Comments
 (0)