Skip to content

Commit a182001

Browse files
spelluruspelluru
committed
Azure Stream Analytics - Freshness Review
1 parent 6079353 commit a182001

4 files changed

Lines changed: 174 additions & 152 deletions

File tree

‎articles/stream-analytics/set-up-cicd-pipeline.md‎

Lines changed: 66 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
11
---
2-
title: Use Azure DevOps to create a CI/CD pipeline for a Stream Analytics job
3-
description: This article describes how to set up a continuous integration and deployment (CI/CD) pipeline for an Azure Stream Analytics job in Azure DevOps
2+
title: Azure DevOps CI/CD Pipeline for Stream Analytics Jobs
3+
description: Learn how to set up a CI/CD pipeline for Azure Stream Analytics jobs in Azure DevOps. Follow step-by-step instructions to automate builds, tests, and deployments.
4+
#customer intent: As a developer, I want to set up a CI/CD pipeline for an Azure Stream Analytics job in Azure DevOps so that I can automate the build and deployment process.
45
author: alexlzx
56
ms.author: zhenxilin
67
ms.service: azure-stream-analytics
78
ms.topic: how-to
8-
ms.date: 12/17/2024
9+
ms.date: 03/05/2026
910
# Customer intent: I want to know how to set up a CI/CD pipeline for an Azure Stream Analytics job in Azure DevOps.
1011
---
1112

1213
# Use Azure DevOps to create a CI/CD pipeline for a Stream Analytics job
1314

14-
In this article, you learn how to create Azure DevOps [build](/azure/devops/pipelines/get-started/pipelines-get-started) and [release](/azure/devops/pipelines/release/define-multistage-release-process) pipelines using Azure Stream Analytics CI/CD tools.
15+
In this article, you learn how to create Azure DevOps [build](/azure/devops/pipelines/get-started/pipelines-get-started) and [release](/azure/devops/pipelines/release/define-multistage-release-process) pipelines by using Azure Stream Analytics CI/CD tools.
1516

1617
## Commit your Stream Analytics project
1718

@@ -23,51 +24,51 @@ The steps in this article use a Stream Analytics Visual Studio Code project. If
2324

2425
In this section, you learn how to create a build pipeline.
2526

26-
1. Open a web browser and navigate to your project in Azure DevOps.
27+
1. Open a web browser and go to your project in Azure DevOps.
2728

2829
2. Under **Pipelines** in the left navigation menu, select **Builds**. Then, select **New pipeline**.
2930

30-
:::image type="content" source="media/set-up-cicd-pipeline/new-pipeline.png" alt-text="Create new Azure Pipeline":::
31+
:::image type="content" source="media/set-up-cicd-pipeline/new-pipeline.png" alt-text="Screenshot of Azure Pipelines Builds page showing no build pipelines found and a New pipeline button." lightbox="media/set-up-cicd-pipeline/new-pipeline.png":::
3132

3233
3. Select **Use the classic editor** to create a pipeline without YAML.
3334

3435
4. Select your source type, team project, and repository. Then, select **Continue**.
3536

36-
:::image type="content" source="media/set-up-cicd-pipeline/select-repo.png" alt-text="Select Azure Stream Analytics project":::
37+
:::image type="content" source="media/set-up-cicd-pipeline/select-repo.png" alt-text="Screenshot of Azure Pipelines source selection page with Azure Repos Git, GitHub, and other options, and repository, branch, and continue button visible." lightbox="media/set-up-cicd-pipeline/select-repo.png":::
3738

3839
5. On the **Choose a template** page, select **Empty job**.
3940

4041
## Install npm package
4142

42-
1. On the **Tasks** page, select the plus sign next to **Agent job 1**. Enter *npm* in the task search and select **npm**.
43+
1. On the **Tasks** page, select the plus sign next to **Agent job 1**. Enter *npm* in the task search, and select **npm**.
4344

44-
:::image type="content" source="media/set-up-cicd-pipeline/search-npm.png" alt-text="Select npm task":::
45+
:::image type="content" source="media/set-up-cicd-pipeline/search-npm.png" alt-text="Screenshot of Azure Pipelines Tasks page with Agent job 1 selected, npm searched, and npm task options displayed to the right." lightbox="media/set-up-cicd-pipeline/search-npm.png":::
4546

46-
2. Give the task a **Display name**. Change the **Command** option to *custom* and enter the following command in **Command and arguments**. Leave the remaining default options.
47+
2. Give the task a **Display name**. Change the **Command** option to *custom* and enter the following command in **Command and arguments**. Don't change the other default options.
4748

4849
```bash
4950
install -g azure-streamanalytics-cicd
5051
```
5152

52-
:::image type="content" source="media/set-up-cicd-pipeline/npm-config.png" alt-text="Enter configurations for npm task":::
53+
:::image type="content" source="media/set-up-cicd-pipeline/npm-config.png" alt-text="Screenshot of Azure Pipelines task editor showing npm task with custom command to install azure-streamanalytics-cicd package." lightbox="media/set-up-cicd-pipeline/npm-config.png":::
5354

54-
Use following steps if you need to use hosted-Linux agent:
55-
1. Select your **Agent Specification**
55+
Use the following steps if you need to use a hosted Linux agent:
56+
1. Select your **Agent Specification**.
5657

57-
:::image type="content" source="media/set-up-cicd-pipeline/select-linux-agent.png" alt-text="Screenshot of selecting agent specification.":::
58+
:::image type="content" source="media/set-up-cicd-pipeline/select-linux-agent.png" alt-text="Screenshot of Azure Pipelines Tasks page showing agent pool set to Azure Pipelines and agent specification set to ubuntu-16.04." lightbox="media/set-up-cicd-pipeline/select-linux-agent.png":::
5859

59-
2. On the **Tasks** page, select the plus sign next to **Agent job 1**. Enter *command line* in the task search and select **Command line**.
60+
2. On the **Tasks** page, select the plus sign next to **Agent job 1**. Enter *command line* in the task search, and select **Command line**.
6061

61-
:::image type="content" source="media/set-up-cicd-pipeline/cmd-search.png" alt-text="Screenshot of searching commandline task. ":::
62+
:::image type="content" source="media/set-up-cicd-pipeline/cmd-search.png" alt-text="Screenshot of searching commandline task. " lightbox="media/set-up-cicd-pipeline/cmd-search.png":::
6263

63-
3. Give the task a **Display name**. enter the following command in **Script**. Leave the remaining default options.
64+
3. Give the task a **Display name**. Enter the following command in **Script**. Don't change the other default options.
6465

6566
```bash
6667
sudo npm install -g azure-streamanalytics-cicd --unsafe-perm=true --allow-root
6768
```
68-
:::image type="content" source="media/set-up-cicd-pipeline/cmd-scripts.png" alt-text="Screenshot of entering script for cmd task.":::
69+
:::image type="content" source="media/set-up-cicd-pipeline/cmd-scripts.png" alt-text="Screenshot of entering script for cmd task." lightbox="media/set-up-cicd-pipeline/cmd-scripts.png":::
6970

70-
## Add a Build task
71+
## Add a build task
7172

7273
1. On the **Variables** page, select **+ Add** in **Pipeline variables**. Add the following variables. Set the following values according to your preference:
7374

@@ -82,17 +83,17 @@ Use following steps if you need to use hosted-Linux agent:
8283
3. Give the task a **Display name** and enter the following script. Modify the script with your repository name and project name.
8384

8485
> [!NOTE]
85-
> It's highly recommended to use the `build --v2` to generate ARM template for deployment. The new ARM template has fewer parameters while preserving the same functionality as the previous version.
86+
> It's highly recommended to use the `build --v2` command to generate the ARM template for deployment. The new ARM template has fewer parameters while preserving the same functionality as the previous version.
8687
>
87-
> Please note that the older ARM template will soon be deprecated, only templates created using `build --v2` will receive updates and bug fixes.
88+
> The older ARM template will soon be deprecated. Only templates created by using `build --v2` receive updates and bug fixes.
8889
8990
```bash
9091
azure-streamanalytics-cicd build --v2 -project $(projectRootPath)/asaproj.json -outputpath $(projectRootPath)/$(outputPath)/$(deployPath)
9192
```
9293
93-
The image uses a Stream Analytics Visual Studio Code project as an example.
94+
The following image uses a Stream Analytics Visual Studio Code project as an example.
9495
95-
:::image type="content" source="media/set-up-cicd-pipeline/command-line-config-build.png" alt-text="Enter configurations for command-line task visual studio code":::
96+
:::image type="content" source="media/set-up-cicd-pipeline/command-line-config-build.png" alt-text="Screenshot of Azure DevOps pipeline editor showing a command line task to build a Stream Analytics Visual Studio Code project.":::
9697
9798
## Add a Test task
9899
@@ -102,7 +103,7 @@ Use following steps if you need to use hosted-Linux agent:
102103
|-|-|
103104
|testPath|Test|
104105
105-
:::image type="content" source="media/set-up-cicd-pipeline/pipeline-variables-test.png" alt-text="Add pipeline variables":::
106+
:::image type="content" source="media/set-up-cicd-pipeline/pipeline-variables-test.png" alt-text="Screenshot of Azure Pipelines Variables page showing a list of pipeline variables and values, with the Add button visible." lightbox="media/set-up-cicd-pipeline/pipeline-variables-test.png":::
106107
107108
2. On the **Tasks** page, select the plus sign next to **Agent job 1**. Search for **Command line**.
108109
@@ -114,58 +115,58 @@ Use following steps if you need to use hosted-Linux agent:
114115
azure-streamanalytics-cicd test -project $(projectRootPath)/asaproj.json -outputpath $(projectRootPath)/$(outputPath)/$(testPath) -testConfigPath $(projectRootPath)/test/testConfig.json
115116
```
116117
117-
:::image type="content" source="media/set-up-cicd-pipeline/command-line-config-test.png" alt-text="Enter configurations for command-line task":::
118+
:::image type="content" source="media/set-up-cicd-pipeline/command-line-config-test.png" alt-text="Screenshot of Azure Pipelines task configuration." lightbox="media/set-up-cicd-pipeline/command-line-config-test.png":::
118119
119120
## Add a Copy files task
120121
121-
You need to add a copy file task to copy the test summary file and Azure Resource Manager template files to the artifact folder.
122+
Add a copy file task to copy the test summary file and Azure Resource Manager template files to the artifact folder.
122123
123124
1. On the **Tasks** page, select the **+** next to **Agent job 1**. Search for **Copy files**. Then enter the following configurations. By assigning `**` to **Contents**, all files of the test results are copied.
124125
125-
|Parameter|Input|
126+
| Parameter | Input |
126127
|-|-|
127-
|Display name|Copy Files to: $(build.artifactstagingdirectory)|
128-
|Source Folder|`$(system.defaultworkingdirectory)/$(outputPath)/`|
129-
|Contents| `**` |
130-
|Target Folder| `$(build.artifactstagingdirectory)`|
128+
| Display name | Copy Files to: $(build.artifactstagingdirectory) |
129+
| Source Folder | `$(system.defaultworkingdirectory)/$(outputPath)/` |
130+
| Contents | `**` |
131+
| Target Folder | `$(build.artifactstagingdirectory)` |
131132
132133
2. Expand **Control Options**. Select **Even if a previous task has failed, unless the build was canceled** in **Run this task**.
133134
134-
:::image type="content" source="media/set-up-cicd-pipeline/copy-config.png" alt-text="Enter configurations for copy task":::
135+
:::image type="content" source="media/set-up-cicd-pipeline/copy-config.png" alt-text="Screenshot of Azure DevOps pipeline task settings showing Copy Files step with source, contents, and target folder fields filled." lightbox="media/set-up-cicd-pipeline/copy-config.png":::
135136
136137
## Add a Publish build artifacts task
137138
138139
1. On the **Tasks** page, select the plus sign next to **Agent job 1**. Search for **Publish build artifacts** and select the option with the black arrow icon.
139140
140141
2. Expand **Control Options**. Select **Even if a previous task has failed, unless the build was canceled** in **Run this task**.
141142
142-
:::image type="content" source="media/set-up-cicd-pipeline/publish-config.png" alt-text="Enter configurations for publish task":::
143+
:::image type="content" source="media/set-up-cicd-pipeline/publish-config.png" alt-text="Screenshot of Azure Pipelines task editor showing Publish build artifacts settings with Control Options expanded." lightbox="media/set-up-cicd-pipeline/publish-config.png":::
143144
144145
## Save and run
145146
146-
Once you have finished adding the npm package, command line, copy files, and publish build artifacts tasks, select **Save & queue**. When you're prompted, enter a save comment and select **Save and run**. You can download the testing results from **Summary** page of the pipeline.
147+
After you finish adding the npm package, command line, copy files, and publish build artifacts tasks, select **Save & queue**. When prompted, enter a save comment and select **Save and run**. You can download the testing results from **Summary** page of the pipeline.
147148
148149
## Check the build and test results
149150
150-
The test summary file and Azure Resource Manager Template files can be found in **Published** folder.
151+
You can find the test summary file and Azure Resource Manager template files in the **Published** folder.
151152
152-
:::image type="content" source="media/set-up-cicd-pipeline/check-build-test-result.png" alt-text="Check build and test result":::
153+
:::image type="content" source="media/set-up-cicd-pipeline/check-build-test-result.png" alt-text="Screenshot of Azure Pipelines summary page showing manual run details, repository info, elapsed time, and a published test result link." lightbox="media/set-up-cicd-pipeline/check-build-test-result.png":::
153154
154-
:::image type="content" source="media/set-up-cicd-pipeline/check-drop-folder.png" alt-text="Check artifacts":::
155+
:::image type="content" source="media/set-up-cicd-pipeline/check-drop-folder.png" alt-text="Screenshot of Azure Artifacts page showing the Published tab with folders and files for build and test results." lightbox="media/set-up-cicd-pipeline/check-drop-folder.png":::
155156
156157
## Release with Azure Pipelines
157158
158159
In this section, you learn how to create a release pipeline.
159160
160-
Open a web browser and navigate to your Azure Stream Analytics Visual Studio Code project.
161+
Open a web browser and go to your Azure Stream Analytics Visual Studio Code project.
161162
162163
1. Under **Pipelines** in the left navigation menu, select **Releases**. Then select **New pipeline**.
163164
164165
2. Select **start with an Empty job**.
165166
166167
3. In the **Artifacts** box, select **+ Add an artifact**. Under **Source**, select the build pipeline you created and select **Add**.
167168
168-
:::image type="content" source="media/set-up-cicd-pipeline/build-artifact.png" alt-text="Enter build pipeline artifact":::
169+
:::image type="content" source="media/set-up-cicd-pipeline/build-artifact.png" alt-text="Screenshot of Add an artifact dialog in Azure Pipelines, showing Build source type selected and build pipeline fields completed." lightbox="media/set-up-cicd-pipeline/build-artifact.png":::
169170
170171
4. Change the name of **Stage 1** to **Deploy job to test environment**.
171172
@@ -174,49 +175,49 @@ Open a web browser and navigate to your Azure Stream Analytics Visual Studio Cod
174175
### Add deploy tasks
175176
176177
> [!NOTE]
177-
> The `Override template parameters` is not applicable for ARM v2 builds since parameters are passed as objects. To address this, it's recommended to include a PowerShell script in your pipeline to read the parameter file as JSON and make the necessary parameter modifications.
178+
> The **Override template parameters** option doesn't apply to Azure Resource Manager v2 builds because the process passes parameters as objects. To work around this limitation, add a PowerShell script to your pipeline that reads the parameter file as JSON and makes the necessary parameter modifications.
178179
>
179-
> For more guidance on adding the PowerShell script, please refer to [ConvertFrom-Json](/powershell/module/microsoft.powershell.utility/convertfrom-json) and [Update Object in JSON file](https://stackoverflow.com/questions/65753594/update-object-in-json-file-using-powershell).
180+
> For more information on adding the PowerShell script, see [ConvertFrom-Json](/powershell/module/microsoft.powershell.utility/convertfrom-json) and [Update Object in JSON file](https://stackoverflow.com/questions/65753594/update-object-in-json-file-using-powershell).
180181

181182
1. From the tasks dropdown, select **Deploy job to test environment**.
182183

183184
2. Select the **+** next to **Agent job** and search for **ARM template deployment**. Enter the following parameters:
184185

185-
|Parameter|Value|
186+
| Parameter | Value |
186187
|-|-|
187-
|Display name| *Deploy myASAProject*|
188-
|Azure subscription| Choose your subscription.|
189-
|Action| *Create or update resource group*|
190-
|Resource group| Choose a name for the test resource group that will contain your Stream Analytics job.|
191-
|Location|Choose the location of your test resource group.|
192-
|Template location| Linked artifact|
193-
|Template| `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.json` |
194-
|Template parameters|`$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.parameters.json` |
195-
|Override template parameters|`-<arm_template_parameter> "your value"`. You can define the parameters using **Variables**.|
196-
|Deployment mode|Incremental|
188+
| Display name | *Deploy myASAProject* |
189+
| Azure subscription | Choose your subscription. |
190+
| Action | *Create or update resource group* |
191+
| Resource group | Choose a name for the test resource group that contains your Stream Analytics job. |
192+
| Location | Choose the location of your test resource group. |
193+
| Template location | Linked artifact |
194+
| Template | `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.json` |
195+
| Template parameters | `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.parameters.json` |
196+
| Override template parameters | `-<arm_template_parameter> "your value"`. Define the parameters by using **Variables**. |
197+
| Deployment mode | Incremental |
197198

198199
3. From the tasks dropdown, select **Deploy job to production environment**.
199200

200201
4. Select the **+** next to **Agent job** and search for *ARM template deployment*. Enter the following parameters:
201202

202-
|Parameter|Value|
203+
| Parameter | Value |
203204
|-|-|
204-
|Display name| *Deploy myASAProject*|
205-
|Azure subscription| Choose your subscription.|
206-
|Action| *Create or update resource group*|
207-
|Resource group| Choose a name for the production resource group that will contain your Stream Analytics job.|
208-
|Location|Choose the location of your production resource group.|
209-
|Template location| *Linked artifact*|
210-
|Template| `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.json` |
211-
|Template parameters|`$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.parameters.json` |
212-
|Override template parameters|`-<arm_template_parameter> "your value"`|
213-
|Deployment mode|Incremental|
205+
| Display name | *Deploy myASAProject* |
206+
| Azure subscription | Choose your subscription. |
207+
| Action | *Create or update resource group* |
208+
| Resource group | Choose a name for the production resource group that contains your Stream Analytics job. |
209+
| Location | Choose the location of your production resource group. |
210+
| Template location | *Linked artifact* |
211+
| Template | `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.json` |
212+
| Template parameters | `$(System.DefaultWorkingDirectory)/_azure-streamanalytics-cicd-demo-CI-Deploy/drop/myASAProject.JobTemplate.parameters.json` |
213+
| Override template parameters | `-<arm_template_parameter> "your value"` |
214+
| Deployment mode | Incremental |
214215

215216
### Create a release
216217

217-
To create a release, select **Create release** in the top-right corner.
218+
To create a release, select **Create release** in the upper-right corner.
218219

219-
:::image type="content" source="media/set-up-cicd-pipeline/create-release.png" alt-text="Create a release using Azure Pipelines":::
220+
:::image type="content" source="media/set-up-cicd-pipeline/create-release.png" alt-text="Screenshot of Azure Pipelines release pipeline with artifacts, stages, and the Create release button highlighted." lightbox="media/set-up-cicd-pipeline/create-release.png":::
220221

221222
## Related content
222223

0 commit comments

Comments
 (0)