You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/app-testing/load-testing/how-to-compare-multiple-test-runs.md
+9-7Lines changed: 9 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,13 +12,13 @@ ms.topic: how-to
12
12
13
13
# Compare load test runs in Azure Load Testing
14
14
15
-
In this article, you learn how you can compare test runs in Azure Load Testing. You can view trends across the last 10 test runs, or you can select and compare up to ten individual test runs. Optionally, you can mark a test run as a baseline to compare against.
15
+
In this article, you learn how you can compare test runs in Azure Load Testing. You can view trends across the last few test runs, or you can select and compare up to 10 individual test runs. Optionally, you can mark a test run as a baseline to compare against.
16
16
17
-
To identify regressions over time, you can use the client-side metrics trends of the last 10 test runs, such as the response time, error rate, and more. In combination with [CI/CD integration](./quickstart-add-load-test-cicd.md), the trends data might help you identify which application build introduced a performance issue.
17
+
To identify regressions over time, you can use the client-side metrics trends of the previous test runs, such as the response time, error rate, and more. In combination with [CI/CD integration](./quickstart-add-load-test-cicd.md), the trends data might help you identify which application build introduced a performance issue.
18
18
19
19
When you want to compare the client-side metrics trends against a specific reference test run, you can mark that test run as your baseline. For example, before you implement performance optimizations in your application, you might first create a baseline load test run, and then validate the effects of your optimizations against your baseline.
20
20
21
-
To compare both client-side and server-side metrics, you can select up to ten test runs, and compare them in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each test run. By also comparing server-side application metrics in the dashboard, you can identify which application component was the root cause for a sudden performance degradation.
21
+
To compare both client-side and server-side metrics, you can select up to 10 test runs, and compare them in the Azure Load Testing dashboard. The dashboard overlays the client-side and server-side metric graphs for each test run. By also comparing server-side application metrics in the dashboard, you can identify which application component was the root cause for a sudden performance degradation.
22
22
23
23
## Prerequisites
24
24
@@ -28,9 +28,9 @@ To compare both client-side and server-side metrics, you can select up to ten te
28
28
29
29
## Compare multiple load test runs
30
30
31
-
To compare test runs in Azure Load Testing, you first have to select up to ten runs within a load test. You can only compare runs that belong to the same load test. After you select the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
31
+
To compare test runs in Azure Load Testing, you first have to select up to 10 runs. These test runs can be from the same load test or from another load test. After you select the test runs you want to compare, you can visually compare the client-side and server-side metrics for each test run in the load test dashboard.
32
32
33
-
A test run needs to be in the *Done*, *Stopped*, or *Failed* state to compare it.
33
+
A test run needs to be in the *Passed*, *Failed*, *Completed*, or *Stopped* state to compare it.
34
34
35
35
Use the following steps to select the test runs:
36
36
@@ -45,7 +45,7 @@ Use the following steps to select the test runs:
45
45
46
46
1. Select two or more test runs, and then select **Compare** to compare test runs.
47
47
48
-
You can choose a maximum of ten test runs to compare.
48
+
You can choose a maximum of 10 test runs to compare.
49
49
50
50
:::image type="content" source="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png" alt-text="Screenshot that shows a list of test runs and the 'Compare' button in the Azure portal." lightbox="media/how-to-compare-multiple-test-runs/compare-test-results-from-list.png":::
51
51
@@ -58,9 +58,11 @@ Use the following steps to select the test runs:
58
58
59
59
:::image type="content" source="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png" alt-text="Screenshot of the load testing dashboard in the Azure portal, comparing two test runs." lightbox="media/how-to-compare-multiple-test-runs/load-test-dashboard-compare-runs.png":::
60
60
61
+
1. Optionally, you can change the test runs selected for comparison by selecting 'Update selection'. You can select test runs from the same load test or from a different one.
62
+
61
63
## View metrics trends across load test runs
62
64
63
-
To view metrics trends across test runs in Azure Load Testing, you need to have at least two test runs in the *Done*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
65
+
To view metrics trends across test runs in Azure Load Testing, you need to have at least two test runs in *Passed*, *Failed*, *Completed*, or *Stopped* state. You can only view trends from runs that belong to the same load test.
64
66
65
67
Use the following steps to view metrics trends across test runs:
Copy file name to clipboardExpand all lines: articles/app-testing/load-testing/how-to-diagnose-failing-load-test.md
+9-12Lines changed: 9 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,13 +14,7 @@ ms.topic: how-to
14
14
15
15
In this article, you learn how to diagnose and troubleshoot failing load tests in Azure Load Testing. Azure Load Testing provides several options to identify the root cause of a failing load test. For example, you can use the load test dashboard, or download the test results or test log files for an in-depth analysis. Alternately, configure server-side metrics to identify issues with application endpoint.
16
16
17
-
Azure Load Testing uses two indicators to determine the outcome of a load test:
18
-
19
-
-**Test status**: indicates whether the load test was able to start successfully and run the test script until the end. For example, the test status is *Failed* if there's an error in the JMeter or Locust test script, or if the [autostop listener](./how-to-define-test-criteria.md#auto-stop-configuration) interrupted the load test because too many requests failed.
20
-
21
-
-**Test result**: indicates the result of evaluating the [test fail criteria](./how-to-define-test-criteria.md). If at least one of the test fail criteria was met, the test result is set to *Failed*.
22
-
23
-
Depending on the indicator, you can use a different approach to identify the root cause of a test failure.
17
+
Azure Load Testing uses the test run status to determine the outcome of a load test. Depending on the indicator, you can use a different approach to identify the root cause of a test failure. For more details on test run status, see [Analyze load tests using the results dashboard](./how-to-understand-test-run-results-dashboard.md#test-run-details)
24
18
25
19
## Prerequisites
26
20
@@ -39,7 +33,7 @@ Use the following steps to get the outcome of a load test:
39
33
40
34
1. Select a test from the list to view all test runs for that test.
41
35
42
-
The list of test runs shows the **Test result**and **Test status** fields.
36
+
The list of test runs shows the **Status**field.
43
37
44
38
:::image type="content" source="media/how-to-find-download-logs/load-testing-test-runs-list.png" alt-text="Screenshot that shows the list of test runs in the Azure portal, highlighting the test result and test status columns." lightbox="media/how-to-find-download-logs/load-testing-test-runs-list.png":::
45
39
@@ -85,20 +79,21 @@ You can use a different approach for diagnosing a load test failure based whethe
85
79
86
80
### Load test failed to complete
87
81
88
-
When the load test fails to complete, the *test status* of the test run is set to *Failed*.
82
+
When the load test fails to complete, the *status* of the test run is set to *Error* or *Stopped*.
89
83
90
84
A load test can fail to complete because of multiple reasons. Examples of why a load test doesn't finish:
91
85
92
86
- There are errors in the test script.
93
87
- The test script uses JMeter features that Azure Load Testing doesn't support. Learn about the [supported JMeter features](./resource-jmeter-support.md).
94
88
- The test script references a file or plugin that isn't available on the test engine instance.
95
-
- The autostop functionality interrupted the load test because too many requests are failing and the error rate exceeds the threshold. Learn more about the [autostop functionality in Azure Load Testing](./how-to-define-test-criteria.md#auto-stop-configuration).
89
+
- The auto-stop functionality interrupted the load test because too many requests are failing and the error rate exceeds the threshold. Learn more about the [autostop functionality in Azure Load Testing](./how-to-define-test-criteria.md#auto-stop-configuration).
96
90
97
91
Use the following steps to help diagnose a test not finishing:
98
92
99
93
1. Verify the error details on the load test dashboard.
100
94
1.[Download and analyze the test logs](#download-apache-jmeter-or-locust-worker-logs-for-your-load-test) to identify issues in the JMeter test script.
101
95
1.[Download the test results](./how-to-export-test-results.md) to identify issues with individual requests.
96
+
1. Run the test in [debug mode](./how-to-run-tests-in-debug-mode.md) to get debug logs and request and response data for failed requests.
102
97
103
98
You can also view AI-based error insights for your test run after a test run completes. The insights generated contain the cause of error and more actionable recommendations. This feature is turned on by default and can be turned off in the error blade settings. Refer to [Microsoft Privacy Statement](https://go.microsoft.com/fwlink/?LinkId=521839).
104
99
@@ -107,7 +102,7 @@ You can also view AI-based error insights for your test run after a test run com
107
102
108
103
### Load test completed
109
104
110
-
A load test might run the test script until the end (test status equals *Done*), but might not pass all the [test fail criteria](./how-to-define-test-criteria.md). If at least one of the test criteria didn't pass, the *test result* of the test run is set to *Failed*.
105
+
A load test might run the test script until the end, but might not pass all the [test fail criteria](./how-to-define-test-criteria.md). If at least one of the test criteria didn't pass, the *status* of the test run is set to *Failed*.
111
106
112
107
Use the following steps to help diagnose a test failing to meet the test criteria:
113
108
@@ -135,7 +130,9 @@ To download the worker logs for an Azure Load Testing test run, follow these ste
135
130
136
131
1. From the list of test runs, select a test run to view the load test dashboard.
137
132
138
-
1. On the dashboard, select **Download**, and then select **Logs**.
133
+
1. On the dashboard, select **Download**, and then select **Logs**. To view the logs in Azure portal, select **View**. You can view the files in the storage container, select the required file and view the file. If the file format is not supported for viewing, you can select **Download** to download the specific file.
134
+
135
+
1. To download the logs files as a zipped folder, select **Download**.
139
136
140
137
The browser should now start downloading a zipped folder that contains the JMeter or Locust worker node log file for each [test engine instance](./concept-load-testing-concepts.md#test-engine).
Copy file name to clipboardExpand all lines: articles/app-testing/load-testing/how-to-export-test-results.md
+14-10Lines changed: 14 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,7 +12,7 @@ ms.topic: how-to
12
12
---
13
13
# Export test results from Azure Load Testing for use in third-party tools
14
14
15
-
In this article, you learn how to export your Azure Load Testing test results and reports. You can download the results by using the Azure portal, as an artifact in your CI/CD workflow, in JMeter by using a backend listener, in Locust by using event hooks or listener plugins, or by copying the results from an Azure storage account. You can use these results for reporting in third-party tools or for diagnosing test failures. Azure Load Testing generates the test results in comma-separated values (CSV) file format, and provides details of each application request for the load test.
15
+
In this article, you learn how to export your Azure Load Testing test results and reports. You can view the results in the Azure portal, download the results using the Azure portal, as an artifact in your CI/CD workflow, in JMeter by using a backend listener, in Locust by using event hooks or listener plugins, or by copying the results from an Azure storage account. You can use these results for reporting in third-party tools or for diagnosing test failures. Azure Load Testing generates the test results in comma-separated values (CSV) file format, and provides details of each application request for the load test.
16
16
17
17
You can also use the test results to diagnose errors during a load test. The `responseCode` and `responseMessage` fields give you more information about failed requests. For more information about investigating errors, see [Diagnose failing load tests](./how-to-diagnose-failing-load-test.md).
1676040230680,101,Homepage,200,OK,172.18.33.7-Thread Group 1-1,text,true,,1591,133,5,5,https://www.example.com/,98,0,94
41
41
```
42
42
43
-
## Access and download load test results and report
43
+
## View and download load test results and report
44
44
45
-
After a load test run finishes, you can access and download the load test results and the HTML report through the Azure portal, or as an artifact in your CI/CD workflow.
45
+
After a load test run finishes, you can view and download the load test results and the HTML report through the Azure portal, or as an artifact in your CI/CD workflow.
46
46
47
47
>[!IMPORTANT]
48
48
>For load tests with more than 45 engine instances or a greater than 3-hour test run duration, the results file is not available for download. You can [configure a JMeter Backend Listener to export the results](#export-test-results-using-listeners) to a data store of your choice or [copy the results from a storage account container](#copy-test-artifacts-from-a-storage-account-container).
49
49
>For tests with samplers greater than 30, the downloaded HTML report will only have graphs for data aggregated over all samplers. Graphs will not show sampler-wise data. Additionally, the downloaded report doesn't support graphs corresponding to server-side metrics.
50
50
51
51
# [Azure portal](#tab/portal)
52
52
53
-
To download the test results and the HTML report for a test run in the Azure portal:
53
+
To view and download the test results and the HTML report for a test run in the Azure portal:
54
54
55
55
1. In the [Azure portal](https://portal.azure.com), go to your Azure Load Testing resource.
56
56
@@ -65,14 +65,18 @@ To download the test results and the HTML report for a test run in the Azure por
65
65
66
66
* Select the ellipsis (**...**) next to the load test run you're working with, and then select **Download results file**.
67
67
68
-
:::image type="content" source="media/how-to-export-test-results/test-run-page-download.png" alt-text="Screenshot that shows how to download the results file for a load test run.":::
68
+
:::image type="content" source="media/how-to-export-test-results/test-run-page-download.png" alt-text="Screenshot that shows how to download the results file for a load test run." lightbox="media/how-to-export-test-results/test-run-page-download.png":::
69
69
70
70
> [!NOTE]
71
-
> A load test run needs to have a *Done*, *Stopped*, or *Failed* status for the results file to be available for download.
71
+
> A load test run needs to have a *Passed*, *Failed*, *Completed*, or *Stopped* status for the results file to be available for download.
72
72
73
-
* Select **Download** on the **Test run details** pane. To download the results, select **Results**. To download the HTML report, select **Report**.
73
+
* Select **Download** on the **Test run details** pane. To download the results, select **Results** and select **Download**. To download the HTML report, select **Report**.
74
74
75
-
:::image type="content" source="media/how-to-export-test-results/download-dashboard.png" alt-text="Screenshot that shows how to download the test results from the 'Test run details' pane.":::
75
+
:::image type="content" source="media/how-to-export-test-results/download-dashboard.png" alt-text="Screenshot that shows how to download the test results from the 'Test run details' pane." lightbox="media/how-to-export-test-results/download-dashboard.png":::
76
+
77
+
* To view the results files in the storage container, select **Download** on the **Test run details** pane, select **Results** and select **View**. You can view the results files from all the test engines in the right context pane. Select the file that you want to download and select **Download**.
78
+
79
+
:::image type="content" source="media/how-to-export-test-results/view-results-file.png" alt-text="Screenshot that shows how to view the test results from the 'Test run details' pane." lightbox="media/how-to-export-test-results/view-results-file.png":::
76
80
77
81
1. You can use any zip tool to extract the folder and access the downloaded files.
78
82
@@ -177,14 +181,14 @@ To copy the test results and log files for a test run from a storage account, in
177
181
>[!TIP]
178
182
> To limit the number of tests to display in the list, you can use the search box and the **Time range** filter.
179
183
180
-
1. On the **Test run details** pane, select **Copy artifacts**.
184
+
1. On the **Test run details** pane, select **Download**, then select **Results** and select **View**. You can see the storage container view with the results files.
181
185
182
186
:::image type="content" source="media/how-to-export-test-results/test-run-page-copy-artifacts.png" alt-text="Screenshot that shows how to copy the test artifacts from the 'Test run details' pane.":::
183
187
184
188
> [!NOTE]
185
189
> A load test run needs to be in the *Done*, *Stopped*, or *Failed* status for the results file to be available for download.
186
190
187
-
1. Copy the SAS URL of the storage account container.
191
+
1. Select **Copy SAS URL** to copy the SAS URL of the storage account container.
188
192
189
193
You can use the SAS URL in the [Azure Storage Explorer](/azure/vs-azure-tools-storage-manage-with-storage-explorer?tabs=windows#shared-access-signature-sas-url) or [AzCopy](/azure/storage/common/storage-use-azcopy-blobs-copy#copy-a-container) to copy the results CSV files and the log files for the test run to your storage account.
0 commit comments