Skip to content

Commit 6ca6f21

Browse files
nkolev92nkolev92
authored
Migrate 10 end to end tests to apex (#7222)
1 parent 45c7a1b commit 6ca6f21

3 files changed

Lines changed: 321 additions & 143 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,153 @@
2626
- **`required` on private/internal types** is cleaner than `null!` field initializers.
2727
- **TryCreate/TryGet patterns** — out params need `?`, callers use `!` after the success guard. Out parameters that are guaranteed non-null when the method returns true should be annotated with `[NotNullWhen(true)]`. Don't annotate `[NotNullWhen]` unless it's actually true for all code paths.
2828
- **Work in batches** — group related files, fix source, fix cascading, build, repeat. If this means we need multiple pull requests for enabling nullable, that's fine. Don't try to do it all in one go.
29+
30+
## Migrating PowerShell E2E Tests to Apex Tests
31+
32+
### Overview
33+
34+
PowerShell E2E tests live in `test/EndToEnd/tests/`. Apex tests live in `test/NuGet.Tests.Apex/NuGet.Tests.Apex/NuGetEndToEndTests/`. The goal is to migrate PS tests to C# Apex tests that run the **exact same scenario**, then remove the PS test function.
35+
36+
### Template mapping
37+
38+
| PowerShell function | Apex `ProjectTemplate` | Package management | Verified |
39+
|---|---|---|---|
40+
| `New-ConsoleApplication` | `ProjectTemplate.ConsoleApplication` | packages.config | ✅ |
41+
| `New-ClassLibrary` | `ProjectTemplate.ClassLibrary` | packages.config | ✅ |
42+
| `New-WebSite` | `ProjectTemplate.WebSiteEmpty` | packages.config | ✅ |
43+
| `New-WebApplication` | `ProjectTemplate.WebApplicationEmpty` | packages.config | ❌ |
44+
| `New-WPFApplication` | `ProjectTemplate.WPFApplication` | packages.config | ❌ |
45+
| `New-MvcApplication` | `ProjectTemplate.WebApplicationEmptyMvc` | packages.config | ❌ |
46+
| `New-FSharpLibrary` | `ProjectTemplate.FSharpLibrary` | PackageReference | ❌ |
47+
| `New-NetCoreConsoleApp` | `ProjectTemplate.NetCoreConsoleApp` | PackageReference | ✅ |
48+
| `New-NetStandardClassLib` | `ProjectTemplate.NetStandardClassLib` | PackageReference | ✅ |
49+
50+
| PowerShell function | Apex equivalent |
51+
|---|---|
52+
| `New-SolutionFolder 'Name'` | `testContext.SolutionService.AddSolutionFolder("Name")` |
53+
54+
### Command execution
55+
56+
| Scenario | Apex API |
57+
|---|---|
58+
| Standard install with `-Version` | `nugetConsole.InstallPackageFromPMC(packageName, packageVersion)` |
59+
| Install with extra flags (`-Source`, `-WhatIf`, `-IgnoreDependencies`) | `nugetConsole.Execute($"Install-Package {packageName} -ProjectName {project.Name} -Source {source}")` |
60+
| Standard uninstall | `nugetConsole.UninstallPackageFromPMC(packageName)` |
61+
| Standard update | `nugetConsole.UpdatePackageFromPMC(packageName, packageVersion)` |
62+
| Any raw PMC command | `nugetConsole.Execute(command)` |
63+
64+
**Rule:** If the PS test does not use `-Version`, use `Execute()` with the raw command string. `InstallPackageFromPMC()` always adds `-Version`.
65+
66+
**Important:** `nugetConsole.Execute()` runs in a live PMC PowerShell session. It can execute **any** PowerShell command, not just NuGet commands. This means PS session state — global variables (`$global:InstallVar`), registered functions (`Test-Path function:\Get-World`), environment checks — can all be queried and asserted via `Execute()` + `IsMessageFoundInPMC()`. Do not skip tests just because they assert PS session state.
67+
68+
### Assertion mapping
69+
70+
| PowerShell assertion | Apex equivalent |
71+
|---|---|
72+
| `Assert-Package $p PackageName Version` (packages.config project) | `CommonUtility.AssertPackageInPackagesConfig(VisualStudio, testContext.Project, packageName, version, Logger)` |
73+
| `Assert-Package $p PackageName` (no version, packages.config) | `CommonUtility.AssertPackageInPackagesConfig(VisualStudio, testContext.Project, packageName, Logger)` |
74+
| `Assert-Package $p PackageName Version` (PackageReference project) | `CommonUtility.AssertPackageInAssetsFile(VisualStudio, testContext.Project, packageName, version, Logger)` |
75+
| `Assert-Throws { ... } $expectedMessage` | `nugetConsole.IsMessageFoundInPMC(expectedMessage)` — PMC errors appear as text, not C# exceptions |
76+
| `Assert-Null (Get-ProjectPackage ...)` / package not installed | `CommonUtility.AssertPackageNotInPackagesConfig(VisualStudio, testContext.Project, packageName, Logger)` |
77+
78+
### Package sources
79+
80+
| PowerShell source | Apex equivalent |
81+
|---|---|
82+
| `$context.RepositoryRoot` or `$context.RepositoryPath` | `testContext.PackageSource` — create packages with `CommonUtility.CreatePackageInSourceAsync()` |
83+
| No `-Source` (uses nuget.org) | Create a local package with `CommonUtility.CreatePackageInSourceAsync(testContext.PackageSource, ...)` — never depend on nuget.org |
84+
| Hardcoded invalid sources (`http://example.com`, `ftp://...`) | Use the same hardcoded strings directly |
85+
86+
### NuGet.Config manipulation
87+
88+
PS tests that use `Get-VSComponentModel` + `ISettings` to modify NuGet config at runtime can be migrated by pre-configuring `SimpleTestPathContext` before passing it to `ApexTestContext`.
89+
90+
**Via Settings API** (preferred):
91+
```csharp
92+
using var simpleTestPathContext = new SimpleTestPathContext();
93+
simpleTestPathContext.Settings.AddSource("PrivateRepo", privatePath);
94+
// ... then pass it in:
95+
using var testContext = new ApexTestContext(VisualStudio, projectTemplate, Logger,
96+
simpleTestPathContext: simpleTestPathContext);
97+
```
98+
99+
**Via raw config file** (for settings not covered by the API like `dependencyVersion` or `bindingRedirects`):
100+
```csharp
101+
using var simpleTestPathContext = new SimpleTestPathContext();
102+
File.WriteAllText(simpleTestPathContext.NuGetConfig,
103+
$@"<?xml version=""1.0"" encoding=""utf-8""?>
104+
<configuration>
105+
<config>
106+
<add key=""dependencyVersion"" value=""HighestPatch"" />
107+
</config>
108+
<packageSources>
109+
<clear />
110+
<add key=""source"" value=""{simpleTestPathContext.PackageSource}"" />
111+
</packageSources>
112+
</configuration>");
113+
114+
using var testContext = new ApexTestContext(VisualStudio, projectTemplate, Logger,
115+
simpleTestPathContext: simpleTestPathContext);
116+
```
117+
118+
### Test structure patterns
119+
120+
**Error-path tests** (no package creation needed) — synchronous:
121+
```csharp
122+
[TestMethod]
123+
[Timeout(DefaultTimeout)]
124+
public void DescriptiveTestName_Fails()
125+
{
126+
using var testContext = new ApexTestContext(VisualStudio, ProjectTemplate.ConsoleApplication, Logger);
127+
128+
var packageName = "Rules";
129+
var source = @"c:\temp\data";
130+
var expectedMessage = $"Unable to find package '{packageName}' at source '{source}'. Source not found.";
131+
132+
var nugetConsole = GetConsole(testContext.Project);
133+
nugetConsole.Execute($"Install-Package {packageName} -ProjectName {testContext.Project.Name} -Source {source}");
134+
135+
Assert.IsTrue(
136+
nugetConsole.IsMessageFoundInPMC(expectedMessage),
137+
$"Expected error message was not found in PMC output. Actual output: {nugetConsole.GetText()}");
138+
}
139+
```
140+
141+
**Success-path tests** (need package creation) — async:
142+
```csharp
143+
[TestMethod]
144+
[Timeout(DefaultTimeout)]
145+
public async Task DescriptiveTestNameAsync(/* or [DataTestMethod] with ProjectTemplate */)
146+
{
147+
using var testContext = new ApexTestContext(VisualStudio, ProjectTemplate.ConsoleApplication, Logger);
148+
149+
var packageName = "TestPackage";
150+
var packageVersion = "1.0.0";
151+
await CommonUtility.CreatePackageInSourceAsync(testContext.PackageSource, packageName, packageVersion);
152+
153+
var nugetConsole = GetConsole(testContext.Project);
154+
nugetConsole.InstallPackageFromPMC(packageName, packageVersion);
155+
156+
CommonUtility.AssertPackageInPackagesConfig(VisualStudio, testContext.Project, packageName, packageVersion, Logger);
157+
}
158+
```
159+
160+
### Style rules
161+
162+
- Use `using var` (inline declaration), not `using (var ...) { }`.
163+
- Place migrated tests before the static helper methods (`GetNetCoreTemplates`, etc.) in the file.
164+
- Method names: `{Action}FromPMC{Scenario}[_Fails|Async]`. Suffix with `_Fails` for error tests, `Async` for async tests.
165+
- Always include `[Timeout(DefaultTimeout)]`.
166+
- Include `nugetConsole.GetText()` in assertion failure messages for diagnostics.
167+
168+
### Tests that should NOT be migrated
169+
170+
Skip PS tests that:
171+
- Use `Assert-BindingRedirect` — binding redirect tests are already `[SkipTest]` in PS and not worth migrating.
172+
- Use `Get-ProjectItem`, `Get-ProjectItemPath`, or other VS DTE project-item inspection not available in Apex.
173+
174+
### After migration
175+
176+
1. Remove the migrated function from the PS test file.
177+
2. If a PS test is already covered by an existing Apex test (duplicate), just delete the PS test — no new Apex test needed.
178+
3. Verify with `get_errors` that the Apex file compiles cleanly.

‎test/EndToEnd/tests/InstallPackageTest.ps1‎

Lines changed: 2 additions & 143 deletions
Original file line numberDiff line numberDiff line change
@@ -1,111 +1,3 @@
1-
# Verify Xunit 2.1.0 can be installed into a net45 project.
2-
# https://github.com/NuGet/Home/issues/1711
3-
4-
function Test-InstallPackageWithInvalidAbsoluteLocalSource {
5-
# Arrange
6-
$package = "Rules"
7-
$project = New-ConsoleApplication
8-
$source = "c:\temp\data"
9-
$message = "Unable to find package '$package' at source '$source'. Source not found."
10-
11-
# Act & Assert
12-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
13-
}
14-
15-
function Test-InstallPackageWithValidAbsoluteLocalSource {
16-
# Arrange
17-
$package = "Rules"
18-
$project = New-ConsoleApplication
19-
$source = pwd
20-
$message = "Unable to find package '$package'"
21-
22-
# Act & Assert
23-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
24-
}
25-
26-
function Test-InstallPackageWithInvalidRelativeLocalSource {
27-
# Arrange
28-
$package = "Rules"
29-
$project = New-ConsoleApplication
30-
$source = "..\invalid_folder"
31-
$message = "Unable to find package '$package' at source '$source'. Source not found."
32-
33-
# Act & Assert
34-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
35-
}
36-
37-
function Test-InstallPackageWithValidRelativeLocalSource {
38-
# Arrange
39-
$package = "Rules"
40-
$project = New-ConsoleApplication
41-
$source = "..\"
42-
$message = "Unable to find package '$package'"
43-
44-
# Act & Assert
45-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
46-
}
47-
48-
function Test-InstallPackageWithInvalidHttpSource {
49-
# Arrange
50-
$package = "Rules"
51-
$project = New-ConsoleApplication
52-
$source = "http://example.com"
53-
$message = "Unable to find package '$package' at source '$source'."
54-
55-
# Act & Assert
56-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
57-
}
58-
59-
function Test-InstallPackageWithInvalidHttpSourceVerbose {
60-
# Arrange
61-
$package = "Rules"
62-
$project = New-ConsoleApplication
63-
$source = "http://example.com"
64-
$escapedSource = [regex]::Escape($source)
65-
$escapedUrl = [regex]::Escape($source+"/FindPackagesById()?id='$package'&semVerLevel=2.0.0")
66-
$message = "\ \ GET\ $escapedUrl\ \ \ NotFound\ $escapedUrl\ [\w]+\ An\ error\ occurred\ while\ retrieving\ package\ metadata\ for\ '$package'\ from\ source\ '$escapedSource'\."
67-
# Act
68-
$result = Install-Package $package -ProjectName $project.Name -source $source -Verbose *>&1
69-
$resultString = [string]::Join(" ", $result)
70-
$compare = $resultString -Match $message
71-
$messageToPrint = "Result string is `n$resultString but expected message was `n$message"
72-
# Assert
73-
Assert-True $compare $messageToPrint
74-
}
75-
76-
function Test-InstallPackageWithIncompleteHttpSource {
77-
# Arrange
78-
$package = "Rules"
79-
$project = New-ConsoleApplication
80-
$source = "http://"
81-
$message = "Unable to find package 'Rules' at source '$source'. Source not found."
82-
83-
# Act & Assert
84-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
85-
}
86-
87-
function Test-InstallPackageWithInvalidKnownSource {
88-
# Arrange
89-
$package = "Rules"
90-
$project = New-ConsoleApplication
91-
$source = "nuget.random"
92-
$message = "Unable to find package 'Rules' at source '$source'. Source not found."
93-
94-
# Act & Assert
95-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
96-
}
97-
98-
function Test-InstallPackageWithValidKnownSource {
99-
# Arrange
100-
$package = "Rules"
101-
$project = New-ConsoleApplication
102-
$source = "nuget.org"
103-
$message = "Unable to find package '$package'"
104-
105-
# Act & Assert
106-
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
107-
}
108-
1091
function Test-InstallPackageWithFtpProtocolSource {
1102
# Arrange
1113
$package = "Rules"
@@ -117,6 +9,8 @@ function Test-InstallPackageWithFtpProtocolSource {
1179
Assert-Throws { Install-Package $package -ProjectName $project.Name -source $source } $message
11810
}
11911

12+
# Verify Xunit 2.1.0 can be installed into a net45 project.
13+
# https://github.com/NuGet/Home/issues/1711
12014
function Test-InstallXunit210WithEmptyBuildFolderSucceeds
12115
{
12216
param($context)
@@ -136,22 +30,6 @@ function Test-InstallXunit210WithEmptyBuildFolderSucceeds
13630
Assert-Package $p xunit.abstractions 2.0.0
13731
}
13832

139-
function Test-SinglePackageInstallIntoSingleProject {
140-
# Arrange
141-
$project = New-ConsoleApplication
142-
143-
# Act
144-
Install-Package FakeItEasy -ProjectName $project.Name -version 1.8.0
145-
146-
# Assert
147-
Assert-Reference $project Castle.Core
148-
Assert-Reference $project FakeItEasy
149-
Assert-Package $project FakeItEasy
150-
Assert-Package $project Castle.Core
151-
Assert-SolutionPackage FakeItEasy
152-
Assert-SolutionPackage Castle.Core
153-
}
154-
15533
# Test install-package -WhatIf to downgrade an installed package.
15634
function Test-PackageInstallWithFileUri {
15735
# Arrange
@@ -169,17 +47,6 @@ function Test-PackageInstallWithFileUri {
16947
Assert-Package $project TestUpdatePackage '2.0.0.0'
17048
}
17149

172-
function Test-PackageInstallWhatIf {
173-
# Arrange
174-
$project = New-ConsoleApplication
175-
176-
# Act
177-
Install-Package FakeItEasy -Project $project.Name -version 1.8.0 -WhatIf
178-
179-
# Assert: no packages are installed
180-
Assert-Null (Get-ProjectPackage $project FakeItEasy)
181-
}
182-
18350
# Test install-package -WhatIf to downgrade an installed package.
18451
function Test-PackageInstallDowngradeWhatIf {
18552
# Arrange
@@ -1489,14 +1356,6 @@ function Test-InstallWithFailingInitPs1RollsBack {
14891356
Assert-Null (Get-SolutionPackage PackageWithFailingInitPs1)
14901357
}
14911358

1492-
function Test-InstallPackageThrowsWhenSourceIsInvalid {
1493-
# Arrange
1494-
$p = New-ConsoleApplication
1495-
1496-
# Act & Assert
1497-
Assert-Throws { Install-Package jQuery -source "d:package" } "Unsupported type of source 'd:package'. Please provide an HTTP or local source."
1498-
}
1499-
15001359
function Test-InstallPackageInvokeInstallScriptWhenProjectNameHasApostrophe {
15011360
param(
15021361
$context

0 commit comments

Comments
 (0)