Skip to content

windows: drop the powershell dependency for user directories, and unify CI on one shell - #624

Merged
oyvindberg merged 4 commits into
masterfrom
improve-windows-more
Jul 29, 2026
Merged

windows: drop the powershell dependency for user directories, and unify CI on one shell#624
oyvindberg merged 4 commits into
masterfrom
improve-windows-more

Conversation

@oyvindberg

Copy link
Copy Markdown
Owner

Windows CI kept a parallel universe of eight shell: cmd / shell: pwsh steps beside the unix ones. The reason was real: bleep could not resolve its own cache and config directories without spawning PowerShell, and nesting that inside a pwsh step is a known way to break it. This removes the cause, then the workaround.

The bug

Every bleep invocation on Windows ran:

powershell.exe -NoProfile -EncodedCommand <base64>

whose script is Add-Type @"...[DllImport("shell32.dll")] SHGetKnownFolderPath..."@ — it compiles C# at runtime to make one Win32 call. Main._main does this before anything else, and the BSP server does it from five more places, none memoized. That is the startup cost, and each spawn is a hard failure point: PowerShell in Constrained Language Mode (#43), AppLocker/WDAC, powershell.exe off PATH (#29), a non-writable TEMP, or a PSModulePath inherited from a parent pwsh that breaks Add-Type (PowerShell#8863).

Why it happened

The fix already shipped in code we depend on — it was just unreachable. coursier-cache 2.1.24 carries an FFM implementation under META-INF/versions/23, but that jar has no MANIFEST.MF at all, so no Multi-Release: true, so the JDK never loads it. And WindowsJni gates the JNI path off for java >= 23 precisely because it expects the multi-release class to take over. Both escape hatches closed; PowerShell every time.

coursier split coursier-paths out in 2.1.25 with a proper Multi-Release: true manifest. Verified rather than assumed:

JDK 21 -> coursier.paths.shaded.dirs.impl.WindowsPowerShell
JDK 25 -> coursier.paths.shaded.dirs.impl.WindowsForeign

coursier.cache.shaded.dirs no longer exists in 2.1.25, so UserPaths had to move regardless.

Native image

FFM downcalls need foreign reachability metadata or the image aborts with MissingForeignRegistrationError — the same failure class as #601, which stayed invisible until a real Windows user hit it. Registered SHGetKnownFolderPath's descriptor, and selftest now prints and validates the resolved directories so CI has to link them. The BSP server JVM gets --enable-native-access=ALL-UNNAMED on JDK 22+, which the native image already passed.

Cost of the bump

73 deprecation sites across 12 files, all load-bearing under -Werror. New internal/coursierDeps.scala converts at the two boundaries where coursier generalised a type bleep cannot express (Gradle Module variants) — throwing rather than defaulting; note coursier's own deprecated dependencyArtifacts silently collects those away. CoursierResolver.Result deliberately stays the narrower shape, keeping the on-disk resolution cache format byte-identical.

The 1094-file snapshot diff is pure reordering — coursier 2.1.25 emits resolution modules in a different order. Zero semantic change under an order-insensitive comparison, and a second run produces no further drift.

CI

Every step is now shell: bash on every OS; the only per-OS difference is the name of the two generated launcher scripts. bash also means set -eo pipefail, so every exit code is load-bearing — the old cmd blocks propagated only the last command's status, which is how 42 Windows test failures stayed hidden behind green runs (#607).

graalvm/setup-graalvm@v1 and the JAVA_HOME override in GenNativeImage are both gone. That override dated to GraalVM 21 and said "the one we install ourselves does not work"; neither reason survives on graalvm-community 25. Read out of the actual Windows distribution bleep provisions:

bin/native-image.cmd   present   <- what NativeImagePlugin resolves
bin/java.exe           present
bin/gu                 ABSENT    <- so the gu-install fallback is dead code

native-image.cmd locates lib/svm/bin/native-image.exe purely relative to itself and never reads JAVA_HOME, and since GraalVM for JDK 17/20 it finds Visual Studio via vswhere and sets up MSVC itself.

Drive-by fix: the Windows cleanup step looked for the BSP classpath cache under %USERPROFILE%\.bleep\cache — a directory bleep has never used on Windows (UserPaths.cacheDir is %LOCALAPPDATA%\bleep\cache, confirmed against a runner log). It matched nothing on every run, so the dev script could resolve bleep-bsp from a pre-publish classpath. It now warns and lists the directory rather than silently no-op'ing.

Verification

Full build green; 906 tests pass, 0 failed including slow ITs; selftest resolves unchanged macOS paths.

Not verified by execution, and the reason to review the CI commit separately: the Git Bash mechanics on a Windows runner (taskkill //F, cygpath, invoking ./ni-build.cmd from bash) and "GraalVM 25 finds VS 2022 on windows-latest". CI is the only way to check those.

Also loses native-image-job-reports — the action's PR size/metrics comment. Replacing it via $GITHUB_STEP_SUMMARY is planned separately.

🤖 Generated with Claude Code

oyvindberg and others added 2 commits July 29, 2026 02:35
… Add-Type

Every `bleep` invocation on Windows shelled out to
`powershell.exe -NoProfile -EncodedCommand` and compiled C# with `Add-Type`
at runtime, just to learn where its own cache and config directories live.
`Main._main` does it before anything else and the BSP server does it from
five more places, none of them memoized. That is the startup cost, and every
one of those spawns is a hard failure point: PowerShell in Constrained
Language Mode, AppLocker/WDAC, powershell.exe off PATH, a non-writable TEMP,
or a PSModulePath inherited from a parent pwsh that breaks Add-Type.

The fix already shipped in the code we depend on; it was just unreachable.
coursier-cache 2.1.24 carries an FFM implementation under
META-INF/versions/23, but that jar has no MANIFEST.MF at all — so no
`Multi-Release: true`, so the JDK never loads it. And WindowsJni gates the
JNI path off for java >= 23 precisely because it expects the multi-release
class to take over. Both escape hatches closed, PowerShell every time.

coursier split coursier-paths out in 2.1.25 with `Multi-Release: true` and
the impl under META-INF/versions/22, so JDK 22+ gets WindowsForeign.
Verified rather than assumed:

  JDK 21 -> coursier.paths.shaded.dirs.impl.WindowsPowerShell
  JDK 25 -> coursier.paths.shaded.dirs.impl.WindowsForeign

`coursier.cache.shaded.dirs` no longer exists in 2.1.25, so UserPaths had to
move regardless of any of this. `fromAppDirs` is now a lazy val.

Native image needs `foreign` reachability metadata for the two downcalls or
it aborts at runtime with MissingForeignRegistrationError — the same failure
class as #601, which stayed invisible until a real Windows user hit it.
Registered SHGetKnownFolderPath's descriptor (CLSIDFromString's shape is
already covered by the native-terminal file; matching is by shape, not name),
and `selftest` now prints and validates the resolved directories so CI has
to link them. The BSP server JVM gets --enable-native-access=ALL-UNNAMED on
JDK 22+, which the native image already passed; without it the JVM warns on
every start and a future JDK will refuse the call outright.

The bump costs 73 deprecation sites across 12 files, all load-bearing under
-Werror: Dependency.version -> versionConstraint.asString, configuration ->
variantSelector, Project.version -> version0, Conflict.*Version ->
*VersionConstraint, Resolution.{projectCache,subset,dependencyArtifacts} ->
the 0 variants, Authentication.user -> userOpt. New internal/coursierDeps.scala
converts at the two boundaries where coursier generalised a type bleep cannot
express (Gradle Module variants) — throwing rather than defaulting, and note
coursier's own deprecated dependencyArtifacts silently collects those away.
bleep's CoursierResolver.Result deliberately stays the narrower shape, which
also keeps the on-disk resolution cache format byte-identical.

Snapshot churn is coursier 2.1.25 emitting resolution modules in a different
order: all 1094 files diff order-only, zero semantic change under an
order-insensitive comparison, and a second run produces no further drift.

906 tests pass, 0 failed, including the slow ITs.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The native-image job carried a parallel universe of eight `shell: cmd` /
`shell: pwsh` Windows steps beside the unix ones. That existed because bleep
could not resolve its own directories without spawning PowerShell, so nesting
one inside a pwsh step was a known way to break it. The parent commit removes
that constraint, so every step is now `shell: bash` — GitHub's Windows runners
ship Git Bash, and it runs .bat/.cmd through the Windows loader. The only
per-OS difference left is the name of the two generated launcher scripts,
hence matrix.dev_script / matrix.ni_script.

bash also means GitHub runs each block with `set -eo pipefail`, so every
command's exit code is load-bearing. The old cmd blocks propagated only the
last command's status, which is how 42 Windows test failures stayed hidden
behind green runs (#607).

graalvm/setup-graalvm@v1 and the JAVA_HOME override in GenNativeImage are both
gone. That override dated to GraalVM 21 and said "the one we install ourselves
does not work"; neither reason survives on graalvm-community 25. Read out of
the actual Windows distribution bleep provisions:

  bin/native-image.cmd          present  <- what NativeImagePlugin resolves
  bin/java.exe                  present
  bin/gu                        ABSENT   <- so the gu-install fallback is dead

and native-image.cmd locates lib/svm/bin/native-image.exe purely relative to
itself, never reading JAVA_HOME. Since GraalVM for JDK 17/20 it also finds
Visual Studio through vswhere and sets up the MSVC environment itself, so no
developer command prompt is needed. The override was inert off Windows anyway
— it filtered on bin/java.exe existing — so this is a no-op for linux/macOS.

Two Windows-only steps remain and neither is about shells: nothing, now, plus
the mandatory-file-lock cleanup. That cleanup had been looking under
%USERPROFILE%\.bleep\cache for the BSP classpath cache, a directory bleep has
never used on Windows — UserPaths.cacheDir is %LOCALAPPDATA%\bleep\cache,
confirmed against a runner log. So it silently matched nothing on every run,
and the dev script could resolve bleep-bsp from a pre-publish classpath. Fixed,
and it now warns and lists the directory instead of no-op'ing if the glob ever
comes up empty again.

Costs `native-image-job-reports`, the action's PR size/metrics comment.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@oyvindberg
oyvindberg marked this pull request as draft July 29, 2026 22:53
Experiment against `Build failed: Input length = 1` in "Compile and publish
locally". That is a MalformedInputException from a strict UTF-8 decode, and the
bleep running there is the released M9 from the setup action -- not this
checkout, whose dev script does not exist until two steps later. M9 read the
server's `output` log with Files.readString for readiness and error messages;
master removed that read in #617 and made the survivors lossy in #626, but
neither reaches a binary that is already on disk.

So change what goes into the log instead. With stdout redirected to a file a
JDK 19+ JVM takes stdout.encoding from native.encoding, which on Windows is the
ANSI codepage -- mappable non-ASCII becomes a single high byte (invalid UTF-8)
and unmappable characters become '?'. This job's log shows exactly that mangling:
4 mangled lines here against 0 in the cmd-shell runs.

Via JAVA_TOOL_OPTIONS because the writer is the server JVM the client spawns, and
it inherits the variable.

Unverified -- it targets the mechanism, but the mechanism is inferred from the
error class plus the mangling, not from the log bytes themselves. #626 collects
Windows telemetry now, so a repeat gives us the actual log.
@oyvindberg
oyvindberg marked this pull request as ready for review July 29, 2026 23:42
@oyvindberg
oyvindberg merged commit 376c1fd into master Jul 29, 2026
9 checks passed
@oyvindberg
oyvindberg deleted the improve-windows-more branch July 29, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant