Skip to content

kotlin: make incremental compilation actually engage - #625

Draft
oyvindberg wants to merge 1 commit into
masterfrom
kotlin-incremental-compilation
Draft

kotlin: make incremental compilation actually engage#625
oyvindberg wants to merge 1 commit into
masterfrom
kotlin-incremental-compilation

Conversation

@oyvindberg

Copy link
Copy Markdown
Owner

Kotlin incremental compilation has never engaged, in any configuration bleep ships. Every Kotlin build is a full compile, and nothing says so.

Independent of the Windows/CI work — touches only KotlinSourceCompiler and its test.

Three independent bugs, each fatal on its own

1. Wrong runner class. compileIncremental looked for a 7- or 6-arg IncrementalJvmCompilerRunner. Actual declared arities (javap -p on the real jars):

Kotlin ctor arity 7-arg 6-arg
2.0.0, 2.1.20 9 (+11 synthetic)
2.2.0, 2.3.0 6 (+8)
2.3.21 9 (+11)

And on 2.2+ that class only supports the classpath-snapshot path — it hard-errors with Unexpected ClasspathSnapshotDisabled for this code path on the ClasspathSnapshotDisabled we pass. The class we actually wanted is BuildHistoryJvmICRunner (the build-history runner, what Maven uses), whose 7-arg signature is what the old code was half-remembering — except slot 5 is a ModulesApiHistory, not a ClasspathChanges.

2. ChangedFiles.Unknown. IncrementalCompilerRunner.compile returns RequiresRebuild on it immediately, before looking at anything, and compileNonIncrementally then calls cleanOrCreateDirectories on the output dir. Full compile, every build, even on the versions where the constructor matched.

3. incrementalCompilation never set. JvmConfigurationPipelinePhase.setupIncrementalCompilationServices returns early unless it is on, so the LookupTracker / ICFileMappingTracker / IncrementalCompilationComponents handed over via Services are dropped, no output→source mappings come back, and caches-jvm/jvm/kotlin/ is never written — leaving the next build nothing to be incremental against.

Plus a fourth: classpath-hash lived inside .kotlin-ic, which is passed to Kotlin as one of outputDirs and therefore wiped by cleanOrCreateDirectories on rebuilds. Cross-language change detection was erasing its own state.

Changes

  • Use BuildHistoryJvmICRunner + EmptyModulesApiHistory.INSTANCE via its 7-arg constructor; dropped the 6-arg attempt.
  • Pass ChangedFiles.DeterminableFiles.ToBeComputed.INSTANCE — the runner diffs sources itself against caches-jvm/inputs/source-snapshot.tab (content hash + length, not mtime, so bleep rewriting sources on every compile is harmless here).
  • Set args.incrementalCompilation = true. Deliberately the compiler argument rather than the JVM-global kotlin.incremental.compilation system property, since this runs in a daemon shared across workspaces.
  • Move classpath-hash to a sibling <outputDir>.kotlin-ic-meta/.
  • Both fallbacks now print to stderr instead of a debug() gated on bleep.kotlin.debug, which is set nowhere in the tree — that gate is why this was invisible.

Version support

BuildHistoryJvmICRunner exists from Kotlin 2.2.0. Below that we warn once and compile non-incrementally — same behaviour as today, now visible. bleep-test-ksp-processor pins 2.1.20 and takes that path.

Follow-up, not in this PR: Versions.Kotlin21 = "2.1.21" is the default when a project omits kotlin.version, so unpinned projects get no IC. Kotlin23 = "2.3.21" (what bleep build create-new writes) is fine.

Verification

New regression test asserts caches-jvm/jvm/kotlin/class-fq-name-to-source.tab and source-to-classes.tab exist after a compile, and that a marker file in the output dir survives a second round — a rebuild wipes it, only the incremental path leaves it. Confirmed it fails without the fix (stashed the compiler change, re-ran, red; restored, green).

Run against this exact base:

suite
IncrementalTrackingTest (incl. new test) 6 passed
IncrementalCompilationTest — change A → break A → fix A, per language 3 passed
CompilerVersionIsolationTest — Kotlin 2.3.0 + 2.2.0, mixed Java/Kotlin 12 passed
CompilerIntegrationTest 7 passed
ClasspathCompilationTest 3 passed
SymbolProcessorDagIntegrationTest (KSP) 7 passed
CancellationTest / PlatformCancellationTest 14 passed

The change/break/fix suite is the one that matters — that is where a wrong IC setup yields stale classes rather than an error.

Caveat: these all drive the compiler in-process. The deployed path (a bleep binary against a forked daemon) still wants a bleep sourcegen && bleep publish local-ivy smoke test.

🤖 Generated with Claude Code

@oyvindberg
oyvindberg marked this pull request as draft July 29, 2026 17:24
Parked on its own branch to keep it out of the GitHub Actions work. Written by
an investigation subagent that was scoped read-only and implemented a fix
anyway; neither the diff nor its test results have been reviewed by a human or
re-run independently. Committed only so it is not carried around loose in a
worktree.

The diagnosis behind it is independently verified and does hold: Kotlin
incremental compilation never engages in any configuration bleep ships.
Blessed versions are Kotlin2=2.0.21, Kotlin21=2.1.21 (the default when a
project omits kotlin.version) and Kotlin23=2.3.21; the runner-constructor
lookup matches none of them. The failure is silent — `bleep.kotlin.debug`
appears exactly once in the tree, as the read. The one IC test pins 2.3.0, a
version bleep does not ship, and asserts only first-compile behaviour.

Claimed but unverified here: the constructor arity table, that
BuildHistoryJvmICRunner is the right class on >=2.2, that ChangedFiles
.DeterminableFiles.ToBeComputed and args.incrementalCompilation are the
remaining two pieces, and the test results.

Open decision this does not settle: BuildHistoryJvmICRunner only exists on
Kotlin >=2.2, so with this fix the default 2.1.21 still gets no incremental
compilation. Moving that default is a compatibility call, not a detail to
bury in a compiler patch.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@oyvindberg
oyvindberg force-pushed the kotlin-incremental-compilation branch from 3e440fd to 9583796 Compare July 29, 2026 22:48
oyvindberg pushed a commit that referenced this pull request Jul 29, 2026
…tEmpty

#626 retried the delete only on DirectoryNotEmptyException, which is how the race
presents on Linux. Windows presents the same race as a blocked unlink instead, so
#625 went from failing PlatformCancellationTest to failing TimeoutAndResourceTest:

  java.nio.file.FileSystemException:
    C:\...\Temp\scalajs-mid-cancel...: The process cannot access the file
    at Files.deleteIfExists
    at TestFiles$package$$anon$1.postVisitDirectory(TestFiles.scala:55)

One cause, two symptoms: a cancelled-but-still-running toolchain either puts a
file back after the walk emptied the directory (ubuntu) or holds a handle that
blocks the unlink (windows). Both are FileSystemException, so retry on that
rather than on either leaf type. Budget raised to 20 × 100ms, enough for a killed
node or kotlinc to exit and drop its handles.

TimeoutAndResourceTest and PlatformCancellationTest both green locally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant