Fix PT Run ThreadPool worker leak from stale query cancellation by crutkas · Pull Request #48394 · microsoft/PowerToys

crutkas · 2026-06-08T23:49:08Z

Summary

Fixes a ThreadPool worker leak in PowerToys Run that can eventually surface as System.OutOfMemoryException from Thread.StartInternal after rapid typing and repeated stale-query cancellation.

Related: #36041 and duplicate reports #45704, #36587, #39942, #20264, and #8878.

Root cause

MainViewModel.QueryResults stored the active cancellation token in a mutable field. When a new query replaced that field, older workers could observe the new, non-cancelled token instead of the token belonging to their own query. The previous CancellationTokenSource was also disposed while its consumers could still be running.

As stale queries accumulated, they continued invoking plugins and consuming ThreadPool workers until the process could no longer create another worker thread.

Changes

Adds QuerySession, which owns one captured token and the complete task lifetime for a query. Superseded sessions are cancelled immediately and their token sources are disposed only after their work completes.
Uses a suspended session start so query state is published before workers can return results.
Adds generation checks before scheduling and applying work so superseded queries cannot enqueue stale plugin tasks or update current results.
Adds a per-plugin execution gate. Calls to the same plugin do not overlap, while unrelated plugins can execute independently; cancelled waiters do not occupy ThreadPool workers.
Preserves legacy IResultUpdated compatibility by correlating generation-0 events using RawQuery.
Preserves the original two-phase query contract: all non-delayed plugin queries complete and their results are applied before delayed queries start. Delayed queries remain globally parallel, and noInitialResults is computed from the complete non-delayed phase.
Cancels and performs a bounded wait for the active query during shutdown.

Tests

Wox.Test: 142/142 passing locally.

Coverage includes:

token ownership, cancellation, deferred disposal, shutdown timeout, and suspended session startup;
current-query generation matching and legacy generation-0 compatibility;
per-plugin execution gating and queued latest-query behavior;
deterministic verification that delayed queries cannot start until every non-delayed query completes.

Manual validation

Hold a key in PowerToys Run for 10–15 seconds and confirm the PowerToys Run process thread count stabilizes instead of growing monotonically.
Exercise normal Calculator, file, web, and indexer queries.
Enable search query tuning and waiting for slow results; confirm results appear and final sorting completes.
Start a slow query and type again before it completes; only the newest query should update results.
Exit PowerToys with a query in flight; shutdown should complete cleanly without orphaned processes.

QueryResults in MainViewModel cancels the previous query by replacing the _updateToken field with a fresh CancellationToken on every keystroke. However, the running Task body and its Parallel.ForEach calls read _updateToken via field access (not closure capture), so after the field is reassigned, ThrowIfCancellationRequested() reads the NEW non-cancelled token and the previous query runs to completion. Over time the leaked in-flight queries (and their per-keystroke fan-out across all plugins via Parallel.ForEach) saturate the ThreadPool worker creation budget, ending in System.OutOfMemoryException raised by Thread.StartInternal. Additionally, _updateSource was Dispose()d immediately after Cancel(), which races with in-flight consumers; they can observe ObjectDisposedException (not caught) instead of OperationCanceledException. Changes: - Capture the new token into a local variable updateToken and use that inside the Task lambda and both Parallel.ForEach calls, so field reassignment does not silently mask cancellation. - Defer Dispose of the previous CancellationTokenSource via ContinueWith on the previous query Task (tracked via a new _currentQueryTask field). - Pass the cancellation token to Parallel.ForEach via ParallelOptions so the loop honors cancellation between iterations as well as inside them. - Pass explicit TaskScheduler.Default to Task.Factory.StartNew / ContinueWhenAll to avoid the TaskScheduler.Current footgun. Related: issue microsoft#36041, plus auto-closed reports microsoft#45704, microsoft#36587, microsoft#39942, microsoft#20264, microsoft#8878. Co-authored-by: Copilot <[email protected]>

crutkas · 2026-06-09T00:24:13Z

@copilot fix the spellbot error. betting as simple as adding a comma to "Otherwise" so it becomes "Otherwise,"

Builds on the original capture-token-locally fix by addressing residual issues surfaced in review: * SRP: new internal sealed QuerySession owns ONE CancellationTokenSource and the tail Task it spawns. Construct-once via QuerySession.Start (pipelineFactory); post-construction the only surface is terminal idempotent ops (Cancel / DisposeWhenComplete / CancelAndWait / Dispose). Token is an immutable struct snapshot bound to THIS session's CTS, so the field-reassignment bug at the root of microsoft#36041 is structurally prevented for any caller that uses session.Token (regression-tested below). * Task.Run instead of Task.Factory.StartNew(None, Default): equivalent scheduler/option pinning but also passes DenyChildAttach, preventing plugin-spawned attached child tasks from extending the parent's lifetime past cancellation. * DenyChildAttach on the doFinalSort ContinueWith for the same reason; the continuation is composed inside the pipeline factory so it is part of the session's Completion and the previous CTS isn't disposed before final sort observes cancellation. * Token capture in QueryHistory and RegisterResultsUpdatedEvent: those paths previously read the field; both now snapshot session.Token into a local before the async work. * Removed ExecuteSynchronously from the QueryResults ContinueWith: per docs (only short-running continuations should run synchronously) it could schedule Dispose on the UI thread when the antecedent was already completed at the point ContinueWith was called. * App shutdown: MainViewModel.Dispose now calls _currentSession?.CancelAndWait(2s) to stop the in-flight query before the CTS is disposed, instead of leaking it. * Corrected the inline rationale comment: CancellationToken.ThrowIfCancellationRequested raises OperationCanceledException, never ObjectDisposedException — only Register / WaitHandle can throw ODE post-dispose. * New Wox.Test/QuerySessionTest.cs (9 tests) including the canonical regression for microsoft#36041 — CapturedToken_StaysBoundToOriginalSourceAcrossReplacement — proving that cancelling session A does not affect session B's token and that a captured local snapshot of A.Token observes A's cancellation even after _currentSession has been reassigned. Wox.Test: 139/139 passing.

yeelam-gordon · 2026-06-12T13:30:12Z

Pushed 2eb9fab33d to this branch (maintainer-can-modify) building on top of your fix. Summary of what changed and why, plus the new tests.

What this commit adds on top of yours

Your local-capture fix (var updateToken = currentUpdateSource.Token) is the right root-cause fix for #36041 and is preserved verbatim. The follow-up addresses residual issues from review:

#	Change	Why
1	New `internal sealed QuerySession` owns one CTS + the tail Task it spawns. Construct via `QuerySession.Start(pipelineFactory)`; post-construction surface is terminal idempotent ops only (`Cancel` / `DisposeWhenComplete` / `CancelAndWait` / `Dispose`).	SRP. The captured-`Token` snapshot is now a structural property of the type, so the field-reassignment bug at the root of #36041 can't be reintroduced by any caller that uses `session.Token`. No mutator surface → no lifecycle race → no lock needed.
2	`Task.Run(...)` instead of `Task.Factory.StartNew(action, token, TaskCreationOptions.None, TaskScheduler.Default)`	`Task.Run` is documented as `StartNew(action, CancellationToken.None, DenyChildAttach, Default)`. Adding `DenyChildAttach` prevents plugin-spawned attached child tasks from extending the parent's lifetime past cancellation — same leak family the PR is fixing.
3	`DenyChildAttach` on the `doFinalSort` `ContinueWith`. The continuation is composed inside the pipeline factory so it's part of the session's `Completion` and the previous CTS is not disposed while final sort is still in flight.	Same reasoning as #2; without this the final-sort path could hold workers past cancellation.
4	Removed `TaskContinuationOptions.ExecuteSynchronously` from the `QueryResults` `ContinueWith`.	Per docs: only short-running continuations should run synchronously. If the antecedent had already completed when `ContinueWith` was created (rare but possible on a fast plugin), the continuation would run on the UI thread.
5	`QueryHistory` and `RegisterResultsUpdatedEvent` now snapshot `_currentSession?.Token` into a local before any async work.	These paths previously read the field; same class of bug as the main one, just in adjacent code.
6	`MainViewModel.Dispose` now calls `_currentSession?.CancelAndWait(TimeSpan.FromSeconds(2))`.	App-shutdown path was leaking the in-flight CTS.
7	Corrected the inline rationale comment.	`CancellationToken.ThrowIfCancellationRequested` raises `OperationCanceledException`, never `ObjectDisposedException` — only `CancellationToken.Register` / `WaitHandle` can throw ODE post-dispose.

New regression tests (`src/modules/launcher/Wox.Test/QuerySessionTest.cs`, 9 tests, all passing)

Test	Asserts
`CapturedToken_StaysBoundToOriginalSourceAcrossReplacement`	Canonical #36041 regression. Creates session A, captures `firstToken`, creates session B, cancels A — asserts `firstToken.IsCancellationRequested == true` AND `B.Token.IsCancellationRequested == false`.
`Cancel_SignalsCapturedTokenAndIsIdempotent`	`Cancel()` is idempotent; captured-local token observes cancellation.
`Start_PassesSessionTokenToPipelineFactory`	The factory receives THIS session's token.
`Start_ThrowsArgumentNullException_WhenFactoryIsNull`	Defensive contract.
`DisposeWhenComplete_WaitsForCompletionTaskBeforeDisposingCts`	CTS is not disposed until the tracked completion task finishes.
`CancelAndWait_ReturnsTrueWhenTaskCompletesWithinTimeout`	Shutdown path success case.
`CancelAndWait_ReturnsFalseWhenTaskExceedsTimeout`	Shutdown path timeout case (buggy plugin doesn't yield).
`Dispose_IsIdempotent_AndSafeAfterCancel`	No double-dispose / no throw.
`Token_ThrowIfCancellationRequested_NeverThrowsObjectDisposedException`	Documents the property #7 above corrects.

Wox.Test: 139/139 passing.

Suggested manual smoke

Bug repro — hold a key in PT Run for 10–15s; watch PowerToys.PowerLauncher.exe thread count in Task Manager. Should stabilize, not climb monotonically.
Normal queries — Calculator (=2+2), file search, web search, indexer.
Final-sort path — Settings → PT Run → enable "Search query tuning" + "Wait for slow results", then query a slow plugin. Results should appear, then re-sort.
Race — type a slow query, then type more before it completes. Only the final query's results should remain.
Shutdown — Exit PowerToys with an in-flight query. Should exit cleanly in ≤2s, no orphaned processes.

Happy to back any of this out if you want a smaller diff — the SRP extraction is the bulk of it.

…ispose on CancelAndWait timeout Two fixes spotted while doing a follow-up read of the cancellation refactor for an unrelated review: 1. MainViewModel.cs — previous session was only cancelled when pluginQueryPairs had at least one entry. A non-empty QueryText whose QueryBuilder.Build() returns an empty dictionary (e.g. all global plugins disabled, no keyword match) fell through both cancellation paths and left the in-flight session running. The pre-refactor code cancelled _updateSource unconditionally before the Count check; this restores that invariant by hoisting previousSession.Cancel() + DisposeWhenComplete() out of the Count > 0 block while keeping _currentSession bound to the just-cancelled session (so late IResultUpdated events continue to suppress via their captured token rather than falling back to CancellationToken.None). 2. QuerySession.CancelAndWait — on timeout, the CTS was disposed unconditionally while the tracked task may still have been running. That violates the refactor's own "tasks never observe a disposed CTS" invariant: any future plugin path that touches token WaitHandles (Register, WaitOne) after timeout would crash with ObjectDisposedException. The fix gates dispose on the completed flag and, on timeout, defers disposal via ContinueWith with DenyChildAttach (mirroring DisposeWhenComplete's pattern). Also softens the QuerySession remarks: the "structural guarantee" only holds for callers that capture Token into a local. The class can't prevent a future author from re-reading _currentSession.Token inside a body, which would reintroduce the original bug. Added regression test CancelAndWait_DoesNotDisposeCtsWhileTaskStillRuns covering the new deferred-disposal behavior; all 10 Wox.Test QuerySessionTest cases pass. --- ADO: https://microsoft.visualstudio.com/DefaultCollection/OS/_workitems/edit/55588441/ Co-authored-by: Copilot <[email protected]>

QueryResults in MainViewModel cancels the previous query by replacing the _updateToken field with a fresh CancellationToken on every keystroke. However, the running Task body and its Parallel.ForEach calls read _updateToken via field access (not closure capture), so after the field is reassigned, ThrowIfCancellationRequested() reads the NEW non-cancelled token and the previous query runs to completion. Over time the leaked in-flight queries (and their per-keystroke fan-out across all plugins via Parallel.ForEach) saturate the ThreadPool worker creation budget, ending in System.OutOfMemoryException raised by Thread.StartInternal. Additionally, _updateSource was Dispose()d immediately after Cancel(), which races with in-flight consumers; they can observe ObjectDisposedException (not caught) instead of OperationCanceledException. Changes: - Capture the new token into a local variable updateToken and use that inside the Task lambda and both Parallel.ForEach calls, so field reassignment does not silently mask cancellation. - Defer Dispose of the previous CancellationTokenSource via ContinueWith on the previous query Task (tracked via a new _currentQueryTask field). - Pass the cancellation token to Parallel.ForEach via ParallelOptions so the loop honors cancellation between iterations as well as inside them. - Pass explicit TaskScheduler.Default to Task.Factory.StartNew / ContinueWhenAll to avoid the TaskScheduler.Current footgun. Related: issue microsoft#36041, plus auto-closed reports microsoft#45704, microsoft#36587, microsoft#39942, microsoft#20264, microsoft#8878. Co-authored-by: Copilot <[email protected]>

Builds on the original capture-token-locally fix by addressing residual issues surfaced in review: * SRP: new internal sealed QuerySession owns ONE CancellationTokenSource and the tail Task it spawns. Construct-once via QuerySession.Start (pipelineFactory); post-construction the only surface is terminal idempotent ops (Cancel / DisposeWhenComplete / CancelAndWait / Dispose). Token is an immutable struct snapshot bound to THIS session's CTS, so the field-reassignment bug at the root of microsoft#36041 is structurally prevented for any caller that uses session.Token (regression-tested below). * Task.Run instead of Task.Factory.StartNew(None, Default): equivalent scheduler/option pinning but also passes DenyChildAttach, preventing plugin-spawned attached child tasks from extending the parent's lifetime past cancellation. * DenyChildAttach on the doFinalSort ContinueWith for the same reason; the continuation is composed inside the pipeline factory so it is part of the session's Completion and the previous CTS isn't disposed before final sort observes cancellation. * Token capture in QueryHistory and RegisterResultsUpdatedEvent: those paths previously read the field; both now snapshot session.Token into a local before the async work. * Removed ExecuteSynchronously from the QueryResults ContinueWith: per docs (only short-running continuations should run synchronously) it could schedule Dispose on the UI thread when the antecedent was already completed at the point ContinueWith was called. * App shutdown: MainViewModel.Dispose now calls _currentSession?.CancelAndWait(2s) to stop the in-flight query before the CTS is disposed, instead of leaking it. * Corrected the inline rationale comment: CancellationToken.ThrowIfCancellationRequested raises OperationCanceledException, never ObjectDisposedException — only Register / WaitHandle can throw ODE post-dispose. * New Wox.Test/QuerySessionTest.cs (9 tests) including the canonical regression for microsoft#36041 — CapturedToken_StaysBoundToOriginalSourceAcrossReplacement — proving that cancelling session A does not affect session B's token and that a captured local snapshot of A.Token observes A's cancellation even after _currentSession has been reassigned. Wox.Test: 139/139 passing.

…ispose on CancelAndWait timeout Two fixes spotted while doing a follow-up read of the cancellation refactor for an unrelated review: 1. MainViewModel.cs — previous session was only cancelled when pluginQueryPairs had at least one entry. A non-empty QueryText whose QueryBuilder.Build() returns an empty dictionary (e.g. all global plugins disabled, no keyword match) fell through both cancellation paths and left the in-flight session running. The pre-refactor code cancelled _updateSource unconditionally before the Count check; this restores that invariant by hoisting previousSession.Cancel() + DisposeWhenComplete() out of the Count > 0 block while keeping _currentSession bound to the just-cancelled session (so late IResultUpdated events continue to suppress via their captured token rather than falling back to CancellationToken.None). 2. QuerySession.CancelAndWait — on timeout, the CTS was disposed unconditionally while the tracked task may still have been running. That violates the refactor's own "tasks never observe a disposed CTS" invariant: any future plugin path that touches token WaitHandles (Register, WaitOne) after timeout would crash with ObjectDisposedException. The fix gates dispose on the completed flag and, on timeout, defers disposal via ContinueWith with DenyChildAttach (mirroring DisposeWhenComplete's pattern). Also softens the QuerySession remarks: the "structural guarantee" only holds for callers that capture Token into a local. The class can't prevent a future author from re-reading _currentSession.Token inside a body, which would reintroduce the original bug. Added regression test CancelAndWait_DoesNotDisposeCtsWhileTaskStillRuns covering the new deferred-disposal behavior; all 10 Wox.Test QuerySessionTest cases pass. --- ADO: https://microsoft.visualstudio.com/DefaultCollection/OS/_workitems/edit/55588441/ Co-authored-by: Copilot <[email protected]>

…n-oom

Copilot/review pull 48394

…ellation-oom fix(run): prevent canceled query task buildup

github-actions · 2026-07-10T02:44:32Z

Thank you for contributing to PowerToys. We've detected that this PR might include a new or modified telemetry event. Please ensure the following before merging:

Add your telemetry events to DATA_AND_PRIVACY.md within this PR.
Reach out to Jessica (@chatasweetie) to follow up on the next steps: https://aka.ms/pt-telemetry-process

fix: resolve StyleCop SA1214, SA1508, and CA1068 errors in PT Run

Co-authored-by: Copilot App <[email protected]> Copilot-Session: 54e1bb28-edae-496b-8211-0e1592ddc985

github-advanced-security AI found potential problems Jun 8, 2026

View reviewed changes

Comment thread src/modules/launcher/PowerLauncher/ViewModel/MainViewModel.cs Fixed

This comment has been minimized.

Sign in to view

crutkas added Product-PowerToys Run Improved app launch PT Run (Win+R) Window Needs-Review This Pull Request awaits the review of a maintainer. labels Jun 9, 2026

github-project-automation Bot added this to Needs Attention Jun 9, 2026

crutkas added the bug Something isn't working label Jun 9, 2026

crutkas and others added 2 commits June 8, 2026 18:51

Update MainViewModel.cs

dd75818

crutkas requested a review from yeelam-gordon June 23, 2026 11:19

Copilot AI and others added 11 commits July 9, 2026 23:05

Harden Peek focused-preview Ctrl+W handling

6ad715a

Use typed Peek keyboard hook data

ff91f13

Update MainViewModel.cs

483b169

Fix final-sort query session lifetime

106cf20

Merge branch 'microsoft:main' into user/crutkas/fix-ptrun-cancellatio…

352bdcb

…n-oom

Revert unauthorized final-sort review changes

d279c18

Merge pull request #12 from microsoft/copilot/review-pull-48394

b5aef35

Copilot/review pull 48394

Revert unrelated Peek changes

04d22da

Copilot AI mentioned this pull request Jul 10, 2026

Revert unrelated Peek changes from PT Run branch crutkas/autoUpgradeAttempt#13

Merged

10 tasks

crutkas and others added 5 commits July 9, 2026 14:49

Merge pull request #13 from crutkas/copilot/usercrutkasrevert-peek-files

d3866c0

Fix PowerToys Run query cancellation lifetime

f2b9d6c

Harden query generation synchronization

5b14c7f

Address cancellation review feedback

581c101

Clarify cancellation safeguards

11c77b2

Copilot AI and others added 5 commits July 10, 2026 01:22

Merge base branch and resolve conflicts

1605058

Fix query cancellation review findings

fb550c1

Correct placement of query helpers

af9edae

Queue latest per-plugin queries safely

3cf676c

Merge pull request #14 from crutkas/copilot/usercrutkasfix-ptrun-canc…

ddb33ac

…ellation-oom fix(run): prevent canceled query task buildup

github-actions Bot requested a review from chatasweetie July 10, 2026 02:44

Copilot AI and others added 2 commits July 10, 2026 05:56

fix: resolve StyleCop SA1214, SA1508, and CA1068 errors in PT Run

962fa4c

Merge pull request #16 from crutkas/copilot/fix-ptrun-stylecop

60acfe7

fix: resolve StyleCop SA1214, SA1508, and CA1068 errors in PT Run

yeelam-gordon mentioned this pull request Jul 11, 2026

[review-mirror] Fix PT Run ThreadPool worker leak from stale query cancellation yeelam-gordon/PowerToys#11

Open

MuyuanMS reviewed Jul 27, 2026

View reviewed changes

Comment thread src/modules/launcher/PowerLauncher/ViewModel/MainViewModel.cs Outdated

MuyuanMS reviewed Jul 27, 2026

View reviewed changes

Comment thread src/modules/launcher/PowerLauncher/ViewModel/MainViewModel.cs

MuyuanMS reviewed Jul 27, 2026

View reviewed changes

Comment thread src/modules/launcher/PowerLauncher/ViewModel/MainViewModel.cs Outdated

MuyuanMS reviewed Jul 27, 2026

View reviewed changes

Comment thread src/modules/launcher/PowerLauncher/ViewModel/MainViewModel.cs Outdated

crutkas and others added 3 commits July 27, 2026 13:51

Fix delayed result selection race

10250b6

Co-authored-by: Copilot App <[email protected]> Copilot-Session: 54e1bb28-edae-496b-8211-0e1592ddc985

Address remaining PT Run query review feedback

fb2a2c0

Co-authored-by: Copilot App <[email protected]> Copilot-Session: 54e1bb28-edae-496b-8211-0e1592ddc985

Restore PT Run query phase ordering

fb29a4c

Co-authored-by: Copilot App <[email protected]> Copilot-Session: 54e1bb28-edae-496b-8211-0e1592ddc985

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix PT Run ThreadPool worker leak from stale query cancellation - #48394

Fix PT Run ThreadPool worker leak from stale query cancellation#48394
crutkas wants to merge 30 commits into
microsoft:mainfrom
crutkas:user/crutkas/fix-ptrun-cancellation-oom

crutkas commented Jun 8, 2026 •

edited

Loading

Uh oh!

Uh oh!

This comment has been minimized.

crutkas commented Jun 9, 2026

Uh oh!

yeelam-gordon commented Jun 12, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

crutkas commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root cause

Changes

Tests

Manual validation

Uh oh!

Uh oh!

This comment has been minimized.

crutkas commented Jun 9, 2026

Uh oh!

yeelam-gordon commented Jun 12, 2026

What this commit adds on top of yours

New regression tests (src/modules/launcher/Wox.Test/QuerySessionTest.cs, 9 tests, all passing)

Suggested manual smoke

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

crutkas commented Jun 8, 2026 •

edited

Loading

New regression tests (`src/modules/launcher/Wox.Test/QuerySessionTest.cs`, 9 tests, all passing)