Make lowering incremental, take 3/N by cjgillot · Pull Request #142830 · rust-lang/rust

cjgillot · 2025-06-21T15:26:00Z

View all comments

Rebase of #88186 that was rebased by #127262

No real change in design, so I'm not expecting a perf miracle.

r? @ghost

cjgillot · 2025-06-21T16:12:09Z

@bors try @rust-timer queue

bors · 2025-06-21T16:13:19Z

⌛ Trying commit 0e2f90b with merge a6b131d...

Make lowering incremental, take 3/N Rebase of #88186 that was rebased by #127262 No real change in design, so I'm not expecting a perf miracle. r? `@ghost`

bors · 2025-06-21T18:43:11Z

☀️ Try build successful - checks-actions
Build commit: a6b131d (a6b131d42573c085b032a153779366b448148b9d)

rust-timer · 2025-06-21T20:41:55Z

Finished benchmarking commit (a6b131d): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.8%	[0.2%, 1.8%]	157
Regressions ❌ (secondary)	1.1%	[0.2%, 4.3%]	69
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.3%]	5
Improvements ✅ (secondary)	-0.3%	[-0.5%, -0.2%]	7
All ❌✅ (primary)	0.7%	[-0.5%, 1.8%]	162

Max RSS (memory usage)

Results (primary 10.5%, secondary 10.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	10.5%	[3.1%, 28.6%]	249
Regressions ❌ (secondary)	10.1%	[2.2%, 31.4%]	142
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	10.5%	[3.1%, 28.6%]	249

Cycles

Results (primary 2.5%, secondary 4.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.5%	[1.4%, 5.5%]	35
Regressions ❌ (secondary)	4.1%	[1.8%, 7.5%]	22
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.5%	[1.4%, 5.5%]	35

Binary size

Results (primary -0.1%, secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	82
Improvements ✅ (secondary)	-0.2%	[-0.9%, -0.0%]	43
All ❌✅ (primary)	-0.1%	[-0.2%, -0.0%]	82

Bootstrap: 690.042s -> 689.298s (-0.11%)
Artifact size: 371.86 MiB -> 371.81 MiB (-0.01%)

cjgillot · 2025-06-22T16:12:05Z

@bors try @rust-timer queue

bors · 2025-06-22T16:13:39Z

⌛ Trying commit 8ab4e63 with merge 31325ff...

Make lowering incremental, take 3/N Rebase of #88186 that was rebased by #127262 No real change in design, so I'm not expecting a perf miracle. r? `@ghost`

bors · 2025-06-22T18:48:06Z

☀️ Try build successful - checks-actions
Build commit: 31325ff (31325ff312ad91d8a2dd156c832cacc0435f9ea1)

rust-timer · 2025-06-22T21:45:31Z

Finished benchmarking commit (31325ff): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.3%]	120
Regressions ❌ (secondary)	0.8%	[0.2%, 3.2%]	77
Improvements ✅ (primary)	-0.7%	[-1.3%, -0.2%]	12
Improvements ✅ (secondary)	-0.6%	[-1.3%, -0.2%]	37
All ❌✅ (primary)	0.4%	[-1.3%, 1.3%]	132

Max RSS (memory usage)

Results (primary 10.8%, secondary 10.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	10.8%	[2.3%, 29.1%]	248
Regressions ❌ (secondary)	10.4%	[1.9%, 31.7%]	143
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	10.8%	[2.3%, 29.1%]	248

Cycles

Results (primary 2.1%, secondary 4.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.1%	[1.2%, 3.8%]	20
Regressions ❌ (secondary)	4.1%	[1.7%, 6.6%]	16
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.1%	[1.2%, 3.8%]	20

Binary size

Results (primary -0.1%, secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	83
Improvements ✅ (secondary)	-0.2%	[-0.9%, -0.0%]	43
All ❌✅ (primary)	-0.1%	[-0.2%, -0.0%]	83

Bootstrap: 689.082s -> 688.79s (-0.04%)
Artifact size: 371.89 MiB -> 371.87 MiB (-0.00%)

Co-authored-by: Camille Gillot <[email protected]>

rustbot · 2026-06-05T08:24:28Z

Some changes occurred in coverage instrumentation.

cc @Zalathar

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

cjgillot · 2026-06-05T08:26:16Z

I don't think I will be able to do much better in terms of perf. Meanwhile, it unblocks making progress on more incremental name resolution.

r? @petrochenkov
cc @oli-obk

petrochenkov · 2026-06-05T14:52:36Z

+#[derive(Debug)]
+pub enum AstOwner {
+    NonOwner,
+    Synthetic(rustc_span::def_id::LocalDefId),


Suggested change

Synthetic(rustc_span::def_id::LocalDefId),

Synthetic(LocalDefId),

View changes since the review

petrochenkov · 2026-06-05T14:55:30Z

+#[derive(Debug)]
+pub enum AstOwner {
+    NonOwner,
+    Synthetic(rustc_span::def_id::LocalDefId),


This is used specifically for nested imports, maybe use a more specific name and add an explanation comment?

View changes since the review

petrochenkov · 2026-06-05T15:00:04Z

-    query resolver_for_lowering_raw(_: ()) -> (&'tcx Steal<(ty::ResolverAstLowering<'tcx>, Arc<ast::Crate>)>, &'tcx ty::ResolverGlobalCtxt) {
+    query resolver_for_lowering_raw(_: ()) -> (
+        &'tcx Steal<ty::ResolverAstLowering<'tcx>>,
+        &'tcx Steal<ast::Crate>,


It would be useful to add a comment telling why the Steals are used here.

View changes since the review

petrochenkov · 2026-06-05T15:00:33Z

    }

+    query index_ast(_: ()) -> &'tcx IndexVec<LocalDefId, Steal<(
+        Arc<ty::ResolverAstLowering<'tcx>>,


It would be useful to add a comment telling why the Arcs are used here.

View changes since the review

petrochenkov · 2026-06-05T15:02:59Z

+    query lower_to_hir(key: LocalDefId) -> hir::MaybeOwner<'tcx> {
        eval_always
-        desc { "lowering the delayed AST owner `{}`", tcx.def_path_str(def_id) }
+        desc { "lowering HIR for `{}`", tcx.def_path_str(key.to_def_id()) }


Suggested change

desc { "lowering HIR for `{}`", tcx.def_path_str(key.to_def_id()) }

desc { "lowering HIR for `{}`", tcx.def_path_str(def_id) }

View changes since the review

petrochenkov · 2026-06-05T15:06:32Z

    /// Pre-computed hash of the full HIR. Used in the crate hash. Only present
    /// when incr. comp. is enabled.
-    pub opt_hash_including_bodies: Option<Fingerprint>,
+    pub opt_hash: Option<Fingerprint>,


Does the hash still include bodies?
Is it not valuable to mention it? (Perhaps it can be mentioned in the comment above.)

View changes since the review

petrochenkov · 2026-06-05T15:12:06Z

Started reviewing today, starting from the easy parts, will continue on Monday.

rust-bors · 2026-06-05T22:40:49Z

☔ The latest upstream changes (presumably #157504) made this pull request unmergeable. Please resolve the merge conflicts.

petrochenkov · 2026-06-08T12:42:02Z

 /// Runs all analyses that we guarantee to run, even if errors were reported in earlier analyses.
 /// This function never fails.
 fn run_required_analyses(tcx: TyCtxt<'_>) {
+    tcx.ensure_done().early_lint_checks(());


Could you add a comment explaining why it's needed here?

View changes since the review

petrochenkov · 2026-06-08T13:00:50Z

            ident_and_label_to_local_id: Default::default(),
            #[cfg(debug_assertions)]
-            node_id_to_local_id: Default::default(),
+            node_id_to_local_id: [(owner, hir::ItemLocalId::ZERO)].into_iter().collect(),


Note: this looks similar to 3a2ae6a, but in the opposite direction.

View changes since the review

petrochenkov · 2026-06-08T13:13:39Z

+                span,
+                vis: Visibility { kind: VisibilityKind::Public, span, tokens: None },
+                // Lacking a better choice, we replace the contents with a macro call.
+                // Unexpanded macros should never reach lowering, so this is not confusing.


Macro expansion uses the same approach (compiler\rustc_expand\src\placeholders.rs), maybe it can even be reused here.

View changes since the review

petrochenkov · 2026-06-08T13:16:17Z

+            node: impl FnOnce(Box<Item<K>>) -> AstOwner,
+        ) {
+            let dummy = self.make_dummy(item.id, item.span, dummy);
+            let item = std::mem::replace(item, *dummy);


Suggested change

let item = std::mem::replace(item, *dummy);

let item = mem::replace(item, *dummy);

Here and in other places (mem is already imported).

View changes since the review

petrochenkov · 2026-06-08T13:17:31Z

 pub type ForeignItem = Item<ForeignItemKind>;

+#[derive(Debug)]
+pub enum AstOwner {


If it's only used in AST lowering, then maybe it's better to move it there.

View changes since the review

petrochenkov · 2026-06-08T13:27:31Z

+                .map(|kind| {
+                    let id = id.take().unwrap_or_else(|| {
+                        let next = self.next_node_id;
+                        self.next_node_id.increment_by(1);


I'm not sure what's going on here.
Does this happen for nested use items?
What the old AST lowerer did in this case?
What are the implications of new node ids being created here?

View changes since the review

Looks like a replacement for the change in compiler/rustc_ast_lowering/src/block.rs.
Deserves a comment in any case.

petrochenkov · 2026-06-08T13:31:00Z

-        } else {
-            lowerer.lower_node(def_id);
+    let ast_index = tcx.index_ast(());
+    let resolver_and_node = ast_index.get(def_id).map(Steal::steal);


In which cases the def_id is not in the map?

View changes since the review

petrochenkov · 2026-06-08T13:32:56Z

+    let fallback_to_parent = |parent_id| {
+        // The item did not exist in the AST, it was created by its parent.
+        let mut parent_info = tcx.lower_to_hir(parent_id);
+        if let hir::MaybeOwner::NonOwner(hir_id) = parent_info {


I'm again trying to understand in which cases it's possible for the parent to be a non-owner.
Could you add a comment?

View changes since the review

petrochenkov · 2026-06-08T13:35:35Z

+        AstOwner::ImplItem(item) => item_lowerer.lower_impl_item(&item),
+        AstOwner::ForeignItem(item) => item_lowerer.lower_foreign_item(&item),
+        AstOwner::Synthetic(parent_id) => fallback_to_parent(*parent_id),
+        AstOwner::NonOwner => fallback_to_parent(tcx.local_parent(def_id)),


In which cases AstOwner::NonOwners appear at all? Only during error recovery?

View changes since the review

petrochenkov · 2026-06-08T13:39:00Z

        let attrs = hir::AttributeMap { map: attrs, opt_hash: attrs_hash, define_opaque };

-        self.arena.alloc(hir::OwnerInfo { nodes, parenting, attrs, trait_map, delayed_lints })
+        let opt_hash = if self.tcx.needs_hir_hash() {


Suggested change

let opt_hash = if self.tcx.needs_hir_hash() {

let opt_hash = self.tcx.needs_hir_hash().then(|| ...);

View changes since the review

rustbot added PG-exploit-mitigations Project group: Exploit mitigations T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jun 21, 2025

This comment was marked as outdated.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 21, 2025

This comment was marked as outdated.

Sign in to view

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jun 21, 2025

cjgillot force-pushed the lower-incr-2 branch from d715668 to 0e2f90b Compare June 21, 2025 16:11