Ensure value is on the on-disk cache before returning from ensure().

[cjgillot]

The current logic for ensure() a query just checks that the node is green in the dependency graph.
However, a lot of places use ensure() to prevent the query from being called later. This is the case before stealing a query result.

If the query is actually green but the value is not available in the on-disk cache, ensure would return, but a subsequent call to the full query would run the code, and attempt to read from a stolen value.

This PR conforms the query system to the usage by checking whether the queried value is loadable from disk before returning.

Sadly, I can't manage to craft a proper test...

Should fix all instances of "attempted to read from stolen value".

[rustbot]

r? @TaKO8Ki

(rustbot has picked a reviewer for you, use r? to override)

[cjgillot]

@bors try @rust-timer queue

[bors]

⌛ Trying commit add6a8420e4f417e4c82d159d91669eb053bb8a1 with merge 089156c8708f6aa6054e63459ce1726976130c3a...

[bors]

☀️ Try build successful - checks-actions
Build commit: 089156c8708f6aa6054e63459ce1726976130c3a (089156c8708f6aa6054e63459ce1726976130c3a)

[rust-timer]

Finished benchmarking commit (089156c8708f6aa6054e63459ce1726976130c3a): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	75.7%	[1.4%, 201.4%]	121
Regressions ❌ (secondary)	56.1%	[0.2%, 633.1%]	88
Improvements ✅ (primary)	-0.2%	[-0.3%, -0.2%]	4
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	73.3%	[-0.3%, 201.4%]	125

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	22.7%	[4.3%, 56.2%]	123
Regressions ❌ (secondary)	19.2%	[1.8%, 108.6%]	70
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.9%	[-1.9%, -1.9%]	1
All ❌✅ (primary)	22.7%	[4.3%, 56.2%]	123

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	85.7%	[1.2%, 188.6%]	122
Regressions ❌ (secondary)	67.3%	[1.4%, 592.2%]	76
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	85.7%	[1.2%, 188.6%]	122

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[cjgillot]

@bors try @rust-timer queue

[bors]

⌛ Trying commit d71f87699257e1483f1058cb010c12b9c9c2e204 with merge c6f196f3b9487da4c5134007caf9c459ccea2f3e...

[bors]

☀️ Try build successful - checks-actions
Build commit: c6f196f3b9487da4c5134007caf9c459ccea2f3e (c6f196f3b9487da4c5134007caf9c459ccea2f3e)

[rust-timer]

Finished benchmarking commit (c6f196f3b9487da4c5134007caf9c459ccea2f3e): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.1%	[3.1%, 3.1%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	3.0%	[2.7%, 3.2%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.6%	[-2.6%, -2.6%]	1
All ❌✅ (primary)	3.0%	[2.7%, 3.2%]	2

[oli-obk]

What do you think about flipping the defaults just to nudge people in the right direction? So renaming ensure to ensure_without_checking_incremental_cache and renaming ensure_with_value to ensure.

[cjgillot]

There are currently 126 uses of ensure() in the codebase, and only <13 have a link with stealing a query result.
Most of them are in rustc_hir_analysis and rustc_interface as ways to drive analysis.

I can rename both to ensure_unchanged (current behaviour, to drive analyses) and ensure_with_value (new behaviour, for steal) to make everything explicit.

[oli-obk]

oof that's gonna be very verbose...

let's land this and I'll think about it a bit more

[oli-obk]

@rustbot author

r=me with CI happy

r? @oli-obk

[cjgillot]

@bors r=oli-obk

[bors]

📌 Commit e955ec0 has been approved by oli-obk

It is now in the queue for this repository.

[bors]

⌛ Testing commit e955ec0 with merge f41927f...

[bors]

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing f41927f to master...

[rust-timer]

Finished benchmarking commit (f41927f): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.6%	[-0.6%, -0.6%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.7%	[-3.7%, -3.7%]	1
Improvements ✅ (secondary)	-3.7%	[-5.1%, -2.3%]	2
All ❌✅ (primary)	-3.7%	[-3.7%, -3.7%]	1

Cycles

This benchmark run did not return any relevant results for this metric.