Experiment with lazy allocation for empty DenseBitSet storage

[Jamesbarford]

View all comments

Looking at; #141325 and seeing what impact a naive implementation of lazily instantiating the Vector backed storage has on performance.

r? @ghost

[Jamesbarford]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 4731326 (473132651821eed122d82a410428d80aa10a47fa, parent: ddc1a64229890506f57bae5b270f9e5f604294ec)

[rust-timer]

Finished benchmarking commit (4731326): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.1%, 0.3%]	8
Regressions ❌ (secondary)	0.4%	[0.2%, 1.0%]	14
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.2%]	6
Improvements ✅ (secondary)	-0.2%	[-0.4%, -0.0%]	17
All ❌✅ (primary)	-0.0%	[-0.4%, 0.3%]	14

Max RSS (memory usage)

Results (primary -2.3%, secondary -5.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.3%, -2.3%]	1
Improvements ✅ (secondary)	-5.7%	[-5.7%, -5.7%]	1
All ❌✅ (primary)	-2.3%	[-2.3%, -2.3%]	1

Cycles

Results (primary 2.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.5%	[2.5%, 2.5%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.5%	[2.5%, 2.5%]	1

Binary size

Results (secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.0%	[0.0%, 0.0%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Bootstrap: 510.991s -> 512.432s (0.28%)
Artifact size: 400.76 MiB -> 400.67 MiB (-0.02%)

[Jamesbarford]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 179987b (179987b8ab7e8e221056f70e19468f1fc6c86bbf, parent: 485ec3fbcc12fa14ef6596dabb125ad710499c9e)

[rust-timer]

Finished benchmarking commit (179987b): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.1%, 0.9%]	34
Regressions ❌ (secondary)	0.5%	[0.1%, 1.4%]	36
Improvements ✅ (primary)	-0.4%	[-0.7%, -0.2%]	5
Improvements ✅ (secondary)	-0.6%	[-0.9%, -0.2%]	11
All ❌✅ (primary)	0.4%	[-0.7%, 0.9%]	39

Max RSS (memory usage)

Results (primary -1.9%, secondary 1.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.3%	[1.9%, 2.6%]	4
Improvements ✅ (primary)	-1.9%	[-2.9%, -0.9%]	2
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	1
All ❌✅ (primary)	-1.9%	[-2.9%, -0.9%]	2

Cycles

Results (primary 2.3%, secondary -0.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	1.9%	[1.9%, 1.9%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.7%	[-3.7%, -3.7%]	1
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Binary size

Results (primary -0.1%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.0%]	56
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.0%]	24
All ❌✅ (primary)	-0.1%	[-0.1%, -0.0%]	56

Bootstrap: 517.006s -> 517.556s (0.11%)
Artifact size: 401.43 MiB -> 400.83 MiB (-0.15%)

[Jamesbarford]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: d6d30fa (d6d30fabae7cf7f3c45f45220773691a6a7a25fd, parent: d2f24127d97d8c795d1cf5fbf6037f9885f34d39)

[rust-timer]

Finished benchmarking commit (d6d30fa): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.1%, 1.1%]	23
Regressions ❌ (secondary)	0.7%	[0.3%, 1.3%]	22
Improvements ✅ (primary)	-0.5%	[-1.1%, -0.2%]	11
Improvements ✅ (secondary)	-0.6%	[-1.9%, -0.2%]	31
All ❌✅ (primary)	0.2%	[-1.1%, 1.1%]	34

Max RSS (memory usage)

Results (primary -2.4%, secondary 0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.9%	[1.2%, 2.5%]	2
Improvements ✅ (primary)	-2.4%	[-2.4%, -2.4%]	2
Improvements ✅ (secondary)	-1.6%	[-1.8%, -1.3%]	2
All ❌✅ (primary)	-2.4%	[-2.4%, -2.4%]	2

Cycles

Results (secondary -4.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.7%	[-4.7%, -4.7%]	1
All ❌✅ (primary)	-	-	0

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 518.142s -> 539.371s (4.10%)
Artifact size: 400.91 MiB -> 401.53 MiB (0.15%)

[Jamesbarford]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 82680c4 (82680c4289d4b656a82bb1f5cc146a7587118b57, parent: d2f24127d97d8c795d1cf5fbf6037f9885f34d39)

[rust-timer]

Finished benchmarking commit (82680c4): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 1.1%]	23
Regressions ❌ (secondary)	0.7%	[0.2%, 1.3%]	22
Improvements ✅ (primary)	-0.4%	[-1.1%, -0.2%]	11
Improvements ✅ (secondary)	-0.6%	[-1.9%, -0.2%]	31
All ❌✅ (primary)	0.2%	[-1.1%, 1.1%]	34

Max RSS (memory usage)

Results (primary 1.3%, secondary -0.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	5.0%	[5.0%, 5.0%]	1
Regressions ❌ (secondary)	1.3%	[1.3%, 1.3%]	1
Improvements ✅ (primary)	-2.5%	[-2.5%, -2.5%]	1
Improvements ✅ (secondary)	-1.5%	[-1.7%, -1.4%]	2
All ❌✅ (primary)	1.3%	[-2.5%, 5.0%]	2

Cycles

Results (primary 1.9%, secondary -2.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.9%	[1.7%, 2.1%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.5%	[-2.5%, -2.5%]	1
All ❌✅ (primary)	1.9%	[1.7%, 2.1%]	2

Binary size

Results (secondary -0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	-	-	0

Bootstrap: 518.142s -> 518.044s (-0.02%)
Artifact size: 400.91 MiB -> 401.53 MiB (0.15%)