Change the hashcounts in raw Lit variants from usize to u16.

[nnethercote]

This reduces the size of Token from 32 bytes to 24 bytes on 64-bit
platforms.

[rust-highfive]

r? @michaelwoerister

(rust_highfive has picked a reviewer for you, use r? to override)

[michaelwoerister]

Thanks, @nnethercote!

Re-assigning to @alexcrichton, since it touches libproc_macro.

r? @alexcrichton

[alexcrichton]

@bors: r+

[bors]

📌 Commit 4d34bfd has been approved by alexcrichton

[Mark-Simulacrum]

Out of interest, it feels like this could go lower -- I can't imagine a valid use case for more than 256 of these; did you investigate whether that would net further savings?

[nnethercote]

Lit now has a 1 byte tag, a 2 byte hash count for the StrRaw and ByteStrRaw variants, and a 4 byte ast::Name, all of which fits into 8 bytes along with one byte of padding. Reducing the hash count to 1 byte wouldn't change things, because it would be replaced by an extra byte of padding.

We could get the size of Token down from 24 bytes to 16 bytes with some more effort, by shrinking the Literal variant. The Option<ast::Name> currently takes up 8 bytes, which is wasteful. I fiddled with making ast::Name use NonZeroU32, but it was a bit of a hassle. Another possibility is to rearrange the Literal/Lit nesting, which I also deemed not worth the effort, because the measurable benefit of shrinking from 32 to 24 bytes was negligible. Others might have more luck.

[bors]

⌛ Testing commit 4d34bfd with merge 3dfda16...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 3dfda16 to master...

[eddyb]

Oh, btw, the Option around the suffix in Literal could be removed (and instead you can check that it's equal to the empty Symbol, instead of None).

[nnethercote]

I did a rustc-perf run for this. It was a measurable win for some of the coercions and html5ever jobs:

coercions
        avg: -0.5%      min: -1.7%      max: 0.1%
html5ever-opt
        avg: -0.7%      min: -1.3%      max: -0.3%
html5ever-check
        avg: -0.7%      min: -1.2%      max: -0.3%
coercions-opt
        avg: -0.3%      min: -1.0%      max: -0.1%

[nnethercote]

Oh, btw, the Option around the suffix in Literal could be removed (and instead you can check that it's equal to the empty Symbol, instead of None).

The downside is that this makes Literal harder to work with -- you can't pattern match easily the way you can with Option, and creating an empty suffix involves a call to intern().

[eddyb]

@nnethercote Note that almost nobody should be touching literals though that internal enum, only type-checking and MIR construction. Also, the empty string is preinterned:

rust/src/libsyntax_pos/symbol.rs

Line 302 in 64e6dda

(0, Invalid, "")

(although "Invalid" is not the best name for it - cc @petrochenkov @jseyfried)

[nnethercote]

The preinterning of the empty string helps; thank you for pointing that out.

But overall it's not worth getting rid of the Option<>. I tried it and got it working, but the code becomes uglier, and it's a very small slowdown -- the time saved by making Token smaller is slightly outweighed by slower "is there a suffix?" checks.