Optimize movz/movk on ARM64

[kunalspathak]

Since ARM64 has an ISA with fixed 32-bit instruction width, the move instructions have space for 16-bit unsigned immediate. To move bigger immediate value, we need to move the value in multiple steps using chunks of 16-bits (movz/movk). Due to this, multiple mov instructions are generated to load a single bigger value in register in contrast to x64 where a single mov can load bigger immediate.

I wanted to see if it has been already evaluated or if it is feasible to store such immediate values in a literal pool and load the values from the pool using ldr instead of moving using multi-step movz/movk. I know this would involve accessing memory instead and might end up being slower. In fact, the examples I have tried on clang/gcc, they use movz/movk mostly.

I have also noticed cases where we try to move 2 immediates back to back that are just bytes apart from each other. E.g. Below is the code generated for Vector4.Add(Vector4, Vector4) method where it tries to load the parameters before performing fadd.

        D29D9900          movz    x0, #0xecc8
        F2A785E0          movk    x0, #0x3c2f LSL #16
        F2C051C0          movk    x0, #654 LSL #32
        F9400000          ldr     x0, [x0]            # <==== loads from address 0x6543c2fecc8
        FD400410          ldr     d16, [x0,#8]
        D29D9800          movz    x0, #0xecc0
        F2A785E0          movk    x0, #0x3c2f LSL #16
        F2C051C0          movk    x0, #654 LSL #32    # <==== loads from address 0x6543c2fecc0
        F9400000          ldr     x0, [x0]
        FD400411          ldr     d17, [x0,#8]
        0E31D610          fadd    v16.2s, v16.2s, v17.2s

The 2nd set of move instructions can be optimized by doing sub x0, x0, 8.

Just to get the stats to see how often we need to generate higher immediate values, I ran crossgen tool on all (or most) of the libraries and dumped the assembly code produced from it for ARM64. Then I extracted out the methods and the portion of code that uses movz/movk pair to load the immediates. There were total 191028 methods crossgened out of which 4578 methods contained pair of movz/movk. In all there were 11856 groups of movz/movk instructions.
You can see the summary in movs.txt.

Some of the questions I am thinking about on seeing the above results:

1. Some of the Utf8Parser methods loads constant multiple times and can it be optimized to reuse the value?
2. Is it possible to re-evaluate constant propagation for certain scenarios where the value of constant is big and since it gets propagated, we end up creating these mov sequence multiple times?
3. Should we tweak our inline heuristics because of the amount of code generated for ARM64 is more than that for Intel?
4. Can we eliminate some of the movz/movk by doing arithmetic manipulation, the way I mentioned in above example?
5. And finally, is it beneficial to use literal pool table for certain scenarios?

References:

• https://dinfuehr.github.io/blog/encoding-of-immediate-values-on-aarch64/
• https://azeria-labs.com/memory-instructions-load-and-store-part-4/
• http://www.keil.com/support/man/docs/armasm/armasm_dom1359731147760.htm

category:cq
theme:optimization
skill-level:intermediate
cost:medium

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. Please help me learn by adding exactly one area label.

[kunalspathak]

@TamarChristinaArm , @BruceForstall , @sdmaclea, @dotnet/jit-contrib

[TamarChristinaArm]

Hmm I wouldn't add yet another indirection here with the literal pool. I'm assuming that the address it's building is the address of the managed heap?

        D29D9900          movz    x0, #0xecc8
        F2A785E0          movk    x0, #0x3c2f LSL #16
        F2C051C0          movk    x0, #654 LSL #32
        F9400000          ldr     x0, [x0]            # <==== loads from address 0x6543c2fecc8
        FD400410          ldr     d16, [x0,#8]

Since it's loading an address into x0 and then load a value based on that address.
So isn't 0x6543c2fecc8 the address of a value in the heap?

The 2nd set of move instructions can be optimized by doing sub x0, x0, 8.

You don't need this if it kept the value live longer. I think this is what you meant with no# 2 in your list right?

The block above could be

        D29D9800          movz    x0, #0xecc0
        F2A785E0          movk    x0, #0x3c2f LSL #16
        F2C051C0          movk    x0, #654 LSL #32    # <==== loads from address 0x6543c2fecc0
        F9400000          ldr     x1, [x0]
        FD400411          ldr     d17, [x1,#8]
        F9400000          ldr     x0, [x0, 8]            # <==== loads from address 0x6543c2fecc8
        FD400410          ldr     d16, [x0,#8]

So essentially you pick an anchor use offsets based on that.

If you know the PC value during assembling you can also use adrp + add.
If Indeed the code above is addressing some kind of heap, and if the heap can be aligned to a 4K boundary then you can do

adrp x0, <pcrel offset to 4K aligned heap>
add x0, x0, <offset into heap>

to replace the three movs. On modern cores the latencies would be the same though, but might be easier to share x0 this way if you can't do it via the addressing mode.

[sdmaclea]

• The literal pool contaminates the DCache with data from the instructions.
• Literal pools are often not packed well so the cache will not be used efficiently
• The literal pool stresses the Load/Store side of the processor. So it would perform worse in Load/Store bound situations
• The mov/movk are nearly free from an execution standpoint. They can be executed early in the processor and will rarely block other instructions.
• However as you note mov/movk are denser. They contaminate the ICache. In an ICache bound app, your approach could be beneficial.

So in general I would suspect moving mov/movk to literal pool would reduce performance.

If you know the PC value during assembling

We would know the PC during live JIT, but not during crossgen. We need relocatable code. The loader could fix up these non-relocatable entries but I would doubt it is worth it.

[sdmaclea]

I thought I had added movn to the mov/movk mix. It helped a bit with code density with negative constants.

[TamarChristinaArm]

We would know the PC during live JIT, but not during crossgen. We need relocatable code. The loader could fix up these non-relocatable entries but I would doubt it is worth it.

Is there no support for relocations?

[CarolEidt]

The register allocator attempts to reuse constants within a BasicBlock but it has no notion of reusing a partial constant - i.e. if we have the same high bits in one constant as another, we could in theory avoid redefining those high bits, but that would require a different kind of analysis, and some way to communicate the partial-reuse state of these constants.

[sdmaclea]

Is there no support for relocations?

There is some support. I am not sure how extensive it is. It would require relocation table entries which would bloat/complicate the code and slow loading. The relocation table would require making the loaded code writable, not share with other processes, and run relocation code. It could be expensive from a memory usage perspective.

That was why I guessed it would work better for live JIT in later optimization tiers.

You mentioned constants using PC relative constants to reach the heap. It seems like that would be a new relocation feature....

[briansull]

The CSE phase currently doesn't try to CSE identical large constants, or constants that are within a short distance to each other. It is something that I want to look into.

[kunalspathak]

@briansull - Once we do this, will it optimize these patterns? Here it loads the same MethodTable address twice to create different array size.

        D2888000          movz    x0, #0x4400
        F2B64920          movk    x0, #0xb249 LSL #16
        F2CFFEE0          movk    x0, #0x7ff7 LSL #32
        D2800121          mov     x1, #9
        97E5B98F          bl      CORINFO_HELP_NEWARR_1_VC
        AA0003F3          mov     x19, x0
        D2888000          movz    x0, #0x4400
        F2B64920          movk    x0, #0xb249 LSL #16
        F2CFFEE0          movk    x0, #0x7ff7 LSL #32
        D2800221          mov     x1, #17
        97E5B989          bl      CORINFO_HELP_NEWARR_1_VC
        AA0003F4          mov     x20, x0

[CarolEidt]

@kunalspathak - that's a case where the register allocator might have reused the constant, except that it's only opportunistic and it doesn't do any analysis ahead of time to figure out that in order to reuse in this case it would need to allocate a callee-save register. I wonder if it might be useful to add some analysis for that (i.e. how often would it help, and what would the additional analysis cost in throughput)?