Arm64: Forward memset/memcpy to CRT implementation

[kunalspathak]

In x64, memset and memmove is forwarded to the CRT implementation as seen below:

runtime/src/coreclr/vm/amd64/CrtHelpers.asm

Line 49 in a85a2f5

jmp memset ; forward to the CRT implementation

runtime/src/coreclr/vm/amd64/CrtHelpers.asm

Line 83 in a85a2f5

jmp memmove ; forward to the CRT implementation

However, in Arm64, they are hand written in assembly as seen in https://github.com/dotnet/runtime/blob/2453f16807b85b279efc26d17d6f20de87801c09/src/coreclr/vm/arm64/crthelpers.asm. Experiment if CRT implementation of memset/memmove for Arm64 is faster and if yes, just use it. We might also need to readjust the heuristics that we do today to unroll the copy block.

Here is the benchmark run difference between x64 (base) and arm64 (diff)

Here is the x64 code for CopyBlock128() benchmark that just uses memcpy:

G_M19447_IG03:
       lea      rcx, bword ptr [rsp+08H]
       lea      rdx, bword ptr [rsp+88H]
       mov      r8d, 128
       call     CORINFO_HELP_MEMCPY
       inc      edi
       cmp      edi, 100
       jl       SHORT G_M19447_IG03

But Arm64 unrolls the loop to do so

G_M19447_IG03:
            ldr     x1, [fp,#152]
            str     x1, [fp,#24]
            ldp     q16, q17, [fp,#160]
            stp     q16, q17, [fp,#32]
            ldp     q16, q17, [fp,#192]
            stp     q16, q17, [fp,#64]
            ldp     q16, q17, [fp,#224]
            stp     q16, q17, [fp,#96]
            ldr     q16, [fp,#0xd1ffab1e]
            str     q16, [fp,#128]
            ldr     x1, [fp,#0xd1ffab1e]
            str     x1, [fp,#144]
            add     w0, w0, #1
            cmp     w0, #100
            blt     G_M19447_IG03

I will perform some experiments and update the results here.

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In x64, memset and memmove is forwarded to the CRT implementation as seen below:

runtime/src/coreclr/vm/amd64/CrtHelpers.asm

Line 49 in a85a2f5

jmp memset ; forward to the CRT implementation

runtime/src/coreclr/vm/amd64/CrtHelpers.asm

Line 83 in a85a2f5

jmp memmove ; forward to the CRT implementation

However, in Arm64, they are hand written in assembly as seen in https://github.com/dotnet/runtime/blob/2453f16807b85b279efc26d17d6f20de87801c09/src/coreclr/vm/arm64/crthelpers.asm. Experiment if CRT implementation of memset/memmove for Arm64 is faster and if yes, just use it. We might also need to readjust the heuristics that we do today to unroll the copy block.

Here is the benchmark run difference between x64 (base) and arm64 (diff)

Here is the x64 code for CopyBlock128() benchmark that just uses memcpy:

G_M19447_IG03:
       lea      rcx, bword ptr [rsp+08H]
       lea      rdx, bword ptr [rsp+88H]
       mov      r8d, 128
       call     CORINFO_HELP_MEMCPY
       inc      edi
       cmp      edi, 100
       jl       SHORT G_M19447_IG03

But Arm64 unrolls the loop to do so

G_M19447_IG03:
            ldr     x1, [fp,#152]
            str     x1, [fp,#24]
            ldp     q16, q17, [fp,#160]
            stp     q16, q17, [fp,#32]
            ldp     q16, q17, [fp,#192]
            stp     q16, q17, [fp,#64]
            ldp     q16, q17, [fp,#224]
            stp     q16, q17, [fp,#96]
            ldr     q16, [fp,#0xd1ffab1e]
            str     q16, [fp,#128]
            ldr     x1, [fp,#0xd1ffab1e]
            str     x1, [fp,#144]
            add     w0, w0, #1
            cmp     w0, #100
            blt     G_M19447_IG03

Author:	kunalspathak
Assignees:	kunalspathak
Labels:	area-CodeGen-coreclr, untriaged
Milestone:	-

[kunalspathak]

@dotnet/jit-contrib

[kunalspathak]

@a74nh - FYI

[kunalspathak]

If we switch to CRT memset we will see some benefits (at least) on linux because LLVM uses DC ZVA for that.

https://github.com/llvm/llvm-project/blob/main/libc/AOR_v20.02/string/aarch64/memset.S

Likewise memcpy is much better - https://github.com/llvm/llvm-project/blob/main/libc/AOR_v20.02/string/aarch64/memcpy.S and https://github.com/llvm/llvm-project/blob/main/libc/AOR_v20.02/string/aarch64/memcpy-advsimd.S

[EgorBo]

From my understanding we do use native memset on windows-arm64, it's just *nix-arm64 - should be easy to fix

[echesakov]

If we switch to CRT memset we will see some benefits (at least) on linux because LLVM uses DC ZVA for that.

This is correct, although, I would not expect to see much of the performance improvement (especially, for smaller block sizes) only due to using DC ZVA. Using DC ZVA requires zeroing up to 64 bytes at the beginning and the end of the block manually. It also requires checking for DC ZVA availability on the target platform (e.g. as in https://github.com/llvm/llvm-project/blob/881350a92d821d4f8e4fa648443ed1d17e251188/libc/AOR_v20.02/string/aarch64/memset.S#L81) at runtime.

It might be interesting to have two JIT_MemSet helpers - one that uses DC ZVA and another that doesn't. JIT will do the check for DC ZVA availability at compile time and emit a call to corresponding helper.

[kunalspathak]

Few years ago, we already switched to using platform memset/memcpy in linux/arm64 in dotnet/coreclr#17536 and likewise for windows and linux on x64 in dotnet/coreclr#25750. The only left out was windows/arm64 which was still using the hand written assembly code. The platform implementation is highly optimized and has ways to use appropriate routines based on the size of block.

[kunalspathak]

All the benchmarks access the array element using ref and currently we don't generate decent addressing mode for them. Tracked in #64819.