Optimizer hints - Unsafe.Assume

[jkotas]

C/C++ compilers allows passing hints to the optimizer using __assume or similar constructs. It would be useful to have same capability in .NET.

One use case was discussed in https://github.com/dotnet/corefx/issues/5474#issuecomment-172653606:

    Unsafe.Assume((p & 0x1F) == 0);
    Unsafe.Write<Vector<double>>(p, v);

category:proposal
theme:optimization
skill-level:expert
cost:large

[jkotas]

cc @CarolEidt

[CarolEidt]

It would be great to design such a feature to capture both the assertion about the expected behavior as well as allowing that behavior to be validated. That is, perhaps in Debug one might check the condition, while when generating optimized code (e.g. in a Release build), one might assume it to be true. We've already got Debug.Assert and Trace.Assert. Maybe this should be Unsafe.Assert, and it would check the condition in Debug builds, but assume the condition for optimization purposes in the Release build.

[nietras]

Sounds interesting. How would this be expressed in IL?

[mikedn]

Just like any other method call. It's just that the JIT assumes that the expression passed as argument always evaluates to true and can attempt to infer useful information from that.

That said, the "infer" part is a bit of a can of worms. Any bool expression can be used but only the simplest ones will be useful.

[GSPP]

This can be tremendously useful because it's a way turn off any JIT safety. x != null turns off a null check. index >= 0 && index < array.Length turns off a bounds check.

This design is very extensible because with one API addition more and more assumptions can be made to work in the future.

There should be a small catalog of patterns that are recognized.

(Update: Removed the builtin_expect thought and posted them elsewhere.)

[nietras]

@GSPP elsewhere would then be https://github.com/dotnet/coreclr/issues/6024 just so we have the link :)

[Daniel-Svensson]

Sounds like you want the exising Contract.Assume but without the tooling.

https://msdn.microsoft.com/en-us/library/system.diagnostics.contracts.contract.assume(v=vs.110).aspx

[redknightlois]

@Daniel-Svensson Using Contract.Assume is not a bad idea at all (it is there already and there are many annotations that the JIT could understand if properly stated).

Another interesting use of Assume is hinting that a particular store on an array should be a non-temporal write. Therefore, you can use non-temporal instructions and avoiding polutting the cache if you are going to write and never read that information in that method. For example:

public int[] Sum ( int[] op1, int[] op2 )
{
    var result = new int[op1.Lenght];
    Contract.Assume( Performance.NonTemporal(result) );

    /// here goes result[i] = op1[i] + op2[i]
}

And that can be done to achieve very interesting results

public int[] Sum ( int[] op1, int[] op2 )
{
    if ( op1.Length > 4096 ) // or whatever it makes sense for the kernel
    {
       Contract.Assume( Performance.NonTemporal(op1) );
       Contract.Assume( Performance.NonTemporal(op2) );
    }
    var result = new int[op1.Lenght];
    Contract.Assume( Performance.NonTemporal(result) );

    /// here goes result[i] = op1[i] + op2[i]
}

This is pretty handy optimization because by the time the kernel ends the cache is poluted with only the last batch of data.

[GrabYourPitchforks]

As an extension to this, it might be useful to specify contracts on return values from methods. A strawman:

[return: AssumeNotNull]
public static object MyMethod();

[return: AssumeNonNegative]
public static int MyOtherMethod();

For example, the expression new Span<T>(Array.Empty<T>()) still performs the null check at the beginning of the ctor because the JIT doesn't have enough information to prove that the return value of Array.Empty<T>() is non-null.

[GSPP]

@GrabYourPitchforks The caller can already specify anything like this:

var result = MyMethod();
Assume(Anything(result));

The advantage of this would be that a single unified system (the assume method call) suffices for anything. This would be per callsite customizable as well.

On the other hand, those attributes would help all callsites and can be library provided.

[GSPP]

A related idea would be to automatically analyze methods and discover invariants such as non-null return value. This seems very expensive to do in the JIT. It could be implemented as a tool that can optionally be run on any assembly or application. The tool would produce a file that records all discovered invariants and data flow information. The JIT would consume that data. This could discover all kinds of useful information such as non-nullness, more precise types, integer ranges and constants.

[redknightlois]

The tool would produce a file that records all discovered invariants and data flow information. The JIT would consume that data. This could discover all kinds of useful information such as non-nullness, more precise types, integer ranges and constants.

You don't need an external file, you can bake that into the assembly itself when building.