[AMP] Vulkan Support for Mixed Precision Pass

[AndrewZhaoLuo]

Solve issues and make modifications to support Vulkan for mixed precision pass here: #8069

Current initial issues as described by @Lunderberg

On the vulkan side, it's something similar with the validation checks failing an alignment rule.

Check failed: res == SPV_SUCCESS (-10 vs. 0) : index=27 error:Structure id 12 decorated as Block for variable in StorageBuffer storage class must follow standard storage buffer layout rules: member 0 contains an array with stride 6 not satisfying alignment to 8
%_struct_12 = OpTypeStruct %_runtimearr_v3half

This issue is completed when unit tests can pass for Vulkan target.

[AndrewZhaoLuo]

cc @Lunderberg

[masahi]

I can confirm that TF2 ssd mobilenet v2 can be converted to fp16 and runs on vulkan (AMD) and opencl (Intel Ice lake), if I disable vectorization on fp16 at https://github.com/apache/tvm/blob/main/python/tvm/topi/cuda/injective.py#L54-L55 (cc @Lunderberg).

But the output from fp16 is a bit off compared to fp32 (on both vk and ocl). Also on other models I got type mismatch float32 vs float16 error at Relay level.

fp32
Mean Squared Error of output 0 and shape (1, 100, 4) is 9.562732618023824e-15
Mean Squared Error of output 1 and shape (1, 100) is 0.0
Mean Squared Error of output 2 and shape (1, 100) is 4.539840725570343e-13
Mean Squared Error of output 3 and shape (1,) is 0.0
Mean Squared Error of output 4 and shape (1, 12804, 4) is 3.1784283863710294e-13
Mean Squared Error of output 5 and shape (1, 12804, 91) is 2.194374375133825

fp16
Mean Squared Error of output 0 and shape (1, 100, 4) is 0.01756046526134014
Mean Squared Error of output 1 and shape (1, 100) is 8.5600004196167
Mean Squared Error of output 2 and shape (1, 100) is 5.59057809823571e-07
Mean Squared Error of output 3 and shape (1,) is 0.0
Mean Squared Error of output 4 and shape (1, 12804, 4) is 5.098227120470256e-07
Mean Squared Error of output 5 and shape (1, 12804, 91) is 2.664001463870136e-09

[Lunderberg]

I ran into a few issues with vectorization when I was running ResNet50 with float16. If you apply PR #8528 , is it still necessary to disable the vectorization?

[Lunderberg]

Regarding the numerical accuracy, I had a few maybe-similar issues when putting together the unittests in #8529. There's a decent number of schedules that perform poorly if the accumulator dtype is float16. I had a short discussion with @AndrewZhaoLuo last week on how best to implement float32 accumulation in the mixed precision pass, but haven't looked into it much yet.

[masahi]

With #8528, I get this error:

  1: tvm::codegen::CodeGenSPIRV::VisitStmt_(tvm::tir::StoreNode const*)
  0: tvm::codegen::CodeGenSPIRV::StorageInfo::CheckContentType(tvm::runtime::DataType, int)
  File "/home/masa/projects/dev/tvm/src/target/spirv/codegen_spirv.h", line 160
TVMError: 
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------

  Check failed: type == expected_type (int32 vs. float32) : Attempted to access buffer T_reshape as element type int32 using an index of size 1 when the element type is float32

[masahi]

Vulkan support for fp16 is fully functional, thanks @Lunderberg