When we use AutoTVM to tune on remote devices, we will introduce data copy to remote devices and bring network overhead: https://github.com/apache/incubator-tvm/blob/9d2c984ab633b372cd31054a5abd84a2e16f6220/python/tvm/autotvm/measure/measure_methods.py#L484
To solve this, we could implement a tvm.nd.non_empty in the tvm runtime, then we can do the random fill on the target device without copying over the network. I have implemented one version of it and plan to port it to upstream.
Open this to track this issue.
When we use AutoTVM to tune on remote devices, we will introduce data copy to remote devices and bring network overhead: https://github.com/apache/incubator-tvm/blob/9d2c984ab633b372cd31054a5abd84a2e16f6220/python/tvm/autotvm/measure/measure_methods.py#L484
To solve this, we could implement a
tvm.nd.non_emptyin the tvm runtime, then we can do the random fill on the target device without copying over the network. I have implemented one version of it and plan to port it to upstream.Open this to track this issue.