[Flaky Test] `tests/python/frontend/pytorch/test_forward.py::test_empty_like`

[driazati]

Test tests/python/frontend/pytorch/test_forward.py::test_empty_like https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/3713/tests is flaky and has failed intermittently on main (as discussed in #11896)

[2022-06-29T08:34:53.751Z] _______________________________ test_empty_like ________________________________
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z]     def test_empty_like():
[2022-06-29T08:34:53.751Z]         def test_func(data):
[2022-06-29T08:34:53.751Z]             return torch.empty_like(data)
[2022-06-29T08:34:53.751Z]     
[2022-06-29T08:34:53.751Z] >       verify_model_with_input(test_func, [torch.rand([1, 3, 10, 10]).float()])
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z] tests/python/frontend/pytorch/test_forward.py:3827: 
[2022-06-29T08:34:53.751Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2022-06-29T08:34:53.751Z] tests/python/frontend/pytorch/test_forward.py:221: in verify_model_with_input
[2022-06-29T08:34:53.751Z]     tvm.testing.assert_allclose(baseline_outputs, compiled_output, rtol=1e-5, atol=1e-5)
[2022-06-29T08:34:53.751Z] python/tvm/testing/utils.py:113: in assert_allclose
[2022-06-29T08:34:53.751Z]     np.testing.assert_allclose(actual, desired, rtol=rtol, atol=atol, verbose=True)
[2022-06-29T08:34:53.751Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2022-06-29T08:34:53.751Z] 
[2022-06-29T08:34:53.751Z] x = array([[[[ 7.7170313e-17,  0.0000000e+00,  5.3485164e-26,
[2022-06-29T08:34:53.751Z]            0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
[2022-06-29T08:34:53.751Z]   ...  0.0000000e+00,
[2022-06-29T08:34:53.751Z]            0.0000000e+00,  0.0000000e+00,  0.0000000e+00,
[2022-06-29T08:34:53.751Z]            0.0000000e+00]]]], dtype=float32)
[2022-06-29T08:34:53.751Z] y = array([[[[0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0....         [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[2022-06-29T08:34:53.751Z]          [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]]]], dtype=float32)
[2022-06-29T08:34:53.751Z] func = <ufunc 'isnan'>, hasval = 'nan'
[2022-06-29T08:34:53.751Z] 

From the diff there doesn't appear to be an actual difference but we should disable anyways to not block while its figured out

cc @shingjan

[shingjan]

@driazati It's possible that empty is flaky as well bcoz it shares a same implementation virtually with empty_like. vvv Altan's comments

[altanh]

empty_like ostensibly returns uninitialized memory so we really shouldn't test this numerically (https://pytorch.org/docs/stable/generated/torch.empty_like.html)

[shingjan]

@altanh What about we instead test if the outputs are of the same shape under the condition that torch.empty_like may return uninitialized memory in this case.

[masahi]

I agree that we should only verify the output shape.

[shingjan]

#11890 is out to fix this.