Unexpected behaviour when running regression test

[kursatyurt]

Describe the bug

During the regression tests of #1977, I noticed that some runs weirdly does not pass the test. After the third attempt regression test passed successfully.

Please see the attempts https://github.com/su2code/SU2/actions/runs/4480654814/jobs/7883573846 hybrid_regression_AD.py.

Looks weird to me.

[pcarruscag]

Maybe a little bit of non determinism because of atomic adjoint updates. @jblueh by chance did you run thread sanitizer after the codi 2 update?

[jblueh]

Could be a data race, thanks for reporting @kursatyurt. Last I ran the thread sanitizer during the CoDiPack 2 update, nothing came up at the time. The failing test is the recently added pywrapper_CFD_AD_MeshDisp. I'll look into it.

[jblueh]

I reran the thread sanitizer analysis and it didn't detect any data races. The deviations are small, so I'm with @pcarruscag that this is probably inherent non-determinism. I relaxed the tolerances in #1984.