Swap Loss not Decreasing in Contrastive Learning

[P1terQ]

In the paper, the authors highlight that using contrastive learning can carry more precious environmental information, and the t-SNE visualization indicates the stronger representation ability of this method.

However, when I attempted to run the open-source code, the swap loss didn't drop, regardless of short or long iterations.

I am wondering if this is an expected behavior for contrastive learning algorithms, or if there might be an issue with my implementation.

[mahanxin]

@Junfeng-Long
I have met the same proplem about swap loss, the curve is as the following figure

and I also conduct t-SNE to analyze the latent space output, but unfortunately it could not identify the terrain.

[CSU-BossYu]

I have encountered exactly the same issue, and I agree with your view that the teacher and student encoders in contrastive learning take shortcuts instead of learning terrain features.