Question about low performance on AV2 (FastNSF)

[jang-1996]

Hello, thank you very much for your impressive work and for making the code publicly available.

I am currently running FastNSF, and I have a question regarding unexpectedly low performance.

While experimenting with the configuration, I observed the following behavior:
• When I increased early_patience in the config to 30, I could see a clear improvement in performance.
• However, when I further increased early_patience to 150, the performance no longer improved and remained low.

Based on my understanding of the Argoverse 2 (AV2) test dataset, a validation set performance around the 0.4 (Dynamic Bucket-Normalized EPE Mean) range seems reasonable, but my results do not consistently reach this level depending on the early_patience setting.

I was wondering if you could provide some guidance on:
• How early_patience is intended to be tuned for FastNSF
• Whether there are recommended values or best practices for AV2 validation set.
• Or if there are other hyperparameters that are particularly sensitive when optimizing on AV2

Thank you again for your great work, and I would really appreciate any advice or clarification.

Best regards.

[Kin-Zhang]

Thanks for your interest.

I'm not an expert or collaborator in FastNSF, in this repo, it's more like a re-implementation (core-code are strictly from the FastNSF repo, you can have a double-check).

In my experience, FastNSF is not that good for small objects. And for iteration number, I set the default setting from their paper, indeed, 30 could be bette,r but the speed is also lower a lot. I never try increased over to 100. But looks like the OTHER_VEHICLES is the reason why it's not work?

For advice:

• Please try to check FastNSF code to see if my implementation aligns (double-check).
• try to tune other learning rate or early_patience from 30 to 50/80/100 (Maybe you can screenshot the results I can check again with these)
• check their paper about how to tune (actually I'm a little bit forget doing the hyper-parameter tuning here, as afterward EulerFlow did a really-good score in the leaderboard although it's really really slow also.. and I'm a little bit to data-driven methods, like SeFlow, SeFlow++, TeFlow as you can also feel. Optimization-based methods typically require more parameter tuning than feed-forward learning. (from my HiMo paper experience, as I inferred them to our datasets, and it needs a little bit of tuning)
• About the score: Validation has fixed ground truth, as mentioned in Fix scene flow ground truth generation in Argoverse 2 #5, all local validation scores will be much lower performance after undistorting and having a full flow. If the test has 0.20 (TeFlow), then the validation set is around 0.26. And I checked the performance rank also align no matter which set you infer on (if local val TeFlow better than SeFlow then the test set also have this same rank). The next leaderboard evaluation I will fix with organizers and update the score. (it will be around May I think)

Let me know your further findings and experiments. I will be out of my desktop next month and may reply to messages later than expected.

[jang-1996]

I solved this problem!
I got about 0.39 (Dynamic Bucket-Normalized EPE Mean)

Here's my setting.

• early_patience : 50
• lr : 0.0002

I'll close this issue since the issue has been resolved.

Thank you again!