Add nuScenes data processing into codebase

[Kin-Zhang]

This pull request introduces major improvements to nuScenes dataset support, including a new extraction pipeline, configuration options, documentation updates, and utility functions. The changes enable standardized 10Hz resampling, ground truth scene flow generation, and robust handling of annotated frames, making the nuScenes processing consistent with other datasets and easier to use.

nuScenes dataset extraction and processing:

• Added a new dataprocess/extract_nus.py script for nuScenes dataset extraction, including 10Hz resampling, GT annotation handling, ground segmentation, and per-object velocity-based scene flow computation. The script supports parallel processing and robust output file management. [1] [2]
• Introduced a TOML configuration file conf/others/nuscenes.toml for nuScenes ground segmentation and point cloud partitioning parameters.
• Added a utility function check_h5py_file_exists and nuScenes-to-Argoverse category mapping (NusNamMap) in dataprocess/misc_data.py to ensure output consistency and correct class mapping. [1] [2]

Documentation updates:

• Updated README.md and dataprocess/README.md to reflect nuScenes dataset support, including extraction instructions, frame rate explanation, and dataset statistics. [1] [2] [3] [4]

Model base class:

• Added a BaseModel class to src/models/basic/__init__.py for standardized model checkpoint loading, improving code reuse and maintainability.

---

[Old message]

As requested by many users, this pull request integrates nuScenes data processing scripts into the codebase.

⚠️ Considerations Before Merging

• Unlike Argoverse 2 and Waymo, which use 10Hz for both annotations and data samples, nuScenes provides annotations at 2Hz, while the actual data runs at 20Hz.
• Directly inferring scene flow at 2Hz from 2Hz annotations may introduce significant errors, especially in cases like car rotations.
• A possible solution could be tracking at 2Hz while maintaining 10Hz ground truth flow.

[Kin-Zhang]

Some images compared:

---

Although nuscene has different coordinates (left-hand) compared to others, ... add more analysis here later.

一些注意点（This will be translated once I done notes here）

• nuscene的数据 20Hz 采样，2Hz annotation；如果直接做10Hz resample gt不一定都能包含进来（试过了 确实会丢掉一些gt 20%） 所以需要以gt为主导做resample
• 【仅训练考虑】在dataloader角度上来看 gt不能在最后一帧 因为最后一帧pc -> 没有流向的帧；但是inference可以像HiMo那样反着考虑 所以问题不大
  结论：每个场景，scene/log下，以第一帧遇到的gt做采样，然后以这个为开始；最后结束帧不能是gt [后面发现这点直接dataloader valid_index一下就行 无需数据预处理解决
• nus不是用的tracking做的scene flow（比如tracking后对中轴这种）；nus是每个object有velocity label，用的这个加两帧时间差做的gt flow；

---

So, at the end, I think I finished the draft script on nuscene data extraction! I will soon update the script here while the training might need a double check as the gt frames number is really small compared to the other two (10%, so the deflow might be similar to seflow fig. 4?) Some frames and scene count:

Dataset	# Total Scene	# Total Frames
train	700	137575 / 27392 (w. gt)
val	150	29126 / 5798 (w.gt)

Updates: Checked with seflow trained on nuscene, it works well. and also the nus gt training on deltaflow works well also. [I will update more results table on nuscene once deltaflow is accepted.)

More result will be added in the paper.

[Kin-Zhang]

More nuScene visualization here (for id: 21, scene_id: scene-0757, timestamp: 1535657118649480):

Extraction code already updated. The training dataloader will be merged through the DeltaFlow project.

---

Training Result from DeltaFlow:

I will check the policy about nuScene to see if we can upload pretrained weights for users, etc.