Add MAN TruckScenes data processing into codebase

[Kin-Zhang]

This pull request introduces support for the MAN-TruckScene dataset, updates documentation and scripts for dataset processing, and refines configuration for TruckScene. The changes improve usability and clarity for users working with multiple datasets, especially with the addition of MAN-TruckScene. The most important changes are grouped below by theme.

MAN-TruckScene Dataset Integration:

• Added support for MAN-TruckScene in the main README.md, including download instructions, citation, and dataset statistics. Also updated dataset lists to include MAN-TruckScene with relevant links. [1] [2] [3]
• Added new SLURM script assets/slurm/data-process.sh to process TruckScene data, and removed the old Argoverse2-specific script. [1] [2]
• Added example command for TruckScene data extraction in documentation and listed the extraction script. [1] [2]
• Added configuration file conf/others/truckscenes.toml for TruckScene dataset preprocessing.

Documentation and Citation Updates:

• Updated dataset processing documentation to clarify sources, encourage proper citation, and specify involved scripts for each dataset, including MAN-TruckScene and DoGFlow.

Configuration and Processing Improvements:

• Improved code in dataprocess/extract_nus.py for pose extraction, data resampling, and parallel processing, making it more general and robust. [1] [2] [3] [4] [5]

---

Old Message:

This pull request integrates MAN-TruckScenes data processing scripts into the codebase.

• Unlike Argoverse 2 and Waymo, which use 10Hz for both annotations and data samples, truckscene follows the nuScenes-style provides annotations at 2Hz, while the actual LiDAR data runs at 10Hz. We will follow the same style in Add nuScenes data processing into codebase #6 for truckscenes.

This data is involved by DoGFlow, please check more info in DoGFlow paper.

[Kin-Zhang]

Some visualization record

Sensor Setup

Here is the man truck setup:

• Lidar Sensor: 2x Hesai Pandar64 ; Sampling Rate: 10 Hz; Resolution: 64 vertical layers; Field of View: 360° x 40°; Range: 200 m @ 10 %
• Lidar Nearfield: 4x Ouster OS0, Rev 7 Sampling Rate: 10 Hz Resolution: 64 vertical layers Field of View: 360° x 90° Range: 35 m @ 10 %

Raw point cloud data

Here are SelectedSensor = ['LIDAR_TOP_FRONT', 'LIDAR_TOP_LEFT', 'LIDAR_TOP_RIGHT', 'LIDAR_REAR'] # 4x32

Here are SelectedSensor = ['LIDAR_LEFT', 'LIDAR_RIGHT'] # 2x64

I visualize these by debugging, part code here, here are the commit for debug:

OpenSceneFlow/dataprocess/extract_truckscenes.py

Lines 180 to 189 in bf3f75a

	# # debug visualization
	# import open3d as o3d
	# pcd = o3d.geometry.PointCloud()
	# pcd.points = o3d.utility.Vector3dVector(points[:,:3])
	# # pcd.colors = o3d.utility.Vector3dVector(np.zeros((points.shape[0], 3)))
	# show_lits = [pcd, o3d.geometry.TriangleMesh.create_coordinate_frame(size=1)] # ego frame
	# for lidar_id_ in range(lidar_center.shape[0]):
	# show_lits.append(o3d.geometry.TriangleMesh.create_coordinate_frame(size=1).transform(lidar_center[lidar_id_]))
	# o3d.visualization.draw_geometries(show_lits)
	# exit(0)

Ground Truth Flow Vis

Looks correct

• But some points are missing from annotation (not sure if we need large the bbx again)! I will check to see if HiMo could be applied here also.

Done. Now all should be good (similar comparison #5):

Estimate Flow

I directly used the DeltaFlow waymo ckpt for inference on truckscenes, here are comparsion with gt flow:

[Kin-Zhang]

Some notes for truckscene, I would strong recommend to run Range-wise EPE for truckscene since as last figure show <35m are few and too short for truckscene perception setup etc.

Here is the brief review of the truckscenes trainval-set:

Dataset	# Total Scene	# Total Frames
train	524	101902 / 20380 (w. gt)
val	75	14625 / 2925 (w.gt)

[Copilot]

Pull Request Overview

This pull request integrates MAN TruckScenes dataset processing scripts into the codebase, adding support for a new dataset that follows nuScenes-style annotation patterns (2Hz annotations with 10Hz LiDAR data). The changes include the addition of a complete data processing pipeline for TruckScenes, configuration updates, and documentation improvements.

• Adds complete TruckScenes data processing script with multi-sensor LiDAR support
• Updates environment configuration to specify exact version of truckscenes-devkit
• Refactors common processing utilities to support multiple dataset formats

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
dataprocess/extract_truckscenes.py	New complete data processing script for MAN TruckScenes dataset
dataprocess/misc_data.py	Adds utility functions and mapping dictionaries for TruckScenes processing
dataprocess/extract_nus.py	Refactors utility functions to be more generic for reuse across datasets
envsftool.yaml	Pins truckscenes-devkit to specific version for reproducibility
dataprocess/README.md	Adds comprehensive documentation for TruckScenes dataset processing
conf/others/truckscenes.toml	Adds ground segmentation configuration for TruckScenes
tools/visualization.py	Updates view file reference for demo purposes
assets/view/demo.json	Adds new demo view configuration file

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The print message has a grammatical error with an incomplete sentence 'all timestamps are , skip...'. Should be 'all timestamps are present, skip...' or similar.

Suggested change

	print(f'{log_id} already exists and all timestamps are , skip...')
	print(f'{log_id} already exists and all timestamps are present, skip...')

[Copilot]

Multiple commented-out lines suggest incomplete refactoring. Either remove these comments if they're no longer needed or implement the functionality if it's required.

Suggested change

	# lidar_dt.append(np.ones(pc.shape[0]) * timestamp_diff * 1e-6) # microsecond to s
	lidar_id.append(np.ones(pc.shape[0]) * lidar_id_cnt)
	# lidar_center.append(ego2lidar_np[:3,3].T) # x, y, z
	lidar_center.append(ego2lidar_np)
	lidar_id_cnt += 1
	points = np.vstack(lidar_list)
	# lidar_dt = np.hstack(lidar_dt)
	lidar_id.append(np.ones(pc.shape[0]) * lidar_id_cnt)
	lidar_center.append(ego2lidar_np)
	lidar_id_cnt += 1
	points = np.vstack(lidar_list)

[Copilot]

The function returns a tuple with three elements but the return type annotation only mentions two in the docstring. The docstring should be updated to mention all three return values: closest_ind, closest_int, and int_diff.

[Kin-Zhang]

To run dufomap on the MAN Truckscene dataset, the processing script must be updated to support its multi-LiDAR setup. This is because dufomap requires the sensor's origin for correct ray casting.

Getting the coordinate transformations right between the ego-vehicle, world, and various sensor frames can be a bit of a headache (even for authors, yes). To save others the trouble, here is a summary of the frames and the transformation logic that works:

Coordinate Frame Summary:

• pc0: The point cloud in the base_link frame (the ego-vehicle's frame).
• pose: The base_link pose relative to the world frame.
• lidar_center: The LiDAR sensor's pose relative to the base_link frame.

Required Transformations:
To generate the map correctly, we must perform two key steps:

1. Transform the point cloud (pc0) from the base_link frame to the corresponding LiDAR sensor's frame. This ensures ray casting originates from (0,0,0) in the sensor's perspective.
2. Provide dufomap with the LiDAR sensor's absolute pose in the world frame.

A working implementation of these transformations for Truckscene can be found at the link below:

OpenSceneFlow/process-mlidar.py

Lines 187 to 204 in a7c10e5

	for lid in range(data['lidar_center'].shape[0]):
	pose_lidar = pose0 @ np.linalg.inv(data['lidar_center'][lid])
	lidar_mask = data['lidar_id']==lid
	points_xyz_ego = data['pc0'][lidar_mask & range_mask][:,:3]
	# to lidar frame
	T_ego_to_lidar = data['lidar_center'][lid]
	R_ego_to_lidar = T_ego_to_lidar[:3, :3]
	t_ego_to_lidar = T_ego_to_lidar[:3, 3]
	points_xyz_lidar = points_xyz_ego @ R_ego_to_lidar.T + t_ego_to_lidar
	# to world frame
	T_lidar_to_ego = np.linalg.inv(T_ego_to_lidar)
	T_lidar_to_world = pose0 @ T_lidar_to_ego
	pose_array_lidar = transform_to_array(T_lidar_to_world)
	pose_array = transform_to_array(pose_lidar)
	mydufo.run(points_xyz_lidar, pose_array_lidar, cloud_transform = True)

Here is a sample of the resulting map (by uncomment here).

OpenSceneFlow/process-mlidar.py

Lines 207 to 210 in a7c10e5

	mydufo.oncePropagateCluster(if_cluster = True, if_propagate=True)
	# NOTE(Qingwen): Just for Qingwen to check the voxel map is correct, for dufomap outputMap if voxel_map=True, there is no need to input points.
	# mydufo.outputMap(np.zeros_like(points_xyz_lidar), voxel_map=True)
	# return

For more on addressing challenges like sparse observations on highways, check out our work on HiMo.