OneBEV++: Towards Unifying Bird's-Eye-View Semantic Mapping with Panoramas

This repository contains the codes of OneBEV (ACCV 2024 Oral) and OneBEV++ (IEEE T-PAMI).

Method	View transformation	Configs	Checkpoints
OneBEV	MVT	configs/onebev/	weights/onebev/
OneBEV++	CRP + MVTv2	configs/onebev++/	weights/onebev++/

Updates

• 08/2026, OneBEV++ is released.
• 11/2024, OneBEV is released.
• 09/2024, repository initialized.

Prerequisites

The reference environment uses Python 3.10, PyTorch 2.1.1, and CUDA 12.1.

Environments

conda create -n onebev python=3.10.13 -y
conda activate onebev

pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 \
  --index-url https://download.pytorch.org/whl/cu121
pip install mmcv==2.1.0 \
  -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.1/index.html
pip install -r requirements.txt

pip install 'setuptools<70'
pip install --no-build-isolation -e \
  onebev/models/backbones/kernels/selective_scan

Download the pretrained VMamba backbone:

bash pretrained/download.sh

Datasets and Checkpoints

We support five panoramic BEV datasets. Download the datasets and checkpoints from Google Drive.

Dataset	Domain	Scene source	Panorama construction	Train / Val / Test	Classes
Stanford-360	Indoor	Real-world	Native ERP	1,040 / 373 / —	14
Matterport-360	Indoor	Real-world	Stitched from 18 views	7,829 / 772 / 2,014	21
NuScenes-360	Outdoor	Real-world	Stitched from 6 cameras	28,130 / 6,019 / —	6
DeepAccident-360	Outdoor	Synthetic (CARLA)	Stitched from 6 cameras	40,619 / 8,193 / —	17
DeepAccident-Native	Outdoor	Synthetic (CARLA)	Native ERP	25,685 / 5,082 / —	10

Usage

Prepare

Generate the MMEngine annotation files:

python tools/create_pkl/create_data.py <DATASET> \
  --root-path <DATASET_PATH> --version trainval

<DATASET> can be nusc, deep, deep_native, stanford, or matterport.

Train

# OneBEV
bash tools/dist_train.sh configs/onebev/model_onebev_nusc_50epochs.py <GPU_NUM>

# OneBEV++
bash tools/dist_train.sh configs/onebev++/model_onebev++_nusc_50epochs.py <GPU_NUM>

Test

# OneBEV
bash tools/dist_test.sh configs/onebev/model_onebev_nusc_50epochs.py \
  weights/onebev/nusc_epoch_50.pth <GPU_NUM>

# OneBEV++
bash tools/dist_test.sh configs/onebev++/model_onebev++_nusc_50epochs.py \
  weights/onebev++/nusc_epoch_50.pth <GPU_NUM>

References

We appreciate the following open-source projects:

• BEVFormer
• VMamba
• MMEngine
• MMSegmentation

Citation

If you find our work useful, please cite:

@article{wei2026onebev++,
  title={OneBEV++: Towards Unifying Bird's-Eye-View Semantic Mapping with Panoramas},
  author={Wei, Jiale and Teng, Zhifeng and Teng, Fei and Zheng, Junwei and Liu, Ruiping and Chen, Yufan and Hu, Jie and Yang, Kailun and Zhang, Jiaming and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2026},
  publisher={IEEE}
}

@inproceedings{wei2024onebev,
  title={OneBEV: Using One Panoramic Image for Bird's-Eye-View Semantic Mapping},
  author={Wei, Jiale and Zheng, Junwei and Liu, Ruiping and Hu, Jie and Zhang, Jiaming and Stiefelhagen, Rainer},
  booktitle={Asian Conference on Computer Vision},
  pages={377--393},
  year={2024}
}

License

This project is released under the MIT License.