Open-World Drone Active Tracking with Goal-Centered Rewards

by Haowei Sun* , Jinwu Hu*, Zhirui Zhang, Haoyuan Tian, Xinze Xie,
Yufeng Wang, Xiaohua Xie, Yun Lin, Zhuliang Yu #, Mingkui Tan #

Abstract

Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations, providing a more practical solution for effective tracking in dynamic environments. However, accurate Drone Visual Active Tracking using reinforcement learning remains challenging due to the absence of a unified benchmark and the complexity of open-world environments with frequent interference. To address these issues, we pioneer a systematic solution. First, we propose DAT, the first open-world drone active air-to-ground tracking benchmark. It encompasses 24 city-scale scenes, featuring targets with human-like behaviors and high-fidelity dynamics simulation. DAT also provides a digital twin tool for unlimited scene generation. Additionally, we propose a novel reinforcement learning method called GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios. Specifically, we design a Goal-Centered Reward to provide precise feedback across viewpoints to the agent, enabling it to expand perception and movement range through unrestricted perspectives. Inspired by curriculum learning, we introduce a Curriculum-Based Training strategy that progressively enhances the tracking performance in complex environments. Besides, experiments on simulator and real-world images demonstrate the superior performance of GC-VAT, achieving an approximately 400% improvement over the SOTA methods in terms of the cumulative reward metric.

Installation

1. Install Webots software: see document for details

git clone https://github.com/SHWplus/DAT_Benchmark.git
cd DAT_Benchmark

2. Python dependencies

conda create -n uav_follow311 python=3.11
conda activate uav_follow311

## Alg requirements
pip install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu121
pip install numpy==1.25.0
pip install opencv-python==4.9.0.80
pip install gym==0.25.1
pip install setproctitle==1.3.3
pip install tensorboard
pip install tqdm
pip install stable-baselines3==2.3.0
pip install sb3_contrib==2.3.0
pip install gymnasium
pip install wandb==0.17.6
pip install memory_profiler==0.61.0
pip install psutil==6.0.0
pip install pandas
pip install openpyxl
pip install tianshou==1.1.0

3. Cpp dependencies

## Simulator requirements
sudo apt-get install libeigen3-dev
sudo apt-get install nlohmann-json3-dev

sudo apt install sumo sumo-tools sumo-doc
pip install lxml pyproj shapely webcolors configparser --user
pip install transforms3d

## config
gedit ~/.bashrc
# Change dir according to your installation
export SUMO_HOME=/usr/share/sumo
export WEBOTS_HOME=/usr/local/webots

Detailed installation method, simulator introduction, and execution pipeline can be found in and .

Citation

If you find this project useful, please consider citing our paper

@misc{crossscenebenchmarkopenworlddrone,
      title={A Cross-Scene Benchmark for Open-World Drone Active Tracking}, 
      author={Haowei Sun and Jinwu Hu and Zhirui Zhang and Haoyuan Tian and Xinze Xie and Yufeng Wang and Zhuliang Yu and Xiaohua Xie and Mingkui Tan},
      year={2024},
      eprint={2412.00744},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2412.00744}, 
}

@inproceedings{gcvat,
 author = {Sun, Haowei and Hu, Jinwu and Zhang, Zhirui and Tian, Haoyuan and Xie, Xinze and Wang, Yufeng and Xie, Xiaohua and Lin, Yun and Yu, Zhuliang and Tan, Mingkui},
 booktitle = {Advances in Neural Information Processing Systems},
 title = {Open-World Drone Active Tracking with Goal-Centered Rewards},
 url = {\url{[URL hidden]}},
 year = {2025}
}

Star History