PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking

📖[Project Page]

This is the official repository for PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking (CVPR 2025).

Introduction

✨ PURA is a test-time adaptation framework for RGB-T tracking, designed to robustly adapt to domain transition scenarios during the testing process.

✨ PURA adapts online through parameter update and recovery mechanisms, avoiding drastic parameter changes and heavy computational burden.

Results

We have released the results of our method on the RGBT234, RGBT210 and GTOT datasets:

Method	RGBT234		RGBT210		GTOT		FPS	Results & Weights
	MPR(%)	MSR(%)	PR(%)	SR(%)	MPR(%)	MSR(%)
No Adapt.	90.8	67.6	88.6	65.1	95.1	78.2	59.5	Google Drive
PURA	93.3	70.3	90.3	66.8	95.7	78.6	42.0

• Note: The full code and weights of our method will be released soon.

Usage

Environment

Create a conda environment and install the required packages:

conda env create -f environment.yml

Datasets

1. The paths should be organized as follows:

-- LasHeR/trainingset
    |-- 1boygo
    |-- 1handsth
    ...
...

2. Edit dataset path and save path in lib/train/admin/local.py and lib/test/evaluation/local.py.

Weights

1. Download the weights from Google Drive.
2. Edit the weights path in lib/test/parameter/tbsi_track.py

    params.checkpoint = "/path/to/weights"

Evaluation

1. Edit which TTA methods you want to apply in lib/test/tracker/tbsi_track.py:

class TBSITrack(BaseTracker):
    def __init__(self, params, dataset_name):
        super(TBSITrack, self).__init__(params)
        ...

        # NOTE: PURA
        pura.replace_batchnorm(self.network.box_head)
        pura.configure_model(self.network)

        # NOTE: AdaBN
        # adabn.replace_batchnorm(self.network.box_head)
        # adabn.configure_model(self.network)

        # NOTE: Tent
        # model = tent.configure_model(self.network)
        # tta_params, tta_param_names = tent.collect_params(model)
        # optimizer = torch.optim.AdamW(tta_params, lr=1e-3)
        # self.network = tent.Tent(model, optimizer)

        # NOTE: ETA
        # model = eata.configure_model(self.network)
        # tta_params, tta_param_names = eata.collect_params(model.box_head)
        # optimizer = torch.optim.SGD(tta_params, lr=0.00025, momentum=0.9)
        # self.network = eata.EATA(model, optimizer, e_margin=math.log(1000)*0.40, d_margin=0.05)

    def track(self, image, info: dict = None):
        ...
        
        with torch.no_grad():
            ...

            # NOTE: Uncomment this block when using Tent or ETA
            # model_inputs = {
            #     "template": cur_template,
            #     "search": [x_dict.tensors[:, :3, :, :], x_dict.tensors[:, 3:, :, :]],
            #     "ce_template_mask": self.box_mask_z
            # }
            # out_dict = self.network(model_inputs)

            # NOTE: Uncomment this block when using PURA or AdaBN
            out_dict = self.network.forward(
                template=cur_template,
                search=[x_dict.tensors[:, :3, :, :], x_dict.tensors[:, 3:, :, :]], ce_template_mask=self.box_mask_z)

        ...

2. Edit the paths and datasets and GPUs in sh/cls_run_test.sh.
3. Run the script to get raw results.
4. Use your own evaluation toolkit to get the final results.

How to apply TTA methods to my own tracker?

For applying PURA to your own RGB-T tracker based on pytracking, you can follow the steps below:

1. Copy pura.py to lib\test\tracker folder.
2. Modify lib\test\tracker\xxx_track.py to include the following code:

from lib.test.tracker.tta import pura
from lib.test.tracker import tent  # import Tent
from lib.test.tracker import eata  # import EATA
from lib.test.tracker import adabn  # import AdaBN

...


class TBSITrack(XXXTracker):
    def __init__(self, params, dataset_name):
        super(XXXTrack, self).__init__(params)
        network = build_xxx_track(params.cfg, training=False)
        network.load_state_dict(torch.load(self.params.checkpoint, map_location='cpu')['net'], strict=True)
        self.cfg = params.cfg
        self.network = network.cuda()
        self.network.eval()

        # NOTE: PURA
        pura.replace_batchnorm(self.network.box_head)
        pura.configure_model(self.network)

        # NOTE: AdaBN
        # adabn.replace_batchnorm(self.network.box_head)
        # adabn.configure_model(self.network)

        # NOTE: Tent
        # model = tent.configure_model(self.network)
        # tta_params, tta_param_names = tent.collect_params(model)
        # optimizer = torch.optim.AdamW(tta_params, lr=1e-3)
        # self.network = tent.Tent(model, optimizer)

        # NOTE: ETA
        # model = eata.configure_model(self.network)
        # tta_params, tta_param_names = eata.collect_params(model.box_head)
        # optimizer = torch.optim.SGD(tta_params, lr=0.00025, momentum=0.9)
        # self.network = eata.EATA(model, optimizer, e_margin=math.log(1000)*0.40, d_margin=0.05)

        self.preprocessor = Preprocessor()
        self.state = None
        ...

3. If Tent or ETA is enabled, please build the data as a dictionary input model of the track function in lib\test\tracker\xxx_track.py:

    def track(self, image, info: dict = None):
        H, W, _ = image.shape
        self.frame_id += 1
        
        ...

        with torch.no_grad():  # enable_grad  no_grad
            ...

            # NOTE: Uncomment this block when using Tent or ETA
            # model_inputs = {
            #     "template": cur_template,
            #     "search": [x_dict.tensors[:, :3, :, :], x_dict.tensors[:, 3:, :, :]],
            #     "ce_template_mask": self.box_mask_z
            # }
            # out_dict = self.network(model_inputs)

            # NOTE: Uncomment this block when using PURA or AdaBN
            out_dict = self.network.forward(
                template=cur_template,
                search=[x_dict.tensors[:, :3, :, :], x_dict.tensors[:, 3:, :, :]], ce_template_mask=self.box_mask_z)

        ...

Acknowledgments

• Our tracking framework is based on OSTrack.
• We use the implementation of the SVD decomposition from the PGrad repo.

Citation

If our work is helpful for your research, please consider citing our paper:

@inproceedings{shao2025pura,
    title={PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking},
    author={Shao, Zekai and Hu, Yufan and Fan, Bin and Liu, Hongmin},
    booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
    pages={22089--22098},
    year={2025}
}