Multi-agent Undercover Gaming: Hallucination Removal through Counterfactual Test for Multimodal Reasoning

Dayong Liang^1,4,*, Xiao-Yong Wei^2,3,4,*, Changmeng Zheng^3,†

¹South China University of Technology   ²Sichuan University   ³The Hong Kong Polytechnic   ⁴Peng Cheng Laboratory
^*Contributed equally    ^†Corresponding author

---

Latest News 🔥

• 🔄 Counterfactual Image Generation Pipeline: In progress.
• ✅ [2026/01] Code Implementation: Inference code is released.
• ✅ [2025/11] Paper Accepted: Our paper has been accepted at AAAI2026! 🎉

Features

• Systematic counterfactual testing for hallucination detection
• Dynamic cross-evidence reasoning capabilities
• Active reasoning paradigms facilitating exploratory dialogue among agents

🛠️ Installation

To set up the environment for MUG, please follow the steps below. We recommend using Conda to manage your dependencies.

1. Clone the repository

git clone https://github.com/YongLD/MUG.git
cd MUG

2. Create a new Conda environment

conda create -n mug python==3.12
conda activate mug

3. Install the dependencies

pip install -e .

4.Setup Qwen3 Environment This project relies on Qwen2.5VL. You must install the dependencies required by the official Qwen3VL repository. Please refer to the Qwen3VL GitHub Repository for the most up-to-date installation instructions.

Generally, you will need to install the necessary libraries (like transformers, flash_attention, accelerate, torch, etc.) compatible with Qwen3VL.

🚀 Usage & Testing

Once the environment is configured and the Qwen3 dependencies are installed, you can run the main testing script.

Run the prediction script:

sh predict.sh

📂 Counterfactual Data

We provide access to a subset of our generated counterfactual images via Google Drive.

• Download Link: Google Drive Link

⚠️ Note on Data Quality: The current batch of counterfactual images was generated using Step1X-Edit. Please note that the quality of these samples is preliminary and may be suboptimal.

🚀 Future Plans: We are actively working on utilizing more advanced generative models to produce high-quality counterfactual images. We will release these improved datasets along with the complete Counterfactual Image Generation Pipeline soon.

❤️ Acknowledgments

• VLMEvalKit: An open-source evaluation toolkit of large vision-language models (LVLMs).

📑 Citation

If this repo is useful to you, please cite using this BibTeX.

@article{liang2025multi,
  title={Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning},
  author={Liang, Dayong and Wei, Xiao-Yong and Zheng, Changmeng},
  journal={arXiv preprint arXiv:2511.11182},
  year={2025}
}