NDAD: Negative-Direction Aware Decoding for Large Language Models via Controllable Hallucination Signal Injection

Project Overview

This project implements NDAD (Negative-Direction Aware Decoding), a method designed to enhance factuality in large language models (LLMs) when generating long-form text. By dynamically injecting negative hallucination signals during decoding, combined with a global consistency and local disparity weighting mechanism, our method effectively reduces hallucination in the generated content.

We would like to express our gratitude to the work titled "Retrieval head mechanistically explains long-context factuality", which serves as the basis for calculating the head score in our approach. If you want to recalculate the head score, you can refer to the source code provided in that work. However, we have already included the results we obtained in the head_score folder of this project, so you can directly use those results.

Dataset Preparation

The data for this project is stored in Data.zip. To use the dataset, extract the contents of this file:

```bash
unzip Data.zip -d data

Installation Guide

1. Install Local Dependencies
   Install the transformers library from the local directory included in the project folder.

   pip install -e transformers
   

2. Install Other Requirements
   Use the requirements.txt file to install the remaining dependencies.

   pip install -r requirements.txt
   

3. Run the Code The --mask_topk and --top_k_layers parameters can be adjusted based on your requirements. For example, you can run the following command to specify values for these parameters:

   python run_gsm8k.py --mask_topk <value> --top_k_layers <value>
   

Acknowledgement

This codebase is based on the official repo of DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models. We also highly recommend reading their excellent work.