Developing Interpretable Style Vectors to Steer Large Language Models towards Group-Specific Explanation Generation
The repository contains the source code for the thesis with the title Developing Interpretable Style Vectors to Steer Large Language Models towards Group-Specific Explanation Generation. It was submitted in June 2025 at the Leibnitz University Hannover.
The thesis explores interpretable style and knowledge representations for analyzing and generating group-specific texts. Building on the method of interpretable style vectors proposed by Patel et al. (2023), the work extends the representation to include knowledge attributes—such as expertise and background familiarity—that vary between groups and influence writing style.
The thesis investigates how well these enhanced representations perform in group membership detection, a task related to but distinct from authorship attribution. It evaluates whether the combination of style and knowledge attributes improves classification performance.
In addition to analysis tasks, the thesis focuses on steering large language models (LLMs) to generate group-specific explanations. Two steering strategies are examined: (1) prompt engineering using attribute vectors, and (2) a novel fine-grained method that modifies model activations to align outputs with target group characteristics.
Overall, this work contributes interpretable and adaptable tools for both text analysis and LLM-based text generation tailored to specific audiences.
The code was written in Python 3.10.13. All dependencies can be installed with python -m pip install -r requirements.txt. During the thesis, the code was excecuted inside the pytorch/pytorch docker image.
Start the program with the command python main.py --enable-llm. Without the parameter enable-llm, all tasks that require ML models will not be excecuted. However, the programm will check if the task have already been excecuted and will not excecute anything twice. If the program has run through once, ML models are not necessary for the most part of the evaluation.
All data is saved in a sqlite file named data.sqlite by default. All configuration parameters can be changed inside the config.py file. The models that are trained are saved in the folder models.