DeepLocalizer

Blog Post, Docs

Quickly find functional specialization in PyTorch models.

Extends The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units to other models and data.

Examples:

• Face Localizer Resnet Report
• Face Localizer Resnet Notebook
• Face Localizer Data Viewer

Task Usage

Tasks are just pandas dataframes with a data, positive, and validation columns. Each row is a different data point.

See face_data_viewer.ipynb to see a real example of a face localizer data (face images from CelebA vs. objects images from COCO).

What do the columns mean?

• data is the data itself (eg text) or points to data (eg image filename)
• positive is True for the task and False for the control (eg face images have True and control images have False)
• validation is technically optional. If you want to notate some rows to only be used later on to test performance and not for the main localization, you can indicate a subset of the rows as True. The main dataset used for localization is then False.

Again see face_data_viewer.ipynb if you're still confused.

Library Usage

Install

uv add deeplocalizer

or

pip install deeplocalizer

API Usage

See exact functions at Docs and see resnet34_example.ipynb for an example of using those functions.

Development

cd deeplocalizer # this git repo

Make sure to have https://docs.astral.sh/uv/ installed.

Install and Run

uv sync
uv run deeplocalizer/deeplocalizer.py

or run an example python notebook within the .env generated.

Run Docs

uv run mkdocs serve

References

papers

• https://arxiv.org/abs/2411.02280
• https://web.mit.edu/bcs/nklab/media/pdfs/Dobs_2022.pdf

code/datasets

• https://huggingface.co/datasets/tpremoli/CelebA-attrs
• https://huggingface.co/datasets/phiyodr/coco2017
• https://huggingface.co/microsoft/resnet-50