🛰️ SatCLIP - A Global, General-Purpose Geographic Location Encoder [2026.3.1.0]

Overview of the pretraining and deployment pipeline for SatCLIP.

Approach

SatCLIP trains location and image encoders via contrastive learning, by matching images to their corresponding locations. This is analogous to the CLIP approach, which matches images to their corresponding text. Through this process, the location encoder learns characteristics of a location, as represented by satellite imagery. For more details, check out our paper.

Installation

This guide provides three methods to install Satclip, depending on your setup and needs.

Option 1: Install from Source (Local Repository)

Use this if you're actively developing or working from a cloned repository.

# Clone the repository (if not already cloned)
git clone https://github.com/your-org/satclip.git
cd satclip

# (Optional) Create and activate a virtual environment
python -m venv .venv
.\.venv\Scripts\activate  # On Windows
# source .venv/bin/activate  # On Unix-based systems

# Install the package and its dependencies
pip install .

Option 2: Install from PyPI (Pre-built Distribution) NOTE: NOT IMPLEMENTED YET!

Use this if you want the latest stable release directly from PyPI.

pip install satclip

Option 3: Use Conda Environment (Reproducible Setup)

Recommended for strict dependency management or deployment environments.

# Create the environment from the provided YAML file
conda env create -f ./resources/env.satclip.yaml

# Activate the environment
conda activate satclip

Overview

Usage of SatCLIP is simple:

import torch

from satclip.model import SatCLIP

model = SatCLIP(
    in_channels=13,
    embed_dim=512,
    image_resolution=224,
    n_channels=13,
    vision_layers=4,
    vision_width=768,
    vision_patch_size=32,
    le_type='sphericalharmonics',
    pe_type='siren',
    legendre_polys=10,
    frequency_num=16,
    max_radius=360,
    min_radius=1,
    harmonics_calculation='analytic',
)

img_batch = torch.randn(32, 13, 224, 224)  # Represents a batch of 32 images
loc_batch = torch.randn(32, 2)  # Represents the corresponding 32 locations (lon/lat)

with torch.no_grad():
    logits_per_image, logits_per_coord = model(img_batch, loc_batch)
    probs = logits_per_image.softmax(dim=-1).detach().cpu().numpy()

Training

You first need to download the S2-100k dataset in /data/s2. First, download the index file:

cd data/s2
wget https://satclip.z13.web.core.windows.net/satclip/index.csv

Within /data/s2, navigate to /images, download all images and unpack them:

cd images
wget https://satclip.z13.web.core.windows.net/satclip/satclip.tar
tar -xf satclip.tar

Now, to train SatCLIP models, set the paths correctly, adapt training configs in satclip/configs/default.yaml and train SatCLIP by running:

satclip

Use of the S2-100K dataset

The S2-100K dataset is a dataset of 100,000 multi-spectral satellite images sampled from Sentinel-2 via the Microsoft Planetary Computer. Copernicus Sentinel data is captured between Jan 1, 2021 and May 17, 2023. The dataset is sampled approximately uniformly over landmass and only includes images without cloud coverage. The dataset is available for research purposes only. If you use the dataset, please cite our paper. More information on the dataset can be found in our paper.

Pretrained Models

Visualization of embeddings obtained by different location encoders for locations around the globe.

We provide six pretrained SatCLIP models, trained with different vision encoders and spatial resolution hyperparameters $L$ (these indicate the number of Legendre polynomials used for spherical harmonics location encoding. Please refer to our paper for more details). The pretrained models can be downloaded as follows:

• SatCLIP-ResNet18-L10: wget https://satclip.z13.web.core.windows.net/satclip/satclip-resnet18-l10.ckpt
• SatCLIP-ResNet18-L40: wget https://satclip.z13.web.core.windows.net/satclip/satclip-resnet18-l40.ckpt
• SatCLIP-ResNet50-L10: wget https://satclip.z13.web.core.windows.net/satclip/satclip-resnet50-l10.ckpt
• SatCLIP-ResNet50-L40: wget https://satclip.z13.web.core.windows.net/satclip/satclip-resnet50-l40.ckpt
• SatCLIP-ViT16-L10: wget https://satclip.z13.web.core.windows.net/satclip/satclip-vit16-l10.ckpt
• SatCLIP-ViT16-L40: wget https://satclip.z13.web.core.windows.net/satclip/satclip-vit16-l40.ckpt

Usage of pretrained models is simple (Note: All models are loaded by default in the CPU):

from satclip import load

# Load the entire SatCLIP model
model = load.get_satclip(r'C:\Users\bari_is\Documents\git\Haight3\Embedding\satclip\weights\satclip-vit16-l40.ckpt')

# Or Load only the location encoder
l_encoder = load.load_location_encoder(r'C:\Users\bari_is\Documents\git\Haight3\Embedding\satclip\weights\satclip-vit16-l40.ckpt')

device = "cuda"

model = model.to(device)
l_encoder = l_encoder.to(device)

c = torch.randn(32, 2)  # Represents a batch of 32 locations (lon/lat)
img_batch = torch.randn(32, 13, 224, 224)  # Represents a batch of 32 images

model.eval()
l_encoder.eval()

with torch.no_grad():
    logits_per_image, logits_per_coord = model(img_batch.double().to(device), c.double().to(device))
    loc = l_encoder(c.double().to(device)).detach().cpu()

You can also access SatCLIP model weights directly via Hugging Face.

Examples

Examples on how to obtain and use pretrained SatCLIP embeddings can be found in the notebooks folder. We provide notebooks (optimized for use with Google Colab) for the following use cases.

Setup:

• A01 - Simple usage example
• A02 - Load models from Hugging Face

Example use cases:

• B01 - Air temperature prediction with SatCLIP
• B02 - Example Image Localization

Use baseline pretrained location encoders:

• C01 - Simple CSP usage
• C02 - Simple GeoCLIP usage
• C03 - Simple GPS2Vec usage

Citation

@article{klemmer2023satclip,
  title={SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery},
  author={Klemmer, Konstantin and Rolf, Esther and Robinson, Caleb and Mackey, Lester and Ru{\ss}wurm, Marc},
  journal={arXiv preprint arXiv:2311.17179},
  year={2023}
}

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.