updated last version : serWarning: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

[anthomontan]

Hi,

I updated to the last Valis repo yesterday, was code was working well before BUT now it doesn't :/

Here is my code and the error : (Also for whatever reason at the error measurement step the dimensions are swapped ! so I added some code to verify and swap it back (no idea why this occurs , BUT if you can have a look that will be great !)

Could you please have a look and fix it , I really need Valis to align images :D

THanks
antho

Code :

VALIS last update :

RUNNING 1:44 8/18/2025

import os
import pathlib
import numpy as np
from pathlib import Path
from tqdm import tqdm
import pyvips
import cv2
from skimage import exposure
from valis import valtils, slide_io, registration
from valis import feature_detectors, feature_matcher # Import the new feature modules
from valis.micro_rigid_registrar import MicroRigidRegistrar
import shutil
import re

Add missing imports

try:
from tqdm.notebook import tqdm
except ImportError:
from tqdm import tqdm
print("tqdm.notebook not found. Using tqdm instead.")

def organize_channels_automatically(base_dir, target_dapi_dir=None, target_all_channels_dir=None):
"""
Automatically organize channel files from a base directory

Args:
    base_dir: Path to the folder containing all channel files (e.g., ".../lb06/R2/")
    target_dapi_dir: Where to move ch00 files (if None, creates dapi subfolder)
    target_all_channels_dir: Where to move ch01/ch02 files (if None, creates all_channels subfolder)
"""
base_path = Path(base_dir)

# Create target directories if not specified
if target_dapi_dir is None:
    target_dapi_dir = base_path / "dapi"
else:
    target_dapi_dir = Path(target_dapi_dir)
    
if target_all_channels_dir is None:
    target_all_channels_dir = base_path / "all_channels"
else:
    target_all_channels_dir = Path(target_all_channels_dir)

# Create directories if they don't exist
target_dapi_dir.mkdir(parents=True, exist_ok=True)
target_all_channels_dir.mkdir(parents=True, exist_ok=True)

print(f"Organizing files from: {base_path}")
print(f"DAPI (ch00) files will go to: {target_dapi_dir}")
print(f"Other channels will go to: {target_all_channels_dir}")
print("-" * 60)

# Get all files in the base directory
all_files = [f for f in base_path.iterdir() if f.is_file()]
moved_files = {"dapi": [], "other_channels": [], "unrecognized": []}

for file_path in all_files:
    filename = file_path.name.lower()
    
    # Check for channel patterns
    if "_ch00" in filename or "_merged_ch00" in filename:
        # This is a DAPI file
        target_path = target_dapi_dir / file_path.name
        shutil.move(str(file_path), str(target_path))
        moved_files["dapi"].append(file_path.name)
        print(f"📁 DAPI: {file_path.name}")
        
    elif "_ch01" in filename or "_ch02" in filename or "_ch03" in filename or "_merged_ch01" in filename or "_merged_ch02" in filename or "_merged_ch03" in filename:
        # This is another channel
        target_path = target_all_channels_dir / file_path.name
        shutil.move(str(file_path), str(target_path))
        moved_files["other_channels"].append(file_path.name)
        print(f"🔬 Channel: {file_path.name}")
        
    else:
        # Unrecognized file pattern
        moved_files["unrecognized"].append(file_path.name)
        print(f"❓ Unrecognized: {file_path.name}")

# Summary
print("\n" + "="*60)
print("ORGANIZATION COMPLETE!")
print("="*60)
print(f"✅ DAPI files moved: {len(moved_files['dapi'])}")
print(f"✅ Other channel files moved: {len(moved_files['other_channels'])}")
print(f"⚠️  Unrecognized files: {len(moved_files['unrecognized'])}")

if moved_files["unrecognized"]:
    print("\nUnrecognized files (not moved):")
    for filename in moved_files["unrecognized"]:
        print(f"  - {filename}")

return moved_files

def get_round_name(src_f):
"""Extract the round name from the filename"""
img_name = valtils.get_name(src_f)
round_name = img_name.lower().split("_merged")[0]
return round_name

def get_channel_number(src_f):
"""Extract the channel number from the filename"""
img_name = os.path.basename(src_f).lower()
if "_ch00" in img_name:
return 0
elif "_ch01" in img_name:
return 1
elif "_ch02" in img_name:
return 2
else:
return None

def pad_image(img, target_width, target_height):
"""Pad image to target dimensions, centering the original content"""
pad_width = max(0, target_width - img.width)
pad_height = max(0, target_height - img.height)

if pad_width > 0 or pad_height > 0:
    pad_left = pad_width // 2
    pad_top = pad_height // 2
    padded = img.embed(pad_left, pad_top, target_width, target_height, extend="black")
    return padded
return img

def apply_clahe_normalization(img_array, clip_limit=2.0, tile_grid_size=(16,16)):
"""Apply CLAHE normalization to improve contrast for registration"""
if img_array.dtype != np.uint8:
# Convert to uint8 for CLAHE
img_normalized = exposure.rescale_intensity(img_array, out_range=(0, 255)).astype(np.uint8)
else:
img_normalized = img_array.copy()

# Apply CLAHE
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=tile_grid_size)
clahe_img = clahe.apply(img_normalized)

return clahe_img

def create_clahe_padded_image(img_path, target_width, target_height, output_path, clip_limit=2.0, tile_grid_size=(16, 16)):
"""Pad image first, then apply CLAHE to the padded result"""
try:
# Load original image
img = pyvips.Image.new_from_file(img_path)

    # Print image format for debugging
    print(f"Image format for {os.path.basename(img_path)}: {img.format}, bands: {img.bands}")

    # Step 1: Pad the image
    padded_img = pad_image(img, target_width, target_height)
    
    # Step 2: Convert padded image to numpy for CLAHE processing
    img_array = np.ndarray(buffer=padded_img.write_to_memory(),
                          dtype=np.uint8 if padded_img.format == 'uchar' else np.uint16,
                          shape=[padded_img.height, padded_img.width, padded_img.bands])
    
    # Step 3: Apply CLAHE to first channel (DAPI)
    if padded_img.bands == 1:
        clahe_array = apply_clahe_normalization(img_array[:, :, 0], clip_limit, tile_grid_size)
        clahe_array = clahe_array[:, :, np.newaxis]
    else:
        # For multi-channel, apply CLAHE to first channel only
        clahe_array = img_array.copy()
        clahe_array[:, :, 0] = apply_clahe_normalization(img_array[:, :, 0], clip_limit, tile_grid_size)
    
    # Step 4: Convert back to pyvips - always use 'uchar' for CLAHE output
    # CLAHE always outputs uint8 data
    clahe_padded_img = pyvips.Image.new_from_memory(clahe_array.data,
                                                   clahe_array.shape[1],
                                                   clahe_array.shape[0],
                                                   clahe_array.shape[2],
                                                   'uchar')
    
    # Step 5: Save CLAHE-processed padded image
    clahe_padded_img.write_to_file(output_path)
    return output_path
    
except Exception as e:
    print(f"Error creating CLAHE-padded image for {img_path}: {str(e)}")
    return None

def resize_warped_images(warped_dir, padded_reference_path):
"""
Resize warped images to match padded reference dimensions.

Args:
    warped_dir: Directory containing warped images
    padded_reference_path: Path to padded reference image to get correct dimensions
"""
print(f"Resizing warped images in {warped_dir}...")

# Load padded reference image to get correct dimensions
ref_img = pyvips.Image.new_from_file(padded_reference_path)
ref_width = ref_img.width
ref_height = ref_img.height
print(f"Padded reference dimensions: {ref_width} x {ref_height}")

# Create backup directory
backup_dir = os.path.join(os.path.dirname(warped_dir), "resize_backup")
os.makedirs(backup_dir, exist_ok=True)

# Process all warped images
resized_count = 0
for root, dirs, files in os.walk(warped_dir):
    for file in files:
        if file.startswith("warped_") and file.endswith(".tif"):
            img_path = os.path.join(root, file)
            
            try:
                # Load warped image
                warped_img = pyvips.Image.new_from_file(img_path)
                
                # Check if dimensions match reference
                if warped_img.width != ref_width or warped_img.height != ref_height:
                    print(f"Resizing image: {file}")
                    print(f"  Current dimensions: {warped_img.width} x {warped_img.height}")
                    
                    # Create backup
                    backup_path = os.path.join(backup_dir, file)
                    shutil.copy2(img_path, backup_path)
                    
                    # Resize to match reference dimensions
                    # Use bilinear interpolation for better quality
                    resized_img = warped_img.resize(ref_width / warped_img.width, 
                                                   vscale=ref_height / warped_img.height,
                                                   kernel="bilinear")
                    
                    # Save with proper format preservation
                    if warped_img.format == 'ushort':
                        resized_img.write_to_file(img_path, compression="none", tile=False)
                    else:
                        resized_img.write_to_file(img_path)
                        
                    print(f"  Resized dimensions: {resized_img.width} x {resized_img.height}")
                    resized_count += 1
                else:
                    print(f"Image dimensions already match reference: {file}")
                    
            except Exception as e:
                print(f"Error processing {file}: {str(e)}")

print(f"\nResized {resized_count} images")
return resized_count

def fix_warped_images(individual_channel_dir, reference_slide_path):
"""
Fix warped images that appear squished by ensuring they maintain the correct aspect ratio
relative to the reference image.

Args:
    individual_channel_dir: Directory containing warped images
    reference_slide_path: Path to reference image to get correct dimensions
"""
print("\nFixing warped images to maintain correct aspect ratio...")

# Load reference image to get correct dimensions
ref_img = pyvips.Image.new_from_file(reference_slide_path)
ref_width = ref_img.width
ref_height = ref_img.height
ref_format = ref_img.format
ref_bands = ref_img.bands

print(f"Reference image: {ref_width}x{ref_height}, format={ref_format}, bands={ref_bands}")

# Create backup directory
backup_dir = os.path.join(os.path.dirname(individual_channel_dir), "warped_backup")
os.makedirs(backup_dir, exist_ok=True)

# Process all warped images in the directory and subdirectories
for root, dirs, files in os.walk(individual_channel_dir):
    for file in files:
        if file.startswith("warped_") and file.endswith(".tif"):
            img_path = os.path.join(root, file)
            
            try:
                # Load warped image
                warped_img = pyvips.Image.new_from_file(img_path)
                
                # Print image details for debugging
                print(f"Processing {file}: {warped_img.width}x{warped_img.height}, format={warped_img.format}, bands={warped_img.bands}")
                
                # Check if dimensions match reference
                if warped_img.width != ref_width or warped_img.height != ref_height:
                    print(f"Fixing dimensions for {file}")
                    
                    # Create backup
                    backup_path = os.path.join(backup_dir, file)
                    shutil.copy2(img_path, backup_path)
                    
                    # For 16-bit images, ensure we preserve the format
                    # Use affine transformation instead of resize to maintain aspect ratio
                    
                    # Calculate the scale factors
                    scale_x = ref_width / warped_img.width
                    scale_y = ref_height / warped_img.height
                    
                    # Use the minimum scale to avoid stretching
                    scale = min(scale_x, scale_y)
                    
                    # Calculate dimensions after scaling
                    new_width = int(warped_img.width * scale)
                    new_height = int(warped_img.height * scale)
                    
                    # Calculate padding to center the image
                    left_pad = (ref_width - new_width) // 2
                    top_pad = (ref_height - new_height) // 2
                    
                    # Create transformation matrix for affine transform
                    # [scale, 0, left_pad, 0, scale, top_pad]
                    transform = [scale, 0, left_pad, 0, scale, top_pad]
                    
                    # Apply affine transformation with proper interpolation
                    interp = pyvips.Interpolate.new('bicubic')
                    fixed_img = warped_img.affine(transform, interpolate=interp)
                    
                    # If the image is smaller than reference after transform, pad it
                    if fixed_img.width < ref_width or fixed_img.height < ref_height:
                        fixed_img = fixed_img.embed(
                            0, 0, 
                            ref_width, ref_height, 
                            extend="black"
                        )
                    
                    # Ensure we crop to exact dimensions if somehow larger
                    if fixed_img.width > ref_width or fixed_img.height > ref_height:
                        fixed_img = fixed_img.crop(0, 0, ref_width, ref_height)
                    
                    # Save with proper format preservation
                    # For 16-bit TIFFs, we need to ensure we save with the right options
                    if warped_img.format == 'ushort':
                        fixed_img.write_to_file(img_path, compression="none", tile=False)
                    else:
                        fixed_img.write_to_file(img_path)
                    
                    print(f"Fixed image saved: {img_path}")
                
            except Exception as e:
                print(f"Error fixing {file}: {str(e)}")

print("✅ Fixed warped images to match reference dimensions")
return backup_dir

def main():
# Set up base directory
base_dir = "####your directory /##"

# Organize files automatically
print("Organizing channel files...")
organize_channels_automatically(base_dir)

# Set up directories using the organized structure
slide_src_dir = Path(base_dir) / "dapi"  # Now points to organized DAPI folder
path_to_all_channels = Path(base_dir) / "all_channels"  # Now points to organized channels folder

# Results directory
results_dst_dir = Path(base_dir) / "registration"
results_dst_dir.mkdir(parents=True, exist_ok=True)
registered_slide_dst_dir = results_dst_dir / "registered_slides"
registered_slide_dst_dir.mkdir(parents=True, exist_ok=True)

# Reference slide - now from the organized DAPI folder
reference_slide_name = "###.tif"  # Update this to your actual reference
reference_slide = str(slide_src_dir / reference_slide_name)

print(f"✅ DAPI images directory: {slide_src_dir}")
print(f"✅ All channels directory: {path_to_all_channels}")
print(f"✅ Results directory: {results_dst_dir}")
print(f"✅ Reference slide: {reference_slide}")

# Process images: PAD FIRST, then CLAHE on DAPI only
print("Starting image processing: PADDING FIRST, then CLAHE on DAPI only...")

# Create output directories
src_dir = str(slide_src_dir)
dst_dir = str(results_dst_dir)
all_channels_dir = str(path_to_all_channels)

# Directory for padded images (original intensities)
padded_dir = os.path.join(dst_dir, "padded_images")
os.makedirs(padded_dir, exist_ok=True)
padded_dapi_dir = os.path.join(padded_dir, "dapi")
os.makedirs(padded_dapi_dir, exist_ok=True)
padded_all_channels_dir = os.path.join(padded_dir, "all_channels")
os.makedirs(padded_all_channels_dir, exist_ok=True)
padded_ref_dir = os.path.join(padded_dir, "reference")
os.makedirs(padded_ref_dir, exist_ok=True)

# Directory for CLAHE-padded DAPI images (for registration only)
clahe_padded_dir = os.path.join(dst_dir, "clahe_padded_dapi")
os.makedirs(clahe_padded_dir, exist_ok=True)

# Directory for individual warped channels
individual_channel_dir = os.path.join(dst_dir, "warped_channels")
os.makedirs(individual_channel_dir, exist_ok=True)

# Get all images
dapi_imgs = [os.path.join(src_dir, f) for f in os.listdir(src_dir)]
all_channel_imgs = [os.path.join(all_channels_dir, f) for f in os.listdir(all_channels_dir)]

print(f"Found {len(dapi_imgs)} DAPI images for registration")
print(f"Found {len(all_channel_imgs)} other channel images")

# STEP 1: Find maximum dimensions across all images
max_width = 0
max_height = 0

print("Finding maximum dimensions across all images...")
for img_path in dapi_imgs + all_channel_imgs:
    try:
        img = pyvips.Image.new_from_file(img_path)
        max_width = max(max_width, img.width)
        max_height = max(max_height, img.height)
        # Print format for the first few images to debug
        if max_width == img.width or max_height == img.height:
            print(f"New max from {os.path.basename(img_path)}: {img.width}x{img.height}, format={img.format}")
    except Exception as e:
        print(f"Error reading {img_path}: {str(e)}")

print(f"Maximum dimensions: {max_width} x {max_height}")

# STEP 2: Create padded images (original intensities) AND CLAHE-padded DAPI images
print("Creating padded images and CLAHE-padded DAPI images...")

# Process reference image
ref_img = pyvips.Image.new_from_file(reference_slide)
print(f"Reference image format: {ref_img.format}, bands: {ref_img.bands}")
padded_ref_img = pad_image(ref_img, max_width, max_height)
ref_basename = os.path.basename(reference_slide)
padded_reference_slide = os.path.join(padded_ref_dir, f"padded_{ref_basename}")

# Save with proper format preservation for 16-bit images
if ref_img.format == 'ushort':
    padded_ref_img.write_to_file(padded_reference_slide, compression="none", tile=False)
else:
    padded_ref_img.write_to_file(padded_reference_slide)

print(f"Padded reference: {padded_reference_slide}")

# Create CLAHE-padded version of reference for registration
clahe_padded_reference = os.path.join(clahe_padded_dir, f"clahe_padded_{ref_basename}")
create_clahe_padded_image(reference_slide, max_width, max_height, clahe_padded_reference)
print(f"CLAHE-padded reference: {clahe_padded_reference}")

# Process DAPI images with existence check
padded_dapi_paths = []
clahe_padded_dapi_paths = []

for img_path in dapi_imgs:
    try:
        basename = os.path.basename(img_path)
        padded_path = os.path.join(padded_dapi_dir, f"padded_{basename}")
        
        # Check if padded image already exists
        if os.path.exists(padded_path):
            print(f"✅ Reusing existing padded DAPI: {basename}")
            padded_dapi_paths.append(padded_path)
            continue  # Skip processing if exists
        
        # Process new image
        img = pyvips.Image.new_from_file(img_path)
        padded_img = pad_image(img, max_width, max_height)
        if img.format == 'ushort':
            padded_img.write_to_file(padded_path, compression="none", tile=False)
        else:
            padded_img.write_to_file(padded_path)
        padded_dapi_paths.append(padded_path)
        
        # Create CLAHE-padded version (for registration)
        clahe_padded_path = os.path.join(clahe_padded_dir, f"clahe_padded_{basename}")
        if create_clahe_padded_image(img_path, max_width, max_height, clahe_padded_path):
            clahe_padded_dapi_paths.append(clahe_padded_path)
        
        print(f"Processed DAPI: {basename}")
        
    except Exception as e:
        print(f"Error processing {img_path}: {str(e)}")

# Process all channel images (only padded, no CLAHE) with existence check
padded_all_channel_paths = []
for img_path in all_channel_imgs:
    try:
        basename = os.path.basename(img_path)
        padded_path = os.path.join(padded_all_channels_dir, f"padded_{basename}")
        
        # Check if padded image already exists
        if os.path.exists(padded_path):
            print(f"✅ Reusing existing padded channel: {basename}")
            padded_all_channel_paths.append(padded_path)
            continue  # Skip processing if exists
        
        # Process new image
        img = pyvips.Image.new_from_file(img_path)
        padded_img = pad_image(img, max_width, max_height)
        if img.format == 'ushort':
            padded_img.write_to_file(padded_path, compression="none", tile=False)
        else:
            padded_img.write_to_file(padded_path)
        padded_all_channel_paths.append(padded_path)
        print(f"Padded channel: {basename}")
    except Exception as e:
        print(f"Error padding {img_path}: {str(e)}")

# STEP 3: Perform registration using CLAHE-PADDED DAPI images
print("Performing registration with CLAHE-padded DAPI images...")

# NEW: Configure feature detector and matcher for the latest Valis version
# Using DISK feature detector with grayscale images (more stable for DAPI)
fd = feature_detectors.DiskFD(rgb=False, num_features=2000, device='cpu')
matcher = feature_matcher.LightGlueMatcher(fd)

registrar = registration.Valis(
    clahe_padded_dir,  # Directory with CLAHE-padded DAPI images
    dst_dir,
    reference_img_f=clahe_padded_reference,
    align_to_reference=True,
    micro_rigid_registrar_cls=MicroRigidRegistrar,
    max_processed_image_dim_px=2000,
    max_non_rigid_registration_dim_px=1500,
    norm_method='img_stats',
    matcher=matcher  # Device is already configured in the matcher
)

rigid_registrar, non_rigid_registrar, error_df = registrar.register()

# Perform micro-registration
print("Performing micro-registration...")
micro_reg, micro_error = registrar.register_micro(
    max_non_rigid_registration_dim_px=2000,
    align_to_reference=True
)

# STEP 4: Apply registration transforms to original padded images
print("Applying registration transforms to original intensity padded images...")

# Create mapping between CLAHE-padded slides and original padded slides
clahe_to_original_mapping = {}
for slide_name, slide_obj in registrar.slide_dict.items():
    # Find corresponding original padded image
    clahe_basename = os.path.basename(slide_obj.src_f).replace("clahe_padded_", "padded_")
    original_path = os.path.join(padded_dapi_dir, clahe_basename)
    clahe_to_original_mapping[slide_obj] = original_path

# Warp original padded DAPI images
dapi_warped_dir = os.path.join(individual_channel_dir, "dapi_warped")
os.makedirs(dapi_warped_dir, exist_ok=True)

for slide_obj, original_dapi_path in clahe_to_original_mapping.items():
    try:
        # Load original padded DAPI (original intensities)
        original_dapi_img = pyvips.Image.new_from_file(original_dapi_path)
        orig_width, orig_height = original_dapi_img.width, original_dapi_img.height
        print(f"Warping {os.path.basename(original_dapi_path)}, format={original_dapi_img.format}, dimensions={orig_width}x{orig_height}")

        # Apply transformation found using CLAHE-padded DAPI
        warped_dapi = slide_obj.warp_img(img=original_dapi_img, non_rigid=True, crop=False)
        
        # CRITICAL CHECK: Verify dimensions have correct orientation
        current_width, current_height = warped_dapi.width, warped_dapi.height
        print(f"  Raw warped dimensions: {current_width}x{current_height}")
        
        # Check if width/height are transposed (width should be larger than height for landscape images)
        orig_orientation = "landscape" if orig_width > orig_height else "portrait"
        warped_orientation = "landscape" if current_width > current_height else "portrait"
        
        # Calculate aspect ratio match
        orig_aspect = orig_width / orig_height
        warped_aspect = current_width / current_height
        aspect_ratio_diff = abs(warped_aspect - orig_aspect)
        aspect_ratio_diff_transposed = abs(warped_aspect - (1/orig_aspect))
        
        # Determine if transposition is needed
        needs_transposition = False
        if orig_orientation == warped_orientation:
            # Same orientation, but check if aspect ratios match better when transposed
            if aspect_ratio_diff_transposed < aspect_ratio_diff * 0.5:
                needs_transposition = True
                print(f"  ⚠️ Aspect ratio mismatch suggests transposition needed")
        else:
            # Different orientation - likely needs transposition
            needs_transposition = True
            print(f"  ⚠️ Orientation mismatch: original={orig_orientation}, warped={warped_orientation}")
        
        # Apply transposition if needed
        if needs_transposition:
            print(f"  💡 Transposing image to fix orientation...")
            warped_dapi = warped_dapi.transpose(pyvips.enums.Orientation.ROT90)
            current_width, current_height = warped_dapi.width, warped_dapi.height
            print(f"  ✅ Fixed dimensions: {current_width}x{current_height}")
        
        basename = os.path.basename(original_dapi_path)
        warped_path = os.path.join(dapi_warped_dir, f"warped_{basename}")
        
        # Save with proper format preservation for 16-bit images
        if original_dapi_img.format == 'ushort':
            warped_dapi.write_to_file(warped_path, compression="none", tile=False)
        else:
            warped_dapi.write_to_file(warped_path)
            
        print(f"Warped original DAPI: {warped_path}, dimensions={current_width}x{current_height}")
        
    except Exception as e:
        print(f"Error warping original DAPI {slide_obj.name}: {str(e)}")

# Warp all other channels (ch01, ch02) with original intensities
print("Warping all other channels with original intensities...")
for slide_obj, round_slides in round_dict.items():
    print(f"\nWarping channels for round: {slide_obj.name}")
    valtils.sort_nicely(round_slides)

    for src_f in round_slides:
        img_name = os.path.basename(src_f)
        print(f"Warping channel: {img_name}")
        
        try:
            # Load original intensity padded channel
            channel_img = pyvips.Image.new_from_file(src_f)
            orig_width, orig_height = channel_img.width, channel_img.height
            print(f"  Original channel dimensions: {orig_width}x{orig_height}, format={channel_img.format}, bands={channel_img.bands}")
            
            # Apply transformation found using CLAHE-padded DAPI
            warped_channel = slide_obj.warp_img(img=channel_img, non_rigid=True, crop=False)
            
            # CRITICAL CHECK: Verify dimensions have correct orientation
            current_width, current_height = warped_channel.width, warped_channel.height
            print(f"  Raw warped dimensions: {current_width}x{current_height}")
            
            # Check if width/height are transposed
            orig_orientation = "landscape" if orig_width > orig_height else "portrait"
            warped_orientation = "landscape" if current_width > current_height else "portrait"
            
            # Calculate aspect ratio match
            orig_aspect = orig_width / orig_height
            warped_aspect = current_width / current_height
            aspect_ratio_diff = abs(warped_aspect - orig_aspect)
            aspect_ratio_diff_transposed = abs(warped_aspect - (1/orig_aspect))
            
            # Determine if transposition is needed
            needs_transposition = False
            if orig_orientation == warped_orientation:
                if aspect_ratio_diff_transposed < aspect_ratio_diff * 0.5:
                    needs_transposition = True
                    print(f"  ⚠️ Aspect ratio mismatch suggests transposition needed")
            else:
                needs_transposition = True
                print(f"  ⚠️ Orientation mismatch: original={orig_orientation}, warped={warped_orientation}")
            
            # Apply transposition if needed
            if needs_transposition:
                print(f"  💡 Transposing image to fix orientation...")
                warped_channel = warped_channel.transpose(pyvips.enums.Orientation.ROT90)
                current_width, current_height = warped_channel.width, warped_channel.height
                print(f"  ✅ Fixed dimensions: {current_width}x{current_height}")
            
            channel_basename = os.path.basename(src_f).replace("padded_", "")
            individual_channel_path = os.path.join(individual_channel_dir, f"warped_{channel_basename}")
            
            # Save with proper format preservation for 16-bit images
            if channel_img.format == 'ushort':
                warped_channel.write_to_file(individual_channel_path, compression="none", tile=False)
            else:
                warped_channel.write_to_file(individual_channel_path)
                
            print(f"Saved warped channel: {individual_channel_path}, dimensions={current_width}x{current_height}")
            
        except Exception as e:
            print(f"Error processing {src_f}: {str(e)}")

# # STEP 5: Resize warped images to match padded reference dimensions
# print("Resizing warped images to match padded reference dimensions...")
# resize_warped_images(individual_channel_dir, padded_reference_slide)

# # STEP 6: Fix any remaining aspect ratio issues
# fix_warped_images(individual_channel_dir, padded_reference_slide)

print("\n" + "="*60)
print("REGISTRATION COMPLETE!")
print("="*60)
print(f"✅ CLAHE applied ONLY to padded DAPI (ch00) for registration")
print(f"✅ All channels warped with ORIGINAL intensities preserved")
print(f"✅ 16-bit image format preserved throughout processing")
print(f"✅ Warped images resized to match padded reference dimensions ({max_width} x {max_height})")
print(f"✅ Warped images fixed to maintain correct aspect ratio")
print(f"📁 Original padded images: {padded_dir}")
print(f"📁 Final warped images: {individual_channel_dir}")
print(f"📁 Backup of original warped images: {os.path.join(dst_dir, 'resize_backup')}")

# Clean up CLAHE-padded directory
print("\nCleaning up temporary CLAHE-padded files...")
shutil.rmtree(clahe_padded_dir)
print("✅ Cleaned up CLAHE-padded temporary files")

# Kill JVM
registration.kill_jvm()

if name == "main":
main()

ERROR TRACE :

Performing registration with CLAHE-padded DAPI images...
Loaded LightGlue model
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: Can't find slide file associated with masks
warnings.warn(warning_msg, warning_type)
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: Can't find slide file associated with processed
warnings.warn(warning_msg, warning_type)
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: max_image_dim_px is 1024 but needs to be less or equal to 2000. Setting max_image_dim_px to 2000
warnings.warn(warning_msg, warning_type)

==== Converting images

Converting images: 100%|██████████| 11/11 [04:22<00:00, 23.87s/image]

==== Processing images

Processing images : 100%|██████████| 11/11 [04:26<00:00, 24.26s/image]
Normalizing images: 100%|██████████| 11/11 [00:04<00:00, 2.24image/s]

==== Rigid registration

Detecting features : 0%| | 0/11 [00:00<?, ?image/s]
detecting features in level 0 with image shape (1019, 2000)
Detecting features : 9%|▉ | 1/11 [00:03<00:38, 3.82s/image]
detecting features in level 0 with image shape (1081, 2000)
Detecting features : 18%|█▊ | 2/11 [00:08<00:38, 4.31s/image]
detecting features in level 0 with image shape (1081, 2000)
Detecting features : 27%|██▋ | 3/11 [00:11<00:30, 3.84s/image]
detecting features in level 0 with image shape (1019, 2000)
Detecting features : 36%|███▋ | 4/11 [00:15<00:27, 3.90s/image]
detecting features in level 0 with image shape (1081, 2000)
Detecting features : 45%|████▌ | 5/11 [00:19<00:22, 3.75s/image]
detecting features in level 0 with image shape (1018, 2000)
Detecting features : 55%|█████▍ | 6/11 [00:23<00:19, 3.88s/image]
detecting features in level 0 with image shape (1081, 2000)
Detecting features : 64%|██████▎ | 7/11 [00:27<00:15, 3.82s/image]
detecting features in level 0 with image shape (878, 2000)
Detecting features : 73%|███████▎ | 8/11 [00:30<00:10, 3.64s/image]
detecting features in level 0 with image shape (1153, 2000)
Detecting features : 82%|████████▏ | 9/11 [00:34<00:07, 3.94s/image]
detecting features in level 0 with image shape (877, 1999)
Detecting features : 91%|█████████ | 10/11 [00:39<00:04, 4.00s/image]
detecting features in level 0 with image shape (1018, 1999)
Detecting features : 100%|██████████| 11/11 [00:41<00:00, 3.80s/image]
QUEUEING TASKS | Matching images : 100%|██████████| 11/11 [00:00<00:00, 211.23image/s]
PROCESSING TASKS | Matching images : 100%|██████████| 11/11 [00:51<00:00, 4.64s/image]
COLLECTING RESULTS | Matching images : 100%|██████████| 11/11 [00:00<00:00, 37755.60image/s]
Images sorted using VggFD features. Will now use LightGlueMatcher to match images using DiskFD features
detecting features in level 0 with image shape (1153, 2000)
Re-matching images: 0%| | 0/10 [00:00<?, ?image/s]
detecting features in level 0 with image shape (1256, 2146)
Re-matching images: 0%| | 0/10 [00:03<?, ?image/s]
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 4736, in register
rigid_registrar = self.rigid_register()
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 3401, in rigid_register
rigid_registrar = serial_rigid.register_images(img_dir=self.processed_dir,
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 1825, in register_images
registrar.rematch(matcher_obj=matcher, valis_obj=valis_obj, keep_unfiltered=False)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 901, in rematch
matcher_obj.match_images(img1=prev_img, img2=rotated_img,
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/feature_matcher.py", line 1458, in match_images
match_distances = match_distances.detach().numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Performing micro-registration...

TypeError Traceback (most recent call last)
Cell In[7], line 698
695 registration.kill_jvm()
697 if name == "main":
--> 698 main()

Cell In[7], line 529
527 # Perform micro-registration
528 print("Performing micro-registration...")
--> 529 micro_reg, micro_error = registrar.register_micro(
530 max_non_rigid_registration_dim_px=2000,
531 align_to_reference=True
532 )
534 # STEP 4: Apply registration transforms to original padded images
535 print("Applying registration transforms to original intensity padded images...")

File ~/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:40, in deprecated_args..deco..wrapper(*args, **kwargs)
37 @functools.wraps(f)
38 def wrapper(*args, **kwargs):
39 rename_kwargs(f.name, kwargs, aliases)
---> 40 return f(*args, **kwargs)

File ~/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py:4901, in Valis.register_micro(self, brightfield_processing_cls, brightfield_processing_kwargs, if_processing_cls, if_processing_kwargs, processor_dict, max_non_rigid_registration_dim_px, non_rigid_registrar_cls, non_rigid_reg_params, reference_img_f, align_to_reference, mask, tile_wh)
4897 non_rigid_registrar_obj = non_rigid_registrar_cls
4899 reg_in_rgb = non_rigid_registrar_obj.rgb
4900 nr_reg_src, max_img_dim, non_rigid_reg_mask, full_out_shape_rc, mask_bbox_xywh, using_tiler =
-> 4901 self.prep_images_for_large_non_rigid_registration(max_img_dim=max_non_rigid_registration_dim_px,
4902 processor_dict=slide_processors,
4903 updating_non_rigid=True,
4904 mask=mask,
4905 rgb=reg_in_rgb)
4907 img_specific_args = None
4908 write_dxdy = isinstance(ref_slide.bk_dxdy, pyvips.Image)

File ~/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py:3892, in Valis.prep_images_for_large_non_rigid_registration(self, max_img_dim, processor_dict, updating_non_rigid, mask, rgb)
3889 s = np.min(max_img_dim/np.array(to_reg_mask_wh))
3891 if s < max_s:
-> 3892 full_out_shape = self.get_aligned_slide_shape(s)
3893 else:
3894 full_out_shape = self.get_aligned_slide_shape(0)

File ~/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py:5095, in Valis.get_aligned_slide_shape(self, level)
5092 else:
5093 s_rc = level
-> 5095 aligned_out_shape_rc = np.ceil(np.array(ref_slide.reg_img_shape_rc)*s_rc).astype(int)
5097 return aligned_out_shape_rc

TypeError: unsupported operand type(s) for *: 'NoneType' and 'float'

[cdgatenbee]

Hi @anthomontan,
Thanks for reporting this. Looking through the error messages, it looks like things start going wrong when valis tries to detect features with DiskFD, which uses pytorch. Based on the line "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: can't convert cuda:0", it looks like at least one problem is that DiskFD and LightGlueMatcher are on the cpu, but the images are on the gpu, which throws an error and stops the process. Could you try setting device="cuda" when initializing DiskFD and LightGlueMatcher? e.g.

fd = feature_detectors.DiskFD(rgb=False, num_features=2000, device='cuda')
matcher = feature_matcher.LightGlueMatcher(fd, , device='cuda')

In the meantime, I'll run some additional tests using our cluster's gpu to check if any other changes need to be made to get valis running smoothly on the gpu. Unfortunately, it's currently under maintenance but will back up in a few days. Once it is, I'll run those tests and make any necessary changes.

I'll also look into the issue of the image dimensions being swapped during error measurement.

Best,
-Chandler

[anthomontan]

Hi Chandler,

Thanks a lot,

Testing / running now,
I let you know ;D

I'll also look into the issue of the image dimensions being swapped during error measurement 👍

This was odd, it did swap with some alignment images but not others (entire multiplex images in folders)...
It does that for the entire wrapped images (either all correctly oriented, or all swapped) ..?

Thanks a lot,
Antho

[anthomontan]

Hi Chandler,

Same issue using cuda :

import os
import torch
import pathlib
import warnings
import numpy as np
from pathlib import Path
from tqdm import tqdm
import pyvips
import cv2
from skimage import exposure
from valis import valtils, slide_io, registration
from valis import feature_detectors, feature_matcher
from valis.micro_rigid_registrar import MicroRigidRegistrar
from valis.non_rigid_registrars import RAFTWarper
import shutil
import re

# Force PyTorch to use CPU globally

os.environ["CUDA_VISIBLE_DEVICES"] = ""

torch.set_default_device('cpu')

torch.backends.cudnn.enabled = False

if torch.cuda.is_available():

torch.cuda.empty_cache()

Import necessary libraries

try:
from tqdm.notebook import tqdm
except ImportError:
from tqdm import tqdm
print("tqdm.notebook not found. Using tqdm instead.")

def organize_channels_automatically(base_dir, target_dapi_dir=None, target_all_channels_dir=None):
"""Organize channel files from a base directory."""
base_path = Path(base_dir)
if target_dapi_dir is None:
target_dapi_dir = base_path / "dapi"
else:
target_dapi_dir = Path(target_dapi_dir)
if target_all_channels_dir is None:
target_all_channels_dir = base_path / "all_channels"
else:
target_all_channels_dir = Path(target_all_channels_dir)
target_dapi_dir.mkdir(parents=True, exist_ok=True)
target_all_channels_dir.mkdir(parents=True, exist_ok=True)
print(f"Organizing files from: {base_path}")
print(f"DAPI files will go to: {target_dapi_dir}")
print(f"Other channels will go to: {target_all_channels_dir}")
all_files = [f for f in base_path.iterdir() if f.is_file()]
moved_files = {"dapi": [], "other_channels": [], "unrecognized": []}
for file_path in all_files:
filename = file_path.name.lower()
if "_ch00" in filename or "_merged_ch00" in filename:
target_path = target_dapi_dir / file_path.name
shutil.move(str(file_path), str(target_path))
moved_files["dapi"].append(file_path.name)
print(f"📁 DAPI: {file_path.name}")
elif any(ch in filename for ch in ["_ch01", "_ch02", "_ch03", "_merged_ch01", "_merged_ch02", "_merged_ch03"]):
target_path = target_all_channels_dir / file_path.name
shutil.move(str(file_path), str(target_path))
moved_files["other_channels"].append(file_path.name)
print(f"🔬 Channel: {file_path.name}")
else:
moved_files["unrecognized"].append(file_path.name)
print(f"❓ Unrecognized: {file_path.name}")
print("\n" + "="*60)
print("ORGANIZATION COMPLETE!")
print("="*60)
print(f"✅ DAPI files moved: {len(moved_files['dapi'])}")
print(f"✅ Other channel files moved: {len(moved_files['other_channels'])}")
print(f"⚠️ Unrecognized files: {len(moved_files['unrecognized'])}")
if moved_files["unrecognized"]:
print("\nUnrecognized files (not moved):")
for filename in moved_files["unrecognized"]:
print(f" - {filename}")
return moved_files

def get_round_name(src_f):
"""Extract the round name from the filename."""
img_name = valtils.get_name(src_f)
round_name = img_name.lower().split("_merged")[0]
return round_name

def get_channel_number(src_f):
"""Extract the channel number from the filename."""
img_name = os.path.basename(src_f).lower()
if "_ch00" in img_name:
return 0
elif "_ch01" in img_name:
return 1
elif "_ch02" in img_name:
return 2
else:
return None

def pad_image(img, target_width, target_height):
"""Pad image to target dimensions."""
pad_width = max(0, target_width - img.width)
pad_height = max(0, target_height - img.height)
if pad_width > 0 or pad_height > 0:
pad_left = pad_width // 2
pad_top = pad_height // 2
return img.embed(pad_left, pad_top, target_width, target_height, extend="black")
return img

def apply_clahe_normalization(img_array, clip_limit=2.0, tile_grid_size=(16, 16)):
"""Apply CLAHE normalization."""
if img_array.dtype != np.uint8:
img_normalized = exposure.rescale_intensity(img_array, out_range=(0, 255)).astype(np.uint8)
else:
img_normalized = img_array.copy()
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=tile_grid_size)
return clahe.apply(img_normalized)

def create_clahe_padded_image(img_path, target_width, target_height, output_path, clip_limit=2.0, tile_grid_size=(16, 16)):
"""Create a CLAHE-padded image."""
try:
img = pyvips.Image.new_from_file(img_path)
print(f"Image format for {os.path.basename(img_path)}: {img.format}, bands: {img.bands}")
padded_img = pad_image(img, target_width, target_height)
img_array = np.ndarray(buffer=padded_img.write_to_memory(),
dtype=np.uint8 if padded_img.format == 'uchar' else np.uint16,
shape=[padded_img.height, padded_img.width, padded_img.bands])
if padded_img.bands == 1:
clahe_array = apply_clahe_normalization(img_array[:, :, 0], clip_limit, tile_grid_size)
clahe_array = clahe_array[:, :, np.newaxis]
else:
clahe_array = img_array.copy()
clahe_array[:, :, 0] = apply_clahe_normalization(img_array[:, :, 0], clip_limit, tile_grid_size)
clahe_padded_img = pyvips.Image.new_from_memory(clahe_array.data,
clahe_array.shape[1],
clahe_array.shape[0],
clahe_array.shape[2],
'uchar')
clahe_padded_img.write_to_file(output_path)
return output_path
except Exception as e:
print(f"Error creating CLAHE-padded image for {img_path}: {str(e)}")
return None

def resize_warped_images(warped_dir, padded_reference_path):
"""Resize warped images."""
print(f"Resizing warped images in {warped_dir}...")
ref_img = pyvips.Image.new_from_file(padded_reference_path)
ref_width, ref_height = ref_img.width, ref_img.height
print(f"Padded reference dimensions: {ref_width} x {ref_height}")
backup_dir = os.path.join(os.path.dirname(warped_dir), "resize_backup")
os.makedirs(backup_dir, exist_ok=True)
resized_count = 0
for root, dirs, files in os.walk(warped_dir):
for file in files:
if file.startswith("warped_") and file.endswith(".tif"):
img_path = os.path.join(root, file)
try:
warped_img = pyvips.Image.new_from_file(img_path)
if warped_img.width != ref_width or warped_img.height != ref_height:
print(f"Resizing image: {file}")
backup_path = os.path.join(backup_dir, file)
shutil.copy2(img_path, backup_path)
resized_img = warped_img.resize(ref_width / warped_img.width,
vscale=ref_height / warped_img.height,
kernel="bilinear")
if warped_img.format == 'ushort':
resized_img.write_to_file(img_path, compression="none", tile=False)
else:
resized_img.write_to_file(img_path)
resized_count += 1
else:
print(f"Image dimensions already match reference: {file}")
except Exception as e:
print(f"Error processing {file}: {str(e)}")
print(f"\nResized {resized_count} images")
return resized_count

def fix_warped_images(individual_channel_dir, reference_slide_path):
"""Fix warped images to maintain correct aspect ratio."""
print("\nFixing warped images to maintain correct aspect ratio...")
ref_img = pyvips.Image.new_from_file(reference_slide_path)
ref_width, ref_height, ref_format, ref_bands = ref_img.width, ref_img.height, ref_img.format, ref_img.bands
print(f"Reference image: {ref_width}x{ref_height}, format={ref_format}, bands={ref_bands}")
backup_dir = os.path.join(os.path.dirname(individual_channel_dir), "warped_backup")
os.makedirs(backup_dir, exist_ok=True)
for root, dirs, files in os.walk(individual_channel_dir):
for file in files:
if file.startswith("warped_") and file.endswith(".tif"):
img_path = os.path.join(root, file)
try:
warped_img = pyvips.Image.new_from_file(img_path)
print(f"Processing {file}: {warped_img.width}x{warped_img.height}, format={warped_img.format}")
if warped_img.width != ref_width or warped_img.height != ref_height:
print(f"Fixing dimensions for {file}")
backup_path = os.path.join(backup_dir, file)
shutil.copy2(img_path, backup_path)
scale_x = ref_width / warped_img.width
scale_y = ref_height / warped_img.height
scale = min(scale_x, scale_y)
new_width = int(warped_img.width * scale)
new_height = int(warped_img.height * scale)
left_pad = (ref_width - new_width) // 2
top_pad = (ref_height - new_height) // 2
transform = [scale, 0, left_pad, 0, scale, top_pad]
interp = pyvips.Interpolate.new('bicubic')
fixed_img = warped_img.affine(transform, interpolate=interp)
if fixed_img.width < ref_width or fixed_img.height < ref_height:
fixed_img = fixed_img.embed(0, 0, ref_width, ref_height, extend="black")
if fixed_img.width > ref_width or fixed_img.height > ref_height:
fixed_img = fixed_img.crop(0, 0, ref_width, ref_height)
if warped_img.format == 'ushort':
fixed_img.write_to_file(img_path, compression="none", tile=False)
else:
fixed_img.write_to_file(img_path)
print(f"Fixed image saved: {img_path}")
except Exception as e:
print(f"Error fixing {file}: {str(e)}")
print("✅ Fixed warped images to match reference dimensions")
return backup_dir

def main():
"""Main function to organize, process, and register images."""
base_dir = ""
print("Organizing channel files...")
organize_channels_automatically(base_dir)
slide_src_dir = Path(base_dir) / "dapi"
path_to_all_channels = Path(base_dir) / "all_channels"
results_dst_dir = Path(base_dir) / "registration"
results_dst_dir.mkdir(parents=True, exist_ok=True)
registered_slide_dst_dir = results_dst_dir / "registered_slides"
registered_slide_dst_dir.mkdir(parents=True, exist_ok=True)
reference_slide_name = ""
reference_slide = str(slide_src_dir / reference_slide_name)
print(f"✅ DAPI images directory: {slide_src_dir}")
print(f"✅ All channels directory: {path_to_all_channels}")
print(f"✅ Results directory: {results_dst_dir}")
print(f"✅ Reference slide: {reference_slide}")

src_dir = str(slide_src_dir)
dst_dir = str(results_dst_dir)
all_channels_dir = str(path_to_all_channels)
padded_dir = os.path.join(dst_dir, "padded_images")
os.makedirs(padded_dir, exist_ok=True)
padded_dapi_dir = os.path.join(padded_dir, "dapi")
os.makedirs(padded_dapi_dir, exist_ok=True)
padded_all_channels_dir = os.path.join(padded_dir, "all_channels")
os.makedirs(padded_all_channels_dir, exist_ok=True)
padded_ref_dir = os.path.join(padded_dir, "reference")
os.makedirs(padded_ref_dir, exist_ok=True)
clahe_padded_dir = os.path.join(dst_dir, "clahe_padded_dapi")
os.makedirs(clahe_padded_dir, exist_ok=True)
individual_channel_dir = os.path.join(dst_dir, "warped_channels")
os.makedirs(individual_channel_dir, exist_ok=True)

dapi_imgs = [os.path.join(src_dir, f) for f in os.listdir(src_dir)]
all_channel_imgs = [os.path.join(all_channels_dir, f) for f in os.listdir(all_channels_dir)]
print(f"Found {len(dapi_imgs)} DAPI images for registration")
print(f"Found {len(all_channel_imgs)} other channel images")

max_width, max_height = 0, 0
print("Finding maximum dimensions across all images...")
for img_path in dapi_imgs + all_channel_imgs:
    try:
        img = pyvips.Image.new_from_file(img_path)
        max_width = max(max_width, img.width)
        max_height = max(max_height, img.height)
        if max_width == img.width or max_height == img.height:
            print(f"New max from {os.path.basename(img_path)}: {img.width}x{img.height}, format={img.format}")
    except Exception as e:
        print(f"Error reading {img_path}: {str(e)}")
print(f"Maximum dimensions: {max_width} x {max_height}")

# Define the padded reference and CLAHE-padded reference paths
padded_reference_path = os.path.join(padded_ref_dir, f"padded_{reference_slide_name}")
clahe_padded_reference_path = os.path.join(clahe_padded_dir, f"clahe_padded_{reference_slide_name}")

# Create padded reference if not already present
if not os.path.exists(padded_reference_path):
    print("Creating padded reference image...")
    ref_img = pyvips.Image.new_from_file(reference_slide)
    print(f"Reference image format: {ref_img.format}, bands: {ref_img.bands}")
    padded_ref_img = pad_image(ref_img, max_width, max_height)
    if ref_img.format == 'ushort':
        padded_ref_img.write_to_file(padded_reference_path, compression="none", tile=False)
    else:
        padded_ref_img.write_to_file(padded_reference_path)
    print(f"Padded reference: {padded_reference_path}")
else:
    print(f"Reusing existing padded reference: {padded_reference_path}")

# Create CLAHE-padded reference if not already present
if not os.path.exists(clahe_padded_reference_path):
    print("Creating CLAHE-padded reference image...")
    create_clahe_padded_image(reference_slide, max_width, max_height, clahe_padded_reference_path)
    print(f"CLAHE-padded reference: {clahe_padded_reference_path}")
else:
    print(f"Reusing existing CLAHE-padded reference: {clahe_padded_reference_path}")

# Get a list of all existing padded and CLAHE-padded files
existing_padded_dapi_files = {f.replace("padded_", "") for f in os.listdir(padded_dapi_dir)}
existing_clahe_padded_files = {f.replace("clahe_padded_", "") for f in os.listdir(clahe_padded_dir)}
existing_padded_all_channels_files = {f.replace("padded_", "") for f in os.listdir(padded_all_channels_dir)}

# Process DAPI and other channels only if not already padded
padded_dapi_paths = []
clahe_padded_dapi_paths = []

for img_path in dapi_imgs:
    try:
        basename = os.path.basename(img_path)
        padded_path = os.path.join(padded_dapi_dir, f"padded_{basename}")
        if basename in existing_padded_dapi_files:
            print(f"✅ Reusing existing padded DAPI: {basename}")
            padded_dapi_paths.append(padded_path)
            clahe_padded_path = os.path.join(clahe_padded_dir, f"clahe_padded_{basename}")
            if basename in existing_clahe_padded_files:
                print(f"✅ Reusing existing CLAHE-padded DAPI: {basename}")
                clahe_padded_dapi_paths.append(clahe_padded_path)
            else:
                if create_clahe_padded_image(img_path, max_width, max_height, clahe_padded_path):
                    clahe_padded_dapi_paths.append(clahe_padded_path)
            continue

        img = pyvips.Image.new_from_file(img_path)
        padded_img = pad_image(img, max_width, max_height)
        if img.format == 'ushort':
            padded_img.write_to_file(padded_path, compression="none", tile=False)
        else:
            padded_img.write_to_file(padded_path)
        padded_dapi_paths.append(padded_path)
        clahe_padded_path = os.path.join(clahe_padded_dir, f"clahe_padded_{basename}")
        if create_clahe_padded_image(img_path, max_width, max_height, clahe_padded_path):
            clahe_padded_dapi_paths.append(clahe_padded_path)
        print(f"Processed DAPI: {basename}")
    except Exception as e:
        print(f"Error processing {img_path}: {str(e)}")

padded_all_channel_paths = []
for img_path in all_channel_imgs:
    try:
        basename = os.path.basename(img_path)
        padded_path = os.path.join(padded_all_channels_dir, f"padded_{basename}")
        if basename in existing_padded_all_channels_files:
            print(f"✅ Reusing existing padded channel: {basename}")
            padded_all_channel_paths.append(padded_path)
            continue

        img = pyvips.Image.new_from_file(img_path)
        padded_img = pad_image(img, max_width, max_height)
        if img.format == 'ushort':
            padded_img.write_to_file(padded_path, compression="none", tile=False)
        else:
            padded_img.write_to_file(padded_path)
        padded_all_channel_paths.append(padded_path)
        print(f"Padded channel: {basename}")
    except Exception as e:
        print(f"Error padding {img_path}: {str(e)}")

# Continue with registration and warping as before...
print("Performing registration with CLAHE-padded DAPI images...")
fd = feature_detectors.DiskFD(rgb=False, num_features=5000, device='cuda')
# class CPUSafeLightGlueMatcher(feature_matcher.LightGlueMatcher):
#     def __init__(self, feature_detector, **kwargs):
#         kwargs['device'] = 'cuda'
#         super().__init__(feature_detector, **kwargs)
#         original_match = self.match_images
#         def safe_match(*args, **kwargs):
#             with torch.no_grad():
#                 torch.set_grad_enabled(False)
#                 return original_match(*args, **kwargs)
#         self.match_images = safe_match
matcher = feature_matcher.LightGlueMatcher(fd, device='cuda')
raft_warper = RAFTWarper(rgb=False, device='cuda')
registrar = registration.Valis(
    clahe_padded_dir,
    dst_dir,
    reference_img_f=clahe_padded_reference_path,
    align_to_reference=True,
    micro_rigid_registrar_cls=None,
    max_processed_image_dim_px=4000,
    max_non_rigid_registration_dim_px=4000,
    norm_method='img_stats',
    matcher=matcher,
    non_rigid_registrar_cls=raft_warper
)
try:
    rigid_registrar, non_rigid_registrar, error_df = registrar.register()
    print("Performing micro-registration...")
    micro_reg, micro_error = registrar.register_micro(
        max_non_rigid_registration_dim_px=4096,
        align_to_reference=True
    )
except Exception as e:
    print(f"Registration error: {str(e)}")
    warnings.warn(f"Registration error: {str(e)}", UserWarning)

print("Applying registration transforms to original intensity padded images...")
clahe_to_original_mapping = {}
for slide_name, slide_obj in registrar.slide_dict.items():
    clahe_basename = os.path.basename(slide_obj.src_f).replace("clahe_padded_", "padded_")
    original_path = os.path.join(padded_dapi_dir, clahe_basename)
    clahe_to_original_mapping[slide_obj] = original_path

dapi_warped_dir = os.path.join(individual_channel_dir, "dapi_warped")
os.makedirs(dapi_warped_dir, exist_ok=True)
for slide_obj, original_dapi_path in clahe_to_original_mapping.items():
    try:
        original_dapi_img = pyvips.Image.new_from_file(original_dapi_path)
        orig_width, orig_height = original_dapi_img.width, original_dapi_img.height
        print(f"Warping {os.path.basename(original_dapi_path)}, format={original_dapi_img.format}, dimensions={orig_width}x{orig_height}")
        warped_dapi = slide_obj.warp_img(img=original_dapi_img, non_rigid=True, crop=False)
        current_width, current_height = warped_dapi.width, warped_dapi.height
        print(f"  Raw warped dimensions: {current_width}x{current_height}")
        orig_orientation = "landscape" if orig_width > orig_height else "portrait"
        warped_orientation = "landscape" if current_width > current_height else "portrait"
        orig_aspect = orig_width / orig_height
        warped_aspect = current_width / current_height
        aspect_ratio_diff = abs(warped_aspect - orig_aspect)
        aspect_ratio_diff_transposed = abs(warped_aspect - (1/orig_aspect))
        needs_transposition = False
        if orig_orientation == warped_orientation:
            if aspect_ratio_diff_transposed < aspect_ratio_diff * 0.5:
                needs_transposition = True
                print(f"  ⚠️ Aspect ratio mismatch suggests transposition needed")
        else:
            needs_transposition = True
            print(f"  ⚠️ Orientation mismatch: original={orig_orientation}, warped={warped_orientation}")
        if needs_transposition:
            print(f"  💡 Transposing image to fix orientation...")
            warped_dapi = warped_dapi.transpose(pyvips.enums.Orientation.ROT90)
            current_width, current_height = warped_dapi.width, warped_dapi.height
            print(f"  ✅ Fixed dimensions: {current_width}x{current_height}")
        basename = os.path.basename(original_dapi_path)
        warped_path = os.path.join(dapi_warped_dir, f"warped_{basename}")
        if original_dapi_img.format == 'ushort':
            warped_dapi.write_to_file(warped_path, compression="none", tile=False)
        else:
            warped_dapi.write_to_file(warped_path)
        print(f"Warped original DAPI: {warped_path}, dimensions={current_width}x{current_height}")
    except Exception as e:
        print(f"Error warping original DAPI {slide_obj.name}: {str(e)}")

# Create mapping between DAPI slides and their corresponding other channels
round_dict = {}
for slide_obj in registrar.slide_dict.values():
    # Get the original filename
    if USE_CLAHE:
        original_name = os.path.basename(slide_obj.src_f).replace("clahe_padded_", "")
    else:
        original_name = os.path.basename(slide_obj.src_f).replace("padded_", "")
    base_round_name = get_round_name(original_name)
    print(f"Finding matches for round: {base_round_name}")
    # Find all padded channels that belong to this round
    matching_channels = []
    for padded_path in padded_all_channel_paths:
        padded_basename = os.path.basename(padded_path).replace("padded_", "")
        if get_round_name(padded_basename) == base_round_name:
            matching_channels.append(padded_path)
    print(f"Found {len(matching_channels)} matching channels")
    round_dict[slide_obj] = matching_channels

# Warp all other channels (ch01, ch02) with original intensities
print("Warping all other channels with original intensities...")
for slide_obj, round_slides in round_dict.items():
    print(f"\nWarping channels for round: {slide_obj.name}")
    valtils.sort_nicely(round_slides)
    for src_f in round_slides:
        img_name = os.path.basename(src_f)
        print(f"Warping channel: {img_name}")
        try:
            channel_img = pyvips.Image.new_from_file(src_f)
            orig_width, orig_height = channel_img.width, channel_img.height
            print(f"  Original channel dimensions: {orig_width}x{orig_height}, format={channel_img.format}, bands={channel_img.bands}")
            warped_channel = slide_obj.warp_img(img=channel_img, non_rigid=True, crop=False)
            current_width, current_height = warped_channel.width, warped_channel.height
            print(f"  Raw warped dimensions: {current_width}x{current_height}")
            orig_orientation = "landscape" if orig_width > orig_height else "portrait"
            warped_orientation = "landscape" if current_width > current_height else "portrait"
            orig_aspect = orig_width / orig_height
            warped_aspect = current_width / current_height
            aspect_ratio_diff = abs(warped_aspect - orig_aspect)
            aspect_ratio_diff_transposed = abs(warped_aspect - (1/orig_aspect))
            needs_transposition = False
            if orig_orientation == warped_orientation:
                if aspect_ratio_diff_transposed < aspect_ratio_diff * 0.5:
                    needs_transposition = True
                    print(f"  ⚠️ Aspect ratio mismatch suggests transposition needed")
            else:
                needs_transposition = True
                print(f"  ⚠️ Orientation mismatch: original={orig_orientation}, warped={warped_orientation}")
            if needs_transposition:
                print(f"  💡 Transposing image to fix orientation...")
                warped_channel = warped_channel.transpose(pyvips.enums.Orientation.ROT90)
                current_width, current_height = warped_channel.width, warped_channel.height
                print(f"  ✅ Fixed dimensions: {current_width}x{current_height}")
            channel_basename = os.path.basename(src_f).replace("padded_", "")
            individual_channel_path = os.path.join(individual_channel_dir, f"warped_{channel_basename}")
            if channel_img.format == 'ushort':
                warped_channel.write_to_file(individual_channel_path, compression="none", tile=False)
            else:
                warped_channel.write_to_file(individual_channel_path)
            print(f"Saved warped channel: {individual_channel_path}, dimensions={current_width}x{current_height}")
        except Exception as e:
            print(f"Error processing {src_f}: {str(e)}")

print("\n" + "="*60)
print("REGISTRATION COMPLETE!")
print("="*60)
print(f"✅ CLAHE applied ONLY to padded DAPI (ch00) for registration")
print(f"✅ All channels warped with ORIGINAL intensities preserved")
print(f"✅ 16-bit image format preserved throughout processing")
print(f"✅ Warped images resized to match padded reference dimensions ({max_width} x {max_height})")
print(f"✅ Warped images fixed to maintain correct aspect ratio")
print(f"📁 Original padded images: {padded_dir}")
print(f"📁 Final warped images: {individual_channel_dir}")
print(f"📁 Backup of original warped images: {os.path.join(dst_dir, 'resize_backup')}")

# Clean up CLAHE-padded directory
print("\nCleaning up temporary CLAHE-padded files...")
shutil.rmtree(clahe_padded_dir)
print("✅ Cleaned up CLAHE-padded temporary files")

# Kill JVM
registration.kill_jvm()

if name == "main":
USE_CLAHE = True # Define whether to use CLAHE enhancement
main()

OUTPUT:

...
Performing registration with CLAHE-padded DAPI images...
Loaded LightGlue model
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: max_image_dim_px is 1024 but needs to be less or equal to 4000. Setting max_image_dim_px to 4000
warnings.warn(warning_msg, warning_type)

==== Converting images

Converting images: 100%|██████████| 11/11 [17:08<00:00, 93.52s/image]

==== Processing images

Processing images : 100%|██████████| 11/11 [15:09<00:00, 82.70s/image]
Normalizing images: 100%|██████████| 11/11 [00:17<00:00, 1.60s/image]

==== Rigid registration

Detecting features : 0%| | 0/11 [00:00<?, ?image/s]
detecting features in level 0 with image shape (2665, 4000)
Detecting features : 9%|▉ | 1/11 [00:10<01:47, 10.70s/image]
detecting features in level 0 with image shape (2399, 4000)
Detecting features : 18%|█▊ | 2/11 [00:24<01:52, 12.45s/image]
detecting features in level 0 with image shape (2797, 4000)
Detecting features : 27%|██▋ | 3/11 [00:36<01:37, 12.18s/image]
detecting features in level 0 with image shape (2665, 4000)
Detecting features : 36%|███▋ | 4/11 [00:49<01:29, 12.72s/image]
detecting features in level 0 with image shape (2439, 4000)
Detecting features : 45%|████▌ | 5/11 [01:00<01:12, 12.16s/image]
detecting features in level 0 with image shape (2588, 4000)
Detecting features : 55%|█████▍ | 6/11 [01:16<01:06, 13.22s/image]
detecting features in level 0 with image shape (2661, 4000)
Detecting features : 64%|██████▎ | 7/11 [01:27<00:50, 12.71s/image]
detecting features in level 0 with image shape (2715, 4000)
Detecting features : 73%|███████▎ | 8/11 [01:37<00:35, 11.74s/image]
detecting features in level 0 with image shape (2544, 4000)
Detecting features : 82%|████████▏ | 9/11 [01:50<00:23, 11.99s/image]
detecting features in level 0 with image shape (2475, 4000)
Detecting features : 91%|█████████ | 10/11 [02:00<00:11, 11.57s/image]
detecting features in level 0 with image shape (2666, 4000)
Detecting features : 100%|██████████| 11/11 [02:09<00:00, 11.76s/image]
QUEUEING TASKS | Matching images : 100%|██████████| 11/11 [00:00<00:00, 145.12image/s]
PROCESSING TASKS | Matching images : 100%|██████████| 11/11 [00:46<00:00, 4.27s/image]
COLLECTING RESULTS | Matching images : 100%|██████████| 11/11 [00:00<00:00, 83130.35image/s]
Images sorted using VggFD features. Will now use LightGlueMatcher to match images using DiskFD features
detecting features in level 0 with image shape (2544, 4000)
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 4736, in register
rigid_registrar = self.rigid_register()
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 3401, in rigid_register
rigid_registrar = serial_rigid.register_images(img_dir=self.processed_dir,
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 1825, in register_images
registrar.rematch(matcher_obj=matcher, valis_obj=valis_obj, keep_unfiltered=False)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 824, in rematch
ref_kp, ref_desc = matcher_obj.feature_detector.detect_and_compute(ref_img)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/feature_detectors.py", line 191, in detect_and_compute
kp_pos_xy, desc = self._detect_and_compute(detect_img, mask)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/feature_detectors.py", line 608, in _detect_and_compute
kp_pos_xy = res.keypoints.detach().numpy()
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Performing micro-registration...
Registration error: unsupported operand type(s) for *: 'NoneType' and 'float'
/tmp/ipykernel_324609/3232423135.py:389: UserWarning: Registration error: unsupported operand type(s) for *: 'NoneType' and 'float'
warnings.warn(f"Registration error: {str(e)}", UserWarning)
Applying registration transforms to original intensity padded images...
Warping padded_.tif, format=ushort, dimensions=42530x27664
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: scaling transformation for image with different shape. However, without knowing all of other image's shapes, the scaling may not be the same for all images, and so may not overlap.
warnings.warn(warning_msg, warning_type)
Error warping original DAPI clahe_padded_ch00: unsupported operand type(s) for *: 'NoneType' and 'float'
Warping padded_, format=ushort, dimensions=42530x27664
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: scaling transformation for image with different shape. However, without knowing all of other image's shapes, the scaling may not be the same for all images, and so may not overlap.
warnings.warn(warning_msg, warning_type)

Error warping original DAPI clahe_padded_SB05 vimentin green 6jun25_R 2_Merged_ch00: unsupported operand type(s) for *: 'NoneType' and 'float'
Finding matches for round: s
Found 1 matching channels
Finding matches for round: s
Found 2 matching channels
Finding matches for round: s
Found 2 matching channels
Finding matches for round: s
Found 2 matching channels
...
Warping all other channels with original intensities...

Warping channels for round: clahe_padded_
Warping channel: padded_
Original channel dimensions: 42530x27664, format=ushort, bands=1
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: scaling transformation for image with different shape. However, without knowing all of other image's shapes, the scaling may not be the same for all images, and so may not overlap.
warnings.warn(warning_msg, warning_type)
Error processing ... registration/padded_images/all_channels/padded__ch01.tif: unsupported operand type(s) for *: 'NoneType' and 'float'

[cdgatenbee]

Hi @anthomontan,
Thanks for reporting back. Once our cluster is back up and running following maintenance, I'll run some tests using our cluster's gpu. Thanks again for brining this to my attention.

Best,
-Chandler

[anthomontan]

Hi Chandler.

I added "detach().cpu().numpy()" in feature_matcher and feature_detectors at every .numpy() .

But now :

Anyway to reduce the batch or size ?

thanks
antho

detecting features in level 0 with image shape (2749, 4151)
Re-matching images: 90%|█████████ | 9/10 [00:20<00:02, 2.28s/image]
/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/valtils.py:29: UserWarning: CUDA out of memory. Tried to allocate 3.41 GiB. GPU 0 has a total capacity of 23.99 GiB of which 2.75 GiB is free. Including non-PyTorch memory, this process has 17179869184.00 GiB memory in use. Of the allocated memory 11.91 GiB is allocated by PyTorch, and 8.03 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
warnings.warn(warning_msg, warning_type)
Traceback (most recent call last):
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 4736, in register
rigid_registrar = self.rigid_register()
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/registration.py", line 3401, in rigid_register
rigid_registrar = serial_rigid.register_images(img_dir=self.processed_dir,
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 1825, in register_images
registrar.rematch(matcher_obj=matcher, valis_obj=valis_obj, keep_unfiltered=False)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/serial_rigid.py", line 893, in rematch
rot_kp, rot_desc = matcher_obj.feature_detector.detect_and_compute(rotated_img)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/feature_detectors.py", line 191, in detect_and_compute
kp_pos_xy, desc = self._detect_and_compute(detect_img, mask)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/valis/feature_detectors.py", line 609, in _detect_and_compute
res = self.disk(tensor_img.to(self.device).float(), n=self.num_features, pad_if_not_divisible=True)[0]
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/kornia/feature/disk/disk.py", line 116, in forward
heatmaps, descriptors = self.heatmap_and_dense_descriptors(images)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/kornia/feature/disk/disk.py", line 75, in heatmap_and_dense_descriptors
unet_output = self.unet(images)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/kornia/feature/disk/_unets/unet.py", line 84, in forward
f_bot = layer(f_bot, f_hor)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/kornia/feature/disk/_unets/blocks.py", line 89, in forward
return self.conv(combined)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/container.py", line 244, in forward
input = module(input)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
return forward_call(*args, **kwargs)
File "/home/m197816/anaconda3/envs/VALIS2/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1524, in forward
return F.prelu(input, self.weight)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.41 GiB. GPU 0 has a total capacity of 23.99 GiB of which 2.75 GiB is free. Including non-PyTorch memory, this process has 17179869184.00 GiB memory in use. Of the allocated memory 11.91 GiB is allocated by PyTorch, and 8.03 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[cdgatenbee]

Hi @anthomontan,
Thanks for finding the solution! I'll update the code to this if the gpu is being used. Regarding the memory usage, feature detection is only done 1 image at a time, so no real way to decrease batch size (unless changing the method to scan the image in tiles and and combine features after processing all tiles...). But you could try decreasing the maximum number of features being detected from 2000 to 1000. This can be done when initializing the feature detector, i.e. fd = feature_detectors.DiskFD(rgb=False, num_features=1000, device='cpu'). If that doesn't work, and the amount of gpu memory requested can't be increased, it should also be possible to have the feature detectors, matchers, and tensors all on the cpu, which probably has more memory available. Once our cluster is back up I can work on making these updates.

Best,
-Chandler

[anthomontan]

Hi Chandler,

Thanks a lot,
Testing today, I'll update later on,

Antho

[anthomontan]

Hi Chandler.

I made an alternative using coarse + micro registration (KORNIA), inspired by Valis, thanks to you ;D
Have a look : https://github.com/anthomontan/Iterative-Immunofluorescence-Imaging-microscopy-registration-pipeline-for-large-images-/tree/main

Feel free to give me any feedback or integrate the code or modify,
It takes about 40min to align 31 tif images using 2 GPus 24gb,

This is my first repo, so not sure how to share/ distribute , acknowledge, but it is inspired using Valis and your inputs, so big thanks !

antho