Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Git
.git
.gitignore

# Docker
Dockerfile*
docker-compose*.yml
.dockerignore

# Node.js development files
ui/node_modules
ui/.vite
ui/coverage

# Python
__pycache__
*.pyc
*.pyo
*.pyd
.Python
env
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis

# IDE
.vscode
.idea
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Documentation
README.md
LICENSE
*.md
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#=====================================================================#
# Server & Setup Configuration #
#=====================================================================#

PUID=1000
PGID=1000
18 changes: 18 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
node_modules/
__pycache__/
dist/
.DS_Store
.env
npm-debug.log*
yarn-debug.log*
yarn-error.log*
node.zip
.vscode/
.claude/

# Ignore Models for testing
data/*
!data/comfy/user/default/workflows

# Ignore generated project files
gpu_config.json
103 changes: 103 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

ComfyUI-Distributed is a Python extension for ComfyUI that enables distributed and parallel processing across multiple GPUs and machines. It allows users to scale image/video generation and upscaling workflows by leveraging multiple GPU resources locally, remotely, or in the cloud.

## Architecture

### Core Components

**Main Modules:**
- `distributed.py` - Core distributed processing nodes and workflow coordination
- `distributed_upscale.py` - Specialized distributed upscaling functionality
- `__init__.py` - Node registration, ComfyUI integration, and execution patching

**Worker System:**
- **Master**: Main ComfyUI instance that coordinates work distribution
- **Workers**: ComfyUI instances that process tasks (local, remote, or cloud-based)
- `worker_monitor.py` - Monitors master process and terminates workers if master dies

**Utilities (`utils/`):**
- `config.py` - Configuration management and worker setup
- `process.py` - Process lifecycle management and monitoring
- `network.py` - HTTP client/server communication utilities
- `image.py` - Tensor/PIL image conversion utilities
- `async_helpers.py` - Async operation wrappers for ComfyUI integration
- `constants.py` - Shared timeout and configuration constants
- `usdu_managment.py` / `usdu_utils.py` - Ultimate SD Upscale distributed processing

**Frontend (`web/`):**
- `main.js` - Primary UI integration with ComfyUI
- `ui.js` - Worker management interface and controls
- `apiClient.js` - Backend API communication
- `workerUtils.js` - Worker process management utilities

### Key Design Patterns

**Distributed Processing:**
- Jobs are distributed to available workers with load balancing
- Results are collected and aggregated on the master
- Supports both parallel generation (multiple seeds) and distributed upscaling (tile-based)

**Process Management:**
- Workers are spawned as separate ComfyUI processes on different ports
- Master-worker communication via HTTP API
- Automatic worker cleanup when master terminates

**ComfyUI Integration:**
- Custom nodes register through `NODE_CLASS_MAPPINGS`
- Execution validation is patched for dynamic output nodes
- Frontend integrates with ComfyUI's existing UI framework

## Development Commands

### Testing
```bash
# No automated tests - manual testing through ComfyUI workflows
# Use the provided workflow JSON files in /workflows for testing different features
```

### Linting
```bash
# No specific linting commands configured
# Follow Python PEP 8 standards for new code
```

### Configuration
Worker configuration is managed through a JSON config file. The system auto-generates local worker configs on first launch.

### Key Workflow Files
- `/workflows/distributed-txt2img.json` - Basic parallel image generation
- `/workflows/distributed-wan.json` - Parallel video generation
- `/workflows/distributed-upscale.json` - Distributed image upscaling
- `/workflows/distributed-upscale-video.json` - Distributed video upscaling

## Important Implementation Details

### Worker Management
- Workers are auto-discovered for local GPUs on first launch
- Each worker runs on a unique port (8189, 8190, etc.)
- Workers require `--enable-cors-header` flag when using remote/cloud workers
- Process monitoring ensures workers terminate when master dies

### Memory and Performance
- Batch processing limited by `MAX_BATCH` constant (default 20 items)
- Heartbeat monitoring with configurable timeout (default 60s)
- Automatic VRAM cleanup between worker tasks

### Network Communication
- HTTP-based master-worker communication
- Chunked transfer for large image data
- Configurable timeouts for different operation types

### Error Handling
- Graceful worker failure handling with automatic retry
- Process cleanup on master termination
- Validation patching for ComfyUI's execution system

## Integration Notes

This is a ComfyUI custom node extension. Development should follow ComfyUI's node development patterns and be tested within a ComfyUI environment. The extension requires multiple NVIDIA GPUs or cloud GPU access to be fully functional.
Loading