Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 47 additions & 3 deletions docs/source/en/using-diffusers/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ diffusers-cli run \
`--pipeline-kwargs` takes a JSON object that's forwarded to `pipeline(**kwargs)`. String values at known
media-input keys are auto-loaded:

- Images (`image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image` via
- Images (`image`, `last_image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image` via
`load_image`.
- Videos (`video`, `control_video`) → `list[PIL.Image]` via `load_video`.
- Audio (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.
Expand Down Expand Up @@ -123,11 +123,17 @@ Configure how the CLI loads model weights and custom pipeline code.

### Optimizations

- `--cpu-offload {model, group}` — `model` calls `enable_model_cpu_offload`; `group` calls
- `--cpu-offload {model, group, auto}` — `model` calls `enable_model_cpu_offload`; `group` calls
`enable_group_offload(offload_type="leaf_level", use_stream=True)`. Onload target comes from `--device-map`
(which must be a plain device string for offload). See
[Model offloading](../optimization/memory#model-offloading) and
[Group offloading](../optimization/memory#group-offloading).
[Group offloading](../optimization/memory#group-offloading). Modular pipelines support only `auto`, which

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually support custom strategy too, see this PR #1435

offloads through their [`ComponentsManager`](../modular_diffusers/components_manager); the standard modes
raise for them, and `auto` raises for standard pipelines.
- `--offload-margin <size>` — device memory kept free for activations under `--cpu-offload auto`, passed to
`enable_auto_cpu_offload` as `memory_reserve_margin` (default `3GB`). Raise it when a large canvas runs out
of memory mid-forward: the offloader keeps components resident while they fit, so on a high-VRAM card the
default margin can leave too little room for the activations of a long video.
- `--attention-backend {default, flash_hub, flash_varlen_hub, flash_4_hub, sage_hub}` — Hub-hosted attention
kernels, auto-downloaded on first use. Transformer-based pipelines only; ignored with a warning on legacy UNet
pipelines. See [Attention backends](../optimization/attention_backends).
Expand All @@ -141,6 +147,44 @@ Configure how the CLI loads model weights and custom pipeline code.
under `--remote` the CLI wraps `torchrun --nproc-per-node=gpu` for you. See
[Context parallelism](../training/distributed_inference#context-parallelism).

### Modular pipelines

`run` detects a [modular repo](../modular_diffusers/overview) automatically — either because it ships a
`modular_model_index.json`, or because its `model_index.json` names a `ModularPipeline` subclass — so no flag
is needed to opt in.

Some modular repos define several **workflows**: named tasks that share components but differ in which blocks
run and which inputs they take. [MiniMax-H3](../api/pipelines/minimax_h3), for example, offers `t2va` (text to
video and audio), `fl2va` (first and/or last keyframe) and `ref2va` (an ordered mix of image, video and audio
references). Pass `--workflow` to select one:

```bash
diffusers-cli run \
--model MiniMaxAI/MiniMax-H3 --workflow fl2va \
--pipeline-kwargs '{
"prompt": "the camera pushes in slowly as rain falls",
"image": "opening-frame.png",
"last_image": "closing-frame.png",
"num_frames": 124
}' \
--output-key videos --output-key audio \
--fps 24 --sampling-rate 32000 \
--cpu-offload auto --dtype bf16
```

Selecting a workflow keeps only that task's blocks, so the pipeline declares only the components it needs and
`load_components` fetches only their subfolders. Omit `--workflow` to keep every workflow available and let the
pipeline pick per call from the inputs it is given.

```bash
diffusers-cli --format json schema --model MiniMaxAI/MiniMax-H3 --trust-remote-code
```

`--workflow` applies to modular pipelines only; it is ignored with a warning on standard pipelines.

A modular pipeline returns a [`PipelineState`](../modular_diffusers/modular_pipeline) rather than a single output object, so `--output-key` names the intermediate to save.


### Outputs

`run` detects the pipeline output type:
Expand Down
7 changes: 6 additions & 1 deletion src/diffusers/commands/custom_blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,12 @@ def run(self):
spec = importlib.util.spec_from_file_location(module_name, str(self.block_module_name))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
getattr(module, child_class)().save_pretrained(os.getcwd())
block = getattr(module, child_class)()
block.save_pretrained(os.getcwd())
Comment on lines -106 to +107

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it for reading convenience?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We reuse block in the subsequent line.

# `ModularPipeline.from_pretrained` (and therefore `diffusers-cli run`) loads a repo
# through `modular_model_index.json`, which only the pipeline-level save writes — without
# it the packaged repo is importable as blocks but not runnable as a pipeline.
block.init_pipeline().save_pretrained(os.getcwd())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nice!


def _choose_block(self, candidates, chosen=None):
for cls, base in candidates:
Expand Down
Loading
Loading