[Modular] Avoid downloading weights when loading from an existing local path

[DN6]

What does this PR do?

As discussed in here.

This PR:

• Checks if a component is being loaded from a local path
• If the component is a model, it checks if weight files exist in the local path. If it is a scheduler/tokenizer etc, it checks for the relevant configs in the local path.
• If the component is determined to exist locally, we opt to load from there instead of the using the hub repo id.
• If the component does not exist locally e.g. A text encoder pointing to another repo, it gets downloaded from the hub

# assuming you have run something like this beforehand
# hf download <repo-id> --local-dir <my local path>

pipe = ModularPipeline.from_pretrained(<my local path>)
pipe.load_components() # no longer re-downloads components

Fixes # (issue)

Before submitting

• Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
  • Did you read the Coding with AI agents guide?
  • Did you run the self-review skill on the diff?
  • Did you share the final self-review notes in the PR description or a comment?
• Did you read the contributor guideline?
• Did you read our philosophy doc? (important for complex PRs)
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?
• Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Instead of hardcoding the name of the components, we could call pipe.components.keys() here?

[DN6]

In the anima test repo, the tokenizers point to an external repos
https://huggingface.co/hf-internal-testing/tiny-anima-modular-pipe/blob/main/modular_model_index.json

So their config files are not present in hf-internal-testing/tiny-anima-modular-pipe. Which is why we check against the specific set of components that exist within the repo.

[sayakpaul]

Should it not error out when cache_dir doesn't have any weight copies?

I ran the following

from diffusers import ModularPipeline
from huggingface_hub import snapshot_download
import tempfile
import pathlib
import torch 


with tempfile.TemporaryDirectory() as tmpdir:
    tmpdir = pathlib.Path(tmpdir)
    local_dir = tmpdir / "local_dir"
    cache_dir = tmpdir / "cache_dir"

    snapshot_download("hf-internal-testing/tiny-anima-modular-pipe", local_dir=local_dir)
    pipe = ModularPipeline.from_pretrained(local_dir)
    print(pipe.components.keys())

    pipe.load_components(names=["vae"], dtype=torch.float32, local_files_only=True, cache_dir=cache_dir)
    print(pipe.vae is not None)

And I got:

Logs

Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files:   8%|▊         | 1/12 [00:00<00:06,  1.63it/s]
Fetching 12 files:  17%|█▋        | 2/12 [00:00<00:04,  2.27it/s]
Fetching 12 files:  42%|████▏     | 5/12 [00:01<00:01,  6.66it/s]
Fetching 12 files:  58%|█████▊    | 7/12 [00:01<00:00,  8.11it/s]
Fetching 12 files:  75%|███████▌  | 9/12 [00:03<00:01,  2.14it/s]
Fetching 12 files:  83%|████████▎ | 10/12 [00:04<00:01,  1.96it/s]
Fetching 12 files:  92%|█████████▏| 11/12 [00:04<00:00,  2.00it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:05<00:00,  1.34it/s]
Fetching 12 files: 100%|██████████| 12/12 [00:05<00:00,  2.01it/s]
Guiders are currently an experimental feature under active development. The API is subject to breaking changes in future releases.
/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:205: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
  warnings.warn(
Failed to create component vae:
- Component spec: ComponentSpec(name='vae', type_hint=<class 'diffusers.models.autoencoders.autoencoder_kl_qwenimage.AutoencoderKLQwenImage'>, description=None, config=None, pretrained_model_name_or_path='hf-internal-testing/tiny-anima-modular-pipe', subfolder='vae', variant=None, revision=None, default_creation_method='from_pretrained', repo=None)
- load() called with kwargs: {'dtype': torch.float32, 'local_files_only': True, 'cache_dir': PosixPath('/var/folders/wg/2xcyr_6j3lgc_y5k0344x2b80000gn/T/tmp2xh3y9ma/cache_dir')}
If this component is not required for your workflow you can safely ignore this message.

Traceback:
Traceback (most recent call last):
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/configuration_utils.py", line 414, in load_config
    config_file = hf_hub_download(
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 88, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1035, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1182, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1910, in _raise_on_head_call_error
    raise LocalEntryNotFoundError(
huggingface_hub.errors.LocalEntryNotFoundError: Cannot find the requested files in the disk cache and outgoing traffic has been disabled. To enable hf.co look-ups and downloads online, set 'local_files_only' to False.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/modular_pipelines/modular_pipeline_utils.py", line 347, in load
    component = load_method(pretrained_model_name_or_path, **load_kwargs, **kwargs)
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 88, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/models/modeling_utils.py", line 1144, in from_pretrained
    config, unused_kwargs, commit_hash = cls.load_config(
  File "/Users/sayakpaul/miniconda3/envs/diffusers/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 88, in _inner_fn
    return fn(*args, **kwargs)
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/configuration_utils.py", line 441, in load_config
    raise EnvironmentError(
OSError: hf-internal-testing/tiny-anima-modular-pipe does not appear to have a file named config.json.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/modular_pipelines/modular_pipeline.py", line 2501, in load_components
    components_to_register[name] = spec.load(**component_load_kwargs)
  File "/Users/sayakpaul/Downloads/diffusers/src/diffusers/modular_pipelines/modular_pipeline_utils.py", line 349, in load
    raise ValueError(f"Unable to load {self.name} using load method: {e}")
ValueError: Unable to load vae using load method: hf-internal-testing/tiny-anima-modular-pipe does not appear to have a file named config.json.

dict_keys(['text_encoder', 'tokenizer', 't5_tokenizer', 'guider', 'vae', 'image_processor', 'text_conditioner', 'transformer', 'scheduler'])
pipe.vae is not None=False

Like the assert just right after should fail. What am I missing?

[DN6]

With the proposed change in this PR ModularPipeline will load weights from local_dir if they are present, and will not attempt to download to cache dir or the hub. So it should not error out.

[sayakpaul]

Why not remove the VAE subfolder directly like the transformer?

[DN6]

It simulates the condition where a component folder name exists in the local dir, but no weights exist. In that case, we download the weights from the repo id listed in the model index.