Support Qwen3 models: bump transformers to >=4.51 and fix optimum/colpali compatibility

[hassanraha]

Related Issue

closes #598
closes #611

Related: #620, #642, #643, #655

Summary

Qwen/Qwen3-Embedding-* (and the Qwen3ForSequenceClassification reranker conversions) fail with KeyError: 'qwen3' because poetry.lock pins transformers==4.47.0 and the docker images additionally install a 4.49.0.dev0 git commit on top. This PR bumps the dependency and fixes the code paths that break with newer transformers / optimum, which was the blocker for #620.

• transformers >=4.51.0 (locked 4.57.6) so the qwen3 architecture loads. Re-locked with poetry lock --no-update: optimum 2.1.0 + optimum-onnx 0.1.0, colpali-engine 0.3.13, tokenizers 0.22.2. torch stays at 2.8.0, sentence-transformers at 3.3.1.
• BetterTransformer: optimum 1.25-1.27 raise a RuntimeError (not an ImportError) on import optimum.bettertransformer with transformers>=4.49, and optimum>=2 removed the module. Both cases are now caught and infinity continues with transformers' default (sdpa) attention and logs a warning with the reason (same intent as Fix BetterTransformer compatibility with transformers >= 4.49 #643).
• ONNX / optimum engine: optimum-onnx requires the ONNX file as subfolder + bare file_name and removed the deprecated OptimizationConfig(optimize_with_onnxruntime_only=...) argument. optimize_model now passes subfolder/file_name split and uses enable_transformers_specific_optimizations=True (the equivalent default). Both changes work with optimum 1.x as well; graph optimization still runs.
• colpali-engine: 0.3.9+ removed ColIdefics2 and added ColQwen2_5/ColIdefics3. Importing all classes unconditionally broke every ColPali-style model, so classes are now resolved by architecture name. This also resolves the TODO: colqwen 2.5 is not supported with colpali-engine 0.3.8.
• aiohttp is imported for image/audio URL fetching but was only installed transitively via optimum<2 -> datasets. It is now an explicit optional dependency of the server/vision/audio/all extras.
• Dockerfile: removed the # TODO: remove this line step that pip-installed transformers from a git commit on top of the locked version; regenerated Dockerfile.*_auto via make template_docker.
• Fixed 4 mypy errors that surface because transformers>=4.5x ships type hints.

Checklist

• I have read the CONTRIBUTING guidelines.
• I have added tests to cover my changes.
• I have updated the documentation (docs folder) accordingly.

Additional Notes

Verification (Linux, Python 3.12, CPU):

• Qwen/Qwen3-Embedding-0.6B via AsyncEmbeddingEngine and via infinity_emb v2 --engine torch: embeddings identical to plain sentence-transformers (max abs diff 0.0); the query/document similarity matrix [[0.7646, 0.1414], [0.1355, 0.6000]] matches the model card. Added as test_sentence_transformer_qwen3_embedding.
• tomaarsen/Qwen3-Reranker-0.6B-seq-cls reranks correctly through the torch crossencoder path (README lists it).
• make lint (ruff + mypy), poetry check --lock, codespell: clean. tests/unit_test: 82 passed; tests/end_to_end: 45 passed.
• Two tests are flaky in my environment but fail identically on unmodified main with the old lock file, so they are pre-existing and unrelated: the deprecated ct2 int8 STS-score test (thread dependent, +-0.2 points) and test_optimum_embedding::test_batch_embedding (the quantized ONNX file produces wrong vectors for padded rows with onnxruntime 1.19.2 on this CPU; mostly gone with onnxruntime 1.22, left out of scope).
• Trade-off: BetterTransformer can no longer be applied on transformers>=4.49 (optimum dropped it), so bert-style models run with sdpa attention instead. WIP: add back better_transformer without version check #641 (vendored better_transformer) could restore it independently of this PR.
• Only tested on Linux / Python 3.12; CI covers Windows and Python 3.9-3.12.

[greptile-apps]

Greptile Summary

This PR adds Qwen3 embedding and reranker support by updating the transformers ecosystem and adapting BetterTransformer, ONNX, and ColPali compatibility paths.

• Raises the transformers requirement and refreshes the associated lock file.
• Removes Docker installations of an unreleased transformers commit.
• Falls back cleanly when BetterTransformer is unavailable.
• Splits nested ONNX paths into subfolder and file_name for optimum-onnx.
• Resolves ColPali model and processor classes lazily across package releases.
• Adds aiohttp explicitly to extras that fetch image or audio URLs.
• Adds a Qwen3 embedding regression test and documents validated Qwen3 models.

Confidence Score: 4/5

The PR should not merge until its colpali-engine lower bound matches the newly supported ColQwen2_5 and ColIdefics3 architectures.

Environments using an allowed colpali-engine 0.3.8 installation will route the newly recognized architectures to classes that version does not export, causing model initialization to fail.

Files Needing Attention: libs/infinity_emb/pyproject.toml, libs/infinity_emb/infinity_emb/transformer/vision/init.py

Important Files Changed

Filename	Overview
libs/infinity_emb/pyproject.toml	Raises transformers and adds aiohttp to relevant extras, but leaves the ColPali lower bound below the version required by newly recognized architectures.
libs/infinity_emb/poetry.lock	Relocks the transformers, optimum, optimum-onnx, colpali-engine, tokenizers, and aiohttp dependency chain.
libs/infinity_emb/infinity_emb/transformer/acceleration.py	Handles removed or incompatible BetterTransformer imports and falls back to transformers' default attention.
libs/infinity_emb/infinity_emb/transformer/utils_optimum.py	Adapts ONNX artifact loading and optimization configuration to optimum-onnx APIs.
libs/infinity_emb/infinity_emb/transformer/vision/torch_vision.py	Replaces unconditional ColPali imports with architecture-based lazy resolution across package versions.
libs/infinity_emb/infinity_emb/transformer/vision/init.py	Recognizes ColIdefics3, exposing a runtime mismatch when the allowed minimum colpali-engine version is installed.
libs/infinity_emb/tests/unit_test/transformer/embedder/test_torch.py	Adds a numerical regression test for Qwen3 embedding shape, normalization, and expected similarities.
libs/infinity_emb/Dockerfile.jinja2	Removes the out-of-band transformers Git installation so generated images use declared dependencies.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Engine configuration] --> B{Model architecture}
    B -->|Qwen3 embedding| C[Torch sentence-transformer path]
    B -->|Qwen3 sequence classification| D[Torch reranker/classifier path]
    B -->|ColPali architecture| E[Lazy colpali class resolution]
    A -->|Optimum engine| F[Locate ONNX artifact]
    F --> G[Split subfolder and filename]
    G --> H[Load and optionally optimize ORT model]
    A -->|BetterTransformer requested| I{Module available?}
    I -->|Yes| J[Apply BetterTransformer]
    I -->|No| K[Warn and use default attention]

Loading

_{Reviews (1): Last reviewed commit: "Add Qwen3-Embedding test and document Qw..." | Re-trigger Greptile}

[greptile-apps]

Minimum ColPali Version Unsupported

Adding ColIdefics3 to the recognized architectures routes it through the new class resolver. However, the declared colpali-engine = "^0.3.8" dependency still permits version 0.3.8, which does not provide ColIdefics3 or ColQwen2_5. With that supported minimum version installed, initializing either new architecture raises ValueError. Please raise the dependency lower bound to the first release that exports these classes.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6be51476bb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Keep local ONNX subfolders relative to the model root

When model_name_or_path is a relative local directory such as models/foo, get_onnx_files() returns a path like models/foo/onnx/model.onnx, so this assigns subfolder="models/foo/onnx". from_pretrained() resolves that subfolder beneath model_name_or_path and consequently looks under models/foo/models/foo/onnx/model.onnx, preventing every Optimum engine from loading locally vendored models through relative paths. For local models, derive the subfolder relative to model_name_or_path rather than directly from the returned path.

Useful? React with 👍 / 👎.