Track the ExecuTorch pin and move it to new nightlies automatically by shoumikhin · Pull Request #4566 · pytorch/TensorRT

shoumikhin · 2026-08-23T14:12:05Z

Problem

ExecuTorch's package version, source commit, development lock, and installation instructions must agree. Updating them separately can test or compile against a different runtime than the installed wheel.

Change

Pin ExecuTorch to 1.5.0.dev20260911 and the source revision recorded in its published wheels. The updater checks its inputs before writing, preserves supported formatting, and reports provenance or subprocess failures. Downgrades require explicit manual permission.

Constrain development resolution to the exact pin while preserving the authoring dependency range. Refresh the development lock before either update workflow reports changes. Pin substitution does not hand-edit resolver-owned artifact URLs or hashes.

Decide the supported CUDA rows from the CUDA major rather than from a hardcoded pair. The delegate links CUDA 13 libraries, so the major is the real requirement and the minor is not. The matrix filter, the test runner, the install helper, the pin updater and the shared build gate all match on it, so the delegate follows whichever CUDA row the wheel matrix builds instead of keeping its own list. Ordinary release CUDA 12 rows, JetPack, and Windows behavior remain separate.

Naming a channel is not the same as that channel existing. PyTorch publishes a new CUDA minor before ExecuTorch does, so the install guidance says which CUDA major is required rather than implying every minor is ready.

Validation

All parent pin and updater tests passed on Linux aarch64: 248 tests, including removed-fix controls and a complete repository pin rewrite that rejects the stale lock.

Checked the resolved install channel for CUDA 13.0, 13.2, 13.4, 13.6 and 13.10, and confirmed 12.6, 14.0, an empty value, a missing minor and a non numeric minor all resolve to no channel. Evaluated the shared build gate for cu126, cu128, cu130, cu132, cu134, cu136 and cu140, and confirmed it accepts only CUDA 13 rows.

Confirmed each new check fails when the thing it protects is removed: reverting the runner to the old pair fails the runner test, and making any single declaration of the CUDA major disagree fails the agreement test.

Compared frozen installer selections for Python 3.10-3.13 across three dependency-group choices. All 24 Linux cases passed without unrelated package changes. The 12 Windows cases retained an identical pre-existing MarkupSafe artifact failure, so Windows development-lock qualification remains incomplete.

The branch is rebased onto current main. Final-head CI is still required; this description does not claim green CI or merge readiness.

shoumikhin · 2026-08-23T14:12:57Z

Follow-up that depends on this: #4567 (ships only the TensorRT delegate in the runtime wheel). That change needs the prebuilt ExecuTorch runtime this pin brings in, so it should land after this one.

test_derived_requirements_match_the_pin extracted the python3 -c one-liner from docgen.yml and ran it. Whatever that line said got executed on every pull request: rewriting it to write a file left the test green and the file written. Same class as the bash -c problem fixed in test_api.py last round, still live here. It now compares the command as text against the exact form that reads __executorch_version__ out of dev_dep_versions.yml. Four mutations caught, including a payload that writes a file and still prints the right version, with nothing executed. The CI reachability guard tested the raw string for "--collect-only", so it accepted "--co", pytest's own documented short form, which collects and asserts nothing. It also could not see an exit status being discarded. Now tokenised: --collect-only, --co, -h, --help, a "||" short-circuit and continue-on-error are all rejected, and all five are caught where four previously survived. The comment exemption for .md/.rst/.txt defeated exactly the threat its docstring names. Install commands live in prose files, so exempting them made a comment count as a pin there: the runtime README's install line gutted to a bare "executorch" passed as long as a decoy "# executorch==<pin>" sat beside it, and failed only with no comment present. The exemption is gone, and trailing comments no longer count either, since a decoy after a live requirement on the same line kept the per-file count satisfied. Five mutations caught, baseline green.

…it resolves The nightly-index guard matched only the named-distribution spelling, so the four sites that write "pip install .[executorch]" were unguarded: docgen.yml and the three export examples. The nightly index could be deleted from all four with the test green. Each of the four is now caught individually. Its second half was a bare substring test for the host, which proves a string sits nearby rather than that the instruction resolves. Rewriting every channel in the tree, 18 files, to a nonexistent cu999 left it green. The CUDA suffix is now checked against the set the project publishes for. Deliberately not compared against __cuda_version__: five sites legitimately say cu130 while the pin says 13.2, and I confirmed against the live index that cu130 and cu132 both carry 38 ExecuTorch wheels while cu999 carries none.

The printed install commands resolved no ExecuTorch. "torch-tensorrt[executorch]" with no version pin resolves the stable PyPI wheel, which carries no executorch extra, so the command exited 0 and installed nothing the feature needs. Add --pre to the six commands that name the extra and assert its presence in the guard that already reads them. Close four ways to neutralise the pin check while its guard stayed green: a ";" or "&" terminator after pytest, continue-on-error or a falsy if: on the owning job, and reducing the workflow trigger so it never runs on pull requests. The trigger check also handles PyYAML reading the unquoted "on" key as the boolean True. Close both ways to strip the pairing check while its guard stayed green: assert the workflow actually calls trt_tier_executorch, and validate suite lane names against the known set so a typo raises at import instead of silently dropping the suite from every matrix. Also: anchor the docgen pin check to a live line so a commented-out install no longer satisfies it; fix the lockfile range check crashing on a legal "==1.4.*" clause; correct the range comment to describe what the range admits; and note in the install advice that the feature is published for Linux only.

The delegate is built against one ExecuTorch: __executorch_version__ selects the wheel it links against and __executorch_commit__ selects the tree it compiles from. Those two values repeat across the build workflows, the bazel modules, the docker and toolchain copies, and the docs, so they can drift apart or fall behind upstream with nothing to notice. Add a script and a daily workflow that move both pins to the newest ExecuTorch wheel on the nightly index. The source commit is read from the chosen wheel's own version.py, so the two pins always name one ExecuTorch rather than two that happen to be close. The update lands as a pull request, so the pin consistency checks and the delegate build and test lane decide whether the new wheel is usable before it reaches main. A day with no new nightly rewrites nothing and opens nothing. On a release branch the schedule is a no-op and the pin moves only by a manual run pointed at the stable line, so a cut release does not drift. Back the mechanism with consistency checks that run under the linter. Every requirement and comment that names ExecuTorch is asserted to match the pinned version, including the variable-index install once the variable's assignment is resolved and extensionless install files like justfile. The source commit is checked against the wheel's own provenance wherever that wheel is installed, and commits left in comments are not mistaken for pins. The wheel-content and CI-invocation checks measure effect, running the workflow's own step against a passing and a failing stub and requiring the exit status to follow, rather than enumerating bypass spellings. Install the built wheel in the runtime README rather than an unpublished package. The guard that checks the pin runs in CI compares the step's command as text rather than executing it. Running the step's own shell body meant whatever that body said ran on every pull request: appending a line that writes a file left the test green and the file written. That is the same defect this file already avoids for the docgen one-liner, and the reasoning there applies here too. The delegate claims both names ExecuTorch has used for its pybind extension. It renamed _portable_lib to _C, and portable_lib.py imports whichever its own version carries, so aliasing only the old name is silently ineffective at the new pin: nothing imports it, the stock extension loads, and the backend is never registered. CI reported that as "TensorRTBackend is not registered" from the native runtime check.

The pin now names executorch 1.5.0.dev20260901 and the source commit that wheel records for itself. Every pin site moves together, which is what the pin checks assert: a version bumped in one place and not another is the failure mode they exist to catch. Carries the pin-site changes CUDA 12.6 support brought with it. The release lane builds the runtime wheel, so it installs ExecuTorch and is a pin site: it arrived naming a stale release off the default index, which resolves no ExecuTorch at all, and is now the pinned nightly from the nightly channel, registered in both the guard and the bumper so a future bump moves it too. Two guard bugs of my own that this surfaced. The trailing-comment strip cut at the first "//", so any line carrying an index URL was truncated before its requirement and the site read as missing rather than as wrong; it now skips a "//" that follows a colon. And the install-command helper still passed --upgrade, which on a named requirement replaces a user's released torch_tensorrt with a nightly when all they asked for was the extra. Also drops an assertion that pinned the runtime wheel's TensorRT distribution to the literal tensorrt-cu13. That was right while only CUDA 13 shipped and wrong once 12.6 returned, since a cu126 row would then declare the CUDA 13 distribution; the value is resolved from the build's own CUDA instead.

Read the commit from the wheel's own version.py rather than assuming it, and applied with the repository's own writer so every pin site moves together. 1.5.0.dev20260902 5afeaa8130f68f2afa800e0743d4a73aec79bf15 Test plan: 12 sites rewritten, 11 files naming the new version, zero references left to either the old version or the old commit. Pin coherence suite passes, 17 passed 1 skipped.

The rewriter's trailing boundary excluded '+', so a requirement written as executorch==<pin>+cu130 did not match and the bump left it on the old version. The guard's own requirement pattern does accept that spelling, so such a site would be counted as a pin and skipped by the rewriter, which is the shape that fails the generated pull request as a mismatch rather than as an operator the rewriter cannot see. No live site is written that way today, so this is latent rather than a present break. Test plan: exercised the rewrite across the label-free pin, the labelled pin, the range form with and without a label, the YAML key, and a bystander bazel_dep version. The label survives the bump rather than being dropped, and the bystander is untouched.

A new release workflow for aarch64 arrived with an ExecuTorch requirement of its own, and the guard caught two problems with it at once. It was not on the list of places the pin lives, so a nightly bump would have moved the ten other sites and left this one behind. Registering it fixes that, and the same registration is what makes the guard check it from now on. It also asked for a different version from every other site, and asked for it without naming the nightly channel: python -m pip install pyyaml "executorch==1.4.1" That resolves against the default index, which carries no ExecuTorch nightly at all, so the release build either picks up an unrelated release or fails outright. Its x86_64 counterpart already had the right shape, so this copies that exactly rather than inventing a third spelling. Test plan: the pin suite goes from two failures to 17 passed, and the two failures were the real ones, a site absent from the census and a requirement that excludes the pinned version. Ran the bumper afterwards and confirmed the new file moves with a simulated bump and comes back cleanly, leaving no stale version anywhere in the tree.

Read the commit from the wheel's own version.py rather than assuming it, and applied with the repository's own writer so every pin site moves together. 1.5.0.dev20260904 9379a885af0544c2ae87bb19b341c1c2b58d80c9 The version is taken from a CUDA channel and stored without its local label. A published wheel is labelled by the CUDA build it came from, for example 1.5.0.dev20260904+cu130, and keeping that label would bind every row to one CUDA version. Dropping it lets the same pin resolve on each CUDA row, which is what the install lines do when they read the channel from the environment. Test plan: 13 sites rewritten, 12 files naming the new version, and no ExecuTorch line left on the old version or the old commit. Pin coherence suite passes, 17 passed 1 skipped.

Read the commit from the wheel's own version.py rather than assuming it, and confirmed both CUDA channels publish the same one, so the pin is not specific to either. 1.5.0.dev20260905 04c314256bdac3744cca35db354fc980186de2cc The version is stored without its local label. A published wheel is labelled by the CUDA build it came from, for example 1.5.0.dev20260905+cu130, and keeping that label would tie every row to one CUDA version. Dropping it lets the same pin resolve on each row, which is what the install lines expect when they read the channel from the environment. Test plan: 13 pin sites rewritten and nothing left naming the previous version or commit. The change is only the version and the commit, 20 lines replaced by 20 with no other difference. Pin coherence suite passes, 17 passed 1 skipped, and all workflow files parse.

PyTorch nightly generation no longer offers CUDA 12.6. Keep nightly builds and pin updates on the validated cu130 and cu132 channels while retaining ordinary test/release and JetPack filtering. Test Plan: The new policy tests exposed eight failures before the fix and passed afterward. Focused pin and updater tests: 64 passed, one skipped without installed ExecuTorch, seven clone-heavy tests not run. No GPU build was run.

The companion runtime now uses the standard Linux build and release artifacts. Track its pin at the shared install site so nightly updates cannot leave builds behind, and preserve the independent runtime version and existing loader. Test Plan: 78 focused pin, updater, and parent loader cases passed; installed ExecuTorch provenance skipped and seven full-clone updater cases excluded. Workflow YAML parses and Black passes. The shared upload expression diagnostic also reproduces on main. Native GPU builds were not run.

Use the newly published nightly while keeping wheel dependencies and source builds on the same revision. Preserve the existing CUDA channel policy and shared build path. Test Plan: 69 focused pin and updater checks passed, including the installed September 11 wheel source check. Four Linux Python 3.12 CUDA wheels matched their published checksums and the pinned source commit.

Preflight every pin site before writing, parse supported requirements and YAML values consistently, and report wheel provenance and subprocess failures. Require manual downgrade permission and separate proposal branches by base and track. Make CI checks enforce both test invocations, their dependencies, actual suite selection, and installation command boundaries. Use small Git fixtures plus one history-free integration instead of repeated full clones. Test Plan: 155 pin and updater tests passed on the staged tree, including snapshot integration and installed September 11 provenance. Regression fixtures exercise rejected inputs and disabled checks. Python formatting and correctness lint passed. No native build or hosted workflow was run.

Missing-extra guidance must match the running CUDA build without promising an upgrade or unchanged dependencies. Give concrete commands for supported channels and clear guidance elsewhere. Correct platform and prerelease explanations while preserving the context of each import error. Test Plan: 125 pin and guidance tests passed on the committed source tree. Nine error-path cases executed the real functions with dependency stubs; unsupported-channel cases produced guidance without invented URLs. Public release metadata confirmed existing extra and platform support. No native or GPU execution.

A failed Windows wheel install must not continue into the appended test script. Propagate that failure explicitly and protect both platform branches with executed success and failure cases. Test Plan: 131 pin and installer tests passed on the staged source tree. The complete Bash installer ran with external commands stubbed. Both branches stopped before the appended caller on installation failure; removing the exit guards made the regression assertions fail.

A pin-only edit can leave uv using cached optional dependency metadata. Include the version source in metadata cache keys while retaining the standard metadata inputs. Keep lockfile regeneration separate. Test Plan: 133 pin tests passed. A uv 0.12.5 fixture reproduced stale metadata after a pin-only edit; an explicit refresh corrected it. With cache keys declared, a subsequent ordinary lock recorded the new pin without refresh. Removing the pin cache key makes the guard fail.

Checking alias constants does not prove activation registers either name. Execute activation, check both module-table entries and repeat it to protect idempotence. Keep this coverage with the portable-runtime implementation. Test Plan: All 11 parent runtime tests passed with dependency stubs. Deleting either alias assignment made the behavioral assertion fail. One warning reports that NumPy is absent from the test environment. No native extension was loaded.

An invalid CUDA channel produced an unusable nightly install URL. Accept only cu130 and cu132, while keeping cu130 as the default for empty or unset CU_VERSION. Test Plan: 186 pin and updater tests and 11 activation tests passed. Seven unsupported-channel cases failed before the fix. Disabling validation makes the rejection test fail.

Describe the pinned wheel provenance and supported local CUDA channel without duplicating a dated version in prose. Keep the independent pin census and development-lock behavior unchanged while making their intent clear. Test Plan: Ran the complete ExecuTorch pin and updater suites: 186 passed.

CPU and ROCm index suffixes could pass the CUDA install guard. Check literal channel names in both direct URLs and preceding assignments. Test Plan: The complete pin and updater suite passed 195 tests with one installed-provenance skip. Twelve recipe cases and two removed-fix controls verify rejection of unsupported channels.

Unrelated references could hide a missing source pin or reject a valid one. Require a name boundary so only the intended references count. Test Plan: The complete pin and updater suites passed 206 tests. Regression cases cover valid and missing pins beside unrelated names, with controls that remove the boundary.

Focused pin and updater checks were omitted by unit selection. Mark them explicitly while leaving installed-wheel provenance and the repository-wide update check outside the unit group. Test Plan: Normal unit selection passed 204 cases. The complementary selection passed both integration cases. Removing the updater markers leaves its 45 cases deselected by unit selection.

The development lock could select an older ExecuTorch than the version used by tests and source builds. Constrain development resolution to the pin and refresh the lock before either update workflow reports its changes. Keep the authoring dependency range unchanged. Pin substitution leaves resolver-owned artifact records untouched and requires a separate refresh. Test Plan: All 241 pin and updater tests passed, including drift and removed-fix controls and the full pin-substitution integration test. Compared frozen installer selections for Python 3.10-3.13 and three dependency-group choices: 24 Linux cases passed without unrelated package changes. All 12 Windows cases retained the same pre-existing MarkupSafe artifact failure; Windows lock qualification remains incomplete.

The ExecuTorch integration only accepted CUDA 13.0 and 13.2, named in five separate hardcoded lists. When the repository moved its build matrix to CUDA 13.4, those lists stopped matching, and the ExecuTorch jobs failed in two different ways at once. The test job failed because the runner refused the row it was given: ValueError: Unsupported CU_VERSION 'cu134' The delegate job failed differently. Its matrix filter dropped the 13.4 row, fell back to a 13.0 row, and then asked for a wheel that no job had built: Artifact not found for name: pytorch_tensorrt__3.10_cu130_x86_64 Both come from the same cause. The supported set was written out by hand, so every CUDA version bump needed the same edit in five places, and missing one left the parts disagreeing. This decides support from the CUDA major instead. The delegate links CUDA 13 libraries, so the major is the real requirement and the minor is not. The matrix filter, the test runner, the install helper, the pin updater and the shared build gate now all match on it, so the delegate follows whichever CUDA row the wheel matrix builds rather than keeping its own list. The major is still spelled out in more than one file, because the runner and the pin updater both execute as plain scripts before the wheel is installed and cannot import it. A test reads every one of those sites, plus the inline prefix in the shared workflow, and fails if any disagrees with the CUDA the repository declares. Note that naming a channel is not the same as that channel existing. PyTorch publishes a new CUDA minor before ExecuTorch does, so the install helper can name a channel that has no ExecuTorch wheels yet. The install guidance says this rather than implying every minor is ready. Test plan: Ran the pin and updater suites on Linux aarch64: 274 passed. Checked the resolved channel for CUDA 13.0, 13.2, 13.4, 13.6 and 13.10, and confirmed 12.6, 14.0, an empty value, a missing minor and a non numeric minor all resolve to no channel. Evaluated the shared build gate for cu126, cu128, cu130, cu132, cu134, cu136 and cu140, and confirmed it accepts only CUDA 13 rows. Confirmed each check fails when the thing it protects is removed. Reverting the runner to the old pair makes the runner test fail. Making any one declaration of the major disagree makes the agreement test fail.

The delegate build accepts several CUDA rows, and each row installs ExecuTorch from its own channel. The pin updater only looked at one channel, so it could adopt a version that channel had and the others did not. That is not hypothetical. ExecuTorch fills a newly added CUDA channel later than the established ones, so the newest version on the oldest channel is routinely absent from the newest channel. Adopting it leaves the newer row asking its own channel for a version that is not there, and the delegate build for that row fails on a missing wheel. The updater now reads the accepted channels from the matrix filter and keeps only versions published in all of them. If nothing is common it says so and stops, rather than writing a pin that cannot work everywhere. Reading the channels from the filter rather than listing them here means a CUDA row added to the build matrix is covered without another edit. Test plan: Ran the pin and updater suites on Linux aarch64: 250 passed. Added a case where a newer version exists in one channel only, and confirmed the pin stays where it is instead of moving to it. Added a case where no version is common to every channel, and confirmed it reports that and stops. Confirmed the existing single channel test now also checks that every accepted channel is queried, and that all of these fail when the cross channel check is removed.

The daily bump opens a pull request, and an unlabelled pull request takes the fast lane. The end-to-end ExecuTorch suite only runs on the nightly lane, so the bump reported green having never loaded the delegate. A reviewer saw the pin guards pass and learned nothing about whether the new pin still works. Label the bump pull request so it selects the nightly lane. The standard backend is named too, because that suite has no RTX variant and the nightly lane would otherwise start both engines. Test plan: Ran the pin and updater suites on Linux aarch64: 251 passed. Drove the suite selector with the files a bump changes. Before, the executorch suite was absent on both the fast and full lanes. With the labels it is present, alongside the rest of the nightly set. Confirmed the new check fails when the label is removed.

The installation guidance named CUDA 13.0 and 13.2 as the supported pair. The delegate now follows whichever CUDA row the wheel matrix builds, so that text excluded rows that in fact work, and it will go stale again on the next CUDA bump. Say what the code decides: the integration links CUDA 13 libraries, so any CUDA 13 build is supported, and the channel is substituted to match the install. Also note that ExecuTorch fills a channel later than PyTorch does, so a very new CUDA minor may not have ExecuTorch wheels yet. That is the same caveat the install helper already carries, and without it the guidance implies every minor is ready. The example scripts keep naming one concrete channel, because they show a command to run rather than the set of supported versions. Test plan: Ran the pin and updater suites on Linux aarch64: 251 passed. Confirmed no remaining text claims the supported set is CUDA 13.0 or 13.2, and that the generated documentation tree does not carry the old sentence.

meta-cla Bot added the cla signed label Aug 23, 2026

github-actions Bot added component: tests Issues re: Tests component: build system Issues re: Build system component: api [Python] Issues re: Python API labels Aug 23, 2026

shoumikhin mentioned this pull request Aug 23, 2026

Ship only the TensorRT delegate in the ExecuTorch runtime wheel #4567

Open

shoumikhin force-pushed the executorch-pin-dev-line branch 2 times, most recently from 2f5c384 to 7666e2a Compare August 23, 2026 18:07

shoumikhin marked this pull request as ready for review August 23, 2026 19:10

github-actions Bot added the documentation Improvements or additions to documentation label Aug 23, 2026

shoumikhin force-pushed the executorch-pin-dev-line branch from 598939b to 1eb0a46 Compare August 23, 2026 19:30

shoumikhin marked this pull request as draft August 23, 2026 19:31

shoumikhin marked this pull request as ready for review August 23, 2026 19:57

shoumikhin force-pushed the executorch-pin-dev-line branch 2 times, most recently from f05e654 to a3a0360 Compare August 24, 2026 00:05

lanluo-nvidia added the ci: nightly Run the nightly lane (all tiers incl. llm / kernels / distributed) on every push label Aug 24, 2026

lanluo-nvidia added this to the v2.15.0 milestone Aug 24, 2026

github-actions Bot requested a review from narendasan August 24, 2026 17:03

shoumikhin force-pushed the executorch-pin-dev-line branch 6 times, most recently from 19b80c9 to d8b7e43 Compare August 25, 2026 22:43

shoumikhin changed the title ~~Track a recent ExecuTorch pin instead of 1.4.1~~ Track the ExecuTorch pin and move it to new nightlies automatically Aug 25, 2026

shoumikhin force-pushed the executorch-pin-dev-line branch 5 times, most recently from cc11a6d to 0e6e40b Compare August 27, 2026 14:34

shoumikhin added 24 commits September 11, 2026 21:27

shoumikhin force-pushed the executorch-pin-dev-line branch from f8876ef to 6ad3960 Compare September 12, 2026 05:24

pytorchbot added 3 commits September 12, 2026 17:26

shoumikhin force-pushed the executorch-pin-dev-line branch from d362a0c to 10d8c12 Compare September 13, 2026 00:33

Navigation Menu

Uh oh!

Track the ExecuTorch pin and move it to new nightlies automatically - #4566

Track the ExecuTorch pin and move it to new nightlies automatically#4566
shoumikhin wants to merge 38 commits into
pytorch:mainfrom
shoumikhin:executorch-pin-dev-line

shoumikhin commented Aug 23, 2026 •

edited

Loading

Uh oh!

shoumikhin commented Aug 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shoumikhin commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Change

Validation

Uh oh!

shoumikhin commented Aug 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shoumikhin commented Aug 23, 2026 •

edited

Loading