From 96a4c3eb5dbe6f90cbeb3b68736ea4dde4c5a293 Mon Sep 17 00:00:00 2001 From: Petr Date: Wed, 29 Jul 2026 18:30:35 +0200 Subject: [PATCH 1/7] fix(update): defer the Windows self-update out of the running environment (#528) `uv tool install` recreates a tool environment by removing it and then building a fresh venv at the same path (`uv-tool/src/lib.rs`: create_environment -> "Remove any existing environment" -> create_venv). It is not atomic and has no rollback. On POSIX that is harmless -- unlinking a file another process holds open leaves that process's inode intact. On Windows uv's `kbagent.exe` trampoline loads the venv interpreter in-process, so those files are locked: the removal deletes what it can, hits a locked file, and aborts, leaving a gutted venv. That is the reported failure -- `rich` present but `rich/_windows.py` gone, `typer` present but `typer/rich_utils.py` gone. Upstream: astral-sh/uv#11930. The v0.76.2 fix reordered discovery ahead of mutation, which was a real ordering bug, but the installer still ran from inside the environment it replaces -- so the corruption survived it and reproduced on 0.76.2 -> 0.76.3. Windows now hands the reinstall to a detached PowerShell helper that waits for every kbagent process to exit and only then installs; the outcome is reported once by the next launch, with a recovery command on failure. When no helper can be spawned the user gets the exact command -- never a fallback to the unsafe inline install. POSIX keeps the inline install plus re-exec unchanged. Also removes a second, independent corruption vector: both paths ran the installer through `subprocess.run(timeout=...)`, which kills the child on expiry -- on Windows a hard TerminateProcess of uv mid-write, producing the same half-deleted environment from our own code. The deadline now bounds only how long kbagent waits. `KBAGENT_DEFER_UPDATE=1|0` overrides the platform default. --- ...528-deferred-windows-self-update-design.md | 184 +++++++ .../skills/kbagent/references/gotchas.md | 33 ++ src/keboola_agent_cli/auto_update.py | 130 ++++- src/keboola_agent_cli/commands/version.py | 13 + src/keboola_agent_cli/constants.py | 34 ++ .../services/version_service.py | 98 +++- src/keboola_agent_cli/update_runner.py | 508 ++++++++++++++++++ tests/test_auto_update.py | 240 +++++++-- tests/test_update_runner.py | 357 ++++++++++++ tests/test_version_service.py | 103 +++- 10 files changed, 1619 insertions(+), 81 deletions(-) create mode 100644 docs/superpowers/specs/2026-07-29-issue-528-deferred-windows-self-update-design.md create mode 100644 src/keboola_agent_cli/update_runner.py create mode 100644 tests/test_update_runner.py diff --git a/docs/superpowers/specs/2026-07-29-issue-528-deferred-windows-self-update-design.md b/docs/superpowers/specs/2026-07-29-issue-528-deferred-windows-self-update-design.md new file mode 100644 index 00000000..61716cd3 --- /dev/null +++ b/docs/superpowers/specs/2026-07-29-issue-528-deferred-windows-self-update-design.md @@ -0,0 +1,184 @@ +# Design: defer the Windows self-update out of the running environment + +**Issue:** [#528](https://github.com/keboola/cli/issues/528) +**Supersedes:** `2026-07-23-issue-528-safe-self-update-design.md` (shipped in v0.76.2) +**Status:** Implemented +**Target:** `kbagent update` and the startup auto-update hook + +## Why the v0.76.2 fix was not enough + +v0.76.2 moved every network call, probe, and command construction ahead of any +mutation, updated the independent MCP environment first, and made the kbagent +reinstall a terminal `uv tool install --force --reinstall` from an exact +release artifact. That was a correct fix for a real ordering bug -- the missing +`certifi/cacert.pem` in incident 2 -- and it did not fix the reported +corruption. The reporter hit it again on v0.76.2 -> v0.76.3. + +The reason is that ordering was never the mechanism. + +`uv tool install` recreates a tool environment **in place**. From +`crates/uv-tool/src/lib.rs`, `create_environment` removes any existing +environment and then calls `create_venv` at the same path. There is no +temporary build directory, no atomic swap, and no rollback. + +On POSIX this is safe by accident of the filesystem semantics: unlinking a file +another process holds open leaves that process's inode intact, so a running +kbagent survives having its own venv deleted and rebuilt underneath it. That is +why nobody on macOS or Linux ever saw this. + +On Windows it cannot work. uv's `kbagent.exe` trampoline loads the tool venv's +interpreter in-process, so those files are locked for as long as kbagent runs. +The removal deletes every file it can, reaches a locked one, and aborts. What +is left is not a mixture of old and new distributions -- it is a **partially +deleted** venv, which is exactly what the reported symptoms describe: `rich` +still present but `rich/_windows.py` gone, `typer` still present but +`typer/rich_utils.py` gone. Upstream tracks the same class of failure as +astral-sh/uv#11930 (`uv tool upgrade` of an in-use tool leaves the environment +inconsistent and the receipt lying about it). + +Running the installer from inside the environment it replaces is therefore +unsafe on Windows *by construction*, with any combination of uv flags. + +### The second, independent corruption vector + +Both update paths ran the installer through `subprocess.run(..., timeout=...)`, +which **kills the child** when the deadline expires -- on Windows a +`TerminateProcess`. Killing uv part-way through recreating a venv produces the +same half-deleted environment a file lock does, from our own code, on every +platform. The default deadline was 300s; a cold resolution of the `[server]` +extra on a Windows machine with real-time AV scanning is not reliably under +that. + +## Goals + +- Never run the installer from a process whose own environment is the target. +- Never terminate an installer mid-transaction. +- Never fall back to the unsafe path when the safe one is unavailable. +- Keep the POSIX behaviour that demonstrably works today. +- Report a deferred outcome exactly once, with recovery guidance on failure. +- Be verifiable in CI on macOS/Linux, where the failure cannot be reproduced. + +## Non-goals + +- Making `uv tool install` atomic. That is uv's to own. +- Updating while a long-lived kbagent (`serve`, `repl`) refuses to exit. The + helper declines and retries later; declining is always safe. +- A persistent background update daemon. +- Reworking the MCP environment, which is separate and never the one we run from. + +## Design + +### 1. `update_runner.run_install` -- bound the wait, not the installer + +A single execution helper for both entry points. Output goes to a log file +rather than a pipe (a pipe whose reader is gone would block a child we intend +to outlive), and on timeout it returns `STILL_RUNNING` **without killing**. The +caller reports that the install continues in the background and deliberately +offers no recovery command: a second installer aimed at an environment a live +uv is rewriting is the corruption, not the cure. + +### 2. `update_runner.request_deferred_update` -- install after we are gone + +On Windows (`should_defer()`, overridable with `KBAGENT_DEFER_UPDATE`) the +prepared install command is handed to a detached helper: + +1. Write a marker file first, so a helper that dies immediately still leaves + evidence a later run can report instead of a silent no-op. +2. Spawn `powershell.exe -NoProfile -NonInteractive -WindowStyle Hidden + -Command