Deferred Windows self-update is intermittent: reports (scheduled), then silently never applies

[padak]

Split out of #528. That issue was about the self-update corrupting the uv tool environment, which is fixed. This one is the symptom that remains: on Windows the update is scheduled, announced, and then silently never applied.

It fails safe — nothing is damaged, the install stays usable — but the user stays on an old version while being told an update is on its way.

Reproduced

Windows 11 (26200), kbagent installed with uv tool install, sleep disabled, nothing else running.

Each trial: fresh uv tool install --force of 0.80.0 → delete every pending_update.* → kbagent update → hold the session open 100 s with no kbagent process alive → check.

trial	before	after	pending_update.exit
1	v0.80.0	v0.80.0	not written
2	v0.80.0	v0.80.0	not written
3	v0.80.0	v0.80.0	not written

kbagent update reports:

kbagent v0.80.0 -> v0.80.1 (scheduled) | keboola-mcp-server v1.74.6 (already up to date)
kbagent v0.80.1 will be installed as soon as every kbagent process has exited

pending_update.json is written every time. No exit file and no install log ever appear, so the helper is not reaching either branch of its try/catch — not the install, and not the "gave up because kbagent is still running" path.

It is intermittent, which is the hard part

Earlier the same day, on the same machine, the same flow succeeded three times in a row (0.79.0 → 0.80.0, with marker, exit file and a full install log). I have not found the variable that flips it. Both of my earlier confident explanations were wrong, so this issue deliberately carries no proposed cause — only measurements.

Impact

• Windows users on a uv install can sit on an old version indefinitely. The single-flight marker then suppresses retries for DEFERRED_UPDATE_STALE_SECONDS (24 h) while every launch still prints "Updating in the background".
• Independently reported: "we never saw the deferred update actually arrive" — @papousek-radan in fix(windows): idempotency store, sliced download, and doctor permission check #566. That steady-state observation is what prompted the retest.
• v0.80.1 contains four Windows fixes that affected users will not receive automatically.

Workaround (verified)

uv tool install --force --reinstall "keboola-cli @ https://github.com/keboola/cli/releases/download/v0.80.1/keboola_cli-0.80.1-py3-none-any.whl"

One lead, explicitly not a conclusion

An isolated probe spawning powershell.exe with DETACHED_PROCESS (the flag request_deferred_update uses) reliably exits 0 within a second having executed nothing, while CREATE_NO_WINDOW runs the same script correctly. Reproduced under a real console, so it is not an SSH artefact.

That looks like the answer and I already shipped it once as a fix before retracting it — the real helper demonstrably does run sometimes, which the probe cannot explain. Treat it as an unexplained data point, not a diagnosis.

Suggested next step

Instrument rather than theorise: have the helper write its marker file as its very first statement, so a run that starts and a run that never starts stop being indistinguishable. Right now "helper died instantly" and "helper never spawned" leave identical evidence, which is why three rounds of investigation have not separated them.

[padak]

Fixed in v0.80.2 (#572), and this time with the experiment I should have run first.

Cause

DETACHED_PROCESS gives the child no console at all, and powershell.exe is a console application whose host cannot start without one — it exits 0, in under a second, having executed nothing. CREATE_NO_WINDOW still creates a console, it just never shows it.

The evidence, this time controlled

One Windows 11 box, back to back, fresh install of the same wheel per trial, only the flag changed:

flag	updated	pending_update.exit
DETACHED_PROCESS (shipped 0.78.0–0.80.1)	0/3	never written
CREATE_NO_WINDOW (v0.80.2)	3/3	written, with full install log

Polling the process list every 1.5 s for 46 s during a DETACHED_PROCESS run never sees the helper at all — and I validated that instrument against a live decoy carrying the same command-line marker before trusting the negative, having already been burned once by an unvalidated measurement.

Verified on the shipped artifact

• A machine on 0.80.1 running kbagent update stays on 0.80.1 — the stuck state, reproduced deliberately.
• The documented recovery one-liner takes it to 0.80.2 first try.
• The published 0.80.2 wheel carries CREATE_NO_WINDOW, checked in the installed package rather than assumed from the source tree.

Why it took three rounds

Worth recording, because the shape of it is the interesting part: "never spawned" and "spawned and died instantly" leave identical evidence on disk — a marker and nothing else. Every investigation that reasoned from the state directory was therefore guessing. The new Windows-only regression test asserts on the process for exactly that reason, and both new tests are confirmed to fail with the old flag restored.

That indistinguishability is also why the existing suite could not catch it: it pins the waiter script's text and mocks Popen, and the Windows CI test runs the script through subprocess.run — which supplies the very console the real flags withheld.

I also stated a cause here twice before and was wrong twice, each time by generalising one batch of runs instead of holding everything constant and flipping one variable. The table above is the correction.

If you are on Windows and stuck

Any version from 0.78.0 to 0.80.1 never auto-updates, and kbagent update will not rescue it — same broken path. One reinstall, once:

uv tool install --force --reinstall "keboola-cli @ https://github.com/keboola/cli/releases/download/v0.80.2/keboola_cli-0.80.2-py3-none-any.whl"

After that, auto-update works normally again.

@papousek-radan — this is yours. "We never saw the deferred update actually arrive" was the single observation that outweighed my test runs and reopened it. Thank you.