Skip to content

emrg: fix Windows cmd-window storm + daemon spawn throttle (v0.2.15 regression hotfix) - #592

Merged
argszero merged 2 commits into
masterfrom
feature/win32-no-window-storm-hotfix
Aug 9, 2026
Merged

emrg: fix Windows cmd-window storm + daemon spawn throttle (v0.2.15 regression hotfix)#592
argszero merged 2 commits into
masterfrom
feature/win32-no-window-storm-hotfix

Conversation

@argszero

@argszero argszero commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

Emergency hotfix for the v0.2.15 Windows regression reported in rant 2026-08-09T13:16:36 (highest priority — host observed continuous cmd popups and had to reboot; daemon startup failure left GUI/scheduler unable to connect).

Root causes

  1. No window-suppression anywhere: grep -rn "CREATE_NO_WINDOW|creationflags" emrg/ had zero hits. Every subprocess.Popen / create_subprocess_* call (git, gh, bash tool, scheduler, daemon spawn) popped a console window on Windows.
  2. GUI infinite respawn loop: ensureConnected respawned emrgd.cmd every ~5s forever on failure. Each spawn raced the previous daemon's startup; the G43 stale-port unlink deleted a healthy daemon's port file → its scheduler logged cannot connect 93× while the GUI kept spawning (emrgd.log evidence: 92 prompt built / 93 cannot connect / 0 connected).
  3. No scheduler connect backoff: failed ticks returned immediately and re-ran at full interval; multiple handlers amplified the storm.

Changes

Python (all 34 subprocess call sites)

  • NEW emrg/_win.py: win32_no_window_kwargs() returns {"creationflags": subprocess.CREATE_NO_WINDOW} on Windows, {} elsewhere (no-op on macOS/Linux).
  • Splatted into: bash_tool.py, scheduler.py (12 sites), daemon.py (5), git_utils.py, client/daemon_manager.py, skills/installer.py, __main__.py (3), client/app.py (9).

Scheduler backoff (rant item ③)

  • Connect-failure exponential backoff: max(30s, interval × 2^n) capped at 10 min — retry storm eliminated while daemon is down.

GUI anti-storm master switch (rant item ⑤)

  • daemon_client.js: spawn throttle — max 3 spawn attempts per connect lifecycle, then throw with the real emrgd.log tail instead of respawning; counter resets on successful auth.
  • daemon_client.js: both spawn-timeout errors now append the emrgd.log tail (_readLogTail, mirrors daemon_manager R124) — host sees the real reason, not failed to start within timeout (rant item ②).
  • main.js: reconnect exponential backoff 1s→2s→4s→…cap 60s; throttled failure surfaces as daemon_stopped status with the real error + log tail (zh/en i18n).

Tests

Verification

  • uv run pytest tests/: 647 passed
  • uv run python -c "from emrg.client.app import run_client": OK
  • uv run python -m emrg --help: OK
  • npm test (emrg/gui): 98 passed
  • macOS/Linux unaffected (win32 branch is an empty-dict no-op); all 34 call sites verified to carry the splat.

Per rant: this is a hotfix — once merged, the v0.2.16 release flow should follow (host #16/#18 pre-authorized release without waiting for real-machine verification).

…egression hotfix)

Rant 2026-08-09T13:16:36 (emergency, highest priority): Windows v0.2.15
host observed continuous cmd popups (had to reboot) + daemon startup
failure leaving GUI/scheduler unable to connect.

Root causes:
1. Zero CREATE_NO_WINDOW anywhere in the codebase — every subprocess
   spawn (git/gh/bash tool/scheduler/daemon spawn) popped a console
   window on Windows.
2. GUI reconnect loop respawned emrgd.cmd every ~5s forever; each
   spawn raced the previous daemon's startup (G43 stale-port unlink
   deleted a healthy daemon's port file → its scheduler logged
   'cannot connect' 93 times while the GUI kept spawning).
3. Scheduler had no connect-failure backoff.

Fixes:
- NEW emrg/_win.py: win32_no_window_kwargs() = {creationflags:
  CREATE_NO_WINDOW} on Windows, {} elsewhere. Splatted into all 34
  subprocess call sites (bash_tool, scheduler x12, daemon x5,
  git_utils, daemon_manager, installer, __main__ x3, client/app x9).
- GUI daemon_client.js: spawn throttle (max 3 attempts per connect
  lifecycle, then throw with emrgd.log tail instead of respawning);
  reset counter on successful auth. Both spawn-timeout errors now
  surface the real emrgd.log tail (readLogTail).
- GUI main.js: reconnect exponential backoff 1s→2s→4s→…cap 60s;
  daemon_stopped status surfaces the real failure to the renderer
  (zh/en i18n) instead of infinite 5s respawns.
- scheduler.py: connect-failure exponential backoff
  max(30s, interval*2^n) capped at 10 min — no more per-tick retry
  storm while the daemon is down.

Tests: +6 Python (win32_no_window_kwargs POSIX/Windows/splat; backoff
zero/exp/cap/floor; 641→647), +2 GUI (spawn throttle + counter reset;
96→98). Docs synced (#511 guard). macOS/Linux unaffected (win32 branch
is a no-op empty dict).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle 2026-08-09T16:49 (Round 186)

Reviewed the full diff on the PR branch:

  • emrg/_win.py is a clean cross-platform design: win32_no_window_kwargs() returns {creationflags: CREATE_NO_WINDOW} on Windows and {} on POSIX (verified via unit tests incl. the splat-compat test).
  • All 34 Python subprocess call sites carry the splat (script-verified, zero gaps).
  • Scheduler exponential backoff max(30s, interval*2^n) cap 10 min — covered by 3 new tests (interval/no-failure, growth+capping, 30s floor).
  • GUI spawn throttle (3 attempts per lifecycle, reset on auth) + reconnect backoff (1s→…→60s) — covered by 2 new tests.
  • _readLogTail on both spawn-timeout paths surfaces the real emrgd.log failure reason (rant acceptance item ②).

Verified on the PR branch: uv run pytest tests/ → 647 passed; npm test → 98 passed; import/--help OK. Doc counts synced (#511 guard). POSIX no-op confirmed — zero macOS/Linux behavior change.

@pm25coder

Copy link
Copy Markdown
Contributor

Tested this PR on Windows (win32, Python 3.13.4) — the hotfix behaves as designed:

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle 2026-08-09T17:11 (Round 189)

Second independent review of the storm-fix foundation PR (re-verified fresh this cycle):

  • emrg/_win.py win32_no_window_kwargs() — clean cross-platform design (CREATE_NO_WINDOW on Windows, empty dict on POSIX; getattr fallback keeps POSIX importable/testable).
  • All 34 Python subprocess call sites carry the splat (script-verified, zero gaps across bash_tool/scheduler/daemon/git_utils/daemon_manager/installer/main/client).
  • Scheduler exponential backoff max(30s, interval*2^n) cap 10min — covered by 3 tests (interval/no-failure, growth+capping, 30s floor).
  • GUI spawn throttle (3 attempts/lifecycle, reset on auth) + reconnect backoff (1s→…→60s) + _readLogTail on both spawn-timeout paths — 2 new tests.
  • POSIX no-op confirmed (empty-dict splat changes nothing on macOS/Linux).

Fresh verification on the PR branch: uv run pytest tests/ → 647 passed; npm test → 98 passed; import + --help OK. Doc counts synced (#511).

… 2026-08-09T13:16:36)

After the spawn throttle exhausts, scheduleReconnect keeps retrying with
backoff capped at 60s — each retry hit the throttle and re-sent
daemon_stopped status, so the renderer appended a duplicate 'run emrg
server' system message every minute forever. Add daemonStoppedNotified:
the warning is sent once per connect lifecycle (reset on successful
connect), symmetric with the TUI _throttle_warned guard (PR #594).
@pm25coder

Copy link
Copy Markdown
Contributor

Re-verified the delta commit 4cb0fe8 (one-time daemonStoppedNotified guard) on this Windows host:

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle 2026-08-09T17:25 (Round 192)

Third independent review (3/3 consecutive LGTMs: C186, C189, C192). Fresh verification on the PR branch:

  • uv run pytest tests/ → 647 passed
  • npm test (emrg/gui) → 98 passed
  • import + --help OK

Final diff audit (2 commits):

  1. 37acdc0 — storm containment: emrg/_win.py win32_no_window_kwargs() splatted into all 34 subprocess call sites (script-verified zero gaps); scheduler connect backoff max(30s, interval*2^n) cap 10min; GUI spawn throttle (3/lifecycle, reset on auth) + reconnect backoff 1s→…→60s + real emrgd.log tail in spawn errors.
  2. 4cb0fe8 — daemon_stopped one-time per connect lifecycle (symmetric with TUI _throttle_warned, PR #594), prevents duplicate system messages every backoff retry.

POSIX no-op confirmed (empty-dict splat). Doc counts synced (#511). Merge-ready.

@argszero
argszero merged commit 9699135 into master Aug 9, 2026
1 check passed
argszero added a commit that referenced this pull request Aug 9, 2026
Resolve mechanical conflicts after #592 merged:
- GUI test file: keep both G43-guard tests (#593) and spawn-throttle tests (#592)
- Doc counts: 647+3=650 py, 98+3=101 gui (take highest)
- Production code (daemon.py, daemon_client.js) merged cleanly
argszero added a commit that referenced this pull request Aug 9, 2026
…ate (Windows v0.2.15 root cause) (#593)

Rant 2026-08-09T13:16:36 root-cause follow-up: PR #592 contains the
window-storm (CREATE_NO_WINDOW + spawn throttle + backoff), but the
underlying deletion race remains: G43 stale-port logic deletes a
healthy daemon's emrgd.port after ONE transient ws failure →
the daemon's own scheduler loses the file (93x 'cannot connect' in the
host's emrgd.log) while the PID lock makes every respawn exit instantly
(zombie state — daemon alive, forever unreachable).

Two complementary fixes:
1. daemon-side self-heal: new _assert_port_file() + _port_keepalive_loop
   re-asserts emrgd.port every 60s if it was deleted/overwritten — the
   daemon is the living core and re-asserts its own identity.
2. GUI-side guard: _daemonProcessAlive() checks emrgd.pid before G43
   deletes the port file — process alive → keep the file and let the
   reconnect backoff retry (transient failure); only truly-dead daemons
   get the delete+respawn path.

Tests: +3 Python (test_daemon.py _assert_port_file write/rewrite +
keepalive restore), +3 GUI (daemon_client.test.js G43 guard alive/dead/
no-pid). 641→644 py, 96→99 gui, docs synced (#511).
argszero added a commit that referenced this pull request Aug 9, 2026
* emrg: TUI daemon spawn throttle — complete the anti-storm fix (rant 2026-08-09T13:16:36)

PRs #592 (CREATE_NO_WINDOW + GUI spawn throttle + scheduler backoff) and
#593 (port-file self-heal + G43 PID guard) contain the Windows v0.2.15
storm fix, but the TUI client had the same storm pattern the GUI had:
app.py _reconnect() loops every 1s calling ensure_connected() → with a
down daemon each iteration calls start_daemon() → spawns a NEW daemon
process every second. On Windows each spawn was a cmd-window source
(pre-#592); even windowless it is process churn + PID-lock exit races
post-#592.

This completes rant acceptance item ① ("启动 GUI/TUI 后零 cmd 窗口弹出"):
- daemon_manager.start_daemon: spawn throttle — max 3 attempts per
  connect lifecycle, then raise with a clear 'run emrg server manually'
  message instead of spawning forever; counter resets on successful
  connect in ensure_connected (mirrors GUI daemon_client.js MAX_SPAWN_ATTEMPTS).
- app.py _reconnect: on throttle, surface a one-time system message +
  status hint so the host knows to start the daemon manually (recovery
  path intact: host starts daemon → is_running True → connect succeeds
  → counter resets).

Tests: +2 (start_daemon throttles after 3 attempts, no 4th spawn;
counter resets on success). 641→643 py, doc counts synced (#511).

* emrg: sync doc test counts to 652 (post-#593 merge + #594 tests)

---------

Co-authored-by: EMRG Evolution <emrg@argszero.dev>
argszero added a commit that referenced this pull request Aug 9, 2026
Resolve doc-count conflicts after #592+#593 merged: 650+2=652 py, 101 gui (take highest). Production code merged cleanly.
argszero added a commit that referenced this pull request Aug 9, 2026
…e self-heal + #594 TUI spawn throttle entries (#595)

Co-authored-by: EMRG Evolution <emrg@argszero.dev>
argszero added a commit that referenced this pull request Aug 9, 2026
…T13:16:36) (#596)

Carries the emergency Windows v0.2.15 regression hotfix (host observed
cmd-window storm + daemon startup failure, had to reboot):
- #592: CREATE_NO_WINDOW across all 34 subprocess sites (emrg/_win.py
  win32_no_window_kwargs) + GUI spawn throttle + scheduler connect backoff
- #593: daemon port-file self-heal (_port_keepalive_loop) + G43 PID guard
  (never delete a live daemon's port file)
- #594: TUI spawn throttle (_MAX_SPAWN_ATTEMPTS=3 + reset on connect)
- #595: evolution_prompt quick-ref entries

Per host authorization (rant #16/#18: release without waiting for
real-machine verification), this bump goes straight to Build Release on
merge.
@argszero
argszero deleted the feature/win32-no-window-storm-hotfix branch August 10, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants