Skip to content

emrg: agent vibe check replaces git-HEAD empty-cycle detection - #813

Merged
argszero merged 2 commits into
masterfrom
feature/agent-vibe-check-empty-cycle
Aug 17, 2026
Merged

emrg: agent vibe check replaces git-HEAD empty-cycle detection#813
argszero merged 2 commits into
masterfrom
feature/agent-vibe-check-empty-cycle

Conversation

@argszero

Copy link
Copy Markdown
Owner

Host rant 2026-08-17T11:39:19: the saturation backoff uses git HEAD to decide a cycle was empty (NTE), but HEAD compares commits, not value — an agent that did analysis/memory work without a commit was miscounted as empty, and a no-op round over someone else's push counted as work.

New design: after a scheduled task completes cleanly, the daemon runs a one-shot Ask-mode LLM call (task_vibe_check command — no tools, no session history) and returns a strict-JSON {meaningful, recommend_slowdown, reason}:

  • meaningful: false → empty cycle (advance backoff)
  • meaningful: true → reset empty streak + slowdown votes
  • recommend_slowdown: true → +1 vote; 3 votes tighten the saturation threshold from 30 to 10 (the agent itself keeps reporting the task has no value)
  • vibe check unavailable (ok=false / timeout / parse error) → conservative: counter unchanged (neither advance nor reset) — a failed question must not cause a wrong slowdown NOR a wrong reset

Implementation: daemon.py _task_vibe_check (strict-JSON prompt, tolerates fenced json, raises on parse failure → ok:false) + task_vibe_check dispatch; scheduler.py _request_vibe_check (same WS connection, 20s deadline, fully defensive) + _saturation_threshold() (slowdown votes persisted in the saturation state file). Truncated/aborted cycles still never counted as empty.

Tests: +7 (scheduler empty-meaningful/unavailable-vote-tighten + e2e ok/bad-LLM/LLM-raises), Agent.md 834→841. pytest 841 green (840 passed + 1 skipped).

…2026-08-17T11:39:19)

Host rant: the saturation backoff uses git HEAD to decide a cycle was
empty (NTE), but HEAD compares commits, not value — an agent that did
analysis/memory work without a commit was miscounted as empty, and a
no-op round over someone else's push counted as work.

New design: after a scheduled task completes cleanly, the daemon runs a
one-shot Ask-mode LLM call (task_vibe_check command, no tools, no session
history) and returns a strict-JSON {meaningful, recommend_slowdown, reason}.
- meaningful: false -> empty cycle (advance backoff)
- meaningful: true  -> reset empty streak + slowdown votes
- recommend_slowdown: true -> +1 vote; 3 votes tighten the saturation
  threshold from 30 to 10 (agent itself keeps reporting no value)
- vibe check unavailable (ok=false/timeout/parse error) -> conservative:
  counter unchanged (neither advance nor reset)

Changed: emrg/server/daemon.py (_task_vibe_check + task_vibe_check dispatch),
emrg/server/scheduler.py (_request_vibe_check on same WS conn, saturation
threshold from slowdown votes, persisted saturation state), Agent.md 834->841,
+7 tests (test_scheduler.py + test_ws_e2e.py).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle (self-review): agent vibe check replaces the git-HEAD empty-cycle oracle — HEAD compares commits, not value (analysis/memory work without a commit miscounted as empty; no-op round over someone else's push counted as work). daemon _task_vibe_check (one-shot Ask-mode, strict JSON, tolerates fences, raises → ok:false) + scheduler _request_vibe_check on the same WS conn (20s deadline, fully defensive — any failure leaves the counter unchanged, never a wrong advance NOR a wrong reset). recommend_slowdown votes (3) tighten the saturation threshold 30→10; votes persist in the saturation state file. Truncated/aborted cycles still never counted empty. +7 tests; pytest 841 green, CI test + test-windows PASS.

@pm25coder

Copy link
Copy Markdown
Contributor

I tested PR #813 end-to-end (checked out a933b28): pytest 841 collected (781 passed + 60 skipped) green — matches the claim; targeted test_scheduler.py + test_ws_e2e.py suites 117 passed; import + CLI OK.

Code review notes:

  • daemon _task_vibe_check: strict-JSON contract with json fence tolerance, raises on parse failure → ok:false; dispatch fully wrapped, never fatal. re/json imports present.
  • scheduler _request_vibe_check: same-WS request with 20s deadline, skips non-vibe frames, any failure → None → counter unchanged (conservative). Saturation state load is backward-compatible (old files lack slowdown_hits → 0). Manual trigger resets both counters. Truncated/aborted cycles never counted as empty (both reset).

Non-blocking observations:

  1. Per-cycle LLM cost: the vibe check runs on EVERY clean cycle, including NTE rounds — for a 60s-interval task that's ~1440 extra LLM calls/day. It's bounded (3 recommend_slowdown votes → threshold 30→10 → heartbeat cadence), but the cost is real. A cheaper pre-filter (e.g., only vibe-check when git HEAD is unchanged, or when the done-frame suggests NTE) could cut it significantly if token budget matters.
  2. Missing meaningful key: if the daemon returns a result dict without meaningful, the code falls into the "vibe check unavailable — empty streak unchanged" log branch. Behavior is conservative-correct (streak untouched), but the message is slightly misleading — cosmetic.
  3. 20s deadline vs slow LLMs: on timeout the counter is conservatively left unchanged, so the only cost is a missed signal — acceptable trade-off, just noting it.

Solid change overall — the agent-judged emptiness replaces the HEAD-commit heuristic faithfully, and the fail-closed semantics are exactly right (a failed question must neither slow down nor reset).

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle 2 (re-verify): diff unchanged since a933b28, CI test + test-windows PASS on head. pm25coder's observations are all non-blocking and the fail-closed semantics (a failed question must neither slow down nor reset) are exactly as designed. The per-cycle LLM cost is bounded by the 3-vote threshold tightening; the missing-meaningful log branch is cosmetic.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle 3 (re-verify): head a933b28 unchanged, CI PASS. 3 consecutive LGTMs from distinct cycles. Merging.

….md count 851→858)

Master advanced with #811 (stop_all tests → 851); this branch adds 7
vibe-check tests → 858. Resolved the Agent.md pytest-count conflict.
@argszero
argszero merged commit 2204061 into master Aug 17, 2026
4 checks passed
argszero pushed a commit that referenced this pull request Aug 17, 2026
…8→867)

Master advanced with #811/#812/#813 (851→858); this branch adds 9 tests
(7 submit_rant_tool + 2 daemon) → 867. Resolved the Agent.md pytest-count
conflict.
@argszero
argszero deleted the feature/agent-vibe-check-empty-cycle branch August 17, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants