DeepSeek flash thinking off + lean 15s bash (ADR 0050 / 0051) - #675
Closed
justrach wants to merge 9 commits into
Closed
DeepSeek flash thinking off + lean 15s bash (ADR 0050 / 0051)#675justrach wants to merge 9 commits into
justrach wants to merge 9 commits into
Conversation
DeepSeek flash SWE one-shot 2/6 because three tasks described a patch and never called a tool. Empty-completion retry only covers whitespace. On lean -p, bounce once instead of stealing Pi's catalog.
Suite run-20260829-063114: 4/6 in 250s (was 2/6 in 199s). The three 4s no-edits now pass. Isolated serial retry recovered config-parse; do not stitch to 5/6. Same label-sort miss.
justrach
marked this pull request as ready for review
August 29, 2026 06:41
DeepSeek flash SWE 4/6 died on a ~110-byte / ~450ms api_error after the first tool batch under -j 6. Isolated serial retry passed. Treat internal/empty generic envelopes as bounded retries; leave invalid_request / auth / quota fail-fast. graff-evals gains opencode-codegraff on the same CODEGRAFF_API_KEY gateway as graff-dev / pi-codegraff.
The -j 6 follow-up is still ~110 B / ~450 ms and was not matching overloaded/internal. apiErrorMessage turns a missing message into \"unknown error\"; retry that and truncated unparseable bodies. Eval failures now keep a stderr tail so the next envelope is visible.
Graff 5/6 in 362s after short-flake retry (was 2/6, then 4/6). OpenCode on the same seat is 5/6 in 261s / 1.0G RSS. Same label-sort miss. Do not steal OpenCode's heap.
reasoning_effort=low still emits reasoning_content (cookie-store dumped 1.9MB / 75s). Official off switch is thinking.type=disabled. /effort high still thinks. GLM stays low-only. Do not steal Pi's catalog or OpenCode's heap.
The DeepSeek -j 6 follow-up is still 110 bytes / ~450 ms, but the envelope is now invalid_request_error / Body must be valid JSON. The invalid needle fail-fasted it. Our stringify just succeeded on call 1; auth/quota/a real invalid prompt stay fail-fast.
After thinking-off, DeepSeek SWE wall was hung foreground tests (json-stream 120s, cookie-store 71s). Interactive stays 120s (ADR 0026). The process is not killed. Do not steal Pi's catalog.
Gateway pong: low still thinks; thinking.disabled bills 0. SWE 5/6 in 353s was hung bash; 15s oneshot wait made cookie-store 30s / json-stream 22s. Later 0-token runs are burnt balance, not a result.
cursor Bot
pushed a commit
that referenced
this pull request
Aug 31, 2026
#677 leftover of #554: /teleport restores a snapshot tar onto another CLI backend; /snapshot gc keeps the newest n trees. ADR 0051 (their 0042 collided with TUI claims). #675: lean -p bounces prose-only first "done"; short Codegraff flakes retry twice; DeepSeek thinking off at default low; lean bash auto-backgrounds at 15s. ADRs remapped 0048-0051 -> 0052-0055. Suite floor 1861. Keep-alive-only bodies share the short-flake retry so the request-scratch split stays. Co-authored-by: yxlyx <85774423+yxlyx@users.noreply.github.com>
Owner
Author
|
Folded into #696 ( Did not take this head's Tier 1 green: suite 1861. Closing as superseded. |
Owner
Author
|
ADR remap correction: 0048–0051 → 0052–0055 (typo in the previous comment). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hillclimb of DeepSeek flash SWE on the Codegraff seat. Make graff the harder DeepSeek harness than OpenCode without stealing Pi’s four-tool catalog or OpenCode’s ~1G heap.
Change
-pbounces a first-turn prose-only “done” (ADR 0048).invalid_request_error/ “Body must be valid JSON”. Auth / quota / a real invalid prompt stay fail-fast.graff-evalsgainsopencode-codegraffon the sameCODEGRAFF_API_KEYseat.thinking.type=disabled(ADR 0050). Official Chat Completions off switch.reasoning_effort=lowstill thinks (cookie-store once dumped 1.9MB / 75s). Live pong: omit/low~20–30reasoning_tokens;thinking.disabledbills 0./effort highstill thinks. GLM stayslow-only.-pbash auto-backgrounds at 15s (ADR 0051). Interactive stays 120s (ADR 0026). After thinking-off, wall was hung tests, not CoT.Confirm —
deepseek-v4-flash,--suite swe072758)072310)reasoning: false074142)074659)075402)validateddied on balance reservationSame
label-sortmiss on every harness. Do not stitch suite scores. 0-token follow-ups after075402are burnt gateway (reservation ~4655 µUSD), not a model result.OpenCode
firstis the first JSONL event (~3.6s). Grafffirstis the stderrcallingline. Do not treat 0.05s as a TUI steal.Tests
Suite 1782 (baseline 1758). Tier 1 green.
Base is
release/v0.0.281(fold into #670). Do not merge to main / do not tag.