Skip to content

[bug] A run is admitted onto a folder the platform has already judged "not a git working tree", and starting it overwrites that verdict with ready — three git pull runs died on it #548

Description

@serge-ivo

git pull failed three times overnight in a folder that has no .git

Investigated from the owner's report "Two git pull runs failed overnight — both timed out (stuck
not resolved in time)"
(the agent's own summary, instance_messages, 2026-08-12 22:38:11Z), plus a
third failure the same night.

Instance a1d3522f-6b91-4cd9-bf7b-380e470192b7 (Repo Coder), repo dev/aipa, workdir ~/dev/aipa.

On the machine, right now:

$ ls -d ~/dev/aipa          → /Users/serge-ivo/dev/aipa
$ ls -d ~/dev/aipa/.git     → ls: No such file or directory
$ ls -la ~/dev/aipa         → one entry: platform/

The platform already knows this. GET /v1/instances/a1d3522f…/coding/repos:

{"id":"repo_145254cf-…","name":"dev/aipa","provider":"local","workdir":"~/dev/aipa",
 "cloneStatus":"needs_attention",
 "cloneError":"The configured checkout `/Users/serge-ivo/dev/aipa` has files but is not inside a git working tree — it is a plain folder, not a clone of a repository.",
 "updatedAt":"2026-08-12 22:41:20"}

That sentence is lib/coding-workdir.ts:111, written by #405's machinery, working exactly as
designed. updated_at is 22:41:20 — twenty seconds after the run's last instruction and
fifteen minutes before it gave up. The verdict was on the row while the run was still going.

The run started anyway, and nothing consulted the verdict

codingDriver.start (workers/api/src/lib/loop-drivers.ts:150-284) is the path behind
start_work, the Loop button and delegate_goal. Its admission checks, in order:

  • pickLoopRepo (:156) — is there a repo, and is it this agent's;
  • runtimeConnectivity + classifySubordinateConnectivity (:163-175) — is a machine reachable;
  • claimSessionDriver (:192) — is somebody else already driving.

There is no fourth check. repo.cloneStatus is never read in this function — grepped across
routes/coding*.ts, lib/coding-store.ts and workflows/coding-session.ts: the only readers of
cloneStatus are the console (ReposList.tsx:150, CodingTab.tsx:405,814), the chat prompt
(lib/repo-status-prompt.ts), diagnostics (routes/coding-diagnostics.ts:233,265) and repo
selection for the local-repo connector (lib/connectors/repo-local.ts:128). Nothing on the run
path.

Worse: starting the run overwrites the verdict with ready

workers/api/src/lib/coding-session-open.ts:129, in startSessionOnRunner, immediately after a
successful POST /coding/start:

await updateRepoClone(env, repo.id, { cloneStatus: "ready", cloneError: null });

Nothing looked at the directory. /coding/start succeeding means the runner could chdir there and
was willing to run a command
— a plain folder satisfies that. So the run's own first act is to
erase the sentence that would have stopped it.

This is the same invariant the file states 4 lines below, in the comment #440 put there:

// A REPO'S STATE IS ONLY EVER WRITTEN BY SOMETHING THAT LOOKED AT THE REPO (#440).

#440 closed the false-error direction (a dropped WebSocket stored as the repo's state; the
isRunnerUnreachable guard at :149 is that fix, and it is in the code). The false-ready
direction, four lines above it, was not addressed — and it is ready, per #405, that makes an
agent invent code.

The third safety net is silent too

workflows/coding-session.ts:587-591 runs a repo-state-start step before the first instruction:

return (await readRepoWorkingState(conn, { repo, sessionId })) ?? null;

readRepoWorkingState (lib/repo-state.ts:120-136) runs git status over the relay and returns
null on any error — which is right for its own purpose (an absent report reads as "unknown", not
as a fabricated clean tree) but means fatal: not a git repository arrives as null, stateNote
is skipped, and the Pilot is told nothing.

The composition

Three individually-defensible decisions:

  1. codingDriver.start checks connectivity, not repo health;
  2. startSessionOnRunner treats a successful spawn as proof the checkout is good;
  3. readRepoWorkingState degrades a git error to "unknown".

Together: a run is admitted onto a non-repository, the evidence against it is deleted, and the one
component that could have said so says nothing. What the owner got was three git pull attempts
(22:40:52, 22:40:56, 22:41:00), an unanswered stuck-handoff, and at 22:56:23 the message
"Loop stopped (failed) — stuck not resolved in time". Nowhere in that chain does the phrase
"not a git repository" appear, though the platform had it in D1 at 22:41:20.

The engine-side half of the same run — every Codex turn exiting 1, unclassified — is filed
separately as #545; either fix alone would have shortened this failure, and neither subsumes
the other.

What to do, cheapest first

1. Refuse the run. In codingDriver.start, after pickLoopRepo and before opening a session:

if (repo.cloneStatus === "needs_attention") {
    return { ok: false, status: 409, error: repo.cloneError || `${repo.name}'s folder is not usable — fix it on the Coding tab.` };
}

cloneError is already written to be relayed verbatim (coding-workdir.ts:47-51 — "an agent can
say it to the owner"). A 409 with that sentence turns 15 minutes and three BYOK decisions into one
honest reply the owner can act on. Only needs_attention, which #405 defines as a definite
runner verdict — unknown and unverified must NOT block, or an offline laptop stops every run.

2. Stop writing ready from a spawn. coding-session-open.ts:129 should either drop the write
or take a real verdict. Since startSessionOnRunner already holds a live conn, the honest form is
checkWorkdirVia(conn, repo.workdir) + cloneStatusForVerdict — the pair routes/coding-repos.ts:105-110
already uses, which returns null (no write) for unverified. For a managed clone (no
workdir) the existing ready write is fine and should stay: something did look, and it cloned it.

3. Let the Pilot see it. readRepoWorkingState returning null for "git said no" and for "the
runner did not answer" is one value for two facts. Distinguish them, and let
describeRepoState/stateNote carry "this checkout is not a git repository" into the prompt — the
channel is already built and already injected at workflows/coding-session.ts:596-603.

Alternatives considered and rejected

  • Re-check the workdir at run start instead of reading the row. A relay round trip (5s timeout)
    on every start_work, to re-derive a verdict the list route already takes. Reading the row is
    free; step 2 is what keeps it trustworthy. Worth revisiting only if [bug] A repo has been marked broken for five days by a dropped WebSocket — a transport failure is stored as the repo's state, and no verdict is ever re-taken #440's "no verdict has been
    written in five days" case turns out to be common.
  • Auto-git init / auto-clone into the folder. No. ~/dev/aipa holds platform/ — quite
    possibly the user's real intent was ~/dev/aipa/platform. Guessing writes into a directory the
    owner did not ask us to touch, and the same argument the workflow already makes about not
    discarding an unexpected branch (coding-session.ts:582-586) applies here.
  • Refuse at repo-add time (400 instead of 201). Explicitly rejected by
    routes/coding-repos.ts:290-296, and correctly: the row is the handle the owner needs to fix the
    path. Nothing here changes that — the refusal belongs at run start, not at add.

Acceptance criteria

  1. start_work / Loop / delegate_goal on a repo whose cloneStatus is needs_attention returns
    409 carrying cloneError, and no session is opened, no loop-run row created, no budget pool
    opened.
  2. cloneStatus unknown or unverified still starts a run (a unit test for each).
  3. A successful /coding/start on a local workdir no longer writes ready without a
    checkWorkdirVia verdict; a managed clone still may.
  4. Replaying this repo's state, the failure the owner sees names the folder and the word "git",
    within seconds, instead of stuck not resolved in time fifteen minutes later.
  5. repo-state-start distinguishes "the runner could not answer" from "git said this is not a
    repository", and the second reaches the Pilot's prompt.

Regression risk

Not from feedback

This one was investigated from the owner's observation of repeated git pull failures, not from a
record_feedback row — there is no feedback row to stamp.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions