fix(learn): fail-closed episodic mirror before candidate publish

[diazMelgarejo]

Summary

• Manual stage() previously wrote evidence_ids: [now] then fail-open-mirrored into AGENT_LEARNINGS.jsonl, so a mirror write error left dangling evidence.
• Publish the candidate via temp file + os.replace only after append_jsonl mirror succeeds (fail-closed).
• Update stdlib unittest accordingly (3/3 pass).

Found and hardened downstream in Perpetua-Tools; contributed back without PT-local path_hygiene.

Test plan

• cd .agent/tools && python3 -m unittest test_learn_episodic_mirror -v (3/3)
• Maintainer review of fail-closed vs prior fail-open posture

Made with Cursor

Note

Make learn candidate staging fail-closed on episodic mirror errors

• Reworks learn.stage so the episodic mirror row is written first and durably, and the candidate JSON is published only after that evidence exists. The mirror append is now append-once under an exclusive lock, so retries reuse one canonical timestamp instead of adding duplicate rows.
• Stages the candidate through a temporary file with flush, fsync, and directory sync, and replaces it into place at publish time. Before each new stage run, leftover temporary candidates are recovered: valid ones are published, stale or corrupt ones are discarded, and identical twins are collapsed to one candidate.
• Adds JSONL helpers in episodic_io.py for append-once, locked timestamp lookup, and fsync-before-unlock. Hooks in on_failure.py and post_execution.py now catch OSError from the episodic append and warn on stderr instead of failing.
• Extensive regression coverage added in test_learn_episodic_mirror.py and test_episodic_hooks.py.
• Behavioral Change: staging errors now cause the learn command to exit with a failure status, and evidence validation uses exact parsed JSONL timestamp fields instead of raw substring matching, so substring-only matches no longer count as evidence.

^{Macroscope summarized 4b56ff0.}

Confidence Score: 3/5

The PR is not yet safe to merge because concurrent recovery can discard candidate content and a malformed episodic entry can stop manual staging.

_{Reviews (4) · Last reviewed commit: "Merge pull request #3 from diazMelgarejo..."}

[greptile-apps]

Distinct candidates can be discarded. If two staging calls produce temps with the same candidate ID but different claims or reviewer sources, they share a mirror timestamp. Recovery treats that timestamp as proof that the candidates are identical, publishes the first temp by filename, and deletes the other. A caller can receive a successful stage result even though its candidate content was discarded.

Prompt To Fix With AI

This is a comment left during a code review.
Path: .agent/tools/learn.py
Line: 238-240

Comment:
**Distinct candidates can be discarded.** If two staging calls produce temps with the same candidate ID but different claims or reviewer sources, they share a mirror timestamp. Recovery treats that timestamp as proof that the candidates are identical, publishes the first temp by filename, and deletes the other. A caller can receive a successful stage result even though its candidate content was discarded.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[diazMelgarejo]

@greptileai fixed this in a01fd1a

[greptile-apps]

Verified in a01fd1a30e9f9cdae380645c6e628a68ac3e5b55: recovery now compares complete loaded temp payloads, not just their shared evidence timestamp. Distinct claims or reviewers raise an ambiguity error and leave every temp intact, while only identical resumable temps are collapsed and published once. The regression coverage exercises the shared-timestamp/different-reviewer case. This resolves the finding.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

[greptile-apps]

Malformed bytes block staging. If an older episodic entry contains a non-UTF-8 byte, decoding the whole log raises UnicodeDecodeError before the corrupt-line handler runs. Manual staging now scans that log, so an unrelated malformed entry stops staging with an uncaught traceback and no candidate is published.

Suggested change

	for line in raw.decode("utf-8").splitlines():
	for line in raw.decode("utf-8", errors="replace").splitlines():

Prompt To Fix With AI

This is a comment left during a code review.
Path: .agent/harness/hooks/_episodic_io.py
Line: 59

Comment:
**Malformed bytes block staging.** If an older episodic entry contains a non-UTF-8 byte, decoding the whole log raises `UnicodeDecodeError` before the corrupt-line handler runs. Manual staging now scans that log, so an unrelated malformed entry stops staging with an uncaught traceback and no candidate is published.

```suggestion
    for line in raw.decode("utf-8", errors="replace").splitlines():
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[diazMelgarejo]

@greptileai fixed this in 91c3dff

[greptile-apps]

Confirmed fixed in 91c3dff09094b35ef04f9020989ea77d28e7f420. _first_matching_action now parses each JSONL line independently via _json_object_from_line, skips non-UTF-8/corrupt lines without replacement decoding, and the regression test covers a non-UTF-8 line before a valid mirror row. This resolves the malformed-byte staging failure.