fix: MiniMax max_tokens forwarding + test_learn_episodic_mirror hygiene - #65
Closed
diazMelgarejo wants to merge 177 commits into
Closed
diazMelgarejo wants to merge 177 commits into
diazMelgarejo wants to merge 177 commits into
Conversation
Added a Twitter handle for updates and collaborations.
onboard_ui.py imported tty and termios unconditionally. Both are Unix-only, so the wizard crashed on import under Windows Python before any logic ran. Branch the raw key reader on sys.platform: use msvcrt.getwch on Windows (handling the 0x00 / 0xe0 arrow-key prefix), keep the existing tty/termios path on POSIX. Add .gitattributes forcing LF on .sh and .py so Git core.autocrlf=true on Windows clones does not rewrite shell script shebangs to CRLF (previously caused /usr/bin/env: 'bash\r': No such file or directory when invoking install.sh from Git Bash or WSL).
fix: Windows compatibility for install.sh and onboarding wizard
Currently the memory system can only surface top-k entries by salience score, but cannot search by topic or keyword. This adds a lightweight SQLite FTS5 search tool that indexes all .md and .jsonl files under .agent/memory/ and returns ranked results with context snippets. - memory_search.py: FTS5 index with auto-rebuild and grep fallback - .gitignore: exclude derived .index/ directory - memory-manager SKILL.md: document the search command Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Written by Minimax-M2.7 in Claude Code Harness.
Shifts the reasoning boundary: Python handles mechanical filing
(cluster, extract, stage, heuristic prefilter, decay, archive).
The host agent (Claude Code, Codex, Cursor, Windsurf) handles
validation via CLI tools using the LLM it already has. The brain
is no longer coupled to a specific provider SDK.
Key modules
- memory/auto_dream.py: staging-only dream cycle. No validation,
no graduation, no git commit. Safe on Stop hooks or cron.
- memory/validate.py: heuristic prefilter (length + exact
duplicate against accepted lessons). Deterministic, no LLM.
- memory/review_state.py: candidate lifecycle (staged/provisional/
accepted/rejected/superseded) + append-only decision log.
Stamps evidence + lessons hash at decision time so re-stage
only fires on real change, not churn.
- memory/render_lessons.py: lessons.jsonl as source of truth;
LESSONS.md rendered from it with sentinel-preserved user
content. Auto-migrates legacy bullets on first run; dedupes
by lesson id (handles provisional -> accepted transitions).
- memory/cluster.py: proper single-linkage Jaccard clustering
with bridge-merge; claim+conditions-stable pattern ids.
- memory/promote.py: re-stage gate keyed on NEW evidence and
specific-blocker presence, not reviewer identity or whole-file
hashes. Decay-only evidence shrinkage doesn't churn. Full
lifecycle respects all three subdirs (staged/rejected/graduated).
- harness/context_budget.py: query-aware retrieval filtered to
status=accepted only; loads AGENTS.md, DECISIONS.md,
REVIEW_QUEUE.md per the stated read order.
- harness/llm.py: used only by standalone conductor path; the
memory layer does not import it.
- harness/text.py: shared word_set + jaccard.
- harness/hooks/_provenance.py: source metadata on every
episodic entry (confidence, source{skill, run_id, commit_sha},
evidence_ids).
- tools/{list_candidates,graduate,reject,reopen}.py: host-agent
CLI. graduate.py requires --rationale and is atomic: lessons
write first, candidate move last. --supersedes exempts the
target lesson from the duplicate check.
Workflow notes
- Same pattern across cluster membership changes keeps the same
id and lifecycle history.
- Accepted lessons are terminal. Provisional acceptances re-enter
review when new evidence arrives.
- Rejected candidates re-enter review only when evidence grows
OR the specific blocking lesson disappears.
- Provisional supersessions do not strike the active lesson
until the superseder is itself accepted.
- Retrieval filters to accepted lessons only; empty filter
returns empty rather than leaking raw markdown.
Not tracked (stays per-user)
- memory/personal/PREFERENCES.md
- memory/working/* (WORKSPACE.md, REVIEW_QUEUE.md)
- memory/episodic/AGENT_LEARNINGS.jsonl
- memory/candidates/ (transient)
- memory/semantic/lessons.jsonl (grows from graduations)
Written by Minimax-M2.7 in Claude Code Harness. - [P1] write_candidates now does a claim-level terminal check against accepted lessons before the slug-based lifecycle lookup. Cluster-membership shifts can change `conditions` (intersection of shared tokens) and therefore the pattern id, so accepted patterns could re-stage under a new slug and bypass the "status=accepted = terminal" guard. Checking by claim text is stable regardless of id drift. Provisional and legacy lessons remain absent from the terminal set, so they still allow re-review. - [P2] graduate.py now detects a partial-completion retry and completes the candidate move without re-appending to lessons.jsonl. Previously, if semantic writes succeeded but mark_graduated crashed, the next retry saw its own prior lesson in LESSONS.md and heuristic_check rejected it as a self-duplicate, leaving the candidate stuck. The retry path now recognizes the lesson_id already exists and skips duplicate-check + append, just finishing the move.
feat: add FTS5 full-text search for memory retrieval
Written by Minimax-M2.7 in Claude Code Harness. Integrates sergi-rz's FTS5 memory search (#2) behind a beta feature flag. Default OFF; users opt in through the onboarding wizard or by editing .agent/memory/.features.json directly. Addresses the three codex review findings on that PR at the same time. Beta feature framework - onboard_features.py (new): read/write .agent/memory/.features.json with a simple {key: {enabled: bool, beta: bool}} schema. - onboard.py: adds a final "Optional features" step to the wizard. Default answer is No — beta features stay off unless the user explicitly opts in. --yes (non-interactive) path writes the features file too, all off. - memory_search.py gates search/rebuild behind feature_enabled(). --status still works regardless so users can see the toggle. Codex review fixes on PR #2 - [P1] needs_rebuild now compares the set of memory files currently on disk with the set already in the index; any previously-indexed file that no longer exists flags the index stale. Without this, deleted/renamed files kept appearing in results until some unrelated file bumped the index. - [P2] search_grep now passes explicit .md/.jsonl target paths to grep instead of scanning the whole .agent/memory/ tree. Source files (archive.py, auto_dream.py, etc.) no longer pollute fallback search results. - [P3] .gitignore now ignores .agent/memory/.index/ correctly. The previous rule was cancelled by the !.agent/memory/** negation placed below it; moved the .index/ exclusion AFTER the negation so the later rule wins. Verified with git status --ignored.
…ETA] Written by Minimax-M2.7 in Claude Code Harness. - Bumps VERSION to 0.5.0 in onboard_render.py and the banner. - README rewritten to cover the host-agent review protocol (CLI tools graduate / reject / reopen / list_candidates), the new structured lessons.jsonl source of truth, BETA FTS5 memory search, and Windows install path. - Formula bumped to v0.5.0 tarball URL; sha256 placeholder gets updated in the follow-up commit once GitHub cuts the archive. install list now includes onboard_features.py; brew test asserts .features.json is written alongside PREFERENCES.md. - install.ps1 (new): PowerShell installer parallel to install.sh, so Windows users don't need Git Bash / WSL. Same adapters, same wizard invocation, same flags (--yes / --reconfigure / --force via PowerShell switches).
Written by Minimax-M2.7 in Claude Code Harness. When SQLite FTS5 is unavailable, memory_search now uses ripgrep (`rg`) if it's on PATH, falling back to grep only when rg is missing. ripgrep is faster, UTF-8 clean by default, and respects gitignore. The .md/.jsonl target restriction from the previous fix is preserved for both tools — implementation files never reach the search path. - Adds _fallback_command() that picks rg vs grep based on PATH, returns (cmd, tool_name). - Adds fallback_tool() surfacing the selected tool in --status. - Renames search_grep to search_fallback; keeps search_grep as a backwards-compat alias. - --status now reports "Mode: FALLBACK (ripgrep|grep|unavailable)" and "Fallback available:" in FTS5 mode too. - README memory_search section mentions ripgrep preference.
Written by Minimax-M2.7 in Claude Code Harness. Replaces the placeholder with the real SHA256 of the v0.5.0 tarball: 6bec75321979828243fbc437843393d85f068affaed8c3370f0531c6086c19bd Computed from the GitHub-generated archive. `brew update && brew upgrade agentic-stack` will now fetch the correct artifact.
Written by Minimax-M2.7 in Claude Code Harness. Embeds the star-history.com SVG at the end of the README so repo growth is visible at a glance. Clickable through to the interactive chart on star-history.com.
Added information about the coding environment and PR review process.
Breaking: .openclient-system.md -> .openclaw-system.md. Existing OpenClient users need to re-run ./install.sh openclaw.
The portable brain is designed to be mounted by any harness, and many
harnesses (Hermes Agent with `--profile`, others in the list that
support multi-identity) run several isolated agents against a single
brain. Without a profile field on each episodic entry, the dream cycle
clusters across all of them as one stream: a lesson graduated from a
quant-focused agent's trial-and-error can end up rendered as
"agent learned X" alongside a leisure-focused agent's unrelated
discovery. Per-agent retrospectives are impossible.
This adds a `profile` field to the episodic `source` block:
"source": {
"skill": "<what skill fired>",
"profile": "<harness profile name>", # new
"run_id": "<run identity>",
"commit_sha": "<agent git hash>"
}
Resolution order:
1. `AGENT_PROFILE` env var (canonical; any harness can export this).
2. `HERMES_HOME` basename under `/profiles/<name>` (Hermes-specific
fallback so existing Hermes installs Just Work without touching
adapter code).
3. "default" when neither applies.
No breaking changes: consumers that ignore `source["profile"]` see the
same behaviour as before. Cluster.py and promote.py can now partition
by profile when that sharpens the review-queue signal; existing code
that reads `source` as an opaque blob keeps working.
Tested four cases:
- default (no env): "default"
- AGENT_PROFILE set: value used verbatim
- HERMES_HOME under /profiles/fin: "fin"
- HERMES_HOME at base: "default"
harness: tag episodic entries with active profile name approved by avid live
Three new host-agent tools ship: learn.py (one-shot lesson teaching), recall.py (proactive lesson retrieval with per-entry source labels), and show.py (colorful brain-state dashboard). All 8 adapter configs wire recall into their prompts so every harness can consult graduated lessons before deploy/migration/timestamp/debug/refactor work. Reliability pass: - Unify pattern_id across cluster.py and learn.py paths; canonicalize conditions (casefold, unicode whitespace, zero-width strip, dedupe). - heuristic_check: require ≥3 content words (blocks raw-length-gate bypass like "!!!abc"). - graduate.py retry: idempotent re-render, honor original metadata, refuse on legacy rows missing reviewer/rationale. - render_lessons + append_lesson now hold fcntl.LOCK_EX on lessons.jsonl; LESSONS.md rewrite is atomic via temp+rename. Seed UTC lesson ships pre-graduated so proactive-recall returns a hit on first install.
Updates LICENSE file to canonical Apache 2.0 text, bumps the README badge + license section, and updates the Homebrew Formula license identifier to Apache-2.0. Copyright holder unchanged (Avid, 2026).
…kie99#59) Add a MiniMax provider branch to .agent/harness/llm.py so the portable harness can call MiniMax directly via its OpenAI- and Anthropic-compatible endpoints instead of relying on undocumented external SDK behavior. - Configure global (api.minimax.io) and CN (api.minimaxi.com) regional endpoints with both OpenAI and Anthropic base URLs. - Register current MiniMax-M3 (1,000,000-token context) and MiniMax-M2.7 (204,800-token context) text models; MiniMax-M3 is the default. - Select region via AGENT_MINIMAX_REGION and wire via AGENT_MINIMAX_WIRE. - Document the new env vars in .env.example and the standalone-python adapter README. - Add tests/test_llm_provider.py covering region/model config, provider availability, and both OpenAI/Anthropic wires for each region. Co-authored-by: octo-patch <266937838+octo-patch@users.noreply.github.com>
codejunkie99#57) * fix(memory): stage() mirrors manual lessons into AGENT_LEARNINGS.jsonl The manual-stage path in .agent/tools/learn.py writes a candidate with evidence_ids: [now], promising an episodic record exists at that timestamp — but never writes one. The auto-derived candidate path mirrors to AGENT_LEARNINGS.jsonl correctly; the manual path does not. The result is a referential-integrity gap: a graduated lesson's evidence_ids points at a timestamp with no matching episodic record. Fix: add _append_episodic_mirror(), called right after the candidate file is written, using the same 'now' timestamp so evidence_ids[0] always resolves. Fail-open on OSError so a mirror-write failure never blocks staging. Minimal and additive — stdlib only, no new dependencies, no change to the auto-derived path. Found and fixed downstream in Perpetua-Tools (which vendors this project); contributing back per that project's dogfooding practice. * test(memory): cover the manual-stage episodic mirror Standalone stdlib unittest (no third-party test dependency) since this project currently ships no test harness — intended as the first test file. Kept as a SEPARATE commit from the fix so it can be cherry-picked out if the maintainer prefers to merge the fix alone. Adapted from the downstream Perpetua-Tools suite that proved this fix. Covers: stage() writes exactly one matching episodic mirror; the candidate's evidence_ids[0] resolves to exactly one episodic record (the regression this targets); and _append_episodic_mirror fails open on write error. Run: python3 -m unittest .agent/tools/test_learn_episodic_mirror.py
…e99#61) Two related Windows-compatibility fixes: - Reconfigure stdout/stderr to UTF-8 (errors=replace) at import, so a lesson claim containing non-ASCII (arrows, em-dashes, accented text) doesn't raise UnicodeEncodeError under a cp1252 console on Windows. - Write the candidate JSON file with explicit encoding="utf-8" rather than the platform default, so candidates round-trip identically across OSes. Both are additive and platform-safe (the reconfigure is guarded by hasattr, a no-op where unavailable). Found and applied downstream in Perpetua-Tools; contributing back. Stacked on top of the episodic-mirror fix (atomic-01).
…e99#62) * fix(io): force UTF-8 on stdout/stderr and candidate writes Two related Windows-compatibility fixes: - Reconfigure stdout/stderr to UTF-8 (errors=replace) at import, so a lesson claim containing non-ASCII (arrows, em-dashes, accented text) doesn't raise UnicodeEncodeError under a cp1252 console on Windows. - Write the candidate JSON file with explicit encoding="utf-8" rather than the platform default, so candidates round-trip identically across OSes. Both are additive and platform-safe (the reconfigure is guarded by hasattr, a no-op where unavailable). Found and applied downstream in Perpetua-Tools; contributing back. Stacked on top of the episodic-mirror fix (atomic-01). * fix(io): use a context manager in _lesson_already_appended The read-only probe opened lessons.jsonl with a bare open() and relied on GC to close the handle. Wrap it in a with-statement (and add explicit encoding="utf-8" for cross-platform consistency) so the file descriptor is released deterministically. Behavior is otherwise unchanged. Found and applied downstream in Perpetua-Tools; contributing back. Stacked on top of atomic-02 (UTF-8 fixes).
…odejunkie99#60) recall.py's _load_structured() already deduped lessons.jsonl by latest row per id (handling retraction, which appends a same-id row rather than editing in place) but had no supersession awareness: supersession creates a NEW id whose supersedes field points at the old one, and the old row's own status stays "accepted" forever since supersession never edits it. Proactive recall could return both the stale and replacement guidance for the same topic. render_lessons.py's _build_auto_section already computed exactly this -- an old-id -> new-id map, accepted-supersessions-only (a provisional --supersedes must not retire the old lesson before its replacement is itself accepted). Extracted that computation into a public superseded_by_map(lessons) function and import it from recall.py, rather than re-deriving the same rule a second time -- retrieval and rendering now share one source of truth for "which lessons are currently retired" instead of two copies that can drift apart. Tests: tests/test_recall_supersession.py (superseded lesson excluded from _load_structured, recall() ranks the replacement not the superseded lesson, provisional supersession does not retire the old lesson) + a direct unit test on superseded_by_map itself. Full suite: 179 passed, 1 pre-existing unrelated failure (confirmed via git stash against a clean checkout: test_unknown_executable_is_a_structured_start_failure, a macOS PermissionError-vs-FileNotFoundError platform quirk, not touched by this change).
…ejunkie99#63) _new_loop_assets() computed each new loop-* skill's destination as dst_skills / src.relative_to(src_agent) -- but src.relative_to(src_agent) already starts with "skills/" (src_agent is .agent/, not .agent/skills/), and dst_skills is already .agent/skills/. Result: every genuinely-new loop-* skill lands at .agent/skills/skills/loop-x/SKILL.md instead of .agent/skills/loop-x/SKILL.md -- both the CLI's own --dry-run report and the real copy were wrong, silently, since no existing test exercised a fresh (non-pre-seeded) loop-* skill destination. Fix: dst_agent / src.relative_to(src_agent), matching the sibling non-loop skill-copy block earlier in the same function (line ~84), which already gets this right. New test: test_upgrade_copies_missing_loop_skills_to_the_correct_path. Confirmed it fails on the old code (asserts the doubled path is absent) and passes on the fix.
Patch release covering codejunkie99#60, codejunkie99#61, codejunkie99#62, codejunkie99#63: superseded-lesson filtering in recall, the doubled skills/ path in upgrade's loop-skill copy, UTF-8 I/O in learn.py, and a leaked file handle. Collapses the stacked per-version README history (v0.9.0 through v0.18.0) into a single pointer at CHANGELOG.md, and relaxes the onboarding docs test from an exact version pin to the 0.19.x series so patch releases keep asserting the loop docs and sandbox caveat without editing the assertion each time.
Verified sha256 a128f83f... across two independent fetches of the published tag tarball. The url assertion is now derived from __version__ so a future bump that forgets the formula fails the suite instead of serving brew users the previous release.
Retargets the draft from v0.19 to v0.20 and reconciles it with what v0.19.0 actually shipped. The original was written against a v0.18 baseline; the loop supervisor has since landed several primitives the draft proposed to build from scratch. Adds section 0.1 mapping every shipped primitive (owned worktrees, budgets, deny-path gates, approval gates, stagnation breaker, atomic checkpoints, content-free event journal, deterministic verification, autonomy tiers) to the subsystem that must now build on it rather than duplicate it. Substantive changes beyond renumbering: - Bus messages drop the inline 4KB payload for a payload_ref path. v0.19 shipped a field whitelist for the loop event journal that excludes task, prompt, command, and output by construction; two append-only journals under .agent/ with opposite privacy guarantees means the weaker one becomes the leak. Both now share one whitelist helper. - Speculative execution becomes N loop runs plus a scoreboard instead of a second worktree/budget/gate implementation. Renamed .agent/spec/ to .agent/trials/ — 'spec' already means the loop contract schema, and .agent/spec/ reads as 'the spec for .agent'. - Background actions become L1 report-only loop contracts; act.py is a scheduler and proposal store, not a second supervisor. Carries over the v0.19 caveat that the supervisor is not an operating-system sandbox. - Eval quarantine reuses the recall exclusion path v0.19.1 added for superseded lessons, so retrieval keeps one filter chain. - Migration section is v0.19.x to v0.20; the Python floor is no longer restated, since the spec must not silently raise it. Moves the document from the repo root to docs/specs/ so the root stays README/CHANGELOG/LICENSE plus entry points.
call_model()'s MiniMax path silently dropped the caller's max_tokens entirely on the OpenAI-compatible wire -- the chat.completions.create() call passed model/temperature/messages but nothing for output length. Every MiniMax call through this wire has been unbounded regardless of what max_tokens the caller specified. The MiniMax OpenAI-compatible Chat Completions API deprecated bare max_tokens in favor of max_completion_tokens; add max_completion_tokens=max_tokens to close the gap. The Anthropic wire is untouched -- it already forwards max_tokens correctly and Anthropic's API keeps that parameter name. Extends the existing test_call_model_openai_wire_global coverage with an assertion on the new kwarg. Verified in both directions: fails with KeyError against the pre-fix code, passes with the fix.
_load_learn() stubs sys.modules["text"]/["cluster"] to isolate learn.py from its two sibling modules, but never removed the stubs after loading -- they stayed process-wide for the rest of the test run. Verified this is a real leak, not a theoretical one: after calling _load_learn() once, both "text" and "cluster" remain in sys.modules; any later test or import in the same process would silently get the throwaway lambda stand-ins instead of the real modules. Save whatever was previously in sys.modules for those two names (or note there was nothing) before stubbing, and restore that exact prior state in a finally block around exec_module() so the stubs never outlive the one learn.py load they exist for. Also adds encoding="utf-8" to the one read_text() call in this file that was using the platform default encoding, per this repo's own "force UTF-8 when reading/writing tracked text" convention (already followed everywhere else _episodic() and learn.py itself read files). The third test in this file, test_append_mirror_fails_open_on_write_error, is untouched: _append_episodic_mirror()'s fail-open behavior on a write error is this file's own explicit, documented design (its docstring says so directly, matching _lesson_already_appended's read-only fail-open posture) -- not a bug.
Contributor
Author
|
Related: #64 (the loop-event-export allowlist fix this PR's fixes were found alongside). |
codejunkie99
force-pushed
the
master
branch
from
September 16, 2026 19:37
dd7aa35 to
8fd5726
Compare
Owner
|
Thanks @diazMelgarejo! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent, unrelated fixes found while working on #64 in this same repo
1.
harness/llm.py— MiniMax OpenAI wire dropsmax_tokenscall_model()'s MiniMax path forwardsmax_tokenscorrectly on the Anthropic wire but silently drops it entirely on the OpenAI-compatible wire —chat.completions.create()passesmodel/temperature/messagesand nothing for output length. Every MiniMax call through that wire has been unbounded regardless of what the caller asked for.MiniMax's OpenAI-compatible Chat Completions API deprecated bare
max_tokensin favor ofmax_completion_tokens; this adds it. Extendedtest_call_model_openai_wire_globalwith an assertion on the new kwarg — verified it fails withKeyErroragainst the pre-fix code and passes with the fix.2.
tools/test_learn_episodic_mirror.py— module-stub leak + missing encoding_load_learn()stubssys.modules["text"]/["cluster"]to isolatelearn.pyfrom its two sibling modules for the test, but never removed the stubs afterward — they stayed process-wide for the rest of the test run. This isn't theoretical: ran it directly and confirmed both names remain insys.modulesafter one call, meaning any later test or import in the same process would silently get the throwaway lambda stand-ins instead of the real modules. Now saved and restored (or removed, if there was nothing there before) in afinallyblock aroundexec_module().Also added
encoding="utf-8"to the oneread_text()call in this file using the platform default, matching the convention this file already follows everywhere else (_episodic(),learn.pyitself).Deliberately not touched:
test_append_mirror_fails_open_on_write_error._append_episodic_mirror()'s fail-open behavior on a write error is this file's own explicit, documented design choice — its docstring says so directly, matching_lesson_already_appended's read-only fail-open posture elsewhere in the same file. That's a design decision, not a bug, and this PR leaves it exactly as-is.Tests
pytest tests/test_llm_provider.py— 8 passed.python3 -m unittest test_learn_episodic_mirror(from.agent/tools/) — 3 passed. Both fixes verified in both directions (fails pre-fix, passes post-fix) before opening this PR.Scope
2 commits, one per fix, each independently revertable. No schema changes, no new dependencies.
Note
Fix
max_tokensforwarding for MiniMax and fix module stub leaks in episodic mirror testsmax_completion_tokensto the OpenAI-compatible wire call in_call_minimax, so the caller'smax_tokensvalue is now respected by the MiniMax provider._load_learn:textandclustermodules injected intosys.modulesare now restored (or removed) in afinallyblock afterexec_modulecompletes.test_call_model_openai_wire_globalverifyingmax_completion_tokens=4096is passed through.Macroscope summarized 20441c2.