fix(windows): make redirected output UTF-8, independent of the codepage

[padak]

kbagent semantic-layer --help and kbagent context exit 1 with UnicodeEncodeError on Windows the moment their output is piped or redirected, and any Rich table truncated by width emits a lone 0x85 where its ellipsis should be.

The split that matters is terminal vs not — not which codepage is active

I assumed at first this was a cp1250/cp852 problem and that the fix would have to trade a crash for mojibake on the console. Measuring it says otherwise. Since PEP 528, CPython writes to a real Windows console through the console API, so an interactive kbagent already reports utf-8 and renders anything:

stdout	encoding	"↔"
real console (isatty=True)	utf-8	encodes fine
pipe / file (isatty=False)	cp1252	UnicodeEncodeError

Redirect it and that path is gone — Python falls back to the locale encoding, which cannot represent an arrow, an em dash, or a box-drawing glyph. Running the same command under four different chcp values produces byte-identical output, because the console codepage has no say once stdout is a pipe.

So the people hit by this are scripts, CI, and AI agents capturing output — this CLI's stated primary audience — while the interactive users who would notice a change never see it.

The fix

Reconfigure stdout/stderr to UTF-8 only when the stream is not a terminal.

Terminals are deliberately left untouched. Forcing UTF-8 bytes at a console whose codepage is cp852 would replace a working display with mojibake, which is why the naive "just force UTF-8 everywhere" version of this fix is wrong. Redirected Windows output now matches POSIX byte for byte, including the box-drawing characters Rich had been downgrading to ASCII.

#546 fixed this same class for --json by writing bytes straight to sys.stdout.buffer. The human/Rich path cannot use that escape hatch, because Rich owns the writes — hence the stream-level fix.

The call sits at import rather than in the root callback: Click renders --help while parsing, before any callback runs, and --help is one of the crashing surfaces.

Verified on a real Windows 11 machine

Piped, before and after, same commands:

	semantic-layer --help	context
before	CRASH, 14309 B of traceback	CRASH, invalid UTF-8
after	clean, 3962 B of actual help	clean, valid UTF-8

Previously-mojibake surfaces (project list, permissions list, --help, --json changelog) all now emit valid UTF-8.

And the no-regression half — under a real console, with the fix in place:

console_handle=True
semantic-layer --help    rc=0 clean
context                  rc=0 clean
project list             rc=0 clean

Tests

Seven tests covering the contract on every platform: a redirected non-UTF-8 stream is switched; a terminal is left alone; an already-UTF-8 stream (POSIX, and a real Windows console) is untouched across three spellings; the error handler is preserved (surrogateescape is what round-trips undecodable filename bytes); and streams that lack reconfigure or refuse it do not take the CLI down.

Local suite: 5410 passed, 0 failed.

Notes

• Split out of fix(windows): idempotency store, sliced download, and doctor permission check #566 on request — that PR is three POSIX-semantics bugs, this is a different class and deserves its own review.
• Version is 0.80.2, assuming fix(windows): idempotency store, sliced download, and doctor permission check #566 (0.80.1) lands first. Renumber if the order changes.

---

[devin-ai-integration]

Devin Review found 2 potential issues.

[padak]

Rebased onto main after #566 landed, and renumbered to ship inside 0.80.1 rather than as its own 0.80.2 — one release for the whole Windows batch.

Three things moved as part of that, none of them mechanical:

• the changelog entry was folded into the existing 0.80.1 list (6 entries now) rather than left as a version of its own;
• the gotchas.md entry is re-tagged (since v0.80.1), including the "on kbagent older than …" workaround line, which would otherwise have pointed at a version that will never exist;
• it is placed immediately after the --json output crashes with UnicodeEncodeError on Windows when data contains non-ASCII characters #546 entry so its "see the next entry" cross-link still resolves — fix(windows): idempotency store, sliced download, and doctor permission check #566 had appended two entries in between.

Version files, all six local gates and 5411 tests are green.