fix(output): force UTF-8 for machine output on non-UTF-8 consoles (#546)

[padak]

What

kbagent --json <anything> crashed with UnicodeEncodeError on Windows whenever the payload contained a non-ASCII character (the reporter hit an arrow inside a flow name via kbagent --json flow list).

Machine output is now written as UTF-8 regardless of the console codepage:

• New write_machine_output() in output.py, used by OutputFormatter.output/error/success, kbagent http, and the agent --stream NDJSON events.
• force_utf8_stdout() reconfigures sys.stdout to UTF-8 (covers every real TextIOWrapper).
• Belt-and-braces: on UnicodeEncodeError the payload is written as UTF-8 bytes to sys.stdout.buffer, which bypasses the text layer's codec. A stream with neither reconfigure nor buffer re-raises — for a machine consumer, a silently mangled payload is worse than a crash.

Why

sys.stdout inherits the console codepage — cp1250 on Czech/Polish/Hungarian Windows 11. pydantic's model_dump_json() emits raw UTF-8 rather than \uXXXX escapes, so the write hits the cp1250 codec and aborts the command. --json exists specifically for machine consumption (piping to a file or another program), so it must not depend on which terminal happens to be attached.

agent.py's stream events serialize with ensure_ascii=False and shared the same exposure; kbagent http used the json.dumps default (ensure_ascii=True) so it never crashed, but it is routed through the same helper for consistency.

Unlike the kbagent serve startup banner (#522, fixed in #526), transliterating to ASCII is not an option here: the banner is decoration, this is data.

How it was tested

• New tests/test_json_output_encoding.py (24 tests): a cp1250 stdout double that raises exactly like a real Windows console, plus a fidelity guard proving the double reproduces the crash (so the passing tests are not vacuous). Covers the reconfigure path, the byte-level fallback, the no-fallback re-raise, and OutputFormatter.output/error/success round-tripping a → through json.loads.
• Live check under PYTHONIOENCODING=cp1250: a raw sys.stdout.write("extract → load") raises UnicodeEncodeError, while OutputFormatter(json_mode=True) now emits valid UTF-8 JSON that parses back with json.load, both to a terminal and through a pipe.
• make check: lint + format + changelog-check + full suite (4707 passed, 8 skipped).

Modern UTF-8 terminals are byte-identical to before — no \uXXXX escaping is introduced.

Fixes #546

[padak]

Closing — superseded, and worth recording why rather than just deleting it.

The bug this fixes is fixed. #546 landed via #549 on 2 Aug and shipped in 0.78.0. write_machine_output() has been on main since then, so the substance of this PR is already in the product; this branch just never got closed behind it.

One piece of it was deliberately not taken, though, and I want that on the record so it does not get revived by accident. This PR also adds force_utf8_stdout(), which reconfigures sys.stdout to UTF-8 unconditionally.

#567 (just merged, in 0.80.1) solves the same remaining problem — human/Rich output crashing on Windows — but only when the stream is not a terminal. That distinction is the whole fix, and measuring it on a real Windows 11 box is what settled it:

stdout	encoding	"↔"
real console (isatty=True)	utf-8	encodes fine
pipe / file (isatty=False)	cp1252	UnicodeEncodeError

Since PEP 528, CPython writes to a real Windows console through the console API, so an interactive session already reports UTF-8 and was never affected. Forcing UTF-8 unconditionally therefore fixes nothing on the console side and actively breaks it: a console whose codepage is cp852 would start receiving UTF-8 bytes and render mojibake where it previously rendered correctly. Trading a crash nobody sees for garbled output everybody sees is a bad trade.

So force_utf8_when_redirected() on main carries an isatty() guard, with a test asserting a terminal is never reconfigured.

Anyone arriving here later: see #549 for the --json half and #567 for the human/Rich half. Nothing from this branch is lost.