Pasted image base64 triggers immediate compaction in a brand-new chat

[yxlyx]

Description

A brand-new chat can immediately print:

[compacting ~398389 tokens…]
[history compacted to a 1665-char summary]

There is no long conversation to compact. The apparent ~398k-token history is the serialized/base64 image attachment being estimated as ordinary text. Auto-compaction then replaces the first image-bearing message with a text summary before the normal agent turn, so the model no longer has the pasted image.

This likely explains or contributes to #259 (Kimi k2.6 cannot see pasted image attachments): the UI accepts the image, but pre-send compaction strips it before the model gets the actionable turn.

Reproduction

1. Start a fresh session (observed process command: graff --yolo, with no --resume).
2. Paste a reasonably large screenshot/image into the first prompt.
3. Submit the prompt.
4. Observe immediate compaction claiming hundreds of thousands of tokens even though this is the first message.
5. Observe that the subsequent agent only receives the compacted text handoff and cannot inspect the original image.

Trace evidence

Observed on harness 0.0.213, fresh run/session, using gpt-5.6-sol High mode:

Time from startup	Agent	Request bytes	Context tokens	Interpretation
27.866s	title	697	8,111	Small title request
38.658s	main	1,566,241	398,469	Image-bearing history / compaction request
50.134s	main	17,022	8,465	First request after history was replaced by summary
54.560s	main	15,379	9,305	Normal continuation

The terminal independently displayed ~398389 tokens, matching the ~1.56 MB request at the implementation's approximate 4-bytes-per-token calculation. The next main request shrinking from 1.56 MB / 398k tokens to 17 KB / 8.5k tokens confirms that compaction happened before useful work proceeded.

This was not a resumed old session: the live command line was graff --yolo, and the run had a fresh session ID.

Likely root cause

src/agent_request.zig currently estimates the full input by serializing the entire message array and dividing its byte count by four:

pub fn fullInputEstimateTokens(self: *Agent) u64 {
    var buf: [512]u8 = undefined;
    var d: Io.Writer.Discarding = .init(&buf);
    var s: std.json.Stringify = .{ .writer = &d.writer };
    s.write(Value{ .array = self.messages }) catch return 0;
    return d.fullCount() / 4;
}

Vision messages created by src/vision.zig:imageMessage embed the full base64 image in those messages (data, image_url, or a data:...;base64,... URL). Therefore fullInputEstimateTokens() counts every base64 byte as text tokens.

inputOverCompactThreshold() adds its prefill baseline and Agent.runTurn() invokes compactOrRecover(true) whenever that estimate crosses provider.compactAt(). A single image can therefore trip the pre-send overflow gate in an otherwise empty session.

The relevant estimation was introduced around 6977877 (fullInputEstimateTokens) and became a pre-send compaction trigger around 5e251a2 (#193). This is likely an interaction/regression in the local overflow protection rather than actual inherited chat history.

Expected behavior

• A fresh chat containing one image should not be treated as a ~398k-token text conversation.
• Token estimation should exclude raw base64/data-URI bytes and use provider-appropriate image token accounting (or a conservative bounded image estimate).
• Auto-compaction must not discard a newly attached image before the model has had a chance to process it.
• The model should receive and interpret the original image attachment.

Possible fix direction

Make fullInputEstimateTokens() content-type-aware:

• Count normal textual fields as text.
• Do not apply serialized_bytes / 4 to image base64/data URIs.
• Add a bounded/provider-specific image token estimate if needed.
• Add a regression test where a large base64 image plus a short first prompt remains below the compaction threshold and reaches the first normal request intact.

[justrach]

Confirmed valid and fixed in draft PR #263.

The root cause was raw Base64 transport bytes being estimated as ordinary text tokens. The fix uses bounded vision accounting for recognized inline images while retaining conservative byte accounting for text/tool output. A permanent 1.6 MB / 262K-window regression now covers the pre-send gate; the High-mode live test sent a 3,874,993-byte request at 12,150 actual provider context tokens without compacting.

[justrach]

Same root cause and same fix as #259 — a pasted image's Base64 was counted as text, so a brand-new chat looked near the context limit and compacted immediately. src/context_tokens.zig on main estimates image blocks properly; the cherry-pick of 317ee7e onto current main is empty.