Skip to content

.NET: Preserve UTF-8 order in HeadTailBuffer#7128

Open
jstar0 wants to merge 1 commit into
microsoft:mainfrom
jstar0:fix/head-tail-buffer-utf8-boundary
Open

.NET: Preserve UTF-8 order in HeadTailBuffer#7128
jstar0 wants to merge 1 commit into
microsoft:mainfrom
jstar0:fix/head-tail-buffer-utf8-boundary

Conversation

@jstar0

@jstar0 jstar0 commented Jul 15, 2026

Copy link
Copy Markdown

Motivation & Context

HeadTailBuffer can silently drop and reorder multi-byte UTF-8 output when a complete rune no longer fits the remaining head bytes but later smaller runes still do. At the exact overall byte cap, the previous logic could evict a tail rune and still report that the output was not truncated. This affects the bounded stdout and stderr capture shared by the local and Docker shell executors.

Description & Review Guide

  • What are the major changes? Seal the head after the first rune spills to the tail, defer whole-rune tail eviction until total input exceeds the overall cap, and add exact-cap and overflow regressions.
  • What is the impact of these changes? Inputs that fit the configured byte cap now round-trip in order without silent loss, while overflow remains bounded and retains only complete UTF-8 runes.
  • What do you want reviewers to focus on? The one-way transition from head to tail and the eviction boundary when total input first exceeds the cap.

Related Issue

Fixes #7112

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Copilot AI review requested due to automatic review settings July 15, 2026 09:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a UTF-8 boundary bug in the .NET HeadTailBuffer used for bounded stdout/stderr capture, ensuring multi-byte runes are never reordered or silently dropped at the head/tail boundary and that truncation reporting matches actual byte loss.

Changes:

  • Seals the head buffer after the first rune that can’t fully fit, forcing all subsequent runes into the tail to preserve byte order.
  • Defers tail eviction until the overall byte cap is exceeded (not merely the tail budget), preventing silent loss when input is exactly at cap.
  • Adds unit tests covering exact-cap and minimal-overflow multi-byte UTF-8 regression scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
dotnet/src/Microsoft.Agents.AI.Tools.Shell/HeadTailBuffer.cs Updates head/tail append and eviction logic to preserve UTF-8 rune order and correct truncation behavior.
dotnet/tests/Microsoft.Agents.AI.Tools.Shell.UnitTests/HeadTailBufferTests.cs Adds regression tests for exact-cap and overflow cases involving 4-byte UTF-8 runes.

@rogerbarreto rogerbarreto requested a review from westey-m July 15, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.NET: [Bug]: Microsoft.Agents.AI.Tools.Shell.HeadTailBuffer can reorder and silently drop UTF-8 runes at the head/tail boundary

3 participants