Skip to content

fix(core): strip BOM from AI-generated content (Issue #368) - #382

Open
moizlatif7 wants to merge 1 commit into
devfrom
anr/368/strip-bom-ai-content
Open

fix(core): strip BOM from AI-generated content (Issue #368)#382
moizlatif7 wants to merge 1 commit into
devfrom
anr/368/strip-bom-ai-content

Conversation

@moizlatif7

Copy link
Copy Markdown

Summary

Fixes #368 by stripping UTF-8 BOMs from AI-generated content while preserving BOMs in existing files.

Problem

On Windows systems, OpenCode was adding UTF-8 BOMs (0xEF 0xBB 0xBF) to newly created files when AI-generated content contained a BOM character (\uFEFF). This broke Linux scripts because the shebang (#!/bin/bash) must be the very first bytes in the file.

Root cause: The logic was desiredBom = source.bom || next.bom

  • source.bom: Does existing file have BOM? ✅ Correct to preserve
  • next.bom: Does AI content have BOM? ❌ Should NOT trust

Solution

Changed BOM handling to: desiredBom = source.bom

This means:

  • ✅ Existing files with BOMs keep their BOMs (preserves round-trips)
  • ✅ New files never get BOMs (fixes Windows issue)
  • ✅ AI-generated BOMs are stripped (fixes BOM in generated File Bug #368)
  • ✅ Works on Windows, macOS, and Linux

Changes

Modified Files (7 changes across 6 files):

All changes include ANRCODE_CHANGE markers adjacent to modified code:

  1. packages/core/src/file-mutation.ts:117 - Core V2 write logic
  2. packages/core/src/patch.ts:80 - Patch derivation
  3. packages/core/src/tool/edit.ts:195 - Core edit tool
  4. packages/opencode/src/tool/write.ts:49 - Write tool
  5. packages/opencode/src/tool/edit.ts:98, 134 - Edit tool (2 locations)
  6. packages/opencode/src/patch/index.ts:338 - Patch application

Test Updates:

  • Updated existing test: "preserves BOM from existing files but strips BOM from new content"
  • Added new test: "strips BOM from AI-generated content in new files (Issue BOM in generated File Bug #368)"

Testing

All tests pass:

Acceptance Criteria Met:

  • New files with BOM in AI content → NO BOM written
  • Existing files with BOM → BOM preserved
  • Existing files without BOM → NO BOM added
  • Linux scripts remain executable

ANR Workflow

This fix was implemented using proper ANR workflow:

  • ✅ ANR Researcher analyzed and created implementation plan
  • ✅ ANR Implementor applied changes with ANRCODE_CHANGE markers
  • ✅ ANR Tester validated all changes and tests
  • ✅ All markers placed adjacent to modified code per AGENTS.md

Verification

To verify the fix works:

  1. Create a new file with AI-generated content containing BOM
  2. Verify the file does NOT have BOM bytes at the start
  3. Existing files with BOMs should preserve them on edit

Closes #368

Fixes #368 by changing BOM preservation logic to only preserve BOMs
from existing files, never from AI-generated content.

Changes:
- Modified 7 locations across 6 files
- Added ANRCODE_CHANGE markers to all modified regions
- Updated test expectations to reflect new behavior
- Added new test case for Issue #368

Behavior:
- Existing files with BOMs: BOM preserved (round-trip safe)
- New files: Never get BOMs (fixes Windows script issue)
- AI-generated content with BOM: BOM stripped

Related to #368
Copilot AI review requested due to automatic review settings July 30, 2026 21:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@moizlatif7 moizlatif7 self-assigned this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BOM in generated File Bug

2 participants