Skip to content

Add utf8, utf16, utf32 as accepted Encoding.GetEncoding aliases - #133710

Merged
tarekgh merged 1 commit into
dotnet:mainfrom
tarekgh:tarekgh/encoding-utf8-utf16-utf32-aliases
Sep 11, 2026
Merged

tarekgh merged 1 commit into
dotnet:mainfrom
tarekgh:tarekgh/encoding-utf8-utf16-utf32-aliases

Conversation

@tarekgh

@tarekgh tarekgh commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds the hyphen-less encoding names utf8, utf16, and utf32 as accepted aliases in Encoding.GetEncoding(string), mapping them to code pages 65001, 1200, and 12000 respectively.

Today Encoding.GetEncoding("utf8") throws:

System.ArgumentException: 'utf8' is not a supported encoding name.

These hyphen-less spellings are already accepted by essentially every other major platform (Java, Python, ICU, glibc/iconv, Node.js) and are required labels for UTF-8 by the WHATWG Encoding Standard. .NET is the outlier in rejecting them. This surfaces in practice when a server returns a Content-Type header such as application/json; charset=utf8 and HttpClient fails to decode the body (see #133319).

Change

Only the CoreLib name-to-code-page lookup table is affected (System.Private.CoreLib):

  • EncodingData.cs: added utf16, utf32, utf8 to the EncodingNames string in ordinal-sorted position, recomputed EncodingNameIndices, and added the matching CodePagesByName entries.

System.Text.Encoding.CodePages needs no change: it owns only legacy code pages and does not participate in resolving Unicode encoding names.

Notes

  • These are input-resolution aliases only. Encoding.GetEncodings() and EncodingInfo.Name/WebName are driven by a separate table (MappedCodePages) and are unchanged, so enumeration returns no new or duplicated entries and still reports the canonical names (utf-8, utf-16, utf-32).
  • Matches the existing pattern used for other non-IANA aliases already accepted, such as unicode, latin1, and ascii.

Testing

  • Extended EncodingGetEncodingTests so the name-to-code-page and EncodingName tests cover utf8/utf16/utf32.
  • Full System.Text.Encoding.Tests suite passes (14,695 tests, 0 failed).

Fixes #133319

Map the hyphen-less names utf8, utf16, and utf32 to code pages 65001,
1200, and 12000 in the CoreLib encoding name table, matching the aliases
already accepted by Java, Python, ICU, glibc, and the WHATWG standard.

Fixes the ArgumentException thrown when a server returns a Content-Type
charset such as "utf8" (dotnet#133319).
Copilot AI lite review requested due to automatic review settings September 11, 2026 17:35
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-text-encoding
See info in area-owners.md if you want to be subscribed.

@tarekgh tarekgh added this to the 12.0.0 milestone Sep 11, 2026
@tarekgh
tarekgh requested a review from MihaZupan September 11, 2026 17:38
@tarekgh

tarekgh commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

CC @kilasuit

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Update the generator source so the autogenerated alias changes are not overwritten.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds utf8, utf16, and utf32 aliases to Encoding.GetEncoding(string) with regression tests.

Changes:

  • Maps aliases to UTF-8, UTF-16, and UTF-32 code pages.
  • Extends encoding lookup tests.
  • Requires updating the generator source for the autogenerated lookup file.
File summaries
File Summary
src/libraries/System.Runtime/tests/System.Text.Encoding.Tests/Encoding/EncodingGetEncodingTests.cs Tests the new alias mappings.
src/libraries/System.Private.CoreLib/src/System/Text/EncodingData.cs Adds alias lookup entries and indices; generated source requires corresponding generator updates.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

@MihaZupan MihaZupan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@tarekgh

tarekgh commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

/ba-g the failures are unrelated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Web Server returning utf8 (not utf-8) for ContentType header throws an error in ReadBufferAsString method

3 participants