Skip to content

Decode %UTF8% parameter values centrally in ParameterCollection.Parse - #54

Open
Andrei-Errapart wants to merge 1 commit into
issus:masterfrom
Andrei-Errapart:FixUtf8ParameterValueDecoding
Open

Decode %UTF8% parameter values centrally in ParameterCollection.Parse#54
Andrei-Errapart wants to merge 1 commit into
issus:masterfrom
Andrei-Errapart:FixUtf8ParameterValueDecoding

Conversation

@Andrei-Errapart

@Andrei-Errapart Andrei-Errapart commented Aug 28, 2026

Copy link
Copy Markdown

Summary

Altium writes non-ASCII parameter values twice: a %UTF8% variant (UTF-8 bytes) first, then a plain-ANSI twin. ParameterCollection.Parse stripped the %UTF8% prefix without decoding the value, so both entries collapsed onto one key and the first-match indexer returned the mojibake variant — "KON 4.7µF" instead of "KON 4.7µF" for SchComponent.Description, and the same for any other %UTF8% key.

Decoding the value at the point the prefix is stripped makes the UTF-8 variant win with the correct string, which in turn lets several downstream workarounds go away.

Changes

  • AltiumRecordGenerator — in the generated ParameterCollection.Parse, decode the value via AltiumEncoding.DecodeUtf8ParameterValue when the %UTF8% prefix is stripped.
  • SchDocReader / SchLibReader — remove the now-redundant per-site decode of the Text parameter. The %UTF8% key is still detected, purely to preserve TextIsUtf8 for the writer.
  • SchLibWriter — promote ComponentDescription to %UTF8%ComponentDescription when Windows-1252 cannot represent the description, matching how SchParameter.Text is written.
  • SchComponentRenderer — stop applying the FixTextEncoding mojibake-repair heuristic to parameter-sourced text (text frames, net labels, harness type labels and entries, string indirection); it could corrupt legitimate strings such as "Não". Binary pin names, which carry no encoding marker, still use it.

Test Plan

  • Existing tests pass (dotnet test) — 853 passed, 10 skipped, 0 failed
  • New tests added for new behavior

Added to Utf8ParameterRoundTripTests:

  • ParameterCollection_Utf8AndAnsiDuplicate_ReturnsDecodedValue — a record carrying both %UTF8%ComponentDescription and its ANSI twin resolves to the decoded string.
  • SchDoc_NonAsciiComponentDescription_RoundTrips — round-trips a Windows-1252-representable description (KON 4.7µF ±10%, plain key) and one that is not (Shunt 5mΩ, forcing the %UTF8% key).

Checklist

  • Code follows existing style and conventions
  • Public API changes are documented with XML doc comments — no public surface added; the reworked FixTextEncoding doc comment now states that it covers binary pin names only
  • No breaking changes (or clearly identified below)

No signatures change, but two behavioural changes are worth calling out:

  • ParameterCollection.Parse now returns decoded values for %UTF8%-prefixed keys. Any caller that repaired the mojibake itself (as SchDocReader/SchLibReader did) would double-decode and should drop its own fix-up.
  • Parameter-sourced text is no longer passed through SchComponentRenderer.FixTextEncoding, so text that was previously "repaired" at render time now renders as stored.

Altium writes non-ASCII parameter values twice: a %UTF8% variant (UTF-8
bytes) first, then a plain-ANSI twin. Parse stripped the prefix without
decoding, so both entries collapsed onto one key and the first-match
indexer returned the mojibake variant ("KON 4.7µF" instead of
"KON 4.7µF" for SchComponent.Description, and any other %UTF8% key).

Decode the value when the prefix is stripped, so the UTF-8 variant wins
with the correct string. Remove the now-redundant per-site decodes of
the Text parameter in SchDocReader/SchLibReader (the %UTF8% key is still
detected to preserve TextIsUtf8 for the writer), promote
ComponentDescription to a %UTF8% parameter on write when Windows-1252
cannot represent it (as with SchParameter.Text), and drop the renderer's
mojibake-repair heuristic for parameter-sourced text (it could corrupt
legitimate strings like "Não"; binary pin names keep it).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant