Sub-task of #208. See the parent tracking issue for the full plan and context. Builds on the output of #209 (the grown baseline).
Goal
Add a --dump <dir> flag to discovery mode in CorpusRoundtripMain.cpp that writes the fully-normalized expected and actual XML documents for every non-PASS file into a developer-local directory. This makes offline diff analysis possible without touching any checked-in file.
Background
runRoundtrip() in src/private/mxtest/api/CorpusRoundtripMain.cpp currently performs the full pipeline (load → api round-trip → normalize both sides → apply fixup sidecar → compare) and returns only a RoundtripResult (status + one-line detail string). The normalized documents are never surfaced to the caller. Phase 1 needs those documents for the failing files so that Phase 2 (root-cause classification) can diff them without re-running the harness and re-parsing output by hand.
What to implement
1. New dumpDocuments() helper
Add a free function (inside the anonymous namespace in CorpusRoundtripMain.cpp) that takes an absolute path, a dump directory, and a RoundtripResult::Status, re-runs just enough of the pipeline to produce the normalized documents, and serializes them to disk:
void dumpDocuments(const std::string& absolutePath,
const std::string& dumpDir,
RoundtripResult::Status status);
Do not refactor runRoundtrip(). It has one well-scoped responsibility (run the trip and report pass/fail). Returning intermediate documents from it would widen its interface and leak pipeline internals into the regression-mode path that never needs them. The discovery loop already calls runRoundtrip() for status; when --dump is active it calls dumpDocuments() immediately after for the same file. The duplication of pipeline work (reloading and re-normalizing) is acceptable: --dump is a developer tool, not a hot path.
dumpDocuments() replicates the normalization sequence from runRoundtrip() in the same order:
- Load the original input with pugixml →
expectedDoc.
- Run
normalizeForComparison(expectedDoc).
- Construct
Fixer(absolutePath) and call fixer.applyToExpected(expectedDoc).
- Run the api pipeline (same as
runRoundtrip()): createFromFile → getData → createFromScore → writeToStream, parse the output string → actualDoc.
- Call
stripMxAttribution(actualDoc.document_element()).
- Run
normalizeForComparison(actualDoc).
Then serialize to <dumpDir>/<sanitized-relpath>.expected.xml and <dumpDir>/<sanitized-relpath>.actual.xml (see filename convention below).
For LOADFAIL / GETDATAFAIL / CREATEFAIL statuses the api pipeline did not produce an actual document. In those cases write the expected file only; do not write an actual file and do not fail — print a short note to stderr ("dump: no actual for <relpath> (<STATUS>)").
2. Filename convention
Convert the corpus-relative path to a flat filename by replacing every / (and \ on Windows) with __, then appending .expected.xml or .actual.xml. Example:
lysuite/Saltarello.xml → lysuite__Saltarello.xml.expected.xml
lysuite__Saltarello.xml.actual.xml
The corpus-relative path is already computed in the discovery loop (fs::relative(absPath, dataRoot).generic_string()). Pass it alongside absolutePath to dumpDocuments().
3. --dump <dir> flag in discovery mode
Extend the main() argument parsing so that discovery mode accepts an optional --dump <dir> argument:
discovery <dataRoot> [--dump <dir>]
Both orderings (--dump before or after <dataRoot>) are acceptable; pick whichever is simplest to parse. The directory is created if it does not exist (std::filesystem::create_directories). If creation fails, print to stderr and exit non-zero.
In the discovery loop, after calling runRoundtrip(), check whether the result is non-PASS and a dump dir is set; if so, call dumpDocuments():
if (dumpDir && r.status != RoundtripResult::Status::pass
&& r.status != RoundtripResult::Status::skip)
{
dumpDocuments(absPath, relPath, *dumpDir);
}
SKIP files: do not dump. SKIP means the harness intentionally bypassed the file (version-gating or similar) — there is no meaningful expected/actual pair.
PASS files: do not dump. They are already captured in the baseline.
4. Makefile target
Add a dump-api-roundtrip target that builds and runs discovery with --dump pointed at $(BUILD_ROOT)/api/roundtrip-dump/:
# Dump normalized expected/actual XML for every failing api round-trip.
# Output goes to build/api/roundtrip-dump/ (build dir, already gitignored).
dump-api-roundtrip: dev
mkdir -p $(BUILD_ROOT)/api/roundtrip-dump
$(BUILD_ROOT)/api/mxtest-api-roundtrip discovery $(CURDIR)/data \
--dump $(CURDIR)/$(BUILD_ROOT)/api/roundtrip-dump
Place it immediately after the discover-api-roundtrip target (around line 179 of the current Makefile).
What gets checked in
src/private/mxtest/api/CorpusRoundtripMain.cpp — the only source file changed.
Makefile — the new dump-api-roundtrip target.
Nothing else. The dump output files are ephemeral developer artifacts. They live in build/api/roundtrip-dump/, which is inside build/ — already gitignored. They must never appear in git status after a run.
Dump output location
$(BUILD_ROOT)/api/roundtrip-dump/ — equivalently build/api/roundtrip-dump/ from the repo root. The build/ tree is already excluded by .gitignore; no new ignore entry is needed. A developer can also pass any other path to --dump directly.
Definition of done
Sub-task of #208. See the parent tracking issue for the full plan and context. Builds on the output of #209 (the grown baseline).
Goal
Add a
--dump <dir>flag todiscoverymode inCorpusRoundtripMain.cppthat writes the fully-normalized expected and actual XML documents for every non-PASS file into a developer-local directory. This makes offline diff analysis possible without touching any checked-in file.Background
runRoundtrip()insrc/private/mxtest/api/CorpusRoundtripMain.cppcurrently performs the full pipeline (load → api round-trip → normalize both sides → apply fixup sidecar → compare) and returns only aRoundtripResult(status + one-line detail string). The normalized documents are never surfaced to the caller. Phase 1 needs those documents for the failing files so that Phase 2 (root-cause classification) can diff them without re-running the harness and re-parsing output by hand.What to implement
1. New
dumpDocuments()helperAdd a free function (inside the anonymous namespace in
CorpusRoundtripMain.cpp) that takes an absolute path, a dump directory, and aRoundtripResult::Status, re-runs just enough of the pipeline to produce the normalized documents, and serializes them to disk:Do not refactor
runRoundtrip(). It has one well-scoped responsibility (run the trip and report pass/fail). Returning intermediate documents from it would widen its interface and leak pipeline internals into the regression-mode path that never needs them. The discovery loop already callsrunRoundtrip()for status; when--dumpis active it callsdumpDocuments()immediately after for the same file. The duplication of pipeline work (reloading and re-normalizing) is acceptable:--dumpis a developer tool, not a hot path.dumpDocuments()replicates the normalization sequence fromrunRoundtrip()in the same order:expectedDoc.normalizeForComparison(expectedDoc).Fixer(absolutePath)and callfixer.applyToExpected(expectedDoc).runRoundtrip()):createFromFile → getData → createFromScore → writeToStream, parse the output string →actualDoc.stripMxAttribution(actualDoc.document_element()).normalizeForComparison(actualDoc).Then serialize to
<dumpDir>/<sanitized-relpath>.expected.xmland<dumpDir>/<sanitized-relpath>.actual.xml(see filename convention below).For LOADFAIL / GETDATAFAIL / CREATEFAIL statuses the api pipeline did not produce an actual document. In those cases write the expected file only; do not write an actual file and do not fail — print a short note to stderr (
"dump: no actual for <relpath> (<STATUS>)").2. Filename convention
Convert the corpus-relative path to a flat filename by replacing every
/(and\on Windows) with__, then appending.expected.xmlor.actual.xml. Example:The corpus-relative path is already computed in the discovery loop (
fs::relative(absPath, dataRoot).generic_string()). Pass it alongsideabsolutePathtodumpDocuments().3.
--dump <dir>flag in discovery modeExtend the
main()argument parsing so thatdiscoverymode accepts an optional--dump <dir>argument:Both orderings (
--dumpbefore or after<dataRoot>) are acceptable; pick whichever is simplest to parse. The directory is created if it does not exist (std::filesystem::create_directories). If creation fails, print to stderr and exit non-zero.In the discovery loop, after calling
runRoundtrip(), check whether the result is non-PASS and a dump dir is set; if so, calldumpDocuments():SKIP files: do not dump. SKIP means the harness intentionally bypassed the file (version-gating or similar) — there is no meaningful expected/actual pair.
PASS files: do not dump. They are already captured in the baseline.
4. Makefile target
Add a
dump-api-roundtriptarget that builds and runs discovery with--dumppointed at$(BUILD_ROOT)/api/roundtrip-dump/:Place it immediately after the
discover-api-roundtriptarget (around line 179 of the current Makefile).What gets checked in
src/private/mxtest/api/CorpusRoundtripMain.cpp— the only source file changed.Makefile— the newdump-api-roundtriptarget.Nothing else. The dump output files are ephemeral developer artifacts. They live in
build/api/roundtrip-dump/, which is insidebuild/— already gitignored. They must never appear ingit statusafter a run.Dump output location
$(BUILD_ROOT)/api/roundtrip-dump/— equivalentlybuild/api/roundtrip-dump/from the repo root. Thebuild/tree is already excluded by.gitignore; no new ignore entry is needed. A developer can also pass any other path to--dumpdirectly.Definition of done
make dump-api-roundtripruns without error and populatesbuild/api/roundtrip-dump/with.expected.xmland.actual.xmlpairs for each FAIL file..expected.xmlonly; no.actual.xmlis written and no error is returned for those.compareElements()would have seen on the expected side:normalizeForComparisonapplied, thenFixer::applyToExpectedapplied.compareElements()would have seen on the actual side:stripMxAttributionapplied, thennormalizeForComparisonapplied.make test-api-roundtrippasses (the regression gate is unaffected).make checkpasses.git statusis clean aftermake dump-api-roundtrip(no dump files, no accidental source changes).