Add `buildDataCsv(variables, submissions): string` — response-data CSV emitter, currently the only thing keeping the app on Pyodide

[jstet]

The gap

@correlaid/formtransform emits DDI metadata only. Given submissions in BuildDdiOptions, it uses their count for <caseQnty> and a filename in <fileDscr>, but it never writes the response data itself:

// dist/ddi/codebook.ts (paraphrased)
/** Response records — only their count (caseQnty) is used. */
submissions?: unknown[];

There is no TypeScript equivalent of the Python survey2ddi_core.data.build_data_csv, which remaps raw Kobo response columns onto the DDI variable names and expands select_multiple into per-choice binary columns.

Why this is on your radar

CorrelAid/formtransform-app is the consumer. The Kobo → DDI tab's metadata mode (#6) now uses buildDdiXml and leaves Pyodide off. The full mode (XLSForm + responses CSV → DDI XML and data.csv) still boots Pyodide and the survey2ddi wheel just to call build_data_csv. Removing Pyodide is the last step of the migration (#10) and is blocked on this.

Tracked from the app side at CorrelAid/formtransform-app#9.

What the Python build_data_csv does (the spec to match)

survey2ddi_core.data (whl) has three pieces; the third is the one to port:

1. get_canonical_columns(variables) → list[str] — the DDI <var name=""> in the order build_ddi_xml emits them. select_multiple expands to <name>_<choice> columns; everything else contributes one column equal to v.name.
2. to_canonical_rows(variables, neutral_rows) → list[dict] — re-key adapter rows from v.data_key to DDI variable names. select_multiple values (space-joined choice codes) become per-choice "0" / "1" columns. Every other variable becomes a single string column.
3. build_data_csv(variables, neutral_rows) → str — RFC 4180 CSV: CRLF line endings, QUOTE_MINIMAL (only fields containing a delimiter, quote, or line break get quoted), header row first, then canonical rows in input order.

get_canonical_columns and to_canonical_rows could be exported separately as getDdiColumnNames / remapSubmissionsToDdi for callers that want to write the CSV themselves. buildDataCsv is the convenience wrapper that does all three.

What I think the TS API should look like

Rough — your call on exact shape:

// New exports from src/index.ts
import { buildDataCsv, getDdiColumnNames, remapSubmissionsToDdi } from '@correlaid/formtransform';

const xml = buildDdiXml(surveyData, choicesData, { settings: settingsData[0] });
const variables = extractVariables(surveyData, choicesByList);
const csv = buildDataCsv(variables, submissions);

Where:

• variables is Variable[] in the same shape the lib's extractVariables already returns (the TS Variable doesn't currently have a data_key field — see below).
• submissions is the raw Kobo row array, keyed by question name ({ q1: 'yes', q2: 'red blue', ... }). Per the Python implementation, select_multiple values arrive space-joined. The function splits on whitespace and produces one 0 / 1 per choice.
• Returns an RFC 4180 CSV string ready to download.

Design decisions for upstream

1. Variable.data_key. The Python Variable has data_key (defaults to ''; Kobo sets it to group/name, LimeSurvey to name). The TS Variable does not. Either add it and document that the caller populates it, or accept submissions keyed by question name and let the function do the path-flattening itself. The Kobo-only use case is simple; LimeSurvey's full mode is what would have to agree.
2. Submissions → DDI key remapping. Same question. The Kobo path is identity ({ q1: ... } → q1 column). The LimeSurvey path needs the [group]/[question] flattening the Python does. If buildDataCsv is Kobo-only for now, that's fine — the issue title and the call site make the scope explicit. If it's meant to serve both, the key remapping is part of the contract.
3. Where the emitter lives in the package. dist/pipelines/xlsform2ddi/data.ts next to variables.ts, mirroring the Python survey2ddi_core.data layout, is the obvious home.
4. Validation surface. buildDdiXml validates by default and throws on subset violations. buildDataCsv should probably accept anything the caller has already validated (i.e. always do its work — no skipValidation toggle), since the canonical-row remap is mechanical and a bad submission row is the caller's problem. Consistent with the lstsvToDdiXml skipValidation policy in the existing emitter would be the alternative; either is defensible.

Out of scope

• limesurvey2ddi integration. The app does not currently wire the LimeSurvey tab to a response-data CSV path (the new Lstsv2DdiTab is metadata-only, on purpose). When the app wires a CSV into that tab, the same buildDataCsv is expected to cover it, but that's a future PR.
• Touching the limesurvey2ddi Python package or xlsform parser.
• Re-implementing the DDI XML side. buildDdiXml already handles the schema side; the gap is purely the data side.

Verification I'd want

• A small unit test (or a script in tests/) that takes a Kobo CSV plus a hand-built Variable[] and asserts: header row matches the DDI XML <var name=""> order, select_multiple expands to N binary columns, empty values become empty cells (not "None"), and the line endings are CRLF.
• An E2E byte-comparison: a Python build_data_csv reference output and the TS buildDataCsv output should be byte-equal for a fixed input, modulo trailing whitespace.

Once it lands

The app's #10 becomes actionable: drop Pyodide, the survey2ddi wheel, pyodide.config.json, scripts/setup-pyodide.mjs, src/lib/pyodide.{ts,worker.ts,svelte.test.ts}, and the pyodide + xlsform2lstsv entries from package.json. The full-mode Kobo branch in Kobo2DdiTab.svelte becomes a one-liner: await XLSFormParser.parseXLSData(xlsx) + buildDdiXml(...) + buildDataCsv(variables, submissions).