Step 5 of 6 for the survey2ddi retirement. Plan: HANDOVER_SURVEY2DDI.md. Independent of the CLI/adapter issues — can run in parallel. Blocks the emit-core deletion in CorrelAid/survey2ddi.
Two pieces of coverage that must not die with survey2ddi's emitter
1. The qwacback equivalence test (permanent)
tests/integration/test_conversion_equivalence.py in CorrelAid/survey2ddi posts the same XLSForm through survey2ddi and qwacback and compares the DDI shape, for every answer type qwacback supports (qwacback/internal/examples/examples.go). It is marked integration (docker + the schematron-worker image, opt-in via -m integration).
qwacback is a listed consumer of this library (see the README's ecosystem table and HANDOVER_QWAC.md), so this comparison is worth more here than it ever was in survey2ddi. Port it to compare buildDdiXml against qwacback, reusing this repo's existing pytest validation setup (tests/validation/, which already runs XSD and Schematron checks) or the TS contract project — whichever fits the docker orchestration better. Note the shape normalization it needs: qwacback returns a bare <var> / <varGrp> when there is only one, so the test synthesizes a <dataDscr> container.
2. The Python↔TS byte-parity script (temporary)
A script that runs both emitters over the fixture corpus and byte-compares XML and CSV. It exists to gate the deletion of survey2ddi_core's emit core, and it gets deleted with it — do not wire it into CI as a permanent job.
Known-good starting point, already verified by hand: on a flat survey (no groups, no select_multiple, no _other), build_data_csv and buildDataCsv are byte-equal — quoting, CRLF, None → empty cell, space-joined multi expansion all agree.
Known divergence, expected and correct: get_canonical_columns returns input order while ddi_xml.py emits bucketed order, so any survey with a grid, a select_multiple or an _other pattern gets a Python CSV whose header does not match its own XML. This library derives columns from the emitter's own bucket walk, so it disagrees with Python and agrees with the XML. The script must classify that difference rather than fail on it — and any other difference is a finding to triage, since a Python behaviour missing here is a bug here.
Acceptance criteria
Step 5 of 6 for the survey2ddi retirement. Plan:
HANDOVER_SURVEY2DDI.md. Independent of the CLI/adapter issues — can run in parallel. Blocks the emit-core deletion inCorrelAid/survey2ddi.Two pieces of coverage that must not die with survey2ddi's emitter
1. The qwacback equivalence test (permanent)
tests/integration/test_conversion_equivalence.pyinCorrelAid/survey2ddiposts the same XLSForm through survey2ddi and qwacback and compares the DDI shape, for every answer type qwacback supports (qwacback/internal/examples/examples.go). It is markedintegration(docker + the schematron-worker image, opt-in via-m integration).qwacback is a listed consumer of this library (see the README's ecosystem table and
HANDOVER_QWAC.md), so this comparison is worth more here than it ever was in survey2ddi. Port it to comparebuildDdiXmlagainst qwacback, reusing this repo's existing pytest validation setup (tests/validation/, which already runs XSD and Schematron checks) or the TS contract project — whichever fits the docker orchestration better. Note the shape normalization it needs: qwacback returns a bare<var>/<varGrp>when there is only one, so the test synthesizes a<dataDscr>container.2. The Python↔TS byte-parity script (temporary)
A script that runs both emitters over the fixture corpus and byte-compares XML and CSV. It exists to gate the deletion of
survey2ddi_core's emit core, and it gets deleted with it — do not wire it into CI as a permanent job.Known-good starting point, already verified by hand: on a flat survey (no groups, no
select_multiple, no_other),build_data_csvandbuildDataCsvare byte-equal — quoting, CRLF,None→ empty cell, space-joined multi expansion all agree.Known divergence, expected and correct:
get_canonical_columnsreturns input order whileddi_xml.pyemits bucketed order, so any survey with a grid, aselect_multipleor an_otherpattern gets a Python CSV whose header does not match its own XML. This library derives columns from the emitter's own bucket walk, so it disagrees with Python and agrees with the XML. The script must classify that difference rather than fail on it — and any other difference is a finding to triage, since a Python behaviour missing here is a bug here.Acceptance criteria
buildDdiXmlin this repo, same type coverage, still opt-in