feat(ddi2xlsform): DDI back to XLSForm, round trips proven on the model (#154) - #159
Merged
Merged
Conversation
…el (#154) - `ddiToXlsform(xml, { onWarning })` / `formtransform ddi2xlsform`: a DDI codebook, or a <dataDscr>/<var>/<varGrp> fragment, → `{ survey, choices, settings }`. `src/instrument/fromDdi.ts` reads standard DDI first, then the cdl: notes; `src/utils/xmlParse.ts` is a dependency-free XML reader, so it runs in the browser bundle too. - Never refuses DDI it can read: a non-CDL codebook gets one `ddi-field-missing` warning per field only CDL carries, an unknown responseDomainType is text (`ddi-type-unknown`), a fragment's reference outside it is `ddi-reference-outside`. Broken XML is `ddi-invalid`. - The Instrument → XLSForm emitter moves to `src/xlsform/fromInstrument.ts` and is shared with lstsv2xlsform: groups keep their names, grid members their own fields, a shared list is written once, exclusive choices come from the choice rows too, form_id/version/per-language titles are kept. lstsv2xlsform: a LimeSurvey group's description comes back as its hint, and a group name that is a valid XLSForm name is kept (`Later`), else slugified and made unique. - Round trips: every survey fixture and registry entity, and 200 random forms (fast-check), give back their Instrument after XLSForm → DDI, and after XLSForm → DDI → XLSForm sheets, up to the losses src/pipelines/ddi2xlsform/README.md lists. Each survey's output is blessed as ddi2xlsform.json and validated with pyxform. - Schematron: the cdl: note vocabulary (known types, the subject each needs, one note per type and language), generated from the registry. - Emitter fixes the round trip found: a note before a semi-open question was dropped, and a per-language form_title was written as "[object Object]"; it is now titl plus parTitl per language. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #154, and with it #155 (#151 logic, #152 structure and #153 fields are merged).
ddi2xlsformCLI:
formtransform ddi2xlsform codebook.xml -o form.json.What qwacback asked for (qwacback#37):
<codeBook>,<dataDscr>or bare<var>/<varGrp>elements work, with no wrapping needed. AvarGrppointing outside the fragment givesddi-reference-outside, not an error.seqNoor anycdl:note, it gets oneddi-field-missingwarning for each field only CDL carries. An unknownresponseDomainTypebecomes text (ddi-type-unknown). Only XML that isn't well-formed throws (ddi-invalid).{ survey, choices, settings }JSON sheets, the shapelstsvToXlsformalready returns. Warnings go throughonWarningasDiagnostic[].cdl:notes only where DDI has no element.src/utils/xmlParse.tsis a small XML reader with no dependency (no DOMParser, no Node). The browser-bundle test runs it.Round trips, on the model
tests/ts/contract/ddiRoundtrip.test.tsruns every survey fixture and registry entity (29 cases) through two paths: XLSForm → DDI → Instrument, and XLSForm → DDI → XLSForm sheets → Instrument. In both, the result must equal the original Instrument up to the documented losses (canonicalInstrument.ts). A control test proves the comparison catches a dropped relevant, constraint or label.ddiRoundtripGenerated.test.tsdoes the same with fast-check: 200 random forms per run, covering every simple type, selects withor_otherand exclusive choices, notes, nested groups, grids, 1–2 languages, relevant, constraint, required, hints, appearances and parameters. It also held at 2000 runs locally.ddi2xlsform.json, pinned by a snapshot test, and validated with pyxform.src/pipelines/ddi2xlsform/README.md:or_othershorthand (it comes back as the explicit pair),Found and fixed by the round trip
_othertext) was dropped from the DDI.form_titlewas written astitl=[object Object]. It is nowtitlin the base language, plusparTitl xml:lang="…"(DDI's parallel title) for each other language.Shared XLSForm emitter
src/xlsform/fromInstrument.ts, so both reverse paths share it; pipelines may not import each other.form_id/version, and per-language titles.hint.Later) instead of lowercased. Any other is slugified and made unique, which also keeps lstsv2ddi's section IDs valid.Schematron
New generated rules for the
cdl:vocabulary:cdl:relevantandcdl:constraintneedsubject="xlsform-xpath", andcdl:settingneeds a subject,stdyDscr).Each rule has a mutation test.
Docs
src/pipelines/README.md: "Why there is noddi2xlsform" is now "The way back from DDI".src/pipelines/ddi2xlsform/README.md.fast-checkis added as a dev dependency.Checks
npm run validateand the drift check are clean.🤖 Generated with Claude Code