feat(ddi)!: a codebook gives back its whole form; round trips through LimeSurvey (#160) - #162
Merged
Merged
Conversation
… LimeSurvey (#160) A CDL codebook now carries what ddi2xlsform used to lose, in cdl: notes where DDI 2.5 has no element (convention:ddiFields): - cdl:list: a select's list name, when it isn't the question's - cdl:or_other: an added other pair, `shorthand` (the type cell's or_other) or `added` (LimeSurvey's other=Y); cdl:other_label for a select_multiple pair's authored other label - cdl:note_names: the note rows a preQTxt joins - stdyDscr: cdl:row (+ cdl:row_label) for rows without data (metadata rows, matrix headers), cdl:row_hint / row_relevant / row_appearance for note rows, cdl:position for orphan notes, rows and groups of notes only (which now get a section varGrp) - cdl:setting for every setting, default_language as authored; cdl:language for the form's language names and order - a range's start/end as authored in cdl:parameters The LimeSurvey TSV → DDI path declares the survey's language (codeBook/@xml:lang), passes its settings (style), and carries exclude_all_others onto the choices. format=G's field-list groups are inferred by the LimeSurvey parser, not the shared XLSForm emitter. New round trips, on every fixture and on generated forms (fast-check): DDI → XLSForm → DDI is byte-identical; XLSForm → DDI → XLSForm → LimeSurvey equals XLSForm → LimeSurvey byte for byte; LimeSurvey → DDI → Instrument equals the TSV's own; XLSForm → LimeSurvey → DDI → XLSForm equals XLSForm → LimeSurvey → XLSForm. The model comparison is strict now. BREAKING CHANGE: lstsv2ddi declares codeBook/@xml:lang for a single-language survey too, so its universe prose is in the survey's language. extractVariables / variablesFromInstrument also return rows without data (Variable.row set). instrumentFromLstsv's settings no longer hold default_language (defaultLanguage does). A CDL codebook's identical category sets are no longer merged into one list. New cdl: note types and section varGrps without members appear in the DDI. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jstet
added a commit
that referenced
this pull request
Sep 27, 2026
…constraint (#160) (#163) Closes the small losses #162 left: - constraint_message is kept without a constraint - cdl:required holds the cell as authored (TRUE, true), yes by default - cdl:parameters holds the cell as authored, guidance_hint included (ivuInstr still has it), not only the parameters DDI has no element for - cdl:no_label marks a group without a label (its txt stays its name) - notes whose joined preQTxt can't be split at blank lines (a blank line in one, a language only some have) keep their own text in cdl:row_label - the form model's languages come from every text column, not only label: a language only a hint has is no longer dropped The generated forms cover all of these. BREAKING CHANGE: cdl:parameters now includes a guidance_hint, and cdl:constraint_message appears without a constraint. ddi2xlsform gives back a guidance_hint inside parameters as it was, which pyxform rejects when it has spaces (hints_survey, as its source). Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #160.
What a codebook carries now
ddi2xlsformused to lose several things. They are now in the codebook, as standard DDI where it has an element and acdl:note otherwise (convention:ddiFields):cdl:list, when the list isn't named after the questionor_othershorthandcdl:or_otheron the pair'svarGrp:shorthandoradded(LimeSurveyother=Y); a select_multiple pair's authored other label:cdl:other_labelcdl:note_namesnext to thepreQTxtcdl:positiononstdyDscr(in=<group> after=<item>)varGrpwithout members, placed bycdl:positioncdl:row_hint,cdl:row_relevant,cdl:row_appearancecdl:row(the type cell) +cdl:row_labellabel::Deutsch (de)) and ordercdl:languagecdl:settingfor every setting,default_languageas authoredcdl:parametersas authored (valrngstays)Remaining losses are listed in
src/pipelines/ddi2xlsform/README.md:calculate)New round trips
Every fixture (29 forms, 28 TSVs), plus generated forms (fast-check, 200 per property;
FT_ROUNDTRIP_RUNS). Locally I also ran 10 000 per property.orOther, metadata rows, language names and all settings.Fixed along the way
The new tests found these bugs:
exclude_all_othersnever reached the DDI.style.field-listforstyle: pagesin the DDI path too. That inference now lives in the LimeSurvey parser.or_othereven when the type cell keeps the shorthand.Breaking
codeBook/@xml:langfor single-language surveys too. As a result, the universe prose is in the survey's language.extractVariables/variablesFromInstrumentalso return rows without data, marked withVariable.row.settingsfrominstrumentFromLstsvno longer includedefault_language;defaultLanguagestill does.cdl:note types, and sectionvarGrps with no members.Consumers: the formtransform-app DDI view should ignore typed
cdl:notes and tolerate empty sections.Checks
ddi2xlsform.json.npm run validateand the drift check pass.🤖 Generated with Claude Code