Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ The canonical `Variable` (`src/ddi/types.ts`) is what survives an emit:

- **`relevant`, `constraint`, `constraint_message`, `required`** survive (#151). DDI Codebook 2.5 has no expression syntax, so each goes in twice: readable (`<universe>` prose, `<valrng>` for a simple numeric range) and exact, in a typed `<notes type="cdl:…">`. `convention:logicMapping` (`ddiEncoding`) defines the notes. A group's own condition is on its `varGrp`; a variable's `<universe>` states its groups' conditions too.
- **Groups and order** survive (#152): every group is a `<varGrp>` (a plain one `type="section"`), nested through `@varGrp`, and the `<var>`s and data columns follow the survey.
- **no `default`, per-question `appearance`, `calculation`, group hint** yet (#153). A question's `hint` and `guidance_hint` do survive, as `<preQTxt>` and `<ivuInstr>`.
- **Every other form field** (#153, `convention:ddiFields`): standard DDI where it has a home (the hint as `postQTxt`, `guidance_hint` as `ivuInstr`, `varFormat/@category` for date and time, `var/@dcml="0"` for integer, `valrng` for a range, `qstn/@seqNo` and `qstn/backward`), else a typed note (`cdl:default`, `cdl:appearance`, `cdl:parameters`, a group's `cdl:hint`, `cdl:exclusive`, `cdl:setting`). A unit test fails when a model field has neither nor a documented loss. Not carried: `calculation` and other unlifted columns, a choice list's name, the `or_other` shorthand as such.

Compare `lstsv2xlsform`, which *is* implemented: a LimeSurvey structure TSV carries `relevance`, `em_validation_q`, `mandatory`, `default` and the `!`/`T` type overrides. It is a form definition in a different dialect, so reversing it is a translation problem. Reversing a codebook without those pieces is a *reconstruction* problem, and they cannot be inferred from it.

Expand Down
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,15 +188,16 @@ records how each maps onto it.
`lstsv2ddi` and `lstsv2xlsform` (the reverse paths). All are lossy for some
types: nested groups flatten in LimeSurvey, choice codes over 5 chars truncate,
`select_multiple` becomes N binary variables, and the reverse paths cannot
recover a select's authored `list_name` or tell `integer` from `decimal`.
recover a select's authored `list_name`.

**DDI has no way back yet:** there is no `ddi2xlsform` or `ddi2lstsv`. A CDL
codebook carries skip logic, validation and `required`: each condition as a
readable `<universe>` sentence (and a simple numeric range as `<valrng>`), plus
the exact expression in a typed note such as `<notes type="cdl:relevant"
subject="xlsform-xpath">` (`convention:logicMapping`). Defaults, appearances
and the group structure are not in it yet, so a reversed form would still look
right and behave wrongly. #155 tracks the rest.
subject="xlsform-xpath">` (`convention:logicMapping`). Groups, order, hints,
defaults, appearances and parameters are in it too, in standard DDI where it
has a place and typed notes where not (`convention:ddiFields`). The reverse
parser that reads them back is #154.

## Errors and warnings

Expand Down
194 changes: 194 additions & 0 deletions registry/conventions/ddiFields.jsonld
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
{
"@context": {
"@version": 1.1,
"@vocab": "https://civic-data.de/survey/",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"skos": "http://www.w3.org/2004/02/skos/core#",
"type": "https://civic-data.de/survey/type/",
"variant": "https://civic-data.de/survey/variant/",
"composite": "https://civic-data.de/survey/composite/",
"appearance": "https://civic-data.de/survey/appearance/",
"convention": "https://civic-data.de/survey/convention/",
"vocab": "https://civic-data.de/survey/vocab/",
"meta": "https://civic-data.de/survey/meta/",
"classes": {
"@id": "meta:classes",
"@type": "@json"
},
"relations": {
"@id": "meta:relations",
"@type": "@json"
},
"rule": {
"@id": "https://civic-data.de/survey/rule",
"@type": "@json"
}
},
"@graph": [
{
"@id": "convention:ddiFields",
"@type": "GlobalConvention",
"skos:prefLabel": "Form fields in DDI Codebook",
"rule": {
"description": "Where each field of a form (the Instrument model, src/instrument/types.ts) goes in a CDL DDI codebook (formtransform#153): a standard DDI 2.5 element or attribute wherever one fits, a typed note `<notes type=\"cdl:<field>\">` (the XSD's controlled-vocabulary extension point, as in convention:logicMapping) only where DDI 2.5 has none, or a documented loss. A unit test fails when a model field has no entry.",
"questionText": {
"preQTxt": "a note row before the question, or a grid's or select_multiple's shared question text",
"qstnLit": "the label",
"postQTxt": "the hint shown to respondents with the question",
"ivuInstr": "guidance_hint (for interviewers and the codebook, not shown)",
"order": "preQTxt, qstnLit, postQTxt, backward, ivuInstr (XSD order)"
},
"backward": "qstn/backward/@qstn lists the IDs of the questions its own relevant refers to (a var, or the varGrp of a select_multiple or other pair).",
"seqNo": "qstn/@seqNo is the question's position among the form's data questions, from 1; a select_multiple's binaries share their question's.",
"notes": {
"default": {
"type": "cdl:default",
"on": [
"var",
"varGrp"
],
"text": "the default cell as authored"
},
"appearance": {
"type": "cdl:appearance",
"on": [
"var",
"varGrp"
],
"text": "the appearance cell, lowercased"
},
"parameters": {
"type": "cdl:parameters",
"on": [
"var",
"varGrp"
],
"text": "the parameters not in a standard element, space-separated key=value"
},
"hint": {
"type": "cdl:hint",
"on": [
"varGrp[@type='section' or @type='grid']"
],
"localized": true
},
"exclusive": {
"type": "cdl:exclusive",
"on": [
"varGrp[@type='multipleResp']"
],
"text": "the exclusive choices' codes, space-separated"
},
"setting": {
"type": "cdl:setting",
"on": [
"stdyDscr"
],
"subject": "the settings key",
"keys": [
"style"
]
}
},
"fields": {
"ItemBase.name": {
"ddi": "var/@name; a group's path in varGrp/@name"
},
"ItemBase.label": {
"ddi": "qstn/qstnLit; a group's varGrp/txt"
},
"ItemBase.hint": {
"ddi": "qstn/postQTxt; a group's in a cdl:hint note (varGrp has no hint element)",
"note": "CDL uses postQTxt for the hint shown to respondents with the question, though the XSD describes it as text about what follows the question: DDI 2.5 has no hint element."
},
"ItemBase.relevant": {
"ddi": "universe + cdl:relevant note (convention:logicMapping)"
},
"ItemBase.appearance": {
"cdlNote": "cdl:appearance",
"note": "On a var (a select_multiple's or pair's on its varGrp), and on a section; a grid's table-list is its type='grid', so only a grid appearance other than that is noted."
},
"ItemBase.row": {
"loss": "Columns the model does not lift (media, calculation, …) are not carried."
},
"GroupItem.kind": {
"ddi": "a group is a varGrp"
},
"GroupItem.children": {
"ddi": "varGrp/@var and @varGrp (nesting), var element order and qstn/@seqNo (order)"
},
"GroupItem.closed": {
"loss": "Every group read back is closed."
},
"QuestionItem.kind": {
"ddi": "a question is a var (a select_multiple: a varGrp of binary vars)"
},
"QuestionItem.type": {
"ddi": "qstn/@responseDomainType, var/@intrvl, varFormat/@type, varFormat/@category (date, time), var/@dcml='0' (integer)"
},
"QuestionItem.rawType": {
"loss": "Rebuilt from type, list and the other pattern; the or_other shorthand comes back as the explicit pair, which behaves the same."
},
"QuestionItem.list": {
"loss": "The list name: questions with identical category sets come back sharing one list."
},
"QuestionItem.file": {
"ddi": "concept/@vocab (the file is <vocab>.csv)"
},
"QuestionItem.orOther": {
"ddi": "varGrp[@type='other'] (convention:other)",
"note": "Comes back as the explicit pair, see rawType."
},
"QuestionItem.guidanceHint": {
"ddi": "qstn/ivuInstr"
},
"QuestionItem.constraint": {
"ddi": "cdl:constraint note, plus valrng/range for a simple numeric range (convention:logicMapping)"
},
"QuestionItem.constraintMessage": {
"cdlNote": "cdl:constraint_message",
"note": "One per language (convention:logicMapping)."
},
"QuestionItem.required": {
"cdlNote": "cdl:required"
},
"QuestionItem.default": {
"cdlNote": "cdl:default"
},
"QuestionItem.parameters": {
"ddi": "range start/end: valrng/range @min/@max; guidance_hint: qstn/ivuInstr; the other parameters (step, randomize, …) in a cdl:parameters note"
},
"QuestionItem.otherLabel": {
"ddi": "the other answer's catgry/labl and the companion's qstnLit"
},
"InstrumentChoice.name": {
"ddi": "catgry/catValu"
},
"InstrumentChoice.label": {
"ddi": "catgry/labl, one per language"
},
"InstrumentChoice.row": {
"ddi": "exclusive: a cdl:exclusive note on the select_multiple's varGrp listing the codes",
"note": "The choice sheet's other columns are not carried."
},
"Instrument.languages": {
"ddi": "xml:lang siblings of every text (convention:languageTagging)"
},
"Instrument.defaultLanguage": {
"ddi": "codeBook/@xml:lang"
},
"Instrument.settings": {
"ddi": "form_title: titl; form_id: IDNo; version: verStmt/version; default_language: codeBook/@xml:lang; style: stdyDscr/notes[@type='cdl:setting'][@subject='style']",
"note": "Other settings are not carried."
},
"Instrument.lists": {
"ddi": "catgry of each var that uses a list"
},
"Instrument.body": {
"ddi": "dataDscr: varGrp and var, in survey order"
}
}
}
}
]
}
2 changes: 2 additions & 0 deletions registry/conventions/languageTagging.jsonld
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,9 @@
"qstnLit",
"labl",
"preQTxt",
"postQTxt",
"ivuInstr",
"universe",
"notes"
],
"baseLanguage": "settings.default_language, else the form's first language; declared as codeBook/@xml:lang whenever the form has more than one language (or default_language is set). Its elements are untagged and come first.",
Expand Down
4 changes: 2 additions & 2 deletions registry/entities/date/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@
</fileDscr>
<dataDscr>
<var ID="V_geburtsdatum" name="geburtsdatum" intrvl="discrete" files="F1">
<qstn responseDomainType="text">
<qstn responseDomainType="text" seqNo="1">
<qstnLit>Wann wurden Sie geboren?</qstnLit>
</qstn>
<concept>Wann wurden Sie geboren?</concept>
<varFormat type="character" schema="other"/>
<varFormat type="character" schema="other" category="date"/>
</var>
</dataDscr>
</codeBook>
2 changes: 1 addition & 1 deletion registry/entities/decimal/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
</fileDscr>
<dataDscr>
<var ID="V_schlafdauer" name="schlafdauer" intrvl="contin" files="F1">
<qstn responseDomainType="numeric">
<qstn responseDomainType="numeric" seqNo="1">
<qstnLit>Wie viele Stunden schlafen Sie durchschnittlich pro Nacht?</qstnLit>
</qstn>
<concept>Wie viele Stunden schlafen Sie durchschnittlich pro Nacht?</concept>
Expand Down
4 changes: 2 additions & 2 deletions registry/entities/grid/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<concept>Vertrauen in Institutionen</concept>
</varGrp>
<var ID="V_vertrauenparlament" name="vertrauenparlament" intrvl="discrete" files="F1">
<qstn responseDomainType="category">
<qstn responseDomainType="category" seqNo="1">
<preQTxt>Vertrauen in Institutionen</preQTxt>
<qstnLit>Das Parlament</qstnLit>
</qstn>
Expand Down Expand Up @@ -54,7 +54,7 @@
<varFormat type="numeric" schema="other"/>
</var>
<var ID="V_vertrauenpolizei" name="vertrauenpolizei" intrvl="discrete" files="F1">
<qstn responseDomainType="category">
<qstn responseDomainType="category" seqNo="2">
<preQTxt>Vertrauen in Institutionen</preQTxt>
<qstnLit>Die Polizei</qstnLit>
</qstn>
Expand Down
4 changes: 2 additions & 2 deletions registry/entities/integer/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
</fileTxt>
</fileDscr>
<dataDscr>
<var ID="V_alter" name="alter" intrvl="contin" files="F1">
<qstn responseDomainType="numeric">
<var ID="V_alter" name="alter" intrvl="contin" dcml="0" files="F1">
<qstn responseDomainType="numeric" seqNo="1">
<qstnLit>Wie alt sind Sie?</qstnLit>
</qstn>
<concept>Wie alt sind Sie?</concept>
Expand Down
8 changes: 6 additions & 2 deletions registry/entities/integer/definition.jsonld
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@
"limesurvey": {
"typeCode": "N",
"supportsOther": false,
"answerClass": null
"answerClass": null,
"integerOnly": {
"attribute": "num_value_int_only",
"whenWhole": []
}
},
"ddi": {
"intrvl": "contin",
Expand All @@ -44,4 +48,4 @@
"useWhen": "When collecting open integer values without predefined categories, where respondents provide a raw number (e.g. age, household size)."
}
]
}
}
14 changes: 7 additions & 7 deletions registry/entities/integer/tsv.tsv
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
class type/scale name relevance text help language validation em_validation_q mandatory other default same_default hidden hide_tip
S language 1 en en
S format 1 A en
S shownoanswer 1 N en
SL surveyls_title 1 Untitled Survey en
G Questions 1 en
Q N alter 1 Wie alt sind Sie? en 1
class type/scale name relevance text help language validation em_validation_q mandatory other default same_default hidden hide_tip num_value_int_only
S language 1 en en
S format 1 A en
S shownoanswer 1 N en
SL surveyls_title 1 Untitled Survey en
G Questions 1 en
Q N alter 1 Wie alt sind Sie? en 1 1
6 changes: 5 additions & 1 deletion registry/entities/range/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,15 @@
</fileDscr>
<dataDscr>
<var ID="V_zufriedenheit" name="zufriedenheit" intrvl="contin" files="F1">
<qstn responseDomainType="numeric">
<qstn responseDomainType="numeric" seqNo="1">
<qstnLit>Wie zufrieden sind Sie insgesamt? (0 = gar nicht, 10 = voll)</qstnLit>
</qstn>
<valrng>
<range min="0" max="10"/>
</valrng>
<concept>Wie zufrieden sind Sie insgesamt? (0 = gar nicht, 10 = voll)</concept>
<varFormat type="numeric" schema="other"/>
<notes type="cdl:parameters">step=1</notes>
</var>
</dataDscr>
</codeBook>
4 changes: 2 additions & 2 deletions registry/entities/select_multiple/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<concept>An welchen Tagen des Wochenendes sind Sie erreichbar?</concept>
</varGrp>
<var ID="V_wochenende_sa" name="wochenende_sa" intrvl="discrete" files="F1">
<qstn responseDomainType="multiple">
<qstn responseDomainType="multiple" seqNo="1">
<preQTxt>An welchen Tagen des Wochenendes sind Sie erreichbar?</preQTxt>
<qstnLit>Samstag</qstnLit>
</qstn>
Expand All @@ -40,7 +40,7 @@
<varFormat type="numeric" schema="other"/>
</var>
<var ID="V_wochenende_so" name="wochenende_so" intrvl="discrete" files="F1">
<qstn responseDomainType="multiple">
<qstn responseDomainType="multiple" seqNo="1">
<preQTxt>An welchen Tagen des Wochenendes sind Sie erreichbar?</preQTxt>
<qstnLit>Sonntag</qstnLit>
</qstn>
Expand Down
2 changes: 1 addition & 1 deletion registry/entities/select_multiple_long_list/ddi.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
</fileDscr>
<dataDscr>
<var ID="V_besuchtelaender" name="besuchtelaender" intrvl="discrete" files="F1">
<qstn responseDomainType="multiple">
<qstn responseDomainType="multiple" seqNo="1">
<qstnLit>Welche dieser Länder haben Sie bereits besucht? Mehrere Antworten möglich.</qstnLit>
</qstn>
<concept vocab="iso_3166_1">Welche dieser Länder haben Sie bereits besucht? Mehrere Antworten möglich.</concept>
Expand Down
Loading
Loading