[ENHANCEMENT](pyspark) Generalize field_path to arbitrary map/array nesting

packages/overture-schema-codegen/docs/design.md

@@ -317,18 +317,33 @@ Four dispatch mechanisms:
 ### Check Builder
 
 `pyspark/check_builder.py` walks `FieldSpec` trees to produce `Check` and `ModelCheck`
-IR. Resolves the mapping from nested field paths to PySpark array iteration patterns,
-producing a `FieldPath` (`ScalarPath` or `ArrayPath`) on each `Check`:
-
-- **Scalar field** -- `ScalarPath`; renders as `F.col("field")`
-- **Top-level array** -- `ArrayPath` with one `ArraySegment`; renders as
-  `array_check("field", lambda el: ...)`
-- **Field inside an array element** -- `ArrayPath` with struct navigation after the
-  array segment; renders as `array_check("array_col", lambda el: el["field"])`
-- **Nested array inside an array** -- `ArrayPath` with multiple `ArraySegment`s;
-  renders as `nested_array_check("outer", lambda el: array_check(el["inner"], ...))`
-- **Multiple nesting levels** -- chained `nested_array_check` with struct segments
-  navigating between array iterations
+IR. Resolves the mapping from nested field paths to PySpark iteration patterns,
+producing a `FieldPath` (`Direct` or `Iterated`) on each `Check`. A `Direct` locates
+a value reached with no iteration; an `Iterated` mixes struct segments with one or
+more iterating segments (`ArraySegment` for a `list`, `MapSegment` for a `dict[K,
+V]` projected to its keys or values); at render time each iterating segment --
+named or anonymous alike -- becomes its own render frame (one `array_check`/
+`map_*_check` call). A container nested directly inside another with no field
+name between them (`list[list[X]]`, `dict[K, list[X]]`, `list[dict]`, a map
+projected inside an array element, an array inside a map value) is an
+*anonymous* iterating segment (`is_anonymous`) -- its render frame carries no
+struct descent from the previous one. `promote_terminal` performs the entry
+into a container at the point the walker reaches it, replacing a struct
+terminal with a named iterating segment or appending an anonymous one when the
+terminal already iterates:
+
+- **Scalar field** -- `Direct`; renders as `F.col("field")`
+- **Top-level array or map** -- `Iterated` with one named iterating segment;
+  renders as `array_check("field", lambda el: ...)` or
+  `map_keys_check`/`map_values_check("field", lambda k_or_v: ...)`
+- **Field inside an array or map element** -- struct segments after the iterating
+  segment; renders as `array_check("col", lambda el: el["field"])`
+- **`list[list[X]]`, `dict[K, list[X]]`, a nested map, or a map projected inside
+  an array** -- an anonymous or named second iterating segment adds another
+  render frame, rendered by folding the flattening variant (`nested_array_check`,
+  `nested_map_{keys,values}_check`) around the inner render frame's helper
+- **Multiple nesting levels** -- chained flattening helpers with struct segments
+  navigating between render frames
 
 Union handling: variant-specific fields are annotated with `ColumnGuard` or
 `ElementGuard` discriminator gates. `Check.guards` is AND-composed at render time.

packages/overture-schema-codegen/src/overture/schema/codegen/pyspark/_render_common.py

@@ -31,7 +31,7 @@
 
 from jinja2 import Environment, FileSystemLoader
 
-from overture.schema.system.field_path import ArrayPath, MapProjection
+from overture.schema.system.field_path import Iterated, MapProjection
 
 from .check_ir import Check, Guard, ModelCheck
 from .constraint_dispatch import ForbidIf, RequireIf, model_constraint_function
@@ -77,21 +77,27 @@
     "check_struct_unique": "_unique",
 }
 
-_MAP_RUNTIME_HELPERS: dict[MapProjection, str] = {
-    MapProjection.KEY: "map_keys_check",
-    MapProjection.VALUE: "map_values_check",
+_MAP_RUNTIME_HELPERS: dict[tuple[MapProjection, bool], str] = {
+    (MapProjection.KEY, False): "map_keys_check",
+    (MapProjection.VALUE, False): "map_values_check",
+    (MapProjection.KEY, True): "nested_map_keys_check",
+    (MapProjection.VALUE, True): "nested_map_values_check",
 }
 
 
-def map_runtime_helper(projection: MapProjection) -> str:
+def map_runtime_helper(projection: MapProjection, *, flatten: bool = False) -> str:
     """Map a projection to its PySpark column-patterns helper name.
 
     `MapProjection.KEY` -> `map_keys_check`;
-    `MapProjection.VALUE` -> `map_values_check`. This is a pyspark-layer
+    `MapProjection.VALUE` -> `map_values_check`. When *flatten* is set (the
+    map holds further iteration, e.g. `dict[K, list]`, so the projected
+    element check returns an `array<string>`), the flattening variant is
+    named instead (`nested_map_keys_check` / `nested_map_values_check`) --
+    the map analogue of `nested_array_check`. This is a pyspark-layer
     concern; the mapping lives here rather than on `MapProjection` itself
     (a system-package enum) to avoid a layering violation.
     """
-    return _MAP_RUNTIME_HELPERS[projection]
+    return _MAP_RUNTIME_HELPERS[(projection, flatten)]
 
 
 _TEMPLATES_DIR = Path(__file__).parent / "templates"
@@ -209,20 +215,25 @@ def _model_check_base_label(check: ModelCheck) -> str:
       carries a single target field (multi-field decorators split at
       dispatch time).
     - Other kinds (`require_any_of`, `radio_group`, `min_fields_set`)
-      name the whole constraint; on `ArrayPath` targets they use the
+      name the whole constraint; on `Iterated` targets they use the
       path itself so anchors are distinguishable across nestings.
+
+    Every `Iterated` target (array, map, or mixed) uses the iterated
+    formula -- the anchor-disambiguation reason that motivates it for
+    arrays applies identically to maps and mixed paths. `Direct` targets
+    keep the row-root formula.
     """
     match check.descriptor:
         case RequireIf():
             kind_suffix = "_required"
         case ForbidIf():
             kind_suffix = "_forbidden"
         case _:
-            if isinstance(check.target, ArrayPath):
+            if isinstance(check.target, Iterated):
                 return str(check.target)
             return check_name(model_constraint_function(check.descriptor))
     target = check.descriptor.field_names[0]
-    if not isinstance(check.target, ArrayPath):
+    if not isinstance(check.target, Iterated):
         return f"{target}{kind_suffix}"
     return f"{check.target}.{target}{kind_suffix}"