Skip to content

fix(spec): reject unequal map default arrays - #3110

Open
mattfaltyn wants to merge 1 commit into
apache:mainfrom
mattfaltyn:fix/map-default-array-lengths
Open

fix(spec): reject unequal map default arrays#3110
mattfaltyn wants to merge 1 commit into
apache:mainfrom
mattfaltyn:fix/map-default-array-lengths

Conversation

@mattfaltyn

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

  • Reject map literals whose parallel keys and values arrays have different lengths instead of silently truncating them with Iterator::zip.
  • Propagate invalid field-default parse errors through NestedField deserialization so malformed metadata cannot be accepted as a missing default.
  • Cover both mismatch directions at the literal boundary and both initial-default and write-default at the schema boundary.

This matches the Java reference implementation's equal-length requirement for map literals.

Are these changes tested?

Yes.

  • cargo +stable test -p iceberg --lib spec::values::tests::json_map_rejects_mismatched_key_value_lengths -- --exact --nocapture
  • cargo +stable test -p iceberg --lib spec::datatypes::tests::nested_field_rejects_invalid_map_defaults -- --exact --nocapture
  • make check
  • make build
  • make unit-test
  • cargo nextest run --all-targets --all-features --workspace --no-fail-fast: 2,107 passed; the only 20 failures were HMS integration cases because the pinned amd64-only Hive 3.1.3 container did not bind port 9083 under ARM64 emulation on this host.
  • cargo nextest run --all-targets --all-features --workspace -E 'not (package(iceberg-catalog-hms) | test(/hms_catalog/))': 2,090/2,090 passed, including REST, Glue, S3Tables, SQL, DataFusion, Spark, S3, GCS, and Hugging Face integrations. HMS unit tests passed under make unit-test.

AI Disclosure

I used Codex to assist with repository inspection, implementation, test execution, and drafting this description. I reviewed the bug reproduction, diff, and test results before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(spec): unequal map default arrays are silently truncated

1 participant