Repository navigation
[BUG](pyspark) Honor string min_length in base-row generation - #578
Merged
Merged
Conversation
Seth Fitzsimmons (sethfitz)
temporarily deployed
to
staging
July 21, 2026 03:31 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
🗺️ Schema reference docs preview is live!
Note ♻️ This preview updates automatically with each push to this PR. |
Seth Fitzsimmons (sethfitz)
marked this pull request as draft
July 21, 2026 03:34
Seth Fitzsimmons (sethfitz)
marked this pull request as ready for review
July 21, 2026 04:16
Victor Schappert (vcschapp)
requested review from
a team and
John McCall (lowlydba)
as code owners
July 21, 2026 17:31
Victor Schappert (vcschapp)
previously approved these changes
Jul 21, 2026
_value_from_check_string_min_length ignored the descriptor's min_length and always returned a single character, so a string field with min_length > 1 would get a base/valid row value that already violates the constraint -- making the generated conformance ::valid row invalid and the ::invalid scenario's baseline wrong. Read the min_length bound from the descriptor and emit a filler of that length. Latent today (no live schema field declares string min_length > 1); surfaced while working the generalize-field-path branch (bd-swy2). Fixes bd-mwy8. Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Seth Fitzsimmons (sethfitz)
dismissed
Victor Schappert (vcschapp)’s stale review
July 21, 2026 18:05
The merge-base changed after approval.
Seth Fitzsimmons (sethfitz)
force-pushed
the
base-row-string-min-length
branch
from
July 21, 2026 18:05
11690ca to
75a34d0
Compare
Victor Schappert (vcschapp)
approved these changes
Jul 22, 2026
Victor Schappert (vcschapp)
left a comment
Victor Schappert (vcschapp)
left a comment
Collaborator
There was a problem hiding this comment.
Re-approved and I got the PR check green by editing the description with "partly closes #517".
Roel Bollens (RoelBollens-TomTom)
self-requested a review
July 23, 2026 15:32
Roel Bollens (RoelBollens-TomTom)
approved these changes
Jul 23, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_value_from_check_string_min_length(base-row test-data synthesis) ignored the descriptor'smin_lengthand always returned a single character. For any string field withmin_length > 1, the synthesized base/valid row value ('a', length 1) already violates the constraint -- so the generated conformance::validrow would be invalid and the::invalidscenario's baseline wrong.The fix reads the
min_lengthbound from the descriptor and emits a filler of that length ('a' * min_length).Scope
Latent on
maintoday: no live schema field declares a stringmin_length > 1, so the conformance suite stays green and generated output is unchanged. The defect surfaced while working the generalize-field-path branch, where a mixed-shape test had to usemin_length=1rather than3to keep its valid row valid. Genuinely orthogonal to that branch -- it does not mask a defect in #570.The analogous array min-length synthesizer (
_min_length_from_shape_constraints) already readsmin_lengthcorrectly and generates that many elements; string max-length and array max-length are not lower bounds and need no base-row value. The string min-length synthesizer was the sole offender.Part of and partly closes #517.