fix: validate ComputeModelStatistics input columns by ranadeepsingh · Pull Request #2633 · microsoft/SynapseML

Rana Singh (ranadeepsingh) · 2026-08-14T22:17:59Z

Related Issues/PRs

Fixes #736
Depends on merged #2635 and #2632 for release-replay baselines

What changes are proposed in this pull request?

Resolve and validate label, scored-label, and score columns before metadata or Catalyst access.
Reject missing, duplicate/ambiguous, empty, and unsupported scalar/vector inputs with actionable parameter-specific errors.
Preserve explicit column overrides when SynapseML scoring metadata is present or incomplete.
Support numeric scalar classification scores plus dense/sparse Spark ML vectors; reject vectors without a positive-class slot.
Ignore null metric rows, safely quote top-level Spark column names, and remove obsolete unbounded input caching.
Align specific regression metric runtime columns with transformSchema.
Preserve configured params through copy, Pipeline fitting, save/load, codegen, and generated Python wrappers.
Keep pipeline.yaml unchanged and replay merged feat(core): add precision-recall AUC metric #2635 followed by fix: resolve current model metadata deterministically #2632 on Spark 4.1, matching target history.

How was this patch tested?

JDK 11 / Spark 3.5: ComputeModelStatisticsValidationSuite 8/8, VerifyComputeModelStatistics 32/32, and VerifyComputePerInstanceStatistics 5/5.
JDK 11: core/compile, core/Test/compile, core/scalastyle, and core/Test/scalastyle passed.
Python: Black 22.3.0 checked 189 files; sbt codegen passed; the generated wrapper passed AST and py_compile checks.
CI scripts: tools/ci/tests/test_pipeline_yaml.py 61/61; the amended prerequisite config format test passed on the final head.
Spark 4.1.1 / Scala 2.13.17 / JDK 17: ordered prerequisites and the final release-relevant patch produced exact expected blobs; full test:compile passed and the focused suite passed 8/8.
Azure build 231493934: succeeded, 66/66 jobs, reason=pullRequest, including Spark 4.1 compatibility and CPU/GPU/Fabric E2E.
Current-head Copilot review covered 6/6 files with 0 new comments; all review threads are resolved and no suppressed findings exist.

Compatibility

No public JVM signatures or serialized parameter shapes were removed or changed. ComputeModelStatistics.copy(ParamMap) retains its Transformer return type and now uses defaultCopy. Generated Python parameter APIs are unchanged.

azure-pipelines · 2026-08-14T22:18:08Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

github-actions · 2026-08-14T22:18:09Z

Hey Rana Singh (@ranadeepsingh) 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

fix: Fix LightGBM crashes with empty partitions
feat: Make HTTP on Spark back-offs configurable
docs: Update Spark Serving usage
build: Add codecov support
perf: improve LightGBM memory usage
refactor: make python code generation rely on classes
style: Remove nulls from CNTKModel
test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Rana Singh (ranadeepsingh) · 2026-08-14T22:18:22Z

/azp run

azure-pipelines · 2026-08-14T22:18:35Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Pull request overview

This PR improves ComputeModelStatistics robustness when scoring metadata is missing by adding explicit input-column validation (label/scored-labels/scores) and expanding tests and docs to cover unannotated DataFrame scenarios, addressing the historical NullPointerException reported in #736.

Changes:

Add schema-based validation to fail fast with actionable IllegalArgumentException messages instead of late Spark expression failures.
Expand VerifyComputeModelStatistics coverage for missing/incorrect configured columns and for classification/regression behaviors without scoring metadata.
Document how labelCol, scoredLabelsCol, and scoresCol are inferred/required for unannotated DataFrames.

Show a summary per file

File	Description
core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.scala	Adds `validateInputColumns` and invokes it before metric computation paths to improve failure modes for missing columns.
core/src/test/scala/com/microsoft/azure/synapse/ml/train/VerifyComputeModelStatistics.scala	Adds regression tests for missing columns and metadata-less DataFrame configurations.
core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.txt	Updates user-facing documentation clarifying column roles and configuration when metadata inference is unavailable.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Files reviewed: 3/3 changed files
Comments generated: 1
Review effort level: Lite

Rana Singh (ranadeepsingh) · 2026-08-14T23:03:38Z

/azp run

azure-pipelines · 2026-08-14T23:03:50Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Files reviewed: 3/3 changed files
Comments generated: 0 new
Review effort level: Lite

Rana Singh (ranadeepsingh) · 2026-08-14T23:06:54Z

/azp run

azure-pipelines · 2026-08-14T23:07:07Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Rana Singh (ranadeepsingh) · 2026-08-15T00:25:55Z

/azp run

azure-pipelines · 2026-08-15T00:26:07Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Codecov Comments Bot (codecov-commenter) · 2026-08-15T01:22:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.40%. Comparing base (60cf93c) to head (dd9cbeb).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2633      +/-   ##
==========================================
- Coverage   88.60%   88.40%   -0.20%     
==========================================
  Files         331      331              
  Lines       18531    18557      +26     
  Branches     1752     1751       -1     
==========================================
- Hits        16419    16405      -14     
- Misses       2112     2152      +40

Files with missing lines	Coverage Δ
...zure/synapse/ml/train/ComputeModelStatistics.scala	`88.88% <100.00%> (+3.46%)`	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

## Summary Replace null and late Spark failures with schema-first column resolution, ambiguity and type checks, metadata-safe overrides, scalar/vector handling, regression schema agreement, and persisted pipeline coverage. Preserve the precision-recall AUC behavior merged in PR microsoft#2635 and add its Spark 4.1 replay prerequisite. ## Prompting Intent Make microsoft#2633 merge-ready by auditing ComputeModelStatistics across missing and ambiguous columns, classification versus regression, Spark metadata, scalar and vector scores, null rows, schema/runtime behavior, copying, persistence, code generation, Python wrappers, performance, and Spark 4.1 compatibility. Rebase on current master, resolve all valid review feedback, and add focused negative and end-to-end tests. ## Linked Sources - GitHub issue microsoft#736: microsoft#736 - Pull request microsoft#2633: microsoft#2633 - Merged prerequisite PR microsoft#2635: microsoft#2635 - Copilot review thread: microsoft#2633 (comment) - SynapseML review checklist: .github/skills/code-review/SKILL.md ## Rationale Validate resolved columns before any metadata or Catalyst access, preserve explicit parameter precedence over metadata, and accept only runtime-supported numeric/vector shapes so transformSchema and transform fail consistently. Cast numeric scalar inputs to doubles, quote top-level Spark column names, remove obsolete unbounded DataFrame caching, retain binary AUC/areaUnderPR semantics from PR microsoft#2635, and use defaultCopy so configured stages survive pipeline copy and persistence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rana Singh (ranadeepsingh) · 2026-08-16T09:51:56Z

/azp run

azure-pipelines · 2026-08-16T09:52:09Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Files reviewed: 6/6 changed files
Comments generated: 1
Review effort level: Lite

## Summary Replace null and late Spark failures with schema-first column resolution, ambiguity and type checks, metadata-safe overrides, scalar/vector handling, regression schema agreement, and persisted pipeline coverage. Preserve the precision-recall AUC behavior merged in PR microsoft#2635 and replay its scoped test baseline before the dependent Spark 4.1 prerequisite. ## Prompting Intent Make microsoft#2633 merge-ready by auditing ComputeModelStatistics across missing and ambiguous columns, classification versus regression, Spark metadata, scalar and vector scores, null rows, schema/runtime behavior, copying, persistence, code generation, Python wrappers, performance, and Spark 4.1 compatibility. Rebase on current master, resolve all valid review feedback, and add focused negative and end-to-end tests. ## Linked Sources - GitHub issue microsoft#736: microsoft#736 - Pull request microsoft#2633: microsoft#2633 - Merged prerequisite PR microsoft#2635: microsoft#2635 - Failed Azure compatibility build: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=231383604 - Copilot review thread: microsoft#2633 (comment) - SynapseML review checklist: .github/skills/code-review/SKILL.md ## Rationale Validate resolved columns before metadata or Catalyst access, preserve explicit parameter precedence, and accept only runtime-supported numeric/vector shapes so transformSchema and transform fail consistently. Cast numeric scalar inputs to doubles, quote top-level Spark names, remove obsolete unbounded caching, retain PR microsoft#2635 AUC semantics, and use defaultCopy for persisted stages. Teach release replay to activate add-only scoped baselines required by later prerequisites and validate each baseline against that dependent commit parent, avoiding broad replay of PR microsoft#2507. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rana Singh (ranadeepsingh) · 2026-08-16T10:24:47Z

/azp run

azure-pipelines · 2026-08-16T10:25:00Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Files reviewed: 8/8 changed files
Comments generated: 1
Review effort level: Lite

## Summary Replace null and late Spark failures with schema-first column resolution, ambiguity and type checks, metadata-safe overrides, scalar/vector handling, regression schema agreement, actionable metric validation, and persisted pipeline coverage. Preserve the precision-recall AUC behavior merged in PR microsoft#2635 and replay its scoped test baseline before the dependent Spark 4.1 prerequisite. ## Prompting Intent Make microsoft#2633 merge-ready by auditing ComputeModelStatistics across missing and ambiguous columns, classification versus regression, Spark metadata, scalar and vector scores, null rows, schema/runtime behavior, copying, persistence, code generation, Python wrappers, performance, and Spark 4.1 compatibility. Rebase on current master, resolve all valid review feedback, and add focused negative and end-to-end tests. ## Linked Sources - GitHub issue microsoft#736: microsoft#736 - Pull request microsoft#2633: microsoft#2633 - Merged prerequisite PR microsoft#2635: microsoft#2635 - Failed Azure compatibility build: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=231383604 - Copilot review threads: microsoft#2633 (comment) and microsoft#2633 (comment) - SynapseML review checklist: .github/skills/code-review/SKILL.md ## Rationale Validate resolved columns before metadata or Catalyst access, preserve explicit parameter precedence, and accept only runtime-supported numeric/vector shapes so transformSchema and transform fail consistently. Cast numeric scalar inputs to doubles, quote top-level Spark names, remove obsolete unbounded caching, retain PR microsoft#2635 AUC semantics, use defaultCopy for persisted stages, and report invalid metrics as IllegalArgumentException with valid choices. Teach release replay to activate add-only scoped baselines required by later prerequisites and validate each baseline against that dependent commit parent, avoiding broad replay of PR microsoft#2507. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Review details

Files reviewed: 8/8 changed files
Comments generated: 0 new
Review effort level: Lite

## Summary Replace null and late Spark failures with schema-first column resolution, ambiguity and type checks, metadata-safe overrides, scalar/vector handling, regression schema agreement, actionable metric validation, Spark-configured case resolution, and persisted pipeline coverage. Preserve the precision-recall AUC behavior merged in PR microsoft#2635 and replay its scoped test baseline before the dependent Spark 4.1 prerequisite. ## Prompting Intent Make microsoft#2633 merge-ready by auditing ComputeModelStatistics across missing and ambiguous columns, classification versus regression, Spark metadata, scalar and vector scores, null rows, schema/runtime behavior, copying, persistence, code generation, Python wrappers, performance, and Spark 4.1 compatibility. Rebase on current master, resolve all valid review feedback, and add focused negative and end-to-end tests. ## Linked Sources - GitHub issue microsoft#736: microsoft#736 - Pull request microsoft#2633: microsoft#2633 - Merged prerequisite PR microsoft#2635: microsoft#2635 - Failed Azure compatibility build: https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_build/results?buildId=231383604 - Copilot review threads: microsoft#2633 (comment), microsoft#2633 (comment), and microsoft#2633 (comment) - SynapseML review checklist: .github/skills/code-review/SKILL.md ## Rationale Validate and canonicalize resolved columns before metadata or Catalyst access, using Spark SQLConf resolver semantics so configured case sensitivity is honored. Preserve explicit parameter precedence and accept only runtime-supported numeric/vector shapes so transformSchema and transform fail consistently. Cast numeric scalar inputs to doubles, quote top-level Spark names, remove obsolete unbounded caching, retain PR microsoft#2635 AUC semantics, use defaultCopy for persisted stages, and report invalid metrics as IllegalArgumentException with valid choices. Teach release replay to activate add-only scoped baselines required by later prerequisites and validate each baseline against that dependent commit parent, avoiding broad replay of PR microsoft#2507. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rana Singh (ranadeepsingh) · 2026-08-16T10:48:17Z

/azp run

azure-pipelines · 2026-08-16T10:48:29Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Files reviewed: 8/8 changed files
Comments generated: 0 new
Review effort level: Lite

GitHub#736 ## Summary Validate ComputeModelStatistics inputs before Spark metadata or Catalyst access, align regression schemas with runtime output, add focused public-path coverage, and replay the target metadata baseline during Spark 4.1 compatibility validation. ## Prompting Intent Make microsoft#2633 engineering-ready on master without broadening sensitive CI behavior. Preserve public JVM and generated Python APIs while validating label, prediction, and score columns; actionable error timing and messages; scalar and vector scores; null and boundary inputs; copy and persistence behavior; and Spark 4.1 release compatibility. ## Linked Sources - GitHub issue microsoft#736: microsoft#736 - Pull request microsoft#2633: microsoft#2633 - Merged prerequisite PR microsoft#2635: microsoft#2635 - Related metadata PR microsoft#2632: microsoft#2632 - Copilot review threads: microsoft#2633 (comment), microsoft#2633 (comment), microsoft#2633 (comment) - SynapseML contributor guidance: AGENTS.md - SynapseML review guidance: .github/skills/code-review/SKILL.md ## Rationale Centralize schema-first validation in a private helper so transform and transformSchema share column resolution, case-sensitivity, ambiguity, and type rules without exceeding the repository source-size limit. Preserve explicit parameter precedence, existing serialized Params and generated wrapper shape, and PR microsoft#2635 metric behavior. Accept only runtime-supported numeric scalars and Spark vectors, quote top-level column names, drop null metric rows, and use defaultCopy for persistence. Keep pipeline.yaml unchanged; add only the missing PR microsoft#2632 prerequisite after PR microsoft#2635 because Spark 4.1 lacks both target baselines and the final patch cannot apply until they are replayed in target-history order. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rana Singh (ranadeepsingh) · 2026-08-17T12:05:44Z

/azp run

azure-pipelines · 2026-08-17T12:05:58Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Files reviewed: 6/6 changed files
Comments generated: 0 new
Review effort level: Lite

Rana Singh (ranadeepsingh) requested a review from Brendan Walsh (BrendanWalsh) as a code owner August 14, 2026 22:18

Copilot AI lite review requested due to automatic review settings August 14, 2026 22:18

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 14, 2026 22:18 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

Comment thread core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.scala

Copilot AI review requested due to automatic review settings August 14, 2026 23:02

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from 67f5cd4 to 1d83994 Compare August 14, 2026 23:02

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 14, 2026 23:03 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from 1d83994 to dd9cbeb Compare August 14, 2026 23:06

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from dd9cbeb to 60739cc Compare August 16, 2026 09:51

Copilot AI review requested due to automatic review settings August 16, 2026 09:51

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 16, 2026 09:52 View session

Copilot AI reviewed Aug 16, 2026

View reviewed changes

Comment thread core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.scala Outdated

Copilot AI review requested due to automatic review settings August 16, 2026 10:24

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from 60739cc to 506f79a Compare August 16, 2026 10:24

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 16, 2026 10:25 View session

Copilot AI reviewed Aug 16, 2026

View reviewed changes

Comment thread core/src/main/scala/com/microsoft/azure/synapse/ml/train/ComputeModelStatistics.scala Outdated

Copilot AI review requested due to automatic review settings August 16, 2026 10:39

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from 506f79a to cd04062 Compare August 16, 2026 10:39

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 16, 2026 10:40 View session

Copilot AI reviewed Aug 16, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 16, 2026 10:47

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from cd04062 to d82903e Compare August 16, 2026 10:47

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 16, 2026 10:48 View session

Copilot AI reviewed Aug 16, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 17, 2026 12:05

Rana Singh (ranadeepsingh) force-pushed the copilot/issue-736-metric-errors branch from d82903e to 45352dc Compare August 17, 2026 12:05

Copilot started reviewing on behalf of Rana Singh (ranadeepsingh) August 17, 2026 12:08 View session

Copilot AI reviewed Aug 17, 2026

View reviewed changes

Uh oh!

Conversation

Rana Singh (ranadeepsingh) commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues/PRs

What changes are proposed in this pull request?

How was this patch tested?

Compatibility

Uh oh!

azure-pipelines Bot commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 14, 2026

Uh oh!

azure-pipelines Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 14, 2026

Uh oh!

azure-pipelines Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 14, 2026

Uh oh!

azure-pipelines Bot commented Aug 14, 2026

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 15, 2026

Uh oh!

azure-pipelines Bot commented Aug 15, 2026

Uh oh!

Codecov Comments Bot (codecov-commenter) commented Aug 15, 2026

Codecov Report

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 16, 2026

Uh oh!

azure-pipelines Bot commented Aug 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 16, 2026

Uh oh!

azure-pipelines Bot commented Aug 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 16, 2026

Uh oh!

azure-pipelines Bot commented Aug 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 17, 2026

Uh oh!

azure-pipelines Bot commented Aug 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Rana Singh (ranadeepsingh) commented Aug 14, 2026 •

edited

Loading