Skip to content

[spark] Improve SQL UDF with body analysis and validation#8315

Open
Zouxxyy wants to merge 2 commits into
apache:masterfrom
Zouxxyy:xinyu/sql-udf-followup
Open

[spark] Improve SQL UDF with body analysis and validation#8315
Zouxxyy wants to merge 2 commits into
apache:masterfrom
Zouxxyy:xinyu/sql-udf-followup

Conversation

@Zouxxyy

@Zouxxyy Zouxxyy commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Purpose

Follow-up to #8299. Aligns Paimon SQL UDF creation with Spark's CreateSQLFunctionCommand:

  • Body analysis: move analyze/validate/derive from parser rule to CreatePaimonSQLFunctionCommand.run() (execution stage), mirroring Spark's approach
  • Validation: reject aggregate/window/generator in scalar function body, duplicate parameter names, non-trailing defaults, NOT NULL parameters, temp object references, invalid default expressions
  • Derivation: infer deterministic and containsSQL from the analyzed plan
  • Return type inference: support omitting RETURNS clause (Spark 4.1+)
  • Properties: capture SQL configs (timezone, ANSI mode) and catalog/namespace at creation time, restore at query time
  • Cleanup: remove redundant options (INPUT_PARAM, RETURN_TYPE), unify DEFAULT metadata key handling in SparkTypeUtils

Tests

  • SQL configs round-trip (ANSI mode preserved across sessions)
  • Non-deterministic function body (rand())
  • Reject aggregate/window/generator in scalar body
  • Reject duplicate parameter names
  • Reject non-trailing defaults
  • Omitting RETURNS clause (Spark 4.1+ only)

Zouxxyy and others added 2 commits June 22, 2026 13:53
…eturn type inference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant