Skip to content

Add EMR Serverless interactive session support to Amazon provider#69207

Open
vgkowski wants to merge 1 commit into
apache:mainfrom
vgkowski:amazon-emr-serverless-interactive-sessions
Open

Add EMR Serverless interactive session support to Amazon provider#69207
vgkowski wants to merge 1 commit into
apache:mainfrom
vgkowski:amazon-emr-serverless-interactive-sessions

Conversation

@vgkowski

@vgkowski vgkowski commented Jul 1, 2026

Copy link
Copy Markdown

Add operators, a sensor, and triggers to manage the lifecycle of EMR Serverless
interactive (Spark Connect) sessions: start a session, wait until it is ready,
fetch the Spark Connect endpoint, and terminate it. The start/stop operators and
the sensor support deferrable mode. GetSessionEndpoint registers the short-lived
auth token with the secrets masker. Includes waiters, docs, a system-test example
DAG, and unit tests.

Requires botocore 1.42.94+.

Was generative AI tooling used to co-author this PR?
  • Yes — Kiro (AI coding assistant)

Generated-by: Kiro, following the Gen-AI guidelines in contributing-docs/05_pull_requests.rst

@vgkowski vgkowski force-pushed the amazon-emr-serverless-interactive-sessions branch from f2bf4d2 to 0da374b Compare July 1, 2026 19:05
Add operators, sensor and triggers to manage the lifecycle of EMR Serverless interactive (Spark Connect) sessions: start, wait, fetch the Spark Connect endpoint, and terminate. Start/Stop operators and the sensor support deferrable mode. The GetSessionEndpoint operator masks the short-lived auth token. Includes waiters, docs, a system-test example DAG, and unit tests. Requires botocore 1.43.0+ (declared separately).

Document botocore/aiobotocore requirements for EMR Serverless sessions

Interactive sessions need botocore 1.43.0+ (and aiobotocore 3.6.0+ for deferrable mode) plus EMR 7.13+. Note this in the docs instead of raising the provider's dependency floor, so users who do not use interactive sessions are not forced onto newer AWS SDK versions.
@vgkowski vgkowski force-pushed the amazon-emr-serverless-interactive-sessions branch from 0da374b to f5b43ec Compare July 1, 2026 19:06
@vgkowski vgkowski marked this pull request as ready for review July 1, 2026 19:08
@vgkowski vgkowski requested a review from o-nikolas as a code owner July 1, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant