Skip to content

PERF: Reduce Python Row indexing overhead - #822

Merged
Jahnvi Thakkar (jahnvi480) merged 2 commits into
mainfrom
jahnvi/python-row-overhead
Sep 28, 2026
Merged

Jahnvi Thakkar (jahnvi480) merged 2 commits into
mainfrom
jahnvi/python-row-overhead

Conversation

@jahnvi480

@jahnvi480 Jahnvi Thakkar (jahnvi480) commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

AB#48366

Summary

Add a two-line fast path in Row.__getitem__ for indexes whose exact type is int, returning the stored value directly. Keep the existing fallback order unchanged for booleans, integer subclasses, slices and column-name access.

Only mssql_python/row.py changes. No public API, native code, tests, build configuration or CI changes are included.

Fast-path exact built-in integer indexes while preserving the existing ordered fallback for other index types.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 28, 2026 04:04
@github-actions github-actions Bot added pr-size: small Minimal code update labels Sep 28, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task Before After Paired change Result
Connection opening 10.390 ms 10.356 ms -0.3% no signal
SELECT queries 1.058 ms 1.237 ms +16.6% no signal
Row insertion 34.410 ms 35.246 ms +1.8% no signal
Executemany inserts 154.786 ms 156.690 ms +1.4% no signal
Fetch-all queries 120.472 ms 121.386 ms +0.8% no signal
Row-by-row fetching 14.473 ms 14.337 ms -0.9% no signal
Batched row fetching 120.258 ms 118.015 ms -1.5% no signal
Transaction commit and rollback 113.345 ms 113.171 ms -2.3% no signal
Arrow row fetching 94.995 ms 95.563 ms -0.2% no signal
100,000-row insertion 442.666 ms 447.702 ms +1.1% no signal
Row fetching in batches of 100 121.265 ms 121.803 ms +0.8% no signal
Row fetching in batches of 10,000 125.235 ms 138.390 ms +10.5% no signal
Repeated positional queries 33.676 ms 34.124 ms -0.5% no signal
Repeated named-parameter queries 36.655 ms 35.539 ms -3.0% no signal
Legacy 100,000-row insertion 349.230 ms 353.975 ms +1.6% no signal
Insertion with explicit input sizes 483.134 ms 482.317 ms +0.5% no signal
Joined aggregation queries 177.425 ms 177.651 ms +0.1% no signal
Large joined-result fetching 180.416 ms 180.188 ms +0.2% no signal
1.2-million-row fetching 3470.772 ms 3527.123 ms +1.2% no signal
Common table expression queries 5.364 ms 5.438 ms +1.0% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.325 ms 1.220 ms -2.9% no signal

Unix / SQL Server 2025

Database task Before After Paired change Result
Connection opening 97.901 ms 97.747 ms +0.3% no signal
SELECT queries 1.091 ms 1.083 ms +0.3% no signal
Row insertion 34.731 ms 34.612 ms -0.1% no signal
Executemany inserts 150.908 ms 151.824 ms +0.1% no signal
Fetch-all queries 124.663 ms 123.074 ms -1.0% no signal
Row-by-row fetching 15.224 ms 14.454 ms -4.2% no signal
Batched row fetching 121.410 ms 119.374 ms -1.4% no signal
Transaction commit and rollback 115.368 ms 115.483 ms +0.1% no signal
Arrow row fetching 95.038 ms 95.323 ms +0.5% no signal
100,000-row insertion 479.485 ms 448.188 ms -5.1% no signal
Row fetching in batches of 100 124.230 ms 123.274 ms -0.5% no signal
Row fetching in batches of 10,000 130.075 ms 129.763 ms -0.3% no signal
Repeated positional queries 34.012 ms 33.517 ms -1.5% no signal
Repeated named-parameter queries 36.468 ms 36.141 ms +0.4% no signal
Legacy 100,000-row insertion 355.478 ms 359.343 ms -0.6% no signal
Insertion with explicit input sizes 487.333 ms 492.532 ms +1.1% no signal
Joined aggregation queries 161.264 ms 161.150 ms +0.2% no signal
Large joined-result fetching 189.173 ms 186.827 ms -1.2% no signal
1.2-million-row fetching 3502.387 ms 3479.956 ms -0.7% no signal
Common table expression queries 5.185 ms 5.270 ms +1.1% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.539 ms 1.555 ms -0.3% no signal
Build and measurement details

ADO build 178639

PR head: 60519f5825091f97a85d26c6edfd14391c1625df
Base: d849a09119c831d945521d5333cfb9bdb6e0561d
Measured merge: ff086676e1ff2bccf41bf4383c9555d5013189da

  • Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4295.3; 5 paired comparisons and 1 warmup.
  • Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5005.3; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Add regression coverage for boolean and integer-subclass indexing and provide current-main performance validation.

Review effort: Lite
Findings: None

What changed in this PR

Adds an exact-int fast path to Row.__getitem__ to reduce positional indexing overhead while preserving fallback behavior.

Changes:

  • Directly returns stored values for exact integer indexes.
  • Retains existing handling for booleans, integer subclasses, slices, and column names.
  • Adds no API, native, build, or CI changes.
File Description
mssql_python/​row.py Adds the positional integer indexing fast path.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

84%


📈 Total Lines Covered: 9402 out of 11085
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

  • mssql_python/row.py (100%)

Summary

  • Total: 2 lines
  • Missing: 0 lines
  • Coverage: 100%

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 79.3%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

@jahnvi480
Jahnvi Thakkar (jahnvi480) marked this pull request as ready for review September 28, 2026 04:22
Copilot AI review requested due to automatic review settings September 28, 2026 04:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The remaining test-coverage comment is a non-blocking nit.

Review effort: Lite
Findings: 1 Low severity

Open (1)

Comment thread mssql_python/row.py
@jahnvi480
Jahnvi Thakkar (jahnvi480) merged commit fead15c into main Sep 28, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: small Minimal code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants