Store NUMERIC columns as NUMERIC

[florinutz]

sa.Numeric(10, 2) created a BIGINT column. Decimal("1.25") went in and Decimal("1.00") came back, with no error at any layer — SQLAlchemy re-applies the declared scale on read, so the truncated integer returned looking like a plausible decimal.

amount = sa.Column(sa.Numeric(10, 2))     # -> amount LONG
session.add(Ledger(amount=Decimal("1.25")))
...
session.query(Ledger).one().amount        # Decimal('1.00')

Closes #292. Also delivers the write half of #163 — see the note on reads below.

Root cause

Two constants in CrateTypeCompiler, from 2018 when CrateDB had no NUMERIC and mapping elsewhere was the only way to make the column creatable:

def visit_DECIMAL(self, type_, **kw):  return "DOUBLE"
def visit_NUMERIC(self, type_, **kw):  return "LONG"

Both discard precision and scale. CrateDB gained NUMERIC as a cast target in 4.4.0 and storage for it in 5.9.0; the mapping was never revisited, and docs/data-types.rst already documents NUMERIC as the intended mapping.

The dialect also leaves supports_native_decimal at False, so SQLAlchemy binds every Decimal through processors.to_float before the driver sees it. On a NUMERIC(38, 24) column:

bound as Decimal -> stored 1.234567890123456789012345
bound as float   -> stored 1.234567890123456700000000

Fix

Render NUMERIC(precision, scale) in DDL and casts. NUMERIC and DECIMAL are the same type in SQLAlchemy and in CrateDB, so visit_DECIMAL defers to visit_NUMERIC.

Bind Decimal values unconverted, via a Numeric entry in colspecs; the driver already serializes them as strings, so CrateDB stores every digit. This stops short of supports_native_decimal = True: that flag would also tell SQLAlchemy to skip result conversion, and since this driver returns floats, Numeric(asdecimal=True) would hand back floats.

Two points worth a maintainer's opinion:

• Declaring the type without a precision now raises CompileError: CrateDB rejects a bare NUMERIC column, so there is nothing correct to emit. Casts may still omit it, since the server accepts CAST(x AS NUMERIC). Emitting bare NUMERIC would only move the refusal to execution time, and keeping LONG would leave exactly this bug in place for anyone writing sa.Numeric().
• This makes CrateDB 5.9 the effective minimum for Numeric columns; older servers get a working (if lossy) BIGINT today and would start seeing server errors. The repository documents no supported-version floor, so I stated the requirement in CHANGES.md and the type map instead. Say the word if you'd rather it degraded on older servers.

Reads are unchanged: responses are parsed as JSON, whose numbers become floats, so a returned value carries at most a double's digits. The server sends the full value, so this is fixable in crate-python rather than here; crate/crate-python#826 carries the measurement.

Tests

• tests/numeric_test.py — DDL and cast rendering for all three spellings, precision without scale, a live round-trip through each spelling, and a live write of a value with more digits than a double holds.
• tests/create_table_test.py — the basic-types table now declares precision on its exact-numeric columns and expects NUMERIC(10, 2).

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 4fe87b30-cb56-4e0b-8ae1-9cdeba64a406

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[bgunebakan]

Hi @florinutz, Could you fix conflicts?

[bgunebakan]

Thanks, Added some comments.

[bgunebakan]

get_columns selects only column_name, data_type, so a reflected column arrives with no precision. That gap was exist before this PR, but it was harmless while NUMERIC mapped to LONG, now that a precision-less NUMERIC raise a CompileError.

[bgunebakan]

Thanks! I checked my review comments with latest changes, and checked my get_columns comment with real crate container. it raises CompileError: CrateDB stores a NUMERIC column only with a precision now. That was already broken on main, so it didn't introduced by this PR. I think would be nice to have follow up issue for that to fully implement, if you don't prefer to fix here.

[florinutz]

already broken on main

indeed. Reported the issue.