Reflection: Report column nullability, and filter views by schema

[aminghadersohi]

Problem

Several reflection methods return wrong results against a live CrateDB (verified on 5.10.16 with SQLAlchemy 2.0.52):

• get_columns hard-codes nullable=True, so a primary key or NOT NULL column reflects as nullable. information_schema.columns.is_nullable reports the real value.
• get_view_names passes the schema as a parameter, but the query has no placeholder for it, so views of every schema are returned for any schema.
• has_table only checks base tables. SQLAlchemy 2.0's Inspector.has_table also reports views.
• get_view_definition is not implemented (NotImplementedError), although information_schema.views.view_definition exists.
• With crate://host/?schema=sales, get_table_names() lists tables from sales, but get_columns, get_pk_constraint and get_view_names query doc. A listed table then reflects no columns and no primary key.

Change

• get_columns selects is_nullable (ordered by ordinal_position) and returns it as nullable. CrateDB up to 6.0 reports is_nullable as a BOOLEAN, while 6.4 and later report 'YES'/'NO' text (where bool('NO') would be True), so both forms are accepted.
• get_view_names filters on table_schema = ?. get_view_definition reads information_schema.views.
• has_table makes one query against information_schema.tables with table_type IN ('BASE TABLE', 'VIEW'). CrateDB lists views there too, so the bulk tests' call counts stay the same.
• A _reflection_schema helper resolves the schema in this order: the explicit argument, then the URL schema parameter, then the default. The reflection methods above use it.

Tests

• Unit tests for nullability (boolean and 'YES'/'NO' forms), view listing and definition, has_table on a view, and URL-schema resolution for get_columns, get_pk_constraint and get_view_names. test_get_view_names and test_has_table now expect the new SQL. The mocked get_columns cursor describes all three selected columns, and the URL-schema tests use crate:///?schema=sales, so they also pass on SQLAlchemy 1.3 and 1.4.
• tests/reflection_test.py: a live test (testcontainers) that a primary key and a NOT NULL column reflect as not nullable.
• The CI workflow's steps, run locally against CrateDB nightly (6.5.0): poe lint is clean and integration.py passes. pytest gives 209 passed on SQLAlchemy 2.1.1, 195 passed on 1.4.54, and 132 passed on 1.3.24. On 1.3 there is one failure, test_schema.py::test_correct_schema, which fails the same way on main.
• I also ran live checks against a CrateDB 5.10.16 container. Before the change, each check above failed. After it, each one passes: nullability, schema-filtered views, has_table on a view, the view definition, and URL-schema column/PK reflection.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 84ffd466-e35a-41f5-b329-ccfc28721b15

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}