Skip to content

Catalog attachment is unreachable from HotdataClient, and the database record discards half the detail response #81

Description

@rohan-hotdata

Two gaps in the same area, both measured against hotdata-framework 0.13.0 with
hotdata 0.9.0. Neither needs a platform change — the endpoints and the fields already
exist and work.

Attaching a registered connection into a managed database is how one query reads across a
database boundary: the attached source's tables become addressable as
<alias>.<schema>.<table> alongside the database's own. That is reachable on the generated
SDK and nowhere on the client.

1. attach_database_catalog / detach_database_catalog are not on the client

>>> [m for m in dir(HotdataClient) if "attach" in m or "catalog" in m]
[]
>>> [m for m in dir(DatabasesApi) if "attach" in m]
['attach_database_catalog', 'detach_database_catalog', ...]

So a consumer wanting a multi-catalog database has to drop to
DatabasesApi(client.api).attach_database_catalog(...). That works, and client.api is a
public property, so it is a legitimate escape hatch rather than a hack — but it means every
downstream package reimplements the same call and its error handling. hotdata-langchain
now does this in two places, which is the signal that the layer below is missing something
rather than that the hatch is being used well.

Suggested shape, following add_managed_table's existing conventions:

def attach_database_catalog(
    self, database: str | ManagedDatabase, *, connection_id: str, alias: str | None = None
) -> None: ...

def detach_database_catalog(
    self, database: str | ManagedDatabase, *, connection_id: str
) -> None: ...

Both endpoints answer 204 with no body, and map 400/404 (plus 409 on attach) to
ApiErrorResponse, so api_error_message handling is the same as everywhere else.

2. managed_database_from_detail drops four fields that are on the response

def managed_database_from_detail(detail: Any) -> ManagedDatabase:
    return ManagedDatabase(
        id=str(detail.id),
        description=detail.name,
        default_connection_id=str(detail.default_connection_id),
    )

DatabaseDetailResponse carries eight fields. Four never survive:

Field Consequence of dropping it
attachments a caller holding a resolved ManagedDatabase cannot ask what is attached to it, so the result of an attach is unverifiable without a second raw call
default_catalog least costly of the four — it reports default whether or not anything answers to that name, so a consumer reads table_catalog from information_schema instead
default_schema mild; callers currently pass a schema they already know
expires_at a database created with a TTL cannot be asked when it expires, so reaping is unobservable from the client

attachments and expires_at are the two that cost something concrete. expires_at is
newly load-bearing now that create_managed_database(expires_at=...) exists: the client can
set a lifetime and then cannot read it back.

This is the more disruptive of the two changes, since ManagedDatabase is in the public
surface. Adding fields with defaults keeps it compatible; if the dataclass should stay
minimal, a separate managed_database_detail() returning the full record would work as well.

Why this matters beyond one consumer

hotdata-langchain, hotdata-ibis and hotdata-dlt-destination all speak to the platform
through this client, and any of them wanting cross-catalog reads has the same gap. A
consumer whose provisioning is ordinary Python — not a tool layer — has no reason to depend
on a LangChain integration package to attach a data source, which is where this currently
leads.

What is not being asked

Attaching one managed database into another. The platform refuses it
(Connection '<id>' is scoped to another database and cannot be attached here), so that is a
product question, not a wrapping one. A registered data source attaches as designed, which is
what this issue is about.

Separately, and only noted in case someone is in here: fork_database is on DatabasesApi
and on no client version either.

Related

That PR is the concrete evidence for the "every consumer reimplements this" point above — it
carries its own 404 translation, its own read-back confirmation and its own attachment type,
none of which belong in a LangChain integration package.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions