Skip to content

Clear empty strings left in nullable JSON columns at boot - #2103

Open
Abhishek-B-R wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
Abhishek-B-R:fix/selfhost-empty-json-columns
Open

Abhishek-B-R wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
Abhishek-B-R:fix/selfhost-empty-json-columns

Conversation

@Abhishek-B-R

Copy link
Copy Markdown

Summary

Databases carried forward from before the FumaDB cutover can hold '' instead of NULL in nullable JSON columns (the report has it in connection.credential_write). The JSON row mapper calls JSON.parse, which throws on an empty string inside the row mapper, so the whole findMany fails. connections.list runs on every toolkit MCP session, so every POST /mcp/toolkits/<slug> initialize returned 500.

This adds a boot-time data migration, modelled on bigintStorageClassSqliteMigration:

  • The columns are derived from the core schema (every column declared json and nullable, nine today across connection, integration, oauth_client, tool and artifact), so a column added later is covered without editing a list.
  • It only touches values that are exactly '' and sets them to NULL, which is what the ORM writes for "no value". It runs inside BEGIN/COMMIT and is idempotent.
  • NOT NULL JSON columns such as item_ids are left alone, since an empty string there has no faithful replacement.
  • It is registered right after the bigint migration in both the self-host and local registries, so the repair lands before anything else reads connection. Local is one step past the issue; it follows fix(local): repair bigint columns left in SQLite's integer storage class #1823, which registered in both, and is easy to drop if you want self-host only.

The migration is named 2026-09-24-empty-json-columns, following the other date-prefixed names.

On the second crash in the issue (tools_synced_at holding a number): that is the shape bigintStorageClassSqliteMigration (#1823) already repairs across every bigint column, and its test covers connection.tools_synced_at, so there is no second migration here. The suggested typeof(tools_synced_at) = 'blob' predicate would also clear healthy values, since BLOB is the current representation. If a number was written into that column after the bigint migration was stamped, that is a separate writer and probably worth its own issue.

I did not treat '' as NULL inside SQLiteTextJson.mapFromDriverValue, since that changes the read path for every JSON column on every driver; a one-time data repair keeps the mapper strict.

Linked issue

Closes #2092

Verification

  • sqlite-empty-json-migration.test.ts seeds legacy rows with raw SQL and checks that the read throws before the migration, that '' becomes NULL while valid JSON and NULL are untouched, that a second run changes nothing, that item_ids is left alone, and that the column list matches every nullable JSON column the schema declares.

  • legacy-empty-json-boot.test.ts in apps/host-selfhost seeds a connection row with '' in credential_write, last_health and provider_state, shows findMany throwing, then runs selfHostDataMigrations and reads the row back. It fails if the entry is removed from the registry.

  • oxfmt --check on the changed files: clean

  • oxlint -c .oxlintrc.jsonc --deny-warnings over packages/core/sdk, apps/host-selfhost, apps/local, and bun run lint:changelog-stubs: clean

  • tsgo --noEmit in packages/core/sdk, apps/host-selfhost, apps/local: exit 0

  • vitest run: packages/core/sdk 949 passed (65 files), apps/host-selfhost 160 passed (27 files), apps/local (bunx --bun vitest run) 104 passed (17 files)

  • e2e: not run; the boot test above covers the self-host registry path

Checklist

  • Added a changeset (bun run changeset), or this change needs none.
  • Added or updated tests for the new behaviour.
  • No secrets, credentials, or private data in the diff.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Self-host: toolkit MCP endpoints 500 on legacy connection rows (empty-string JSON columns, bigint in blob tools_synced_at) — boot migration suggestion

1 participant