Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,40 @@
# Changelog

## Unreleased

### Bug Fixes

- add `titles` column migration for `person_observations` — existing
databases predate the `title` → `titles` rename and CREATE TABLE IF NOT
EXISTS is a no-op on already-created tables; the first write hits `table
has no column named titles`. Wired before `PersonObservationRepo.setupDatabase()`.
- add `loi_date` column migration for `spac`, `spac_deal`, `spac_history` —
same CREATE TABLE IF NOT EXISTS problem after the LOI lifecycle stage
landed. `spac.status = "loi"` needs no DDL (`TypeStringEnum` emits plain
TEXT with no CHECK constraint).
- bump the person resolver to `2.0.0` — `PersonNormalization` now strips
periods/commas and folds typographic apostrophes, which changes the
`(normalized_first, normalized_middle, normalized_last, normalized_suffix)`
tuple `PersonResolver.personKey` looks up by. Post-fold observations under
the same `resolver_version` silently miss pre-fold canonicals and mint
duplicates.

### Operator Ceremony (person resolver 2.0.0)

Existing DBs must run through the version-bump ceremony to migrate
identities across the fold. Aliases must be re-applied manually against the
fresh 2.0.0 canonical UUIDs (`sec canonical person alias-list` before the
ceremony; re-issue `sec canonical person alias` after).

```sh
sec version start-dev resolver person 2.0.0 --bump major \
--notes "PersonNormalization punctuation/typographic fold changes key tuple"
sec resolve --kind person --resolver-version 2.0.0 --all
sec version coverage resolver person
sec version promote resolver person
sec version drop-previous resolver person
```

## 0.0.9

### Chores
Expand Down
22 changes: 22 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,28 @@ sec version drop-previous resolver company
sec version drop-previous extractor <extractor-id>
```

#### Resolver version bumps

When the resolver's normalized key tuple changes (e.g. `PersonNormalization`
now strips periods/commas and folds typographic apostrophes → the
`normalized_first/middle/last/suffix` tuple `PersonResolver.personKey` looks up
by), bump the resolver's semver so post-change lookups do not silently miss
pre-change canonicals and mint duplicates under the same `resolver_version`.
Worked example for the person resolver 1.0.0 → 2.0.0:

```bash
sec version start-dev resolver person 2.0.0 --bump major \
--notes "PersonNormalization punctuation/typographic fold changes key tuple"
sec resolve --kind person --resolver-version 2.0.0 --all
sec version coverage resolver person
sec version promote resolver person
sec version drop-previous resolver person
```

Aliases must be re-applied manually against the fresh 2.0.0 canonical UUIDs
(`sec canonical person alias-list` before the ceremony; re-issue
`sec canonical person alias` after).

### S-1 extraction

S-1 prospectuses are narrative HTML (not structured XML), so the `S-1` extractor
Expand Down
28 changes: 28 additions & 0 deletions src/cli/groups/version.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,34 @@ describe("sec version CLI", () => {
}
});

it("start-dev resolver person 2.0.0 --bump major snapshots observation count", async () => {
const dir = mkdtempSync(join(tmpdir(), "sec-version-test-"));
try {
const setup = await runCli(["db", "setup"], dir);
expect(setup.exitCode).toBe(0);

const result = await runCli(
["version", "start-dev", "resolver", "person", "2.0.0", "--bump", "major"],
dir
);
expect(result.exitCode).toBe(0);
// Must NOT be refused as "not yet supported".
expect(result.stderr + result.stdout).not.toMatch(/not yet supported/);

// No observations were seeded, so target_count === PersonObservationRepo.count() === 0.
const coverage = await runCli(
["version", "coverage", "resolver", "person", "--format", "json"],
dir
);
expect(coverage.exitCode).toBe(0);
const parsed = JSON.parse(coverage.stdout);
// computeResolverCoverage returns denominator (observation count).
expect(parsed.denominator).toBe(0);
} finally {
rmSync(dir, { recursive: true, force: true });
}
}, 15000);

it("rejects start-dev for an unknown extractor id", async () => {
const dir = mkdtempSync(join(tmpdir(), "sec-version-test-"));
try {
Expand Down
37 changes: 25 additions & 12 deletions src/cli/groups/version.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@

import type { Command } from "commander";
import { globalServiceRegistry } from "workglow";
import type { ResolverId } from "../../resolver/resolverIds";
import { isFamilyResolverId, type ResolverId } from "../../resolver/resolverIds";
import { FILING_REPOSITORY_TOKEN } from "../../storage/filing/FilingSchema";
import { CompanyObservationRepo } from "../../storage/observation/CompanyObservationRepo";
import { PersonObservationRepo } from "../../storage/observation/PersonObservationRepo";
import {
dropNext as dropNextCeremony,
dropPrevious as dropPreviousCeremony,
Expand Down Expand Up @@ -61,20 +63,31 @@ function assertFormat(s: string): asserts s is "table" | "json" {
}

/**
* For a major-bump extractor start-dev, snapshot the count of filings
* handled by this extractor. Stored on the next-slot row; used as the
* promote-gate denominator.
* For a major-bump start-dev, snapshot the count of items the ceremony's
* coverage gate will be denominated over. Stored on the next-slot row.
*
* This is extractor-specific today. When resolvers gain a major
* dev-cycle, they will need their own kind-aware snapshot strategy
* (count of observations? count of canonical identities? TBD). Add a
* dispatch table keyed on ComponentKind when resolver support lands.
* Extractor snapshots count the filings the extractor handles (via
* `FORM_TO_EXTRACTOR_ID` → filing forms). Resolver snapshots count the
* observations the resolver ranges over (person / company). Family-tier
* resolvers have no observation model and are refused, matching the
* behavior of `computeResolverCoverage`.
*/
async function snapshotTargetCount(kind: ComponentKind, id: string): Promise<number> {
if (kind !== "extractor") {
throw new Error(
`snapshotTargetCount: kind '${kind}' is not yet supported; only 'extractor' has a snapshot strategy. Add resolver support when implemented.`
);
if (kind === "resolver") {
if (isFamilyResolverId(id)) {
throw new Error(
`Coverage snapshots not supported for family-tier resolvers (got '${id}'). ` +
`Family resolvers track membership, not observation identity-links.`
);
}
const resolverId = id as ResolverId;
if (resolverId === "person") {
return await new PersonObservationRepo().count();
}
if (resolverId === "company") {
return await new CompanyObservationRepo().count();
}
throw new Error(`snapshotTargetCount: unknown resolver id '${id}'`);
}
const filingRepo = globalServiceRegistry.get(FILING_REPOSITORY_TOKEN);
const forms = Object.entries(FORM_TO_EXTRACTOR_ID)
Expand Down
11 changes: 11 additions & 0 deletions src/config/setupAllDatabases.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ import { SPAC_REPOSITORY_TOKEN } from "../storage/spac/SpacSchema";
import { SPAC_DEAL_REPOSITORY_TOKEN } from "../storage/spac/SpacDealSchema";
import { SPAC_EVENT_REPOSITORY_TOKEN } from "../storage/spac/SpacEventSchema";
import { SPAC_HISTORY_REPOSITORY_TOKEN } from "../storage/spac/SpacHistorySchema";
import { migrateSpacLoiColumns } from "../storage/spac/SpacLoiColumnsMigration";
import { SPAC_MERGER_EXTRACTION_REPOSITORY_TOKEN } from "../storage/spac/SpacMergerExtractionSchema";
import { SPAC_REDEMPTION_EXTRACTION_REPOSITORY_TOKEN } from "../storage/spac/SpacRedemptionExtractionSchema";
import { SPAC_LOI_EXTRACTION_REPOSITORY_TOKEN } from "../storage/spac/SpacLoiExtractionSchema";
Expand Down Expand Up @@ -90,6 +91,7 @@ import { PERSON_IDENTITY_LINK_REPOSITORY_TOKEN } from "../storage/canonical/Pers
import { CURRENT_CANONICAL_VIEW_DDL } from "../storage/canonical/views";
import { COMPANY_OBSERVATION_REPOSITORY_TOKEN } from "../storage/observation/CompanyObservationSchema";
import { PERSON_OBSERVATION_REPOSITORY_TOKEN } from "../storage/observation/PersonObservationSchema";
import { migratePersonObservationTitles } from "../storage/observation/PersonObservationTitlesMigration";
import { OBSERVATION_PROVENANCE_REPOSITORY_TOKEN } from "../storage/provenance/ObservationProvenanceSchema";
import { BENEFICIAL_OWNERSHIP_REPOSITORY_TOKEN } from "../storage/beneficial-ownership/BeneficialOwnershipSchema";
import { RELATED_PARTY_TRANSACTION_REPOSITORY_TOKEN } from "../storage/related-party/RelatedPartyTransactionSchema";
Expand Down Expand Up @@ -146,6 +148,11 @@ export async function setupAllDatabases(): Promise<void> {
await globalServiceRegistry.get(REGA_SERVICE_PROVIDER_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(REGA_FINANCIAL_DATA_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(REGA_EQUITY_CLASS_REPOSITORY_TOKEN).setupDatabase();
// Add loi_date to the SPAC tables before setupDatabase() creates the current
// shape; CREATE TABLE IF NOT EXISTS is a no-op on an existing table, so an
// upgraded DB never gains loi_date on its own. The status enum's new "loi"
// value needs no DDL — TypeStringEnum emits plain TEXT with no CHECK.
await migrateSpacLoiColumns();
await globalServiceRegistry.get(SPAC_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(SPAC_DEAL_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(SPAC_EVENT_REPOSITORY_TOKEN).setupDatabase();
Expand All @@ -160,6 +167,10 @@ export async function setupAllDatabases(): Promise<void> {
await globalServiceRegistry.get(COMPONENT_VERSION_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(EXTRACTOR_RUN_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(VERSION_EVENT_REPOSITORY_TOKEN).setupDatabase();
// Migrate legacy `title TEXT` → `titles TEXT` before setupDatabase() creates
// the current shape; CREATE TABLE IF NOT EXISTS is a no-op on an existing
// table, so an upgraded DB never gains the renamed column on its own.
await migratePersonObservationTitles();
await globalServiceRegistry.get(PERSON_OBSERVATION_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(COMPANY_OBSERVATION_REPOSITORY_TOKEN).setupDatabase();
await globalServiceRegistry.get(OBSERVATION_PROVENANCE_REPOSITORY_TOKEN).setupDatabase();
Expand Down
68 changes: 68 additions & 0 deletions src/resolver/PersonResolver.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,74 @@ describe("PersonResolver.resolve", () => {
expect(result).toBe("alias-target");
});

it("cross-version isolation: 1.0.0 canonical is not returned under 2.0.0 lookup", async () => {
// Seed a canonical person from the pre-fold tuple (a "1.0.0" resolver
// would have written this). Now resolve an observation whose normalized
// tuple is the post-fold shape under a fresh 2.0.0 resolver — the
// resolver_version scope keeps the 1.0.0 row out of scope and mints a
// new canonical id.
const preFoldRow: CanonicalPerson = {
canonical_person_id: "canon-1.0.0",
resolver_version: "1.0.0",
display_first: "Richard",
display_middle: "J.",
display_last: "Boyle",
display_suffix: "Jr.",
cik: null,
normalized_first: "richard",
normalized_middle: "j.",
normalized_last: "boyle",
normalized_suffix: "jr.",
source_filing_issuer_cik: 100,
created_at: "2026-05-22T00:00:00.000Z",
};
await setup.canonRepo.create(preFoldRow);

const v2Resolver = new PersonResolver({
canonicalPersonRepo: setup.canonRepo,
canonicalPersonAliasRepo: setup.aliasRepo,
activeResolverVersion: "2.0.0",
});
const v2Id = await v2Resolver.resolve(
obs({
normalized_first: "richard",
normalized_middle: "j",
normalized_last: "boyle",
normalized_suffix: "jr",
source_filing_issuer_cik: 100,
})
);
expect(v2Id).not.toBe("canon-1.0.0");
});

it("2.0.0 idempotency: two equal post-fold keys return the same UUID", async () => {
const v2Resolver = new PersonResolver({
canonicalPersonRepo: setup.canonRepo,
canonicalPersonAliasRepo: setup.aliasRepo,
activeResolverVersion: "2.0.0",
});
const a = await v2Resolver.resolve(
obs({
normalized_first: "richard",
normalized_middle: "j",
normalized_last: "boyle",
normalized_suffix: "jr",
source_filing_issuer_cik: 100,
})
);
const b = await v2Resolver.resolve(
obs({
normalized_first: "richard",
normalized_middle: "j",
normalized_last: "boyle",
normalized_suffix: "jr",
source_filing_issuer_cik: 100,
observation_id: 2,
})
);
expect(a).toBe(b);
});

it("serialises alias lookup inside the per-key mutex (no overlapping alias.resolve calls)", async () => {
// The previous regression test stubbed `aliasRepo.resolve` to always
// return a constant id regardless of input, so both pre-fix (lookup
Expand Down
Loading