#18 follow-ups: sync guard, live pending count, one status vocabulary, wire pin, tested feature configs

[YellowSnnowmann]

Summary

Every remaining finding from #18's end-to-end review, in one branch: a per-connection guard so two Composio syncs cannot clobber each other's cursor and budget, per-source status read from the tables that actually hold embeddings, one FreshnessLabel and one source-id prefix instead of two of each, a serde pin on the tinymemory-sources twin, and CI that runs each engine configuration rather than only compiling it.

Verifying item 2 turned up a sixth defect the issue does not know about, and fixing it was a precondition for item 2 meaning anything — see below.

Related issue

Closes #59.

API or behavior changes

Behaviour, tinymemory-core:

• A Composio sync of a connection that is already syncing now returns immediately with note: Some(SYNC_ALREADY_RUNNING) and records_ingested: 0, instead of running concurrently. This applies to the Gmail and Slack backfills too — they key their SyncState on the same (toolkit, connection_id) as the periodic sync. An operator-initiated backfill fired during a periodic sync is now skipped rather than corrupting the shared state; the note says so, and a retry after the run succeeds.
• Synced Composio items are keyed {toolkit}:{connection_id}:{document_id} with tree scope {toolkit}:{connection_id}, restoring the engine adapter's scheme (see the regression note). Items ingested under Sync moves onto the memory API, with its acceptance test (#18 §B1/§B2/§B5/§E4) #48's composio:* scheme are orphaned by this and re-ingest once under the correct id. Sync moves onto the memory API, with its acceptance test (#18 §B1/§B2/§B5/§E4) #48 merged the same morning, so the exposure is under a day of syncs; no migration ships here.
• chunks_pending in the memory-sources status now reports real in-flight work rather than always equalling chunks_synced. Expect the numbers in that UI to move.
• A Composio source's chunks are matched per connection, not per toolkit, so two connections of one toolkit stop reporting each other's chunks.

Public API: additive — sync::pipelines::host::SYNC_ALREADY_RUNNING. sources::status::FreshnessLabel is now a re-export of sync::sync_status::FreshnessLabel; the path still resolves and the serde shape is unchanged. Nothing downstream names it today.

The regression this uncovered

The issue states "None is a regression introduced by #18." One is — 29f7c30, inside #48.

host.rs's tree reconnect claims in its own comment to mirror the engine adapter's. It does not:

	engine adapter (engine/sync.rs)	host.rs before this PR
source_id	{toolkit}:{conn}:{doc}	composio:{toolkit}:{conn}:{doc}
path_scope	Some("{toolkit}:{conn}")	None
owner	{toolkit}-sync:{conn}	""
provider	composio:{toolkit}	{toolkit}

Verified in the engine: retrieval/cover.rs derives a chunk's tree scope as path_scope.unwrap_or(source_id), and retrieval/source.rs's PLATFORM_KINDS classifies a scope by its platform prefix — gmail: is email, slack: is chat. With no path_scope, every synced item became its own single-item tree under composio:gmail:conn:msg-7, a prefix matching no platform. Items were stored and then unreachable: precisely the #5473 defect the reconnect exists to fix.

It also broke both source_id LIKE '{toolkit}:%' prefixes, so Composio chunks_synced read zero as well — fixing item 2's pending count alone would have left the UI at 0/0.

#48's own acceptance test asserts only that rows exist, not their addressing, which is why the drift passed. The engine adapter has a test pinning this scheme; the host that replaced it on the live path had none. It has one now.

Validation

Commands actually run, with their outcome:

• cargo fmt --all -- --check — pass
• cargo clippy --all-targets --all-features -- -D warnings — pass
• cargo build --all-targets --all-features — pass
• cargo test --all-features — pass, 23 suites, 0 failures

Every row of the new feature-configs matrix was also run locally and passes: --no-default-features, tinycortex, tinycortex,memory-git, mem0, supermemory, cognee, --all-features.

Tests

• Guard — one connection admits one run; distinct connections are independent; the key normalises the toolkit exactly as the pipeline gate does; the Slack backfill shares the Slack sync's guard; and end to end, a held connection returns the note without ticking the pipeline.
• Tree scope — a synced document is keyed by its connection scope, with path_scope and owner asserted rather than a row count, so this drift cannot recur silently. An item with a blank toolkit or connection skips the tree and still lands in the skill store.
• Pending — four chunks, one of each resolution state, assert 4 synced / 1 pending. The test first asserts the legacy embedding column is still NULL for all four, so it fails rather than passes vacuously if the predicate ever collapses back onto the dead column. A source with no chunks reports zeroes rather than tripping on SUM over no rows.
• Pin — MemorySourceEntry populated and empty (the second catches a skip_serializing_if dropped from one copy), plus SourceItem and SourceContent, which cross the same seam.

Deliberately untested: the contacts feature is checked, not executed. Its only behaviour is a macOS CNContactStore reader whose dependencies sit behind a cfg(target_os = "macos") table, so on ubuntu there is nothing to run but the empty stub; executing it needs a macos-latest runner, deliberately not spent. The powerset job already enumerates the feature as a subset of size one.

Documentation

Module and item docs updated in the same commits as the behaviour:

• sync/pipelines/host.rs — why the addressing scheme is a contract rather than an implementation detail, and what passing no path_scope actually does.
• sources/status.rs — a "Where pending lives" section: mem_tree_chunks.embedding is a migration artefact nothing writes, and resolution has three terminal forms.
• diff/source.rs — the prefix map now resolves through the one shared definition.
• .github/workflows/ci.yml — what the matrix adds over the powerset pass (cargo check never links and never runs a binary), and that §E2's sync-composio names a feature existing nowhere in the workspace.

Notes on the issue's items

• Item 1 uses a non-blocking guard rather than the "held for the run" mutex the issue describes. Queueing would stall the periodic loop's whole tick behind a long manual sync and then run a redundant sync of a connection just synced, which is the Composio spend the guard exists to avoid.
• Item 2's "consolidate onto core::sync::sync_status" is done for FreshnessLabel, which is what item 4 names. SourceStatus stays where it is: it is per-source, while MemorySyncStatus is per-provider and carries batch_total/batch_processed. Merging them would mean inventing a shape neither caller wants. The engine's third FreshnessLabel is vendored and stays.
• Item 5 — two of §E2's nine configurations name features the facade does not have. contacts belongs to tinymemory-core; sync-composio exists nowhere in the workspace, because the Composio sync is unconditional there. Both are written into the workflow rather than quietly dropped, since a missing row in a matrix reads as covered.

Worth stating plainly about the matrix: the facade's 54 tests are the same in every configuration, because DriverRegistry admission is a static policy table rather than a function of which adapters were compiled in. These rows earn their minutes by linking and running — which cargo check never does, and where feature-unification bites, as the root Cargo.toml's own links = "git2" warning records.

Checklist

• The change is focused on one logical change
• No new #[allow(...)], #[ignore], or relaxed lints
• No secrets, tokens, or .env contents in the diff or the description

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fbd5915-1032-4cb0-ba40-d24a91736e2b

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[tinysweeper]

tinysweeper found nothing blocking. Approving.

$0.0000 · 0 in / 0 out · 726 embedded · openrouter/openai/text-embedding-3-small

[tinysweeper]

How this change flows

4 changed behaviours across 19 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 42 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["source_id_prefix<br/>changed"]:::changed
  n1["source_status<br/>changed"]:::changed
  n2["status_list<br/>changed"]:::changed
  n3["run_composio_connection_with_caps<br/>changed"]:::changed
  n4["a_held_connection_short_circuits_the_run"]:::impacted
  n5["new"]:::impacted
  n6["join"]:::impacted
  n7["run_pipeline"]:::impacted
  n8["...lved_chunks_not_the_dead_embedding_column"]:::impacted
  n9["...document_is_keyed_by_its_connection_scope"]:::impacted
  n1 -->|calls| n0
  n2 -->|calls| n1
  n3 -->|calls| n5
  n3 -->|calls| n7
  n4 -->|calls| n5
  n4 -->|tests| n5
  n4 -->|calls| n6
  n4 -->|tests| n6
  n4 -->|calls| n7
  n4 -->|tests| n7
  n7 -->|calls| n5
  n8 -->|calls| n1
  n8 -->|tests| n1
  n8 -->|calls| n6
  n8 -->|tests| n6
  n9 -->|calls| n5
  n9 -->|tests| n5
  n9 -->|calls| n6
  n9 -->|tests| n6
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff

Loading

_{Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.}