Repository navigation
Time out a stalled auto-push - #4
Merged
Merged
Conversation
git runs `git maintenance run --auto --detach` after every commit. Once the loose-object count passes gc.auto, each commit starts its own detached full repack, and nothing stops a second one from starting while the first is still running. An archive's pack is mostly zstd or git-crypt blobs, so each repack holds gigabytes. One ingest tick makes one commit per changed session, which stacked repacks until a 16 GB Mac with a 7.3 GiB archive ran out of memory and had to be force-restarted. Set maintenance.auto=false and gc.auto=0 in the archive's local git config. init.py sets them after `git init`. ingest.py sets them under the ingest lock before the first commit, which migrates existing archives on their next tick. Loose objects now wait for a manual `git repack -d`. The selftest checks both the init path and the migration path.
On a real archive 1,832 compressed deposits failed the sweep fsck, yet every one decompressed to its canonical sha256. The v2.3.0 pure migration copied sha256_compressed from the prior manifest instead of hashing the file, so stale values survived, and the fsck compares only the compressed hash. About 40 false failures per sweep slice buried any real one. Hash the file during a pure migration. In the fsck, fall back to the canonical hash when the compressed one disagrees and count a match as stale, not failed. Add scripts/repair_compressed_hashes.py to rewrite the stale values in one commit, touching only that field and only after the canonical hash matches.
The preferred Claude Code source on the maintainer's machine is a backup mirror in ~/Documents, which iCloud syncs. iCloud had evicted 3,556 of its 11,311 transcripts to the cloud. The launchd tick may not download an evicted file, so every read failed with EDEADLK and each sweep logged dozens of "Resource deadlock avoided" worker failures. Dedup now prefers a later path whose copy is on disk over an evicted first path. A deposit whose source is still evicted returns skipped-offline and leaves the prior deposit in place.
git push ran with no timeout. A push that stalls rather than fails holds the ingest lock, so every later tick exits early and nothing is deposited until someone kills it by hand. Run pushes in their own process group with a one-hour limit and kill the whole group on timeout, so ssh and pack-objects exit too. A timed out push is reported like any other push failure.
13 of 28 tasks
PEEKPerformer
changed the base branch from
fix/icloud-evicted-sources
to
main
September 17, 2026 12:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3 (#1, then #2, then #3). This PR's diff is only its own commit.
What this PR does
Auto-push in
ingest.pynow runs with a one-hour timeout. On timeout it kills the push's whole process group.Why
git pushran with no timeout. A push that stalls instead of failing (a connection that stops sending but never resets, a laptop that sleeps mid-push) holds the ingest lock indefinitely. Every later tick then exits with "another ingest is running", and nothing is deposited until someone kills the push by hand.subprocess.run(timeout=...)would not be enough, because it kills only the directgitchild and leaves ssh andpack-objectsrunning.run_with_timeoutstarts the command in a new session and kills the whole group.One hour (
PUSH_TIMEOUT_SECONDS) covers a--max-pack-gibchunk (default 1.5 GiB) on a slow uplink. A timed-out push is reported like any other push failure, and the next tick with changes retries.Type of change
Non-negotiables check
Selftest
python scripts/selftest.pypasses locallyNew assertion:
run_with_timeout(["sh", "-c", "sleep 37 & sleep 37"], timeout=1)returns within seconds with a non-zero code and a timeout message, andpgrepfinds no leftoversleep 37.Manifest / archive-format impact
manifest_versionbumped from N to N+1index_schema_versionbumped from N to N+1Documentation
CHANGELOG.mdupdated (under the next-release section, 2.4.1)README.mdupdated if user-facing behavior changedSKILL.mdupdated if wizard flow changeddocs/updated as appropriateCross-platform impact
start_new_sessionandkillpgare POSIX; no Windows runner configured)AI assistance
Fix and draft written by Claude Code. Selftest run locally.
Anything else reviewers should know
Not observed in the incident that prompted #1. This closes the one remaining way a single tick can block all later ticks.
repush_chunked.pyis interactive and still pushes without a timeout.