What's wrong
Program.cs:24 loads the whole store into memory once, when the process starts. Every later save writes that process's entire in-memory copy back to disk:
Verbs/Scan.cs:141-142 saves after each image.Scan.cs:106 and Import.cs:65 save as well.
The lock at Scan.cs:137 only serializes threads inside one process. Nothing locks the file across processes, and nothing re-reads and merges the file before writing. So whichever process saves last wins, and the other process's additions are thrown away.
Failure scenario
Two Scan runs started together on different folders (A and B, 3 images each), sharing one store:
- Actual: both print "Scan complete. Total descriptions in database: 3". Afterwards
Stats shows 3 descriptions, and persistent_state.json contains only folder A's paths. B's 3 descriptions are lost with no message, along with the model time spent generating them.
- Expected: 6 descriptions, or at least a refusal to start the second run.
The same thing silently undoes an Import or a Configure change made while a Scan is running: the Scan's next per-image save writes back its stale copy.
I reproduced this with two real processes against a stub Ollama server that delays each response by 0.7 s. Long, model-bound scans make the overlap window large in real use.
Suggested fix
Either of these would do:
- Take an exclusive lock (e.g. a lock file opened with
FileShare.None) for the whole run of any verb that writes to the store. A second run should fail fast with a clear message.
- Under a short cross-process lock, re-load the file and merge into it before every save: take the union of
Descriptions by hash and of KnownPaths.
Option 1 is simpler. Option 2 lets parallel scans of different folders work.
Acceptance criteria
- Two concurrent Scans over separate folders either both end up in the store (union), or the second one refuses to run with a clear message. No description is silently lost.
- An Import or Configure made during a Scan is not reverted by the Scan's later saves.
What's wrong
Program.cs:24loads the whole store into memory once, when the process starts. Every later save writes that process's entire in-memory copy back to disk:Verbs/Scan.cs:141-142saves after each image.Scan.cs:106andImport.cs:65save as well.The
lockatScan.cs:137only serializes threads inside one process. Nothing locks the file across processes, and nothing re-reads and merges the file before writing. So whichever process saves last wins, and the other process's additions are thrown away.Failure scenario
Two
Scanruns started together on different folders (A and B, 3 images each), sharing one store:Statsshows 3 descriptions, andpersistent_state.jsoncontains only folder A's paths. B's 3 descriptions are lost with no message, along with the model time spent generating them.The same thing silently undoes an
Importor aConfigurechange made while a Scan is running: the Scan's next per-image save writes back its stale copy.I reproduced this with two real processes against a stub Ollama server that delays each response by 0.7 s. Long, model-bound scans make the overlap window large in real use.
Suggested fix
Either of these would do:
FileShare.None) for the whole run of any verb that writes to the store. A second run should fail fast with a clear message.Descriptionsby hash and ofKnownPaths.Option 1 is simpler. Option 2 lets parallel scans of different folders work.
Acceptance criteria