Repository navigation
feature(passport service): preprocessing caching - #78
Merged
Merged
Conversation
ClemDoum
changed the title
ClemDoum
force-pushed
the
feature(passport-service)/preprocessing-caching
branch
from
September 2, 2026 11:06
0fddb39 to
1c37696
Compare
ClemDoum
force-pushed
the
feature(passport-service)/preprocessing-caching
branch
from
September 2, 2026 11:18
1c37696 to
5dc8e15
Compare
ClemDoum
marked this pull request as ready for review
September 2, 2026 11:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implement global preprocessing cache inside the passport worker to be ISO with the passport service.
The caching strategy is global and not very fine grained.
All preprocessing results are shared as long as the config matches and ignoring potential code updates.
For reprocessing, we can just disable caching in arguments and this will update the cache with using the latest code even if config is unchanged.
This is has a lot of shortcomings but this was how the passport service worked and it felt very useful implemented this way.
More advance/careful strategy will be implemented if problem pop up or usage evolves.
Changes
datashare-pythonAdded
BaseModel.__hash__determinitics rather than runtimespecifigcaching_keyin theactivity_workdir. This allows workers to cache intermediate results which are shared globally rather than storing them in an activity specific locationpassport-workerASRArgs