perf(chart): reduce history candle merge and gap fill work - #736
Merged
Merged
Conversation
Opening a coin, switching timeframe and backfilling history all run merge_bases and subtract_covered over the cached candles. merge_bases inserted every resampled row into a map one by one, and subtract_covered rescanned the cover intervals from the start for every hole. merge_bases now appends the parts, stable-sorts them on the i64 key and keeps the last row per key, so the same row wins as before. subtract_covered keeps one cover index across holes, which arrive ascending. The composed series is unchanged; tests compare it against a frozen copy of the old algorithm. CPU preparation drops 23-49% on five of six cache-state cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Opening a coin, switching timeframe and backfilling history all run
merge_basesandsubtract_coveredinmarket/candles.rsover the cached candles.merge_basesinserted every resampled row into a map one at a time. It now appends theparts, stable-sorts them on the i64 key and keeps the last row per key, so the same row wins
as before.
subtract_coveredrescanned the cover intervals from the start for every hole. It now keepsone cover index across the holes, which arrive in ascending order.
The composed series is unchanged. The new tests compare it against a frozen copy of the old
algorithm. CPU preparation drops 23-49% on five
of six cache-state cases (open, timeframe switch and backfill, cache hit and miss); the
timeframe-switch miss was already cheap and is unchanged. The numbers are CPU preparation only,
not I/O or end-to-end latency.
How to verify
cargo test -p moon-core --lib market::cargo test -p moon-core --release --lib market::candles::tests::cache_compose_preparation_timing -- --ignored --nocaptureprints the before/after medians