Conversation
…, and server pagination
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
My instance of Thunderhub is deployed on a server and manages a routing node with around 160 channels. Thunderhub is unusable on my node, because of the tremendous amounts of data that are pulled to the front-end. For many versions, I patched it locally so that the forwards summary stops at 1 month, the year total crashes my browser. The root cause it repeated requerying of 365 days of forwards, and summing them on the frontend. This takes 28 MB of data, about 14 seconds to get it to the front-end and then another 14 seconds to compute the summary, all the while blocking the tab. The forwards page has the same timings. This is with 1D history alone, the 7D and 1M I can open with great pain, the 2M or 6M are impossible to open. If I leave the dashboard open, it will run out of memory or the tab will get killed by Chrome due to it needing to re-do all the queries and calculations on every forward.
This PR moves all the calculations, summaries and aggregations to the backend, into specialised GraphQL queries. When coupled with #778, a complete reload of the dashboard now takes less that one second and transfers 160kb from the server to the front-end, including the year aggregates. Switching to the 6M view takes about 1.5 seconds. Instead of pulling over >50k forwards, only a limited set and aggregates/time series numbers are sent to the frontend.
The summary of the most important changes:
by_timeandsummaryI tried to limit this PR as much as possible, but it's a bit of a monolithic change to get to a usable system for a somewhat busy forwarding node. There are still some further improvements possible:
TLDR: I've been using Thunderhub for many years and this finally scratches my itch. I hope you accept my contribution. There's more to follow if you want.
Proof of forwards:
Fixes #590