Skip to content

Optimize search performance - #11

Open
robcodedev wants to merge 1 commit into
OnionUI:mainfrom
robcodedev:optimize-search
Open

Optimize search performance#11
robcodedev wants to merge 1 commit into
OnionUI:mainfrom
robcodedev:optimize-search

Conversation

@robcodedev

Copy link
Copy Markdown

Improves SearchFilter search performance by reducing unnecessary work during search.

Built with the Miyoo toolchain and tested on device.

Build search results in /tmp, then swap them onto the card

Every result row was a separate INSERT compiled and committed straight
to the SD card, so a search across a large collection spent most of its
time waiting on the SD card. The old database was deleted up front too,
so an interrupted search left you with nothing.

The list is now built in /tmp with one prepared statement inside a
single transaction, copied over once, and renamed into place. The
previous results stay readable until that rename. If anything fails we
drop the old database so MainUI rebuilds an empty list instead of
showing hits for a keyword that is no longer in active_search.
@robcodedev

Copy link
Copy Markdown
Author

Here is some more info,

Build search results in /tmp, then swap them onto the card

Every result row was earlier a separate INSERT compiled and committed straight to the SD card, so a search across a large collection spent most of its time waiting on the card rather than searching. The old database was also deleted up front, so an interrupted search left you with nothing.

The list is now built in /tmp using one prepared statement inside a single transaction, copied over once, and renamed into place. The previous results stay readable right up to the rename.

Failure behaviour

If the build fails at any point the old database is dropped rather than left in place. main.cpp has already written the new keyword to active_search by then, so keeping the old one would show hits for a keyword the user can no longer see. Dropping it makes MainUI rebuild the plain "Enter search term..." list, which is what the previous code did on failure.

Testing

  • Searched across 10 systems on hardware, db results identical to before
  • Up to 10x faster when returning 100+ hits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant