feat(blog): add SQLite FTS5 + Dense hybrid retrieval article - #251
feat(blog): add SQLite FTS5 + Dense hybrid retrieval article#251ishwar170695 wants to merge 2 commits into
Conversation
919d215 to
0e21673
Compare
Signed-off-by: Ishwar <ishwarcm@iitbhilai.ac.in>
0e21673 to
9e914a0
Compare
saiyam1814
left a comment
There was a problem hiding this comment.
Thanks for putting this together. I read through the article and the added assets. Overall, this is a strong technical draft: it has a clear retrieval problem, a concrete architecture, code snippets, benchmark numbers, screenshots, and a useful tradeoff section. I think it can fit the KubeSimplify audience well if we frame it as AI infrastructure / backend retrieval engineering rather than only as a generic RAG article.
A few changes I would suggest before publishing:
-
Use the architecture diagram in the article
The PR addspublic/img/blog/sqlite-fts5-dense-hybrid-retrieval/architecture.png, but the post currently shows the pipeline as an ASCII diagram. This topic benefits a lot from a visual architecture diagram, so I would embed the image in the Architecture Overview section and optionally keep the text explanation around it. -
Fix the author image
public/img/authors/ishwar.jpgappears to be the same file as the blogcover.jpg. That looks accidental. Please replace it with an actual author avatar or a suitable placeholder image. -
Add more context around the benchmark numbers
The reported improvements are compelling (466 msto12 ms,320 MBto48 MB,68%to91%Top-5 retrieval), but they need enough context to be credible and reproducible. Please add details such as corpus size, hardware, Node/SQLite versions, embedding model/vector dimension, how the 100-query benchmark was constructed, and how relevance was judged. -
Verify math rendering for the RRF formula
The post uses$$...$$for the RRF equation. Please verify this renders correctly in the KubeSimplify site. If math rendering is not enabled, convert it to a plain-text/code-block formula so readers do not see raw LaTeX. -
Make the KubeSimplify fit more explicit
I would add a short section on where this belongs operationally: local-first backends, edge apps, small/medium corpora, Kubernetes workloads with persistent storage, serverless constraints, and when teams should move to Qdrant/Milvus/Elasticsearch. This would connect the article more strongly with KubeSimplify’s cloud-native/infrastructure audience. -
Add a note on FTS5 table population/sync
The SQLite FTS5 external-content table example is useful, but readers may wonder howsections_ftsstays in sync withsections. A short ingestion snippet or note about triggers/rebuild flow would make the implementation more complete. -
Polish the style for KubeSimplify
Consider removing emojis from section headings and using a cleaner tutorial/deep-dive style. The current content is good, but simpler headings would make it feel more consistent with existing KubeSimplify technical posts.
Overall: this is close and worth publishing after the above polish. The idea is useful, the implementation is practical, and the performance/tradeoff framing is exactly the kind of detail readers will appreciate once the claims and visuals are tightened up.
|
@saiyam1814 Thanks for the detailed review. I've addressed all the suggested changes:
I also verified that the site builds successfully with these changes. Thanks again for the suggestions! |
|
Did you test this on real hardware? |
Yes. I tested everything on my local machine. |
|
for some reason I am not able to see the preview on cloud flare, let me pull in locally and check |
|
@saiyam1814 @ishwar170695 Here's the preview for the current state of the PR https://post-sqlite-fts5-dense-hybri.website-dab.pages.dev/sqlite-fts5-dense-hybrid-retrieval |
|
@shkatara can you add feedback for this blog? |
|
I'm writing this from my phone so keeping it short. Architecture diagram is not understandable at all to people who are new to this. So many flows and nodes make it confusing than clearer. Flow representation is also not clear. How does one query branch off to different endpoints to search. Where is this configured. FTS5. / BM25. I'm guessing searching something over text so full text search. People not working with db have no idea what this is. Avoid using short forms or at least write full form after them in () Synchronizing the FTS5 Virtual Table: this is worded in a way someone from not a db background would fail to understand. External content table ? What is that. What is a virtual table ? We load only the id (string) and the coordinate list—pre-processed into a compact Float32Array object—into memory: what id is this of ? What is in the float32 array Explain the sql queries in english Because the architecture or packet flow is not clear, I can't put a mental model of why memory usage comes down by 85% What is RRF. What is BM25. The problem statement is good. And one that makes sense. The answer goes into a lot of depth and jargons that makes it hard to follow. I had to try at least three times to read it and always I could not finish. The blog should focus on keeping it simple. Keeping in mind people have no idea what we are writing about and would need help at each layer to put a mental model. If they are fighting to understand what these short forms are, the purpose is defeated. It's better to give them ideas that something like this is possible if the topic is dense, and let them explore their own cases. |
Changes proposed
Adds a new, technical, practitioner-led article to the Kubesimplify blog focusing on hybrid retrieval architectures in local RAG systems.
content/blog/sqlite-fts5-dense-hybrid-retrieval.md— Deep-dive into sparse vs. dense search limitations, SQLite FTS5 BM25 configurations, Reciprocal Rank Fusion (RRF), deterministic domain reranking, and vector cache memory footprints.public/img/blog/sqlite-fts5-dense-hybrid-retrieval/— Static assets (architecture diagrams, benchmarks, UI screenshots).content/authors.json— Add author profile entry forishwar.public/img/authors/ishwar.jpg— Author avatar placeholder.No other changes are made to the site's code.
Note to reviewers
This post focuses on systems-level search engineering and database schemas (SQLite), which is well-suited for KubeSimplify's backend, cloud-native, and infrastructure audience.