Fast LLM + Human-in-the-Loop classification post; refresh featured post & series by anishk123 · Pull Request #160 · scribd/scribd.github.io

anishk123 · 2026-07-11T17:49:42Z

This adds a new ML Data Engineering blog post and refreshes the homepage's Featured Post and Featured Series.

What changed

New post: _posts/2026-07-11-fast-llm-human-in-the-loop-classification.md — "How a Three-Person Team Used Cheap, Fast LLMs + Human-in-the-Loop to Classify Hundreds of Millions of Documents in ~2 Months." Content adapted from the source doc; formatting cleaned up for the web (bulleting, code/table blocks, the recurring "Fast Model → Judge Model → SME Review → Ingest → Iterate" mantra styled as callouts) with the wording unchanged.
Tagged featured (dated 2026-07-11, so it becomes the newest Featured Post) and content-trust-series (so it joins the Featured Series).
PhotoDNA post: removed the content-trust-series tag from _posts/2026-01-20-photodna-csam-detection.md so it drops out of the featured series.

Visuals

Three on-brand SVG diagrams (Scribd palette) are embedded in the post:

The repeatable loop — under Workflow Overview (the spot flagged for a diagram): Fast Model → Judge → SME → Ingest → Iterate.
Breadth-to-precision funnel — under Sampling Strategy: 100% fast-model labels → 1% validation → 0.1% judge → SME reviews only disagreements.
Human-in-the-loop — under The Core Point: model proposes, humans decide, deterministic rules apply.

Assets live in post-images/2026-fast-llm-hitl/.

Review fixes

Added tag/llm/index.md so the new llm tag no longer 404s (Cursor Bugbot).
Fixed the Appendix B Mermaid (quoted node labels containing parentheses so it renders) and pointed it at the rendered diagram above.

Resulting homepage state

Featured Post: this new post.
Featured Series (site.featured_series: content-trust-series, shows 2): Dual-Embedding Trust Scoring + this new post. PhotoDNA is no longer in the series.

No _config.yml change needed — featured_series is already content-trust-series.

Note

Low Risk
Static Jekyll content and a small Liquid sort tweak; no auth, data paths, or build-critical logic beyond normal site generation.

Overview
Adds a new featured ML Data Engineering article on a repeatable fast model → judge → SME → ingest → iterate workflow for corpus-scale LLM classification, with three branded SVG diagrams and appendix material (Sheets schema, Mermaid flow, checklist).

Homepage Featured Series now sorts series posts newest first (| reverse on the tag list), so the two slots under content-trust-series surface the latest entries. A tag/llm index page is added so the post’s llm tag does not 404.

^{Reviewed by Cursor Bugbot for commit 1966641. Bugbot is set up for automated code reviews on this repo. Configure here.}

Pull in head repo main

…ries New post 'How a Three-Person Team Used Cheap, Fast LLMs + Human-in-the-Loop to Classify Hundreds of Millions of Documents in ~2 Months', tagged both 'featured' (becomes the featured post) and 'content-trust-series' (joins the featured series). Removes 'content-trust-series' from the PhotoDNA post so the series now shows Dual-Embedding Trust Scoring + this new post. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- Add tag/llm/index.md so the new `llm` tag no longer 404s (Cursor Bugbot; matches the CONTRIBUTING generate-tags output format). - Fix the Appendix B Mermaid: quote node labels containing parentheses so it renders (previously broken syntax). Heading and text otherwise unchanged from the source doc. - Add three on-brand SVG diagrams (Scribd palette) grounded in the post text: - Workflow Overview: the repeatable Fast Model -> Judge -> SME -> Ingest -> Iterate loop. - Sampling Strategy: breadth-to-precision funnel (100% -> 1% -> 0.1% -> SME reviews the disagreements). - The Core Point: human-in-the-loop (model proposes labels, humans have the final say, corrections grow a versioned golden set, feeding back). - Add horizontal-rule dividers between top-level sections for readability (mirrors the section breaks in the source doc). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

rtyler

In its current form this is pretty listicle like, I would like to see some more narrative before this is published

rtyler · 2026-07-13T16:35:21Z

 - aws
 - lambda
 - databricks
- content-trust-series


Why was this removed from the series? The later post will appear on the home page

Yes, I decided to remove it because it this blog post has been there from the beginning of the year. I do understand your point. Are you recommending that we don't remove it from the series until a new featured blog shows up?

Ok, I now actually understood that posts in the series will just age out normally and they don't need to be removed from the actual series, and I have reverted this change.

rtyler · 2026-07-13T16:36:20Z

@@ -0,0 +1,325 @@
+---
+layout: post
+title: "How a Three-Person Team Used Cheap, Fast LLMs + Human-in-the-Loop to Classify Hundreds of Millions of Documents in ~2 Months"


This title is way too long, a good pithy title goes a long way.

Thanks @rtyler — took your point on length to heart. We've shortened it to "Stop Building Models. Start Building Systems.", which is pithier and also intentionally matches the title of Jeanie's TrustCon 2026 presentation that this post accompanies. The longer descriptive framing now lives in the subtitle.

rtyler · 2026-07-13T16:43:28Z

+author: anishk123
+---
+
+*A repeatable "Fast Model → Judge → SME → Iterate" workflow using familiar tools*


This somewhat incoherent sentence fragment is sitting our here all alone, for no clear reason.

rtyler · 2026-07-13T16:48:01Z

+
+So instead of building a one-off system tied to a specific model or a single "perfect prompt," we focused on a **repeatable process** that we successfully applied to two large-scale projects (Slideshare and Scribd). That repeatable process is what we're sharing.
+
+**Key learnings:**


I will try to spare you some ranting and raving of mine about article structure but I personally find the use of "tl;dr" by authors to be a major crutch to avoid crafting coherent flow. I always recommend people think about their reader as somebody who you must convince paragraph by paragraph, to continue reading.

As the writer you are a salesman trying to get them to continue to buy your words.

Some rules I try to follow in the intro:

The first sentence is the headline, if all they read is that, they should be able to answer what the post is about.

The first paragraph is the summary, don't hold anything back, don't bury the lede, if all they read is that, they should be able to tell their friend the gist of the post.

The last sentence is the sales pitch to keep reading to the next paragraph. it doesn't have to be "read on to find out more" or some junk like that, it can also just leave some suspense in the air compelling the reader forward.

@rtyler I have re-organized the entire post to follow the rules above with very minor changes to the actual text. Please review again. Thanks.

@rtyler

- Revert removing `content-trust-series` from the PhotoDNA post (per @rtyler). The Featured Series again shows PhotoDNA + Dual-Embedding Trust Scoring; the new post remains the Featured Post at the top of the home page. - Shorten the title to "Stop Building Models. Start Building Systems." — pithy, and it matches the title of Jeanie Lam's TrustCon 2026 presentation that this post accompanies. The longer framing lives in the subtitle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 8344f1a. Configure here.}

…tyler

…tyler) Per Tyler's note on article flow: drop the "TL;DR" heading and the up-front bullet dump so the opening reads as prose (headline -> gist -> mantra hook) that flows straight into the concrete results in "What We Achieved". Move the five key-learnings down into a "Key Learnings" recap just before the Summary. No article wording changed — the body has legal approval, so this only removes the "TL;DR" heading + redundant "Key learnings:" label and reorders blocks (verified via a token-multiset diff: zero words added, only those labels removed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- Rename "The Core Point" -> "Hypothesis" so the section reads as the idea we set out to test. - Move the whole "Workflow Overview" section down to sit after Key Learnings, just before the Summary — it's reference material that broke the narrative. - Drop the redundant intro mantra restatement (the subtitle already states the Fast Model -> Judge -> SME -> Iterate loop). No approved body copy reworded (verified via token-multiset diff): only the "TL;DR"/mantra labels removed and the "Hypothesis" heading added; every sentence is verbatim and Workflow Overview moved intact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ra refrain - Remove the orphaned subtitle line (Tyler r3572461161); the post now opens on the "LLM models change..." hook. - "That process is:" -> "This idea is:" under Hypothesis (frames the steps as the idea we set out to test). - Heading beats: "What We Achieved (With a Small Team)" -> "What We Achieved"; "Step-by-Step" -> "How the loop works"; "Workflow Overview" -> "The workflow, end to end". - Trim the "Fast Model -> Judge -> SME -> Ingest -> Iterate" refrain from 6 to 2 (kept at Hypothesis + Summary as a bookend); removes the dangling "Repeatable mantra (again):". No substantive copy reworded (token-multiset diff: only new heading words and "This idea" added; subtitle, mantra repeats, and old heading labels removed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@rtyler

…3323856) The Featured Series block sorted ascending and took the first two, so it always showed the two OLDEST `content-trust-series` posts (PhotoDNA + Dual-Embedding) and never surfaced newer entries. Add `| reverse` so it shows the two most recent series posts — consistent with the Featured Post logic just above, and with @rtyler's expectation that "the later post will appear on the home page." PhotoDNA keeps its `content-trust-series` tag (per @rtyler); it simply ages off the two-item display now that a newer series post exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

anishk123 · 2026-07-13T22:05:47Z

                <ul class="post-list text-length-sm">
                    <!-- Pick a series to features using its unique tag - i.e. di-series -->
-                    {% assign featured-series = site.tags[site.featured_series] | sort: 'date' %}
+                    {% assign featured-series = site.tags[site.featured_series] | sort: 'date' | reverse %}


@rtyler just checking to ensure that not being reverse wasn't intentional, was it?

anishk123 and others added 2 commits July 11, 2026 09:49

Merge pull request #1 from scribd/main

65144fa

Pull in head repo main

cursor Bot reviewed Jul 11, 2026

View reviewed changes

Comment thread _posts/2026-07-11-fast-llm-human-in-the-loop-classification.md

anishk123 force-pushed the fast-llm-hitl-classification branch from 6b27f7f to b159a27 Compare July 11, 2026 19:53

rtyler requested changes Jul 13, 2026

View reviewed changes

cursor Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread _posts/2026-07-11-fast-llm-human-in-the-loop-classification.md

anishk123 and others added 4 commits July 13, 2026 11:56

anishk123 commented Jul 13, 2026

View reviewed changes


		So instead of building a one-off system tied to a specific model or a single "perfect prompt," we focused on a repeatable process that we successfully applied to two large-scale projects (Slideshare and Scribd). That repeatable process is what we're sharing.

		Key learnings:

Uh oh!

Conversation

anishk123 commented Jul 11, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Visuals

Review fixes

Resulting homepage state

Uh oh!

Uh oh!

rtyler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

anishk123 commented Jul 11, 2026 •

edited by cursor Bot

Loading