Repository navigation
Conversation
A dated catalog of how MCP servers are being built: host limits that constrain every server, what spec 2026-07-28 changed, and practices by area, each with its source, date and evidence type. External sources only; host limits re-checked against primary pages and, for Claude Code, the shipped binary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Docs-only PR (no generated files, tools, or templates touched), so none of the CLAUDE.md § Review guidelines items apply — no Tool.ui, char-budget, tool-wiring, WORKFLOWS.md, test, or secrets concerns. No Leadbay-internal data leaked, matching the PR description.
However, spot-checking the "Host facts" table (the section the PR says was "re-read on the primary pages on 2026-09-18") against its own cited source turned up fabricated specifics sitting next to real ones: the 2,048-character description/instructions truncation claim, the "10% of context window" auto-deferral trigger, and 3 of 4 claimed _meta keys don't appear on the cited code.claude.com/docs/en/mcp page, while the neighboring claims in the same rows (25k/500k token-output caps, 2-min backgrounding, 5-min idle timeout, anthropic/maxResultSizeChars) check out exactly. Since this doc's stated purpose is to inform future design decisions here, and real/fabricated facts are interleaved under the same "Verified" citation, I'd want at least section 1 re-audited before merging — a reader can't tell which specific numbers to trust.
The docs state the 2KB cut, the 10% tool-search threshold and anthropic/requiresUserInteraction. The exact 2,048 figure, the '… [truncated]' marker and the anthropic/searchHint and _meta anthropic/alwaysLoad keys come from the 2.1.247 binary and are now cited to it; the two undocumented keys are labelled Unverified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Follow-up on the prior CHANGES_REQUESTED review (5254151929), which flagged the 2,048-char truncation figure/marker and 3 of 4 _meta keys as not appearing on the cited docs page.
5c0b9c4 addresses this: the exact figure and … [truncated] marker are now cited to the 2.1.247 binary rather than the docs page, the 10%-threshold auto-deferral claim is quoted from the docs' tool-search section, and the _meta row now separates documented keys (maxResultSizeChars, requiresUserInteraction) from binary-only/undocumented ones (searchHint, alwaysLoad) — correctly noting the docs' own alwaysLoad is a different, transport-config field, not a _meta key. I re-fetched the docs page and this distinction checks out. No new issues in this diff (docs-only, no CLAUDE.md § Review guidelines items apply).
Note: I couldn't dismiss review 5254151929 programmatically — gh api dismissal calls aren't available in this environment — a maintainer should dismiss it manually since its issues are resolved.
[Claude]: Resolved in 5c0b9c4; the bot confirmed it in its follow-up review.
There was a problem hiding this comment.
Docs-only PR (a new reference file, no generated files/tools/templates touched) — none of the CLAUDE.md § Review guidelines items (Tool.ui, char budget, tool wiring, WORKFLOWS.md, tests, secrets) apply. A prior review round already caught fabricated specifics in the section 1 host-facts table; that was addressed in 5c0b9c4 and checks out now. Spot-checking the rest of the doc, two more claims break the doc's own citation convention (every other quantitative claim has a linked, bibliography-listed primary source) — flagging those for a source check before merge.
| | Practice | Evidence | Label | | ||
| |---|---|---| | ||
| | Assume the lethal trifecta: private data, untrusted content, an exfiltration path. | Willison, [2025-06-16](https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/). Invariant Labs GitHub exploit (2025-05-26); General Analysis on Supabase; Block "Operation Pale Fire" red team via zero-width characters (2026-01-15). | Opinion, Measured | | ||
| | Tool poisoning works: poisoned descriptions succeeded 36.5% of the time on average, refusals under 3%. | MCPTox. | Measured | |
There was a problem hiding this comment.
Unlike every other 'Measured' row in this table, this one has no link, and "MCPTox" never appears in the Sources section at the bottom (all other named papers/tools do, e.g. the arXiv IDs). This is the same pattern (specific stat, thin/absent citation) already caught and fixed elsewhere in this PR for the host-facts table — worth the same verify-or-cite treatment before merge.
| |---|---|---| | ||
| | Consolidated tools or granular ones? | GitHub −50% tokens on Projects | GitHub re-added granular tools for per-operation permissions | | ||
| | Server-side tool discovery or host tool search? | Sentry and Stripe keep search/execute | GitHub removed its dynamic toolsets; Cursor and Claude Code defer on the host | | ||
| | MCP or CLI plus skills? | Playwright now recommends CLI plus skills for coding agents | Checkly measured 48-50k versus 45-48k tokens once deferral is on; chat hosts have no shell | |
There was a problem hiding this comment.
Same issue: "Checkly" is cited with a specific token-count comparison but isn't linked and doesn't appear in the Builders list in Sources, unlike Sentry/PostHog/Datadog/etc. which are all listed there. Please add a source or drop the specific numbers.
Adds
docs/mcp-emerging-practices.md: a dated reference of how MCP servers are being built, for future design decisions in this repo.Contents:
External sources only. No Leadbay telemetry or internal numbers are in the file, because the repo is public. The assessment of this repo against the catalog is not in this PR.
Verified: the host limits in section 1 were re-read on the primary pages on 2026-09-18. The Claude Code description and instructions cut at 2,048 characters and the 25,000-token output cap were also read in the shipped 2.1.247 binary. Items the survey could not confirm from a primary source are labelled Unverified.
No code or generated file changes.
🤖 Generated with Claude Code