Skip to content

Add a curated llms.txt generated from the docs.json nav - #47

Open
billlevine wants to merge 2 commits into
mainfrom
bill/ai-500-llms-txt
Open

Add a curated llms.txt generated from the docs.json nav#47
billlevine wants to merge 2 commits into
mainfrom
bill/ai-500-llms-txt

Conversation

@billlevine

Copy link
Copy Markdown
Contributor

Publishes a curated llms.txt at /docs/llms.txt, generated from the docs.json navigation tree so it cannot drift.

Part of the llms.txt reconciliation tracked in AI-500. A companion PR updates the site-level file in flox/floxwebsite.

Why

Mintlify auto-hosts an llms.txt. It is one flat alphabetical list with no preamble, and 40 of its 100 links carry no description at all — every man page, plus flox-vs-containers and organizations — because man/*.mdx has no description: frontmatter. Committing an llms.txt at the repo root overrides it.

Separately, the MkDocs→Mintlify migration dropped a set of agent guardrails that used to ship in floxdocs' llms.txt preamble and now exist nowhere on the docs site.

Approach: generate, don't hand-maintain

A hand-curated file drifts the moment someone adds a page. This derives it from the nav instead.

docs.json is already where curation happens — the CLI reference is already grouped into Overview / Setup / Environments / Packages / Sharing / Build & Publish / Services / Generations / Other — and check-man-nav.sh already fails PRs that add a page without a nav entry. So a page cannot reach the site without also reaching llms.txt. Curation happens once, in the place it already happens.

Mintlify's generated file This PR
Preamble none summary, agent rules, command surface, skills install, glossary
Links without descriptions 40 of 100 0 of 99
Sections 2 (Docs, OpenAPI Specs) 20, from nav groups
Man pages one flat list of 37 9 grouped sections
Back-link to /llms.txt none in the preamble
Drift when a page is added n/a CI fails until regenerated

What ships

  • scripts/generate-llms-txt.sh — walks the docs.json nav, emits one H2 per group in nav order. Link text is the page's frontmatter title; the description is its frontmatter description, falling back to the man page's ## NAME line, which is the canonical one-liner shipped with each command. Exits non-zero if the nav references a page with no .mdx.
  • llms.txt.header — the hand-written preamble, copied verbatim ahead of the generated sections.
  • llms.txt — generated and committed, since Mintlify serves it from the repo rather than building it.
  • .github/workflows/check-llms-txt.yml — regenerates and fails on drift. Mirrors check-man-nav.yml.
  • Descriptions added to concepts/flox-vs-containers.mdx and concepts/organizations.mdx, the only two non-man nav pages that lacked one. These also improve search and the sidebar.

Restored agent guardrails

These shipped in floxdocs and were lost in the migration:

  • The two non-interactive flox activate forms. Bare flox activate launches an interactive sub-shell that hangs a non-interactive harness; -c runs with hooks and profile scripts, -- is exec mode and skips profile scripts.
  • Use flox install, not apt/yum/dnf/brew.
  • Edit .flox/env/manifest.toml directly rather than flox edit, which opens \$EDITOR.
  • Catalog pkg-paths are not always nixpkgs attribute paths — verify with flox search / flox show.
  • There is no curl | bash installer; install.flox.dev and flox.dev/install do not exist. This one had its own bugfix commit in floxdocs (e3c2bf6), so it was a response to observed hallucination.

Note the old floxdocs rule was a blanket "NEVER run `flox activate` interactively." Reading flox-activate.mdx, interactive is one of four documented, legitimate modes — so this scopes the rule to non-interactive harnesses instead of contradicting the man page or the site-level llms.txt.

The preamble sits ahead of the first H2 deliberately. Strict llms.txt parsers extract only link lists from H2 sections, so prose under a heading gets dropped. Same choice floxwebsite made.

MCP server removed

The flox MCP server is deprecated in favour of the skill library, so it is gone from the preamble and from install-flox/ide-extensions.mdx. That page's "Skills and MCP" tab is now "Agent skills", repointed from flox/flox-agentic to flox/flox-skills (pushed 2026-07-27 vs flox-agentic's 2026-07-02), covering the flox and floxify skills with Claude Code, Codex, and skills.sh install paths.

No references to mcp or flox-agentic remain anywhere in the repo.

Known gap

The generated file omits Mintlify's ## OpenAPI Specs section. That link resolves to "title": "OpenAPI Plant Store" served against sandbox.mintlify.com — Mintlify's sample spec, tracked in DEV-203. Restore the section here once that is fixed.

Verification

  • All 105 URLs in llms.txt return 200
  • 99 pages emitted across 20 sections, 99 with descriptions, 0 bare links
  • Page set diffed against Mintlify's generated list: 99/99 covered, nothing dropped, nothing invented
  • Generator is idempotent — a second run produces no diff
  • grep -ri 'mcp\|flox-agentic' over the repo returns nothing

🤖 Generated with Claude Code

Mintlify auto-hosts an llms.txt, but it is a single flat alphabetical
list with no preamble, and 40 of its 100 links carry no description at
all because man/*.mdx has no `description:` frontmatter. Committing an
llms.txt at the repo root overrides it.

Rather than hand-maintain a curated file, generate it from the nav.
docs.json is already where curation happens, and check-man-nav.sh
already fails PRs that add a page without a nav entry, so a page cannot
reach the site without also reaching llms.txt. Sections come from nav
groups; descriptions come from frontmatter, falling back to the man
page's `## NAME` line, which is the canonical one-liner shipped with
each command. Result: 99 pages, 20 sections, 0 missing descriptions.

The preamble in llms.txt.header restores agent guardrails that were
lost when the docs moved off MkDocs — the non-interactive `flox
activate` forms, `flox install` over the system package manager,
editing manifest.toml directly, and the negation for the hallucinated
`install.flox.dev` / `flox.dev/install` URLs. It sits ahead of the
first H2 on purpose: strict llms.txt parsers extract only link lists
from H2 sections, so prose under a heading gets dropped.

Also drops the flox MCP server, which is deprecated in favour of the
skill library, and repoints flox-agentic at flox-skills.

The generated file omits Mintlify's `## OpenAPI Specs` section, which
links a spec that currently serves Mintlify's plant-store sample
(DEV-203). Restore it here once that is fixed.

Refs AI-500
@mintlify

mintlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
flox 🟢 Ready View Preview Jul 28, 2026, 2:14 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@@ -0,0 +1,151 @@
#!/usr/bin/env bash

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense ✅ My one real concern is that this needs a human to run a script, and I don't think we've priced that in.
The CI check finds drift, it doesn't fix it. That's fine when a person is editing docs by hand. It's less fine for sync-man-pages.yml, which runs on cron every morning and opens a bot PR. Descriptions come from the ## NAME line, and those get synced from flox/flox. So the day someone upstream rewords a NAME line, FloxBot's PR goes red and whoever's on rotation has to know the fix is "run the generator and push to the bot's branch." We'd be adding a manual step to a job that currently needs nobody.

Worth noting the check isn't enforced either. I get a 404 on branch protection for main, so a red check doesn't block anything right now (might just be my token lacking admin scope, worth a look in settings). check-man-nav.yml has the same gap so this isn't new, but "CI fails until regenerated" only does work if failing means something.

I've added a build script for the one on the website maybe we do something similar?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, and you're right that it wasn't priced in. Fixed in 835d4f8: the sync job now regenerates llms.txt after sync-man-pages.sh, with llms.txt added to add-paths (without that, create-pull-request drops it from the commit). The job that causes the drift is the one that can fix it, so FloxBot's PR arrives correct and nobody on rotation needs to know the incantation. The step sits outside flox activate on purpose — that environment ships node, vale and pandoc, and the generator needs python3 from the runner.

Verified by rewording a ## NAME line locally: it propagates to exactly one llms.txt line, which the new step now carries into the bot's commit.

On the website build script — that pattern doesn't port, unfortunately. floxwebsite generates llms-full.txt in postbuild because Vercel runs its build. Mintlify builds this site itself from repo contents and gives us no arbitrary build hook, so the file has to be committed and the automation has to happen at commit time. Regenerating in the producing job is the closest equivalent we have.

On branch protection: it wasn't your token. main is protected by a ruleset, not classic protection, and /branches/main/protection 404s for everyone in that case. The active "Main" ruleset has deletion, non_fast_forward, merge_queue and pull_request (1 approval) — and no required_status_checks rule at all, so you're right that red blocks nothing. Two traps before we add one, both of which bite this repo specifically:

  • Both checks use paths: filters. A required check that gets skipped sits "expected" forever and blocks the PR.
  • The ruleset has a merge queue, and neither workflow has a merge_group: trigger — a required check that never reports there times out at 60 minutes and ejects the entry.

So it's drop the path filters (both jobs run in seconds) plus add merge_group:, then require both checks. Happy to do that as its own PR since it affects check-man-nav too — it's a settings change and shouldn't ride along here.

Worth flagging: check-man-nav has the identical shape on the bot PR — a new command's page lands with no nav entry and goes red. That one I'd leave manual, since picking the group is a curation call, but we could auto-insert into "Other" so the reviewer re-groups rather than unblocks.

If hand-edited drift ever shows up in practice, the next step is a post-merge self-heal on main (regenerate, open a FloxBot PR on drift), which covers forks and web edits too. Not worth it until we see it.

llms.txt takes each man page's description from its `## NAME` line, and
those lines are synced from flox/flox. So the morning a command's NAME
is reworded upstream, FloxBot's sync PR goes red on check-llms-txt and
whoever is on rotation has to know the fix is "run the generator and
push to the bot's branch" — a manual step added to a job that needs
nobody today.

Regenerate in the sync job instead. The job that causes the drift is
the one that can fix it without a human, so the bot's PR arrives
correct. `llms.txt` joins `add-paths`, without which
create-pull-request drops the regenerated file from the commit.

The step sits outside `flox activate` deliberately: that environment
ships node, vale and pandoc, and the generator needs python3, which
comes from the runner.

check-llms-txt.yml stays as the net for hand-edited PRs, where the
author is in a local checkout and the error names the command to run.

Refs AI-500

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants