[harlan-zw]

🔗 Linked issue

❓ Type of change

• 📖 Documentation (updates to the documentation or readme)
• 🐞 Bug fix (a non-breaking change that fixes an issue)
• 👌 Enhancement (improving an existing functionality)
• ✨ New feature (a non-breaking change that adds functionality)
• 🧹 Chore (updates to the build process or auxiliary tools and libraries)
• ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

Cloudflare's Markdown converter allows AI training when the origin sends no Content-Signal header.
With robots.header: true, consistent wildcard policies now supply that header on origin responses and Cloudflare static assets.
Explicit response headers, static policies, and header removals take precedence.
Unspecified permissions stay unspecified.
Scoped groups and conflicting policies remain in robots.txt without a derived HTTP header.
If the fallback exceeds Cloudflare's 100-rule limit, the build fails after every header compiler finishes.

AI Ready #208 preserves source headers during Markdown conversion.
AI Ready can use automatic emission after upgrading to a Robots release containing this change.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

[harlan-zw]

🤖 Harlan Agent Kit checked Cloudflare's local asset server because CI only checks generated header rules.

With the app mounted at /docs/, prerendered HTML and static Markdown returned Content-Signal: ai-train=no, search=yes.
The explicit JSON policy returned only search=no.
The explicit removal returned no Content-Signal header.
All four responses returned HTTP 200.

This used the actual Cloudflare asset runtime through Wrangler local mode. The live edge was not exercised.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@nuxtjs/robots@363

commit: eac93b5

[github-actions]

Nuxt Module Size Analyzer

Runtime size changed.

Module	Client gzip	Server raw
@nuxtjs/robots	608 B	165 kB 🔴 +506 B (+0.3%)

Package files (6)

Rows overlap and include unused code. These totals do not measure deployed output or npm downloads.
Excludes dependency files, types, source maps, and other non-code files.

Package files	Summed gzip	Raw	Δ gzip
@nuxtjs/robots · export .	8.2 kB	31 kB	🔴 +1.0 kB (+14.4%)
@nuxtjs/robots · export ./content	292 B	503 B	—
@nuxtjs/robots · export ./util	487 B	817 B	🔴 +20 B (+4.3%)
@nuxtjs/robots · built code files	46 kB	150 kB	🔴 +2.9 kB (+6.7%)
@nuxtjs/robots · app source files	3.3 kB	7.2 kB	—
@nuxtjs/robots · server source files	9.4 kB	22 kB	🔴 +269 B (+3.0%)

Dependencies (8)

Declared dependencies. Their installed size does not show runtime cost.

Package	Dependency	Requested
@nuxtjs/robots	@fingerprintjs/botd	catalog:
@nuxtjs/robots	@nuxt/kit	catalog:
@nuxtjs/robots	@vueuse/core	catalog:
@nuxtjs/robots	consola	catalog:
@nuxtjs/robots	defu	catalog:
@nuxtjs/robots	nuxt-site-config	catalog:
@nuxtjs/robots	nuxtseo-shared	catalog:
@nuxtjs/robots	ufo	catalog:

How this is measured

• Compare the same app with and without the module.
• Client: all emitted JS/CSS, summed per-file gzip. This includes lazy chunks.
• Server: deployed files, including external dependencies. Source maps are excluded.
• Default module settings, Node server preset. Memory and request speed are outside this report.

_{Base: main @ b2a4dbe · 2026-10-13 · differences below 16 B are ignored}

[harlan-zw]

Harlan Agent Kit posted this automated status. AI open source policy.

Release @nuxtjs/robots@7.1.0 after this pull request merges and default branch checks pass.
Includes all unreleased changes since v7.0.2.
Selection applies to head eac93b5db91a878c393e869f4f7d83c27d0b83c4. A changed head or release version clears it.
Clear the checkbox to cancel before merge.

• Release minor after merge

[harlan-zw]

🤖 READY · 89/100

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. A person still decides the merge.

• Merge gate: Passed.

• Review gate: Passed. No finding above 80/100.

• CI gate: Passed.

• Logged (62/100): Content-Signal-only policies from merged robots files are discarded. View code Optional follow-up: Treat content preference directives as valid group rules during validation instead of filtering those groups out.

• Logged (32/100): Duplicate wildcard declarations unnecessarily suppress automatic header emission. View code Optional follow-up: Accept groups whose user-agent entries are all *, or deduplicate user agents during normalization.

[harlan-zw]

Harlan Agent Kit verification of the final static budget guard:

A standalone Cloudflare build with 100 existing header rules failed after the fallback created rule101.
The error retained policy rules and instructed the operator to reduce rules or share a policy.
A later header compiler could reduce optional rules before the final guard ran.
The PR Lens SVG was rasterized and its pixels inspected locally. Labels and arrows remain visible.