Sitemap: chunks return their URLs again, and /sitemap.xml serves an index

[koutst]

The sitemap is effectively empty in production today. This fixes the two causes.

Changes

• Empty chunks: Next 16 passes the sitemap id as a Promise, so Number(id) was NaN and every chunk fell through to return []. app/sitemap.ts now awaits it. /sitemap/0.xml returns its 14 static URLs again and chunks 1–4 run their API fetches.
• Missing /sitemap.xml: generateSitemaps only serves /sitemap/<id>.xml, so the URL robots.ts advertises was a 404. A new route returns a sitemap index listing the chunks from generateSitemaps().
• Why the index is at app/sitemap-index.xml plus a rewrite: Next 16.3.4 still reserves app/sitemap.xml when app/sitemap.ts exists, and a route there fails the build with "Conflicting route and metadata at /sitemap.xml". The rewrite in next.config.js serves the index at /sitemap.xml. It is also reachable at /sitemap-index.xml.

Verified

• next build + next start, and next dev against a mock API: /sitemap.xml returns the index, /sitemap/0.xml to /sitemap/4.xml return the expected URLs, /sitemap/5.xml is a 404.
• Vercel preview: /sitemap.xml returns the index, all five chunks return URLs, and /sitemap/5.xml is a 404, so the rewrite holds there too.

Known gap, not fixed here

Against the production API, chunks 1–4 return only a fraction of the content:

Chunk	URLs returned	Total in the API
Papers	700	no count returned
Topics	1,240	91,121
Proposals	30	806
Grants	10	290

The API ignores page_size (it returns 10–100 per page), fetchAllPages stops after 8 seconds, and repeated paging draws 429s. A timeout or 429 ends the loop and the chunk is served short or empty with a 200. This predates the PR and was hidden while every chunk was empty. Closing it needs a backend change, such as a lightweight id-and-slug endpoint or honouring a large page_size.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
web	Ready	Preview	Oct 8, 2026 11:25pm UTC

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[koutst]

@codex

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-09T16:53:43.804978Z	1d0d9d1	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 1d0d9d1662

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".