Add support for llms.txt to provide LLM-friendly content metadata (like robots.txt)

[1chooo]

Goals

1. support MetadataRoute.LLMs in next/metadata
2. Or allow dropping public/llms.txt to be auto-served
3. Or even a file-based API route helper: pages/api/llms.txt.ts that can return dynamically generated values

Non-Goals

Background

llms.txt is an emerging convention (see: https://llmstxt.org/) proposed to standardize how websites communicate their preferences regarding how Large Language Models (LLMs) like ChatGPT or Gemini should crawl, ingest, and use site content.

This format is similar in spirit to robots.txt, but tailored for LLMs, and can include:

• Which pages or subdomains are designed for LLM use
• Licensing details (e.g., Creative Commons)
• Contact or attribution preferences
• Specific model permissions or disallowances

Proposal

Introduce built-in support in Next.js for serving a llms.txt file from the root of the site, similar to how robots.txt is handled, to explicitly indicate which parts of a site are designed to be LLM-friendly or LLM-restricted.

Next.js already emphasizes performance, SEO, and modern web practices. By enabling an easy way to serve llms.txt, developers can:

• Support content transparency for AI usage
• Protect intellectual property or attribution rights
• Indicate which parts of their site are curated for machine understanding

This aligns with Next.js’s commitment to modern, responsible web development.

[1chooo]

Sorry for duplicating idea in #80692 (comment) and #81182

[icyJoseph]

Hi,

We have gotten these requested here too:

• Support for llm.txt files #81182
• Support for llms.js to dynamically generate llms.txt similar to robots.txt and sitemap.xml #80692 (comment)

The last link includes a comment on how to do this, on user land for now.

We have also implement these here:

• https://nextjs.org/docs/llms.txt
• https://vercel.com/docs/llms.txt
• https://ai-sdk.dev/llms.txt