Two Route Handlers for llms.txt, Zero Drift

I was staring at a pull request diff that added two files I didn't expect: app/llms.txt/route.ts and app/[locale]/llms.txt/route.ts. No UI, no components, just route handlers returning plain text. I ran the app, requested the new path, and got text/plain; charset=utf-8 back: a short markdown document with an H1 and a couple of link lists.

My first guess was wrong. I assumed llms.txt was basically sitemap.xml for LLMs: one root file, every URL listed, and some crawler would maybe pick it up. I also assumed I'd hand-write it and watch it slowly drift, like a README nobody updates.

I thought it was a sitemap for LLMs

It's not. sitemap.xml tries to list everything for a search engine to crawl [1][3]. llms.txt is the opposite: a small curated overview meant to be fetched on demand by an agent during inference [1][3]. Treat it like a sitemap and you'll dump 200 links in there and defeat the point. The v2 spec even recommends rel="alternate" type="text/markdown" links for markdown versions of pages and rel="describedby" pointing at the covering llms.txt [1]. That's the intended flow: the agent reads the tiny overview, then decides which page to fetch next.

The sitemap mental model also made me think search engines would care. They don't, at least not today. The honest consumers right now are coding agents and IDE tools that fetch context while they work. That's my take after shipping this, and it's why I wouldn't pitch this file as an SEO win.

The spec is smaller than I expected

The v2 spec says the file can live at the site root or at any subpath, and it covers the URLs under that path [1]. When multiple files match, the agent uses the most specific one [1]. Structure is strict in a minimal way: an H1 with the site name is the only required section, then an optional blockquote, then H2 link lists with an Optional section for links an agent can skip [1]. The proposal started at answer.ai in September 2024 because assembling website context for an LLM was ambiguous, and markdown was chosen because plain parsers or even regex can read it [3].

Next.js route handlers are custom request handlers built on the Web Request and Response APIs [2]. The docs call out non-UI content and mention sitemap.xml and robots.txt as built-in flavors of the same idea [2]. Segment config like dynamic controls caching behavior [2].

Two routes, one source of truth

Both handlers return text/plain; charset=utf-8 with Cache-Control: public, max-age=3600 and export const dynamic = "force-static" so the output is statically generated. The locale handler awaits params, a Promise in the current App Router, and falls back to id when the locale isn't supported:

// app/[locale]/llms.txt/route.ts
import { hasLocale } from "@/config/nav";
import { llmsTxt } from "@/lib/llms";

export const dynamic = "force-static";

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ locale: string }> },
) {
  const { locale } = await params;
  return new Response(llmsTxt(hasLocale(locale) ? locale : "id"), {
    headers: {
      "Content-Type": "text/plain; charset=utf-8",
      "Cache-Control": "public, max-age=3600",
    },
  });
}

The important part isn't in this file at all. It's in lib/llms.ts, which imports the same content modules the pages render, so llmsTxt() builds the markdown from the source of truth instead of duplicating copy. I've seen teams generate the file with a separate script that copies strings out of pages. It works until the first content update, and then you have two places to change and you will forget one. That anti-drift setup is the only part I'd insist on if you copy this pattern.

If you're adding this to your own site, start by checking what you already have. Request /llms.txt and /en/llms.txt on your dev server and confirm the response is text/plain with a cache header. If you see text/html, your route isn't matching. If there's no Cache-Control, you forgot the segment config [2]. Then open the body: the H1 should be your site name and the links should actually resolve. Keep it short. Curate, don't dump.

I still don't know whether this file will matter in a year. Generating it from the same modules as your pages costs almost nothing and keeps it honest. That's enough for me to keep both routes.

Sources

[1] llmstxt.org: The /llms.txt file, v2
[2] Next.js docs: route.js / Route Handlers
[3] Answer.AI: /llms.txt proposal