Markdown for Agents: What It Is and How to Set It Up

An agent asks for your page.

Your server sends Markdown. Not HTML.

Same URL. Browsers still get HTML.

That's markdown for agents.

The client sends Accept: text/markdown. The server answers in Markdown. The mechanism is HTTP content negotiation. It's decades old.

Why bother? An agent pays for every token it reads.

Coding agents such as Claude Code and Cursor get the page without the navigation, scripts and wrappers.

On the pages measured below, the Markdown copy is 80% to 99.7% smaller than the HTML.

And for AI search? The short answer: it's plumbing for coding agents today.

Public tests found no clear gain in citations.

One more thing. "Markdown for Agents" is also the name of Cloudflare's feature. The practice works on any stack.

How markdown for agents works

A client lists the formats it wants in the Accept header.

A server that supports Markdown answers with Content-Type: text/markdown.

Any other client gets HTML.

Claude Code, for example, sends Accept: text/markdown, text/html, */*.

RFC 7763 registered the text/markdown type in 2016.

This is a trimmed response from Cloudflare's documentation:

HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: accept
x-markdown-tokens: 725
x-original-tokens: 12345
content-signal: ai-train=yes, search=yes, ai-input=yes

---
title: Markdown for Agents · Cloudflare Agents docs
---

## What is Markdown for Agents

What Cloudflare's converter does:

• Drops headers, footers, navigation, scripts and styles
• Builds YAML frontmatter from the title, description and og:image meta tags
• Keeps JSON-LD at the end

Always send Vary: Accept.

Without it, a CDN can hand HTML to an agent. Or Markdown to a browser. (AcceptMarkdown)

There are three ways to ship it.

• Negotiation on the canonical URL. The header picks the format. The agent needs no knowledge of your site. Vercel says this beats separate .md URLs for that reason.
• A sibling .md URL. The agent must know it or find it through a link. Roots notes that a .md URL that ignores the header is not content negotiation.
• Edge conversion. A CDN does the first way for you by converting your HTML.

What this site does: the second way.

A Markdown copy of this post lives at /markdown/blog/markdown-for-agents with noindex.

A request to the blog URL with the header still returns HTML. (Checked with curl on 2 October 2026.)

The LLMOnly component post covers the user agent route and when it counts as cloaking.

Do not confuse the term with:

• AGENTS.md: a file of instructions for coding agents.
• llms.txt: an index of your site for LLMs.

Which agents ask for Markdown

Mostly coding agents ask. Most answer engines don't.

AcceptMarkdown.com keeps a matrix of which AI tools send Accept: text/markdown. It tests each tool's own browse or fetch tool.

The page says "Last updated 2026-06-22". Brackets show each row's verified date:

Support moves.

In February 2026, a Checkly test found that GitHub Copilot's fetch tool did not send the header.

The June matrix says Copilot does.

So check your own tool.

One limit. These tests cover each tool's built-in fetch. Not the search crawlers that OpenAI or Google run.

The tools that ask are coding agents.

The answer engines that most people want citations from did not ask.

How much smaller the page gets

Five published measurements. Plus this site's own.

Units differ.

Cloudflare's 80% is the lowest figure here. Not the typical one.

The saving depends on how much of the page is markup, navigation and scripts. So measure your own pages.

The copies on this site are each page's source. Not a conversion of the HTML.

Pick your setup

$49 report

Does AI name you?

See if AI mentions your brand. Then get the pages to fix.

Your website

Get my $49 report

Check one question freeHow it works

What's the best AI SEO tracker for a startup?

Most answers name Semrush. You aren't cited.

Ask ChatGPT

Set it up

On Cloudflare

1. In the dashboard, select the zone.
2. Open AI Crawl Control.
3. Enable Markdown for Agents.

Cloudflare's availability note lists Pro, Business and Enterprise plans, and SSL for SaaS. At no cost.

Only one subdomain or path? Add a configuration rule with an expression such as starts_with(http.request.uri.path, "/blog/").

One thing to do first. If your origin sets its own content-signal header, Cloudflare keeps it.

Want a stricter policy than the default? Set it before you enable the feature.

In a Next app

Vercel's blog shows a rewrite. It sends requests with a Markdown Accept header to a Markdown route. Trimmed:

// next.config.ts
const nextConfig = {
  async rewrites() {
    return {
      beforeFiles: [
        {
          source: '/blog/:path*',
          has: [{ type: 'header', key: 'accept', value: '(.*)text/markdown(.*)' }],
          destination: '/md/blog/:path*',
        },
      ],
    }
  },
}

export default nextConfig

The route handler at /md/blog/... returns Markdown with Content-Type: text/markdown; charset=utf-8.

Posts already in Markdown or MDX? Return the source.

Send Vary: Accept on both responses.

A rewrite needs no middleware file. And Next.js 16 renamed that convention to proxy.

On Nginx

AcceptMarkdown's recipe pre-renders a .md file next to each page. Nginx picks one from the Accept header. Trimmed:

map $http_accept $preferred_ext {
  default ".html";
  "~*text/markdown" ".md";
}

server {
  root /var/www/yoursite;
  add_header Vary Accept always;

  location / {
    try_files $uri$preferred_ext $uri/index$preferred_ext $uri.html $uri/index.html =404;
  }

  location ~* \.md$ {
    default_type text/markdown;
    charset utf-8;
    add_header Vary Accept always;
  }
}

The map matches a substring. So a client that sends text/markdown;q=0 still gets Markdown.

For strict handling, the recipe suggests a small app that parses Accept.

AcceptMarkdown also has Caddy and Apache recipes.

From Markdown files or WordPress

Already write Markdown? Publish the source.

A warning from Flavio Copes. He converted his HTML back to Markdown at the edge. Every request became a metered Workers invocation.

His static site started to use up the free quota.

He now builds a .md file next to each post.

Advertise the file with a link in the page head:

<link rel="alternate" type="text/markdown" href="/page.md">

Codex CLI follows this tag, per AcceptMarkdown. Pronovix also sends it as an HTTP Link header.

On WordPress, pick a plugin that reads the header on the canonical URL.

• Roots found that AIOSEO's .md feature never looks at the header.
• The Markdown for AI Agents plugin says it reads the header and sets Vary: Accept.
• Roots publishes roots/post-content-to-markdown.

Those are the projects' own descriptions. Test with curl before you trust either.

Test it

Run three commands.

(If your stack does not answer HEAD requests, use curl -s -D - -o /dev/null instead of curl -sI.)

curl -sI -H "Accept: text/markdown" https://yoursite.com/page | grep -i -E "content-type|vary"
curl -sI -H "Accept: text/html" https://yoursite.com/page | grep -i content-type
curl -s -H "Accept: text/markdown" https://yoursite.com/page | head -30

A pass looks like this:

• The first returns text/markdown and a vary header with Accept.
• The second still returns text/html.
• The body starts with a title or frontmatter and has no navigation or script text.
• Headings, prices and links match the HTML page.

Then check the cache.

Request Markdown, then HTML. Twice each.

A CDN that ignores Vary: Accept can hand the wrong body to the next client.

Next, find out who asks.

• Ask your own coding agent to fetch https://httpbin.org/headers. Read the Accept line. (Checkly's method.)
• Add $http_accept to your access log. AcceptMarkdown shows formats for Nginx, Apache and Caddy.

John Mueller's advice is the same: log the header and check the metrics first.

Three free tools help:

• AcceptMarkdown.com checks a URL for Markdown, the Vary header and q-values.
• Is Your Site Agent-Ready? scans for Markdown negotiation and other agent standards.
• markdown.new previews a public URL as Markdown. Prepend markdown.new/ to the URL.

Want to see what AI can read from your HTML today? Page Inspector fetches a public URL and shows the extract: title, meta description, headings, copy and visible page text.

What it does for SEO and AI visibility

It lowers what an agent downloads.

That's the proven part.

AI bot visits? Citations? Public tests found no clear gain.

Google.

In July 2026 John Mueller replied that a separate "agent-friendly" version of a site builds technical debt. (Search Engine Journal)

In August, Search Engine Journal quoted him again: on his test sites, only SEO tools claim to accept Markdown. (Source)

Cloaking.

Google's spam policy defines cloaking as showing users and search engines different content to manipulate rankings.

In February 2026, Search Engine Journal noted two things. Mueller's criticism targeted bot pages served by user agent. And Google had not said whether header negotiation counts.

The safe pattern, from Pronovix and Roots:

• The same facts in both formats
• No user agent checks
• The HTML page canonical
• rel="alternate" on any .md copy

Bing.

Fabrice Canel wrote in February 2026 that Bing ranks on what customers see. And he asked who fixes a Markdown copy that breaks. (Search Engine Roundtable)

The rule.

Ship it if coding agents read your site (docs, an API, developer tools). Or if you already write in Markdown.

Do not ship it to win ChatGPT or Perplexity citations.

Fix the HTML first:

• Put the answer in the page text.
• Let the bots in.
• Say the thing plainly.

Then read AI citations, how to get cited in AI answers, generative engine optimization and answer engine optimization.

A live example. The review of The Prompting Company records that its site returns Markdown for /pricing with the header, as of 2 October 2026.

If you ship Markdown, check whether AI answers change. AI SEO Tracker shows how ChatGPT, Perplexity, Gemini, Copilot and Google AI Overviews mention and cite a brand.