AEO tools for AI agents: crawler permissions, llms.txt, structured data, AI-readiness audit.
An MCP server that gives AI agents — Claude, Claude Code, Cursor, anything speaking the Model Context Protocol — tools to inspect a website's visibility to AI answer engines. Crawler permissions, llms.txt, structured data, on-page signals, and a full 29-check AI-readiness audit (powered by answer-audit), each one tool call away. Read-only, no API keys, no telemetry.

Real session: Claude Desktop discovers The Verge allows GPTBot (OpenAI licensing deal) while blocking ClaudeBot, PerplexityBot, Google-Extended, and CCBot — one tool call.
Claude Code
claude mcp add aeo -- npx -y aeo-mcp
Claude Desktop (claude_desktop_config.json)
{ "mcpServers": { "aeo": { "command": "npx", "args": ["-y", "aeo-mcp"] } } }
Cursor (.cursor/mcp.json)
{ "mcpServers": { "aeo": { "command": "npx", "args": ["-y", "aeo-mcp"] } } }
| Tool | When the agent uses it | Input | Returns |
|---|---|---|---|
audit_url | "How visible is this site to ChatGPT/Perplexity?" — full AEO health check | { url } | 0-100 score, grade, category breakdown, failed/warned checks with fixes |
check_ai_crawlers | "Is GPTBot blocked?" — robots.txt policy per AI bot | { url } | Per-bot allow/block verdicts with the matching rule, plus sitemaps |
inspect_llms_txt | "Does this site have llms.txt? Is it valid?" | { url } | Title, summary, sections with links, structural issues, raw excerpt |
extract_structured_data | "What schema markup does this page have?" | { url } | JSON-LD types, key fields per node, parse errors |
extract_page_signals | "Why do engines misread this page?" | { url } | Title, meta description, h1, canonical, lang, text ratio, heading outline, client-side-rendering flag |
Bots checked by check_ai_crawlers: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot, Bingbot.
The server fetches model-supplied URLs, so it ships with an SSRF guard: http/https only, DNS resolution checked against private and special ranges (loopback, RFC1918, link-local, unique-local, IPv4-mapped forms), re-validation on every redirect hop, 10 s timeout, 2 MB body cap, max 5 redirects. Violations return a clean tool error, never an exception. All tools are read-only and no LLM calls happen inside the server.
npm install
npm run build # required before npm test (the protocol test runs dist/)
npm test
Tests include a scripted stdio round-trip (initialize → tools/list → tools/call) against the built server. To inspect interactively:
npx @modelcontextprotocol/inspector node dist/index.js
MIT © Arthur Carlson
Source-derived launch command. Check the maintainer’s required arguments and credentials before running:
npx -y aeo-mcpMerge this template into ~/Library/Application Support/Claude/claude_desktop_config.json. Keep existing servers. Add any arguments, credentials, and permissions required by the maintainer; this template has not been install-tested.
{
"mcpServers": {
"io-github-arthurcarlsonn-aeo-mcp": {
"command": "npx",
"args": [
"-y",
"aeo-mcp"
]
}
}
}Restart Claude Desktop completely for changes to take effect. Confirm the server appears connected in the client’s tool list, then try a read-only example from its documentation.
Claude Desktop setup referenceaeo-mcpnpmio.github.arthurcarlsonn/aeo-mcp works with any MCP-compatible client. Copy the config snippet from the Configuration section above and add it to the file shown for your client, then restart the application.
~/Library/Application Support/Claude/claude_desktop_config.jsonRestart Claude Desktop completely for changes to take effect.~/.cursor/mcp.jsonRestart Cursor for changes to take effect..vscode/mcp.jsonReload VS Code window for changes to take effect.~/.codeium/windsurf/mcp_config.jsonRestart Windsurf for changes to take effect..mcp.jsonSave at the project root, then start Claude Code in that project and review the MCP server approval prompt. Keep real credentials out of shared files.