Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.
AI SEO tools inside Claude, Cursor, and other MCP-compatible clients.
Ask Claude "is my site AI-readable?" and it'll actually check — running a live audit against 50+ AI crawlers (ChatGPT, Claude, Perplexity, Google AI, and more). Then ask it to fix what's broken, and it can generate a proper llms.txt, robots.txt, or JSON-LD schema and write it straight into your project.
No more copy-paste between your editor and yet another SEO tool.
Eight tools your AI assistant can call:
Audit (no signup):
check_ai_readiness — Score a site 0–100 against 50+ AI crawlers, with a prioritized fix list; extras that don't affect the score are marked as suchvalidate_schema — Audit JSON-LD structured data on any URL (20+ types, per-type rules)validate_robots — Line-by-line robots.txt auditcheck_content_parity — Compare what humans see vs what AI crawlers seeGenerate (requires free API key):
generate_llms_txt — Auto-crawl a site and produce a properly formatted llms.txtgenerate_robots_txt — AI-crawler-aware robots.txt with preset policiesgenerate_schema — Organization / WebSite / Article JSON-LD ready to pasteMonitor (read-only, requires free API key):
get_monitor_trend — See whether ChatGPT, Claude, Perplexity & Google AI mention your brand vs competitors, and how that's trending week over week (reads your LLM Monitor projects; doesn't trigger runs)Open your config file:
~/Library/Application Support/Claude/claude_desktop_config.json%APPDATA%\Claude\claude_desktop_config.jsonAdd this to the mcpServers block:
{
"mcpServers": {
"crawl-readiness": {
"command": "npx",
"args": ["-y", "crawl-readiness-mcp"]
}
}
}
Restart Claude Desktop. You'll see "crawl-readiness" listed in the tools panel.
Open Cursor Settings → Features → Model Context Protocol → Add new global MCP server.
{
"crawl-readiness": {
"command": "npx",
"args": ["-y", "crawl-readiness-mcp"]
}
}
Same shape — add crawl-readiness to your MCP servers list with npx -y crawl-readiness-mcp as the command.
The four audit tools work out of the box, no signup. The three generator tools need an API key — free at crawlreadiness.com/dashboard (50 checks/month on the free tier, no credit card).
Once you have a key, add it via env in the same config:
{
"mcpServers": {
"crawl-readiness": {
"command": "npx",
"args": ["-y", "crawl-readiness-mcp"],
"env": {
"CRAWL_READINESS_API_KEY": "cr_live_xxxxxxxxxxxx"
}
}
}
}
Restart the client. All eight tools are now callable.
Once installed, try any of these in your AI client:
/public/llms.txt"Your AI can chain tools automatically — audit, identify issues, fix them, and write files to your project without you copy-pasting anything.
| Env var | Purpose | Default |
|---|---|---|
CRAWL_READINESS_API_KEY | Free API key for generator tools | none |
CRAWL_READINESS_API_BASE | Override the API base URL (for testing) | https://www.crawlreadiness.com |
Each tool returns the full JSON response from the Crawl Readiness API — the same data the web dashboard renders. Your AI has access to:
extra: true are optional and not scored)Crawl Readiness is a suite of AI-SEO tools that check whether AI systems can access your website and help you fix what's blocked. Every generator tool in this MCP server is also available on the web at crawlreadiness.com/tools. The MCP server just makes the same tools callable from inside your AI assistant, so the "audit → fix → apply" loop happens in one conversation without leaving your editor.
npm run release validates server.json and prints the publish steps. The last step uses the mcp-publisher CLI, which is not part of this repository: download it from the MCP Registry releases and keep it on your PATH or in the repo root (it is git-ignored there).
MIT © 2026 Crawl Readiness / Tundrastone
Source-derived launch command. Check the maintainer’s required arguments and credentials before running:
npx -y crawl-readiness-mcpMerge this template into ~/Library/Application Support/Claude/claude_desktop_config.json. Keep existing servers. Add any arguments, credentials, and permissions required by the maintainer; this template has not been install-tested.
{
"mcpServers": {
"io-github-grubbomatic-crawl-readiness": {
"command": "npx",
"args": [
"-y",
"crawl-readiness-mcp"
]
}
}
}Restart Claude Desktop completely for changes to take effect. Confirm the server appears connected in the client’s tool list, then try a read-only example from its documentation.
Claude Desktop setup referencecrawl-readiness-mcpnpmio.github.Grubbomatic/crawl-readiness works with any MCP-compatible client. Copy the config snippet from the Configuration section above and add it to the file shown for your client, then restart the application.
~/Library/Application Support/Claude/claude_desktop_config.jsonRestart Claude Desktop completely for changes to take effect.~/.cursor/mcp.jsonRestart Cursor for changes to take effect..vscode/mcp.jsonReload VS Code window for changes to take effect.~/.codeium/windsurf/mcp_config.jsonRestart Windsurf for changes to take effect..mcp.jsonSave at the project root, then start Claude Code in that project and review the MCP server approval prompt. Keep real credentials out of shared files.