SEO AI Crawlers Auditor
Audit whether AI search bots can crawl and cite your page via robots.txt
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 19, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Jul 30, 2026 · c3e8aff
⚠ This skill is no longer available upstream. Our re-check on Aug 10, 2026 couldn't find it any more (repo unreachable/deleted). The test below is what we measured on Jul 19, 2026 and we're leaving it up as a record — but there is nothing left to install, so we've removed the command.
Ran its parse-robots-sitemap.mjs live against theverge.com and it correctly resolved 106 robots groups into training/retrieval/user/search buckets, surfacing a real asymmetry (OAI-SearchBot allowed while Claude-SearchBot and PerplexityBot are blocked) that an eyeball pass over 106 groups would miss. The SKILL.md honesty section is unusually candid: it scores llms.txt impact at 0 and warns that blocking GPTBot does not block OAI-SearchBot. Caveat: the skill folder ships only SKILL.md, so a plain skill-copy leaves its references table, schema and verification script behind. Install the whole repo/plugin to get the parser.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 4/5
- Triggers reliably 5/5
- Output vs. baseline 8/10
- Docs & honesty 5/5
What SEO AI Crawlers Auditor does
Audits AI crawler access and citability for a page: confirms retrieval and citation bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed, checks the Googlebot vs Google-Extended split, classifies each user-agent as training vs search vs user-fetch, verifies non-JS renderability, and validates llms.txt. Triggers when the user asks whether AI search engines like ChatGPT or Perplexity can crawl and cite their site, or wants a robots.txt AI-bot audit. Ships a robots.txt parser that resolves the effective directive per bot.
How to install SEO AI Crawlers Auditor
Nothing to install: the source repository no longer has this skill. If the author brings it back, our daily re-check will pick it up and the command will reappear here.
Commands — how to trigger SEO AI Crawlers Auditor
-
/seo-ai-crawlersAudit whether AI search bots can crawl and cite your page via robots.txt
It also activates on plain-language prompts like these:
-
Can ChatGPT and Perplexity actually crawl and cite my site? -
Audit my robots.txt for AI crawler access and blocked bots. -
Check if my llms.txt setup is actually helping AI visibility.
Frequently asked questions
- Is the SEO AI Crawlers Auditor skill free?
- Yes. The skill itself is free from Hainrixz/claude-seo-ai. SkillProof publishes the install command and an independent test verdict at no cost.
- Does SEO AI Crawlers Auditor work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 19, 2026. Verdict: Tested · Works. Ran its parse-robots-sitemap.mjs live against theverge.com and it correctly resolved 106 robots groups into training/retrieval/user/search buckets, surfacing a real asymmetry (OAI-SearchBot allowed while Claude-SearchBot and PerplexityBot are blocked) that an eyeball pass over 106 groups would miss. The SKILL.md honesty section is unusually candid: it scores llms.txt impact at 0 and warns that blocking GPTBot does not block OAI-SearchBot. Caveat: the skill folder ships only SKILL.md, so a plain skill-copy leaves its references table, schema and verification script behind. Install the whole repo/plugin to get the parser.
- What is the SEO AI Crawlers Auditor SkillProof Score?
- 8.8/10 — installs cleanly 4/5, triggers reliably 5/5, output vs. baseline 8/10, docs & honesty 5/5.
- How do I install SEO AI Crawlers Auditor?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use SEO AI Crawlers Auditor with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.