SEO AI Crawlers Auditor

Audit whether AI search bots can crawl and cite your page via robots.txt

Tested · Works

Test report

Verdict
Tested · Works
Score
8.8/10
Tested
Jul 19, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Jul 30, 2026 · c3e8aff

This skill is no longer available upstream. Our re-check on Aug 10, 2026 couldn't find it any more (repo unreachable/deleted). The test below is what we measured on Jul 19, 2026 and we're leaving it up as a record — but there is nothing left to install, so we've removed the command.

Ran its parse-robots-sitemap.mjs live against theverge.com and it correctly resolved 106 robots groups into training/retrieval/user/search buckets, surfacing a real asymmetry (OAI-SearchBot allowed while Claude-SearchBot and PerplexityBot are blocked) that an eyeball pass over 106 groups would miss. The SKILL.md honesty section is unusually candid: it scores llms.txt impact at 0 and warns that blocking GPTBot does not block OAI-SearchBot. Caveat: the skill folder ships only SKILL.md, so a plain skill-copy leaves its references table, schema and verification script behind. Install the whole repo/plugin to get the parser.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 4/5
  • Triggers reliably 5/5
  • Output vs. baseline 8/10
  • Docs & honesty 5/5

What SEO AI Crawlers Auditor does

Audits AI crawler access and citability for a page: confirms retrieval and citation bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed, checks the Googlebot vs Google-Extended split, classifies each user-agent as training vs search vs user-fetch, verifies non-JS renderability, and validates llms.txt. Triggers when the user asks whether AI search engines like ChatGPT or Perplexity can crawl and cite their site, or wants a robots.txt AI-bot audit. Ships a robots.txt parser that resolves the effective directive per bot.

How to install SEO AI Crawlers Auditor

Nothing to install: the source repository no longer has this skill. If the author brings it back, our daily re-check will pick it up and the command will reappear here.

Commands — how to trigger SEO AI Crawlers Auditor

  • /seo-ai-crawlers Audit whether AI search bots can crawl and cite your page via robots.txt

It also activates on plain-language prompts like these:

  • Can ChatGPT and Perplexity actually crawl and cite my site?
  • Audit my robots.txt for AI crawler access and blocked bots.
  • Check if my llms.txt setup is actually helping AI visibility.

Frequently asked questions

Is the SEO AI Crawlers Auditor skill free?
Yes. The skill itself is free from Hainrixz/claude-seo-ai. SkillProof publishes the install command and an independent test verdict at no cost.
Does SEO AI Crawlers Auditor work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 19, 2026. Verdict: Tested · Works. Ran its parse-robots-sitemap.mjs live against theverge.com and it correctly resolved 106 robots groups into training/retrieval/user/search buckets, surfacing a real asymmetry (OAI-SearchBot allowed while Claude-SearchBot and PerplexityBot are blocked) that an eyeball pass over 106 groups would miss. The SKILL.md honesty section is unusually candid: it scores llms.txt impact at 0 and warns that blocking GPTBot does not block OAI-SearchBot. Caveat: the skill folder ships only SKILL.md, so a plain skill-copy leaves its references table, schema and verification script behind. Install the whole repo/plugin to get the parser.
What is the SEO AI Crawlers Auditor SkillProof Score?
8.8/10 — installs cleanly 4/5, triggers reliably 5/5, output vs. baseline 8/10, docs & honesty 5/5.
How do I install SEO AI Crawlers Auditor?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use SEO AI Crawlers Auditor with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.