Skill Evaluator

Scores a SKILL.md against Anthropic best practices with a validator script and weighted rubric

Tested · Works

Test report

Verdict
Tested · Works
Score
8.4/10
Tested
Jul 21, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Aug 10, 2026 · e8d142a

Cloned the repo and ran scripts/validate_skill.py against a deliberately weak test skill (name "pdf", vague description); it emitted valid JSON flagging the non-gerund name and missing activation triggers, exit 0. I noticed a real gap: the path_style check only inspects markdown-link references, so the bare Windows path "references\extra.md" was NOT flagged. Then I produced two artifacts: a baseline free-form review (6 lines, no score) vs a skill-followed report (weighted 1.50/5.0 with a per-dimension table, reproduced validator warnings, and a "Poor" recommendation band) - the skill demonstrably adds a reproducible number and forces the checks to run. All 3 referenced files returned HTTP 200; no security smells.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 5/5
  • Triggers reliably 5/5
  • Output vs. baseline 7/10
  • Docs & honesty 4/5

What Skill Evaluator does

Evaluates an agent skill's SKILL.md against Anthropic's authoring best practices, combining an automated Python validator (naming, description length, body length, reference depth) with a six-dimension weighted rubric that yields a 1-5 score and a templated report. Triggers when the user asks to review, evaluate, assess, or audit a skill for quality. Produces per-dimension scores, detected anti-patterns, and prioritized fix recommendations.

How to install Skill Evaluator

git clone --depth 1 https://github.com/gotalab/skillport.git /tmp/skill-evaluator-src
mkdir -p ~/.claude/skills
cp -R /tmp/skill-evaluator-src/.skills/experimental/skill-evaluator ~/.claude/skills/skill-evaluator
# No deps: scripts/validate_skill.py is pure Python 3 stdlib (argparse, re, json, pathlib).
# Run the validator: python3 ~/.claude/skills/skill-evaluator/scripts/validate_skill.py <path/to/skill> --json

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger Skill Evaluator

  • /skill-evaluator Scores a SKILL.md against Anthropic best practices with a validator script and weighted rubric

It also activates on plain-language prompts like these:

  • Evaluate this skill's SKILL.md for quality
  • Review my skill description for clarity
  • Audit this skill for structural anti-patterns

Frequently asked questions

Is the Skill Evaluator skill free?
Yes. The skill itself is free from gotalab/skillport. SkillProof publishes the install command and an independent test verdict at no cost.
Does Skill Evaluator work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 21, 2026. Verdict: Tested · Works. Cloned the repo and ran scripts/validate_skill.py against a deliberately weak test skill (name "pdf", vague description); it emitted valid JSON flagging the non-gerund name and missing activation triggers, exit 0. I noticed a real gap: the path_style check only inspects markdown-link references, so the bare Windows path "references\extra.md" was NOT flagged. Then I produced two artifacts: a baseline free-form review (6 lines, no score) vs a skill-followed report (weighted 1.50/5.0 with a per-dimension table, reproduced validator warnings, and a "Poor" recommendation band) - the skill demonstrably adds a reproducible number and forces the checks to run. All 3 referenced files returned HTTP 200; no security smells.
What is the Skill Evaluator SkillProof Score?
8.4/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 7/10, docs & honesty 4/5.
How do I install Skill Evaluator?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use Skill Evaluator with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.