Skill Evaluator
Scores a SKILL.md against Anthropic best practices with a validator script and weighted rubric
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 21, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Aug 10, 2026 · e8d142a
Cloned the repo and ran scripts/validate_skill.py against a deliberately weak test skill (name "pdf", vague description); it emitted valid JSON flagging the non-gerund name and missing activation triggers, exit 0. I noticed a real gap: the path_style check only inspects markdown-link references, so the bare Windows path "references\extra.md" was NOT flagged. Then I produced two artifacts: a baseline free-form review (6 lines, no score) vs a skill-followed report (weighted 1.50/5.0 with a per-dimension table, reproduced validator warnings, and a "Poor" recommendation band) - the skill demonstrably adds a reproducible number and forces the checks to run. All 3 referenced files returned HTTP 200; no security smells.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 7/10
- Docs & honesty 4/5
What Skill Evaluator does
Evaluates an agent skill's SKILL.md against Anthropic's authoring best practices, combining an automated Python validator (naming, description length, body length, reference depth) with a six-dimension weighted rubric that yields a 1-5 score and a templated report. Triggers when the user asks to review, evaluate, assess, or audit a skill for quality. Produces per-dimension scores, detected anti-patterns, and prioritized fix recommendations.
How to install Skill Evaluator
git clone --depth 1 https://github.com/gotalab/skillport.git /tmp/skill-evaluator-src
mkdir -p ~/.claude/skills
cp -R /tmp/skill-evaluator-src/.skills/experimental/skill-evaluator ~/.claude/skills/skill-evaluator
# No deps: scripts/validate_skill.py is pure Python 3 stdlib (argparse, re, json, pathlib).
# Run the validator: python3 ~/.claude/skills/skill-evaluator/scripts/validate_skill.py <path/to/skill> --json
Skills live in ~/.claude/skills/ (global) or .claude/skills/
(per-project). Restart Claude Code after installing.
Commands — how to trigger Skill Evaluator
-
/skill-evaluatorScores a SKILL.md against Anthropic best practices with a validator script and weighted rubric
It also activates on plain-language prompts like these:
-
Evaluate this skill's SKILL.md for quality -
Review my skill description for clarity -
Audit this skill for structural anti-patterns
Frequently asked questions
- Is the Skill Evaluator skill free?
- Yes. The skill itself is free from gotalab/skillport. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Skill Evaluator work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 21, 2026. Verdict: Tested · Works. Cloned the repo and ran scripts/validate_skill.py against a deliberately weak test skill (name "pdf", vague description); it emitted valid JSON flagging the non-gerund name and missing activation triggers, exit 0. I noticed a real gap: the path_style check only inspects markdown-link references, so the bare Windows path "references\extra.md" was NOT flagged. Then I produced two artifacts: a baseline free-form review (6 lines, no score) vs a skill-followed report (weighted 1.50/5.0 with a per-dimension table, reproduced validator warnings, and a "Poor" recommendation band) - the skill demonstrably adds a reproducible number and forces the checks to run. All 3 referenced files returned HTTP 200; no security smells.
- What is the Skill Evaluator SkillProof Score?
- 8.4/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 7/10, docs & honesty 4/5.
- How do I install Skill Evaluator?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Skill Evaluator with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.