Is the Agent Benchmark skill free?

Yes. The skill itself is free from vibeeval/vibecosystem. SkillProof publishes the install command and an independent test verdict at no cost.

What is the Agent Benchmark SkillProof Score?

5.6/10 — installs cleanly 3/5, triggers reliably 5/5, output vs. baseline 3/10, docs & honesty 3/5.

How do I install Agent Benchmark?

Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.

Can I use Agent Benchmark with Cursor, Copilot, Gemini CLI, Codex or other AI tools?

The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.

Agent Benchmark

Methodology for scoring agent output quality and catching regressions against baselines

by vibeeval · vibeeval/vibecosystem

Works with setup

Install — copy the command Download source ↗

Test report

Verdict: Works with setup
Score: 5.6/10
Tested: Jul 21, 2026
Environment: Claude Code 2.x (agent harness)
Upstream re-checked: Aug 10, 2026 · 4937080

Listed the repo tree via the GitHub API and fetched skills/agent-benchmark/SKILL.md raw; the directory contains only SKILL.md. Frontmatter parses cleanly with name+description and no security smells (no curl|sh, base64, or exfiltration). Spot-checked referenced scripts: run.mjs returns 404 at both skills/agent-benchmark/run.mjs and benchmarks/run.mjs, and no fixtures/ground-truth/rubrics/baselines directories ship — so every documented `node ~/.claude/benchmarks/run.mjs` command is non-functional out of the box (canavar-cli.mjs, referenced for the ledger integration, does exist at hooks/dist/). I could not produce a benchmark-run artifact because the scoring engine is absent and cannot be derived from the skill, so outputMeasured=false and the missing manual step is authoring run.mjs plus all fixtures/ground-truth.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

Installs cleanly 3/5
Triggers reliably 5/5
Output vs. baseline 3/10
Docs & honesty 3/5

What Agent Benchmark does

A methodology document for measuring agent response quality with scoring rubrics, ground-truth fixtures, and timestamped baselines to detect regressions. Triggers when evaluating agent definition changes, auditing quality, or establishing performance baselines. Ships only the spec — every documented `node run.mjs` command depends on a runner script that must be authored first.

How to install Agent Benchmark

git clone --depth 1 https://github.com/vibeeval/vibecosystem.git /tmp/agent-benchmark-src
mkdir -p ~/.claude/skills
cp -R /tmp/agent-benchmark-src/skills/agent-benchmark ~/.claude/skills/agent-benchmark
# SKILL.md is the only file that ships. It is a methodology/spec, NOT a runnable tool.
# The runner it invokes everywhere (node ~/.claude/benchmarks/run.mjs) does NOT exist in the repo (HTTP 404).
# No fixtures/, ground-truth/, rubrics/, or baselines/ ship either — you must author run.mjs and all
# benchmark data yourself using the JSON/rubric templates in the SKILL body before any bash command works.
# The Canavar integration it mentions (~/.claude/hooks/dist/canavar-cli.mjs) does ship separately under hooks/dist/.

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger Agent Benchmark

/agent-benchmark Methodology for scoring agent output quality and catching regressions against baselines

It also activates on plain-language prompts like these:

Benchmark this agent change against the baseline
Check for quality regressions in the latest agent update
Establish a performance baseline for this agent

Frequently asked questions

Is the Agent Benchmark skill free?: Yes. The skill itself is free from vibeeval/vibecosystem. SkillProof publishes the install command and an independent test verdict at no cost.
Does Agent Benchmark work with Claude Code?: We tested it with Claude Code 2.x (agent harness) on Jul 21, 2026. Verdict: Works with setup. Listed the repo tree via the GitHub API and fetched skills/agent-benchmark/SKILL.md raw; the directory contains only SKILL.md. Frontmatter parses cleanly with name+description and no security smells (no curl|sh, base64, or exfiltration). Spot-checked referenced scripts: run.mjs returns 404 at both skills/agent-benchmark/run.mjs and benchmarks/run.mjs, and no fixtures/ground-truth/rubrics/baselines directories ship — so every documented `node ~/.claude/benchmarks/run.mjs` command is non-functional out of the box (canavar-cli.mjs, referenced for the ledger integration, does exist at hooks/dist/). I could not produce a benchmark-run artifact because the scoring engine is absent and cannot be derived from the skill, so outputMeasured=false and the missing manual step is authoring run.mjs plus all fixtures/ground-truth.
What is the Agent Benchmark SkillProof Score?: 5.6/10 — installs cleanly 3/5, triggers reliably 5/5, output vs. baseline 3/10, docs & honesty 3/5.
How do I install Agent Benchmark?: Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use Agent Benchmark with Cursor, Copilot, Gemini CLI, Codex or other AI tools?: The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.