Agent Benchmark

Metodika pro hodnocení kvality výstupu agenta a zachycování regresí proti základním liniím

Funguje s nastavením ★ 5.6/10

Co umí

A methodology document for measuring agent response quality with scoring rubrics, ground-truth fixtures, and timestamped baselines to detect regressions. Spouští se, když evaluating agent definition changes, auditing quality, or establishing performance baselines. Ships only the spec — every documented `node run.mjs` command depends on a runner script that must be authored first.

Testovací report

Vypsán the repo tree via the GitHub API and fetched skills/agent-benchmark/SKILL.md raw; the directory contains only SKILL.md. Frontmatter se parsuje cleanly with name+description and no security smells (no curl|sh, base64, or exfiltration). Spot-checked referenced scripts: run.mjs returns 404 at both skills/agent-benchmark/run.mjs and benchmarks/run.mjs, and no fixtures/ground-truth/rubrics/baselines directories ship — so every documented `node ~/.claude/benchmarks/run.mjs` command is non-functional out of the box (canavar-cli.mjs, referenced for the ledger integration, does exist at hooks/dist/). I could not produce a benchmark-run artifact because the scoring engine is absent and cannot be derived from the skill, so outputMeasured=false and the missing manual step is authoring run.mjs plus all fixtures/ground-truth.

Testováno: 2026-07-21 · Claude Code 2.x (agent harness)

Instalace

git clone --depth 1 https://github.com/vibeeval/vibecosystem.git /tmp/agent-benchmark-src
mkdir -p ~/.claude/skills
cp -R /tmp/agent-benchmark-src/skills/agent-benchmark ~/.claude/skills/agent-benchmark
# SKILL.md is the only file that ships. It is a methodology/spec, NOT a runnable tool.
# The runner it invokes everywhere (node ~/.claude/benchmarks/run.mjs) does NOT exist in the repo (HTTP 404).
# No fixtures/, ground-truth/, rubrics/, or baselines/ ship either — you must author run.mjs and all
# benchmark data yourself using the JSON/rubric templates in the SKILL body before any bash command works.
# The Canavar integration it mentions (~/.claude/hooks/dist/canavar-cli.mjs) does ship separately under hooks/dist/.

Příkazy a ukázkové prompty

/agent-benchmarkMetodika pro hodnocení kvality výstupu agenta a zachycování regresí proti základním liniím

Skilly se spouštějí běžnými požadavky — žádné příkazy k zapamatování. Po instalaci ho aktivují prompty jako tyto (anglicky):

Benchmark this agent change against the baseline
Check for quality regressions in the latest agent update
Establish a performance baseline for this agent

Kompletní testovací report (anglicky) →