Cultivar
CLI that measures whether an agent skill actually beats a no-skill baseline
Test report
- Verdict
- Works with setup
- Score
- Tested
- Jul 18, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Jul 30, 2026 · 8368e5b
⚠ This skill is no longer available upstream. Our re-check on Aug 10, 2026 couldn't find it any more (repo unreachable/deleted). The test below is what we measured on Jul 18, 2026 and we're leaving it up as a record — but there is nothing left to install, so we've removed the command.
Installed the CLI with uv and its packaged 'cultivar hello' smoke task ran end-to-end against the real Claude runner, writing per-run JSON, Markdown, and JSONL traces plus a workdir exactly as documented. The harness itself is solid, but the actual payoff (a graded with-skill vs without-skill comparison) needs an authed agent CLI plus an ANTHROPIC_API_KEY for the grader; here the runner returned a 401 and grading was skipped, so the headline delta stays unmeasured. The skill is a faithful, accurate command reference over a genuinely working tool. Real setup friction, a paid key plus a logged-in runner, sits between install and any result.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 5/10
- Docs & honesty 5/5
What Cultivar does
Drives the cultivar CLI to evaluate whether an agent skill improves behavior: scaffolds tasks, runs each with-skill, without-skill, and with-docs across Claude/Copilot/Gemini locally or in Modal sandboxes, then an LLM grader scores runs against a rubric. Triggers when a user wants to create, run, or interpret cultivar skill evals.
How to install Cultivar
Nothing to install: the source repository no longer has this skill. If the author brings it back, our daily re-check will pick it up and the command will reappear here.
Commands — how to trigger Cultivar
-
/cultivarCLI that measures whether an agent skill actually beats a no-skill baseline
It also activates on plain-language prompts like these:
-
Does this skill I wrote actually help, can you eval it properly? -
Run this skill with and without it enabled, then grade the results. -
Help me interpret these eval results comparing skill performance.
Frequently asked questions
- Is the Cultivar skill free?
- Yes. The skill itself is free from pinecone-io/cultivar. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Cultivar work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 18, 2026. Verdict: Works with setup. Installed the CLI with uv and its packaged 'cultivar hello' smoke task ran end-to-end against the real Claude runner, writing per-run JSON, Markdown, and JSONL traces plus a workdir exactly as documented. The harness itself is solid, but the actual payoff (a graded with-skill vs without-skill comparison) needs an authed agent CLI plus an ANTHROPIC_API_KEY for the grader; here the runner returned a 401 and grading was skipped, so the headline delta stays unmeasured. The skill is a faithful, accurate command reference over a genuinely working tool. Real setup friction, a paid key plus a logged-in runner, sits between install and any result.
- What is the Cultivar SkillProof Score?
- 8.0/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 5/10, docs & honesty 5/5.
- How do I install Cultivar?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Cultivar with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.