We Built an MCP Server So Claude Can Check a Skill Before Installing It

Installing a Claude skill is a leap of faith. You find a SKILL.md on GitHub, the README sounds confident, the star count looks reassuring, and you copy it into ~/.claude/skills/. Whether it actually does anything useful, you find out later — usually in the middle of the work you were trying to get done.

We spend our days closing that gap by hand: install the skill, run one real task twice, once with it and once without, and publish the difference. 743 skills so far. The awkward part was always the last mile. The verdicts lived on a website, and websites are for humans. The thing that installs your skills is an agent.

So we put the verdicts where the agent already is.

One line

claude mcp add skillproof -- npx -y skillproof-mcp

That’s the whole setup. No API key, no account, no config file to hand-edit. It works in any MCP client — Claude Code, Claude Desktop, Cursor, Windsurf, Zed.

Now, before your agent clones anything, it can ask.

What it actually answers

The server exposes two tools, and they map to the two questions people actually have.

find_skill — “is there a tested skill for this?” You describe the job; you get back matches ranked by how they scored, each with the verdict, the /10 score, what our test found, and the install command.

check_skill — “someone recommended this one, is it any good?” You name a skill, a slug, or a GitHub repo; you get our verdict on it, or an honest “we have not tested this — treat it as unverified.”

Every answer carries one of three verdicts:

• pass — installed cleanly, triggered when it should, and beat plain Claude on a real task.
• setup — it delivers, but not out of the box. The note says exactly what you have to do first.
• didn’t pass — it scored below the no-skill baseline. Either it couldn’t run at all, or it ran and left you worse off than not installing it.

That third verdict is the one that makes this worth installing.

The answer we’re proudest of

Ask the server for a skill that converts Markdown to Confluence wiki markup, and here is what comes back:

Confluence — DIDN’T PASS What our test found: ran the bundled convert_markdown_to_wiki.py on a real sample doc. It silently turns **bold** text into wiki italic — a genuine regex bug, not a style choice — and leaves standard GitHub-style tables completely unconverted, despite SKILL.md explicitly listing “tables” among the elements it handles.

A directory that only lists winners would have shown you that skill with a friendly description and let you find the bold-becomes-italic bug yourself, in a document you’d already sent to your team. Ours tells your agent not to bother.

There are 31 skills like that in the catalog right now — tested, published, and marked in red. Another 59 tie with plain Claude: they run, they produce something, and the something is no better than what you’d have gotten anyway. Nobody else publishes either number, because neither one flatters the ecosystem.

Where the data comes from

The server holds no data of its own. It reads skillproof.dev/api/skills.json — the same live catalog the website renders — and caches it for fifteen minutes. When a skill gets re-tested after a Claude release, the answer your agent gets changes with it. Nothing to update, nothing to re-install.

The scoring rubric is public: install /5, triggering /5, output-vs-baseline /10, docs and honesty /5, normalized to a score out of ten. It’s written up in full on our methodology page, including why output is worth as much as everything else combined.

If you’re building your own tooling, the catalog is open under CC BY — take the JSON, or the whole thing as plain text, and cite skillproof.dev.

The honest limits

Two things this does not do.

It does not cover everything. We’ve tested 743 skills out of the 16,682 we’ve indexed — the top slice by stars and by our automated screen. If your job is niche, the answer may well be “nothing tested matches this yet,” and the server says exactly that instead of inventing a recommendation. We’d rather return nothing than vouch for something we never ran.

And a verdict has a date on it. Skills drift; a skill that passed in June can break when its dependency moves. Every card carries the date we ran it, and the server passes that date through. Old verdicts are a fact about the world, not a bug we can code around — but you’ll always know how old the one you’re reading is.

Install it, then forget about it

The best case for this thing is that you never think about it again. Your agent quietly checks before it installs, skips the three skills that would have wasted your evening, and picks the one that was actually tested on the job you’re doing.

claude mcp add skillproof -- npx -y skillproof-mcp

Source is on GitHub, MIT licensed. It’s listed in the official MCP registry as io.github.Skillproofdev/skillproof. If it gives you a bad answer, that’s a bug in our testing, not in the plumbing — tell us and we’ll re-run the skill.