How Many Claude Skills Are Actually Better Than Plain Claude? We Measured.

The promise of Claude skills is compelling: a library of tools that can be installed to give the model new capabilities, from interacting with APIs to generating complex code. The official catalog and third-party repositories list thousands of them. But this raises a critical question for any developer whose time is valuable: do Claude skills actually improve output in a measurable way?

It’s easy to find skills that claim to be revolutionary. It’s much harder to find objective proof. Most skill directories are just that—directories. They list skills based on the author’s description, but they don’t validate the claims. A skill might be broken, outdated, or, in many cases, no better than what the base model can do on its own. This creates a significant signal-to-noise problem.

At SkillProof, we don’t list skills; we test them. Because we ran every catalog skill against a baseline, we can state the actual fraction that beats no-skill Claude with the evidence behind it. This article presents that evidence. We measure the performance of each skill against the same model with no skill installed to determine if it provides a real, quantifiable benefit.

The Signal-to-Noise Problem in Skill Discovery

If you’ve tried to integrate skills into your workflow, you have likely encountered the discovery problem. You have a task in mind, perhaps generating Terraform configurations or interacting with a specific SaaS API. You search a catalog, find a skill with a promising name, and read its SKILL.md file, which describes its function and provides usage examples.

You install it and try the example prompt. Sometimes it works. More often, the process involves friction. The skill might throw an error, require undocumented environment variables, or produce output that looks nothing like the example. You might spend an hour debugging someone else’s tool only to find it was poorly written or abandoned months ago.

This trial-and-error cycle is inefficient. The core issue is that most skill catalogs function as package managers without a CI/CD pipeline. They index what exists but provide no guarantee of quality. There is no independent verification to confirm that a skill works as advertised, let alone that it offers an improvement over a well-crafted prompt to the base model. The burden of testing falls entirely on the end user.

This is the problem we set out to solve. To determine if Claude code skills are worth installing, you need a consistent, repeatable testing methodology and a clear baseline for comparison.

How We Measure “Better”: The No-Skill Baseline

To answer the question, “Is this skill better than nothing?” you need a rigorous definition of “nothing.” For us, “nothing” is the base Claude model itself—what we call the no-skill baseline. Our entire methodology is built on comparing a skill’s performance against this control.

The process is straightforward and designed to mirror a real-world use case. For each skill, we perform the following steps:

1. Define Test Cases: We analyze the skill’s intended function and create a set of representative tasks. For a Kubernetes manifest generator, this might involve prompts to create Deployments, Services, and Ingress objects with varying complexity.

2. Run the Baseline: We run these test prompts against the plain Claude model with no skill installed. We save the output as our control case. This shows what a competent user could achieve with prompting alone.

3. Run the Skill: We install the skill and run the exact same set of test prompts. This is our experimental case.

4. Score the Outputs: A human reviewer compares the baseline output and the skill’s output side-by-side. We use a detailed rubric to score them on correctness, completeness, adherence to instructions, and efficiency. The final verdict is a single /10 score that measures the lift provided by the skill over the baseline.

A high score (8-10/10) indicates a significant improvement. A medium score (6-7/10) indicates a functional skill that offers a marginal benefit. A low score (1-5/10) indicates a skill that is buggy, difficult to use, or performs worse than the baseline. You can read the full details of our scoring system on our /methodology page.

This claude skills vs no skill baseline comparison is the only way to generate objective data on a skill’s true value. Marketing claims and author descriptions are irrelevant; the only thing that matters is the measured performance on a real task.

The Verdict: What Percentage of Claude Skills That Work?

So, what does the data say? After applying our methodology to the public skill ecosystem, a clear picture emerges. As of this writing, we have installed and executed 1416 unique skills.

The results show that a majority of skills do provide some value, but a very significant portion—over a third—are either broken, require complex setup, or are actively detrimental to the model’s performance.

Here is the top-level breakdown of our findings:

Verdict	Count	Percentage of Total	Description
Pass & Listed	889	63%	The skill installs cleanly, works as described, and scores higher than the no-skill baseline.
Needs Manual Setup	467	33%	The skill is functional but requires undocumented setup (e.g., environment variables, API keys) or has major caveats.
Scores Below Baseline	60	4%	The skill is actively harmful, producing output that is less accurate, less complete, or more error-prone than plain Claude.

These numbers are sobering. While it’s good that nearly two-thirds of skills pass our verification, it means that if you pick a skill at random from a public catalog, you have a 1 in 3 chance of it being a waste of your time.

More alarming is the 4% that score below the baseline. These are skills that not only fail to help but actively make the model’s output worse. Installing one of these is a downgrade for your system. This data provides a clear answer to the question of what percentage of Claude skills that work: it’s far from 100%.

Anatomy of a Failed Skill

Understanding why skills fail is as important as knowing which ones succeed. The failures we record generally fall into two categories: those that are actively harmful and those that are simply incomplete.

Category 1: Scoring Below the Baseline

The 60 skills in this category represent the worst-case scenario. They promise to add a capability but instead introduce errors, constraints, or regressions. For example, we tested a SQL query generator that was intended to write complex queries from natural language. On our test prompts, it consistently produced syntactically invalid SQL. The plain Claude baseline, given the same prompts, produced correct SQL every time. The skill’s internal logic was flawed, actively steering the model toward a worse outcome.

Another common failure mode is over-constraining. A skill designed to enforce a specific JSON schema might be so rigid that it causes the model to refuse to answer legitimate prompts that fall slightly outside its narrow definition, whereas the base model would have handled the request gracefully. These skills are worse than useless; they are a liability.

Category 2: Requiring Manual Setup

This is a much larger category, comprising 467 of the skills we tested. These skills are not necessarily poorly designed, but they are poorly documented. They represent a massive hidden time cost for developers.

A typical example is a skill that acts as a client for a third-party API. The code might be perfectly functional, but the SKILL.md file fails to mention that the user must first sign up for an account, generate an API key, and set it as an environment variable named THIRD_PARTY_API_KEY. Without this information, the skill fails with a generic AuthenticationError.

Our team does the work to discover these hidden requirements, documenting them in our findings. But for an average user, this is a dead end. The skill appears broken, and they uninstall it after a frustrating half-hour of debugging. This is not a failure of the model but a failure of developer experience. Good skills must be usable out of the box, with all dependencies and configuration steps clearly documented.

Characteristics of a High-Scoring Skill

If a third of skills are problematic, what do the other two-thirds—the successful ones—look like? Are Claude code skills worth installing? Yes, if they belong to the high-scoring group.

High-scoring skills share several common traits:

1. They Provide True Tooling: The best skills don’t just rephrase a prompt. They give the model access to new capabilities. A skill that can check a URL for a 200 OK status, a file patcher tool that can apply a diff to a local file, or a skill that interacts with a live cloud provider’s API are all examples of true tooling. They allow the model to take actions in the world, not just talk about them. This provides a clear, undeniable lift over the baseline.

2. They Are Atomic and Reliable: Top-tier skills focus on doing one thing well. A skill to convert a timestamp to an ISO 8601 string is more likely to be robust and useful than a monolithic “DevOps assistant” skill that tries to do twenty different things.

3. They Have Excellent Documentation: The SKILL.md is treated as a critical part of the tool. It contains clear instructions, working examples for common use cases, and explicit documentation of any required setup, like environment variables or authentication.

When a skill meets these criteria, the improvement is not subtle. It transforms the model from a text generator into an interactive agent that can execute tasks, saving significant time and effort. These are the skills that deliver on the original promise.

Finding Skills That Actually Improve Your Workflow

The central takeaway from our research is that the raw number of available skills is a vanity metric. The ecosystem’s value is not in its size but in the density of high-quality, verified tools. Blindly installing skills based on their descriptions is an inefficient and frustrating strategy.

The question developers should ask is not “do Claude skills actually improve output?” but rather “which skills improve output, and by how much?”

Answering that question is why we built SkillProof. We run the tests and publish the results—including the failures—so you can adopt skills with confidence. Our catalog is not a comprehensive list of every skill in existence. It is a curated directory of skills that have been proven to work and provide a measurable benefit over the no-skill baseline.

Related reading: why so many skills fall short · whether GitHub stars predict a good skill.

The point of this data is not to discourage using skills, but to encourage using the right ones. We’ve done the work of testing 1416 skills so you don’t have to. You can browse the 889 skills that passed our baseline tests in our full catalog. If you want to skip the browsing, we also offer a curated pack of the top 50 highest-impact skills for $10.