The Claude Skill That Audits Its Own OpenAPI Spec

Ask Claude to design an API and you get something that looks right: plural nouns, a pagination param, a version prefix. Then you read closely and one endpoint uses page_size where every other list uses limit, a validation error is documented as a 500, and the spec ships with a nullable: true that doesn’t exist in OpenAPI 3.1. Each mistake is small. Together they’re the difference between an API that works and an API that stays consistent under change — and nothing in a “here are the principles” prompt forces the second one.

The result is api-discipline, and the point isn’t that it knows REST conventions — every skill in this niche does. The point is that it checks its own output before handing it back: validator-clean OpenAPI 3.1, a mandatory cross-endpoint consistency pass, and a breaking-change diff on every edit. It’s free and MIT-licensed: github.com/Skillproofdev/api-discipline.

The gap: everyone teaches the principles, nobody enforces them

Before writing a line we surveyed 84 API-design and OpenAPI skills across our 16,682-skill index plus the standalone web ecosystem. The pattern is consistent. The biggest repo (37.6k stars) is a concepts textbook with zero enforcement. The best-engineered one (10.5k stars) names a linter and stops there.

And that matters for what we could honestly claim. Validator-clean output alone is already a solved, competitive space — the 10.5k-star skill gets you there. Shipping “we run a linter too” would have been noise. So we looked for what nobody enforces, and found four things that existed in none of the 84:

1. A cross-endpoint consistency audit as a mandatory pass. Ten defined checks — casing, pluralization, one shared error schema, identical pagination params, uniform id/timestamp formats, operationId pattern, same-action-same-status-code — run across every endpoint before delivery. Every competitor has, at most, one “be consistent” bullet.
2. Breaking-change discipline that fires on edits. Every spec edit gets an enumerated breaking-change pass, backed mechanically by oasdiff breaking when available. The tooling is mature; no surveyed skill wires it in.
3. HTTP semantics as rules, not trivia. PUT replaces, PATCH partials, POST creates with 201 + Location, DELETE returns 204 — enforced with a status-code table, not listed as “concepts to know.”
4. A review/extend output contract. “Review this spec” returns findings keyed to the checklist with locations and fixes; “add an endpoint” returns a diff that inherits the existing spec’s conventions plus a breaking-changes block. Competitors only define greenfield output.

That’s the uncontested ground: not validation, but the audit that runs after validation, plus a published benchmark to back it.

The benchmark: measured, with the losses left in

Seven tasks — two greenfield designs, two spec extensions, two reviews of flawed specs with 22 seeded violations between them, one conventions question. Each ran twice: one Claude Sonnet agent bare, one reading the SKILL.md first, identical prompts. Specs were scored mechanically with redocly lint and spectral lint, edits diffed with oasdiff breaking, and seeded-violation catches judged by independent verifier agents.

Metric (lower is better)	base	skill
Validator errors, greenfield (redocly)	18	0
Consistency violations, all 4 design tasks	6	0
HTTP-semantics errors, all 4 design tasks	3	0
Seeded violations caught, T6 review (higher better)	10/10	9/10

Measured 2026-07-10 with Redocly CLI 2.38.0, Spectral 6.16.1, and oasdiff 1.23.0. Full per-finding detail is in bench/results/verdict.md.

The validator gap is one clean story: both bare runs emitted OpenAPI 3.0 nullable: true into documents declared openapi: 3.1.0 — a structural error in 3.1, which uses type: [x, 'null']. Twelve occurrences in the first greenfield task, six in the second. The skill run used the 3.1 form throughout and validated clean. The consistency and semantics wins are the same shape: the bare runs shipped verb-in-path endpoints (/tasks/{id}/complete), a second ad-hoc error schema alongside the shared one, and a create that returned 200 instead of 201. The skill run modeled actions as noun sub-resources and reused one error schema everywhere — 0 across all four design tasks.

Where the skill lost — and one result we won’t take credit for

Two honest notes, because our methodology requires the losses next to the wins.

The skill lost T6 by one seeded violation. On the semantics-heavy review, the bare agent’s free-form pass walked every operation exhaustively and caught a 201 Created on POST /articles with no Location header. The skill agent’s review, organized around the consistency checklist, flagged all four HTTP-method defects and all five consistency seeds but didn’t scan each 201 for its Location — 9/10 vs 10/10. Structured review under-covered what an exhaustive read caught. That’s now fixed in the checklist with an explicit “every 201 has Location” line.

One breaking-change result is excluded from the headline. On the spec-extension task, the skill agent reported that it had seen a ground-truth hint leaked into the task file (“both changes are breaking”) before analyzing — a protocol slip, since the skill arm should read only the SKILL.md. So that result is not claimed as an independent win, even though it looks good on paper. Two things make the underlying finding robust anyway: the bare arm, which never reads the task file, independently concluded both changes were breaking; and oasdiff mechanically confirmed the breaking surface regardless of what either agent believed. The headline claims rest on validation, consistency, and semantics — none of which the leak touches.

Install

git clone https://github.com/Skillproofdev/api-discipline ~/.claude/skills/api-discipline

Restart Claude Code. It triggers on “design an API,” “add/extend an endpoint,” “review this OpenAPI spec,” and REST convention questions — and stays out of the way for GraphQL-only work, client SDK codegen, and API security testing. It joins token-discipline, which cuts what multi-step work costs, and research-discipline, which cuts what research gets wrong — this one cuts what your API contracts drift into.

FAQ

A 10.5k-star skill already produces valid OpenAPI. Why this one? Because valid isn’t the same as consistent. A linter catches a broken $ref; it doesn’t catch one endpoint paginating with page_size while the rest use limit, or a create returning 200. That cross-endpoint consistency audit is the uncontested ground — it’s what the popular skills don’t enforce, and it’s where the bare runs racked up 6 violations to the skill’s 0.

Does it handle edits to an existing spec, not just greenfield design? Yes, and it treats them differently. New endpoints added to an existing spec inherit its conventions even when they conflict with the skill’s defaults — consistency with the contract other people depend on beats the skill’s preferences. Every edit also gets an enumerated breaking-change pass, backed by oasdiff breaking when the tool is available.

Does it need oasdiff or a validator installed to work? No. When redocly/spectral or oasdiff can run, it uses them and reports the command and result. When they can’t, it says so explicitly and runs a defined self-check fallback — all $refs resolve, unique operationIds, every path parameter declared, every response has a description. It never silently skips the check.

Is the benchmark reproducible? Yes. Seven tasks, two arms, mechanical scoring where possible, and the pre-registered ground truth is checked into the repo under bench/ground-truth/. The full method, per-finding detail, and the T6 loss and T4 integrity footnote are all in bench/results/verdict.md — nothing is hidden.