We Cut Claude Research Hallucinations 59% With One Skill

Ask Claude a research question and you get something dangerous: a confident, well-written answer. Whether it’s true is a separate property, and you can’t tell by reading it. We run a directory that bench-tests Claude skills for a living, and we decided to measure this properly: how many factual claims in a typical researched answer are actually wrong — and how many of them survive when you force the model to follow verification rules?

The result is research-discipline, our third skill, and the first skill in its niche to ship with a measured before/after benchmark: 13.0% of baseline claims were wrong or stale; with the skill, 5.4%. A 59% cut. It’s free and MIT-licensed: github.com/Skillproofdev/research-discipline.

The gap: pipelines don’t verify, verifiers don’t research

Before writing a line we surveyed 15 published research and fact-checking skills, plus our own index of 16,682 discovered skills. The landscape splits cleanly into two camps that never overlap.

Deep-research pipelines — the popular ones, up to 1.6k stars — orchestrate parallel searches and generate long reports. Read their rule sets and you find workflow ergonomics, not epistemics: no per-claim citations, no source-quality rules, no requirement to check anything twice. Fact-check auditors — mostly journalism and academia tools, two orders of magnitude less popular — do enforce verdicts and citations, but only post-hoc, on a finished draft someone hands them.

Nobody ships the middle thing: a lightweight discipline that governs the agent’s claims while it researches, whether the answer is three paragraphs or thirty. And notably, nobody in either camp publishes measurements. The loudest competitor claims to “outperform OpenAI, Gemini and Claude Desktop” with zero numbers attached; another states “95%+ accuracy” as a target it never tested. The niche runs on vibes.

Eight rules, three of them novel

The skill is ~1,500 tokens of enforceable rules (full SKILL.md). The familiar parts: every factual claim carries an inline citation resolving to a dated source list; unsupported claims get an explicit [unverified] flag instead of confident phrasing; sources are tiered, with content farms rejected outright. The parts nobody else enforces:

1. Memory is a hypothesis. For anything volatile — versions, prices, “current” anything — training-data recall is a lead to verify, never evidence. Retrieve the date, search, then assert.
2. Two independent sources for load-bearing facts. Independence is defined: two articles rewriting the same press release count as one source. The only prior implementation of this check we found anywhere was a gist with one star.
3. Numbers are quoted, not manufactured. Verbatim value, unit, and date; when sources disagree, report the range with both citations — never a silent compromise value.

Plus an adversarial pass with a forcing function borrowed from the best prior art we found: search once for the opposite of your conclusion, find at least two weaknesses in your own draft, or re-examine your most load-bearing claims.

The benchmark: every claim judged against live sources

Six research questions with objectively checkable answers, chosen before any run: a version lookup, a pricing comparison, a fact digest, a GitHub repo query, and two traps. Each question ran twice — one Claude Sonnet agent bare, one reading the skill first, both with full web access. Then independent verifier agents established ground truth from live primary sources and judged all 110 factual claims across the 12 answers, one by one.

Question	Baseline wrong/stale	With skill
Claude model catalog (staleness trap)	3	0
Bun production-readiness (real-event trap)	1	1
Deno fact table	1	0
GitHub repo facts	1	0
Astro release history	1	1
Email pricing comparison	0	1
Total	7 of 54 (13.0%)	3 of 56 (5.4%)

The two traps are where the skill earned its keep.

The staleness trap. Asked for the current Claude model catalog, the baseline agent answered from a cached reference without one live check — three context-window figures came out wrong. The skill agent’s Rule 1 forced a fetch of the live docs page; the verifier’s verdict: “matches the live documentation on every single figure.” Same model, same question, same tools available. The only difference was a rule saying memory doesn’t count as evidence.

The real-event trap. Asked whether Bun is production-ready in 2026, the baseline agent ran into the fact that Anthropic acquired Bun in December 2025 — and dismissed it as an unverified SEO rumor. Its sources were third-party blogs; it never opened bun.com or anthropic.com, where both primary announcements sit. The skill agent cited both, and the verifier confirmed them live. Read that again: the baseline’s only uncertainty flag in the entire benchmark was aimed at a true event. Skepticism without verification is just a different way to be wrong.

Where the skill lost — published anyway

Our methodology requires the losses next to the wins, and there were two.

On the email-pricing question, the baseline actually beat the skill: it priced all three Postmark tiers and found the cheapest, while the skill agent priced one tier, called it the “cheapest path,” and was wrong by $2.50/month. Citation discipline doesn’t do your arithmetic — incomplete enumeration is a research failure the rules don’t catch. And on the Bun question, the skill agent repeated a tech outlet’s incorrect version number for a real release instead of cross-checking the vendor’s release notes — a tier-2 source trusted one level too far.

Also honest: the discipline costs about +10% tokens — the live fetches and the adversarial pass aren’t free. And across all twelve answers, the skill’s ~8 proactive [unverified]/[single-source] flags were all judged appropriate, which is its own kind of result: the skill doesn’t just cut wrong claims, it tells you which of the surviving ones to double-check.

Install

git clone https://github.com/Skillproofdev/research-discipline ~/.claude/skills/research-discipline

Restart Claude Code. It triggers on research questions, “what’s the latest”, comparisons, and fact-check requests — and stays out of the way for coding, opinions, and creative work. It joins token-discipline in our discipline series: that one cuts what research costs, this one cuts what it gets wrong.

FAQ

Is there a live demonstration, not just the benchmark? Yes — we asked one real question twice, before and after installing the skill, and verified every number against the GitHub API in public. The baseline missed 3 of the top-5 repos it was asked to rank.

Does this replace Claude’s built-in deep research? No — it complements it. Built-in research features are report generators; this is a discipline layer that applies to every research-shaped answer, including short ones, and it’s inspectable: you can read all eight rules in one screen.

Won’t Claude just refuse to answer when it can’t verify? The skill explicitly forbids that: thin evidence means answering with flags, not shrugging. “I couldn’t confirm this” attached to a specific claim is more useful than either false confidence or refusal.

Why only six questions? Because every claim was verified by hand against live primary sources — 110 claims across 12 answers, each with a checkable verdict. We prefer a small benchmark where the ground truth is real over a large one scored by an unverified judge. The question set and per-claim verdicts are in the repo README.

Is 5.4% still too high? Yes. The skill cuts wrong claims by more than half; it doesn’t make Claude infallible, and the two residual errors in our own benchmark are documented above. If a decision is expensive to get wrong, verify the load-bearing facts yourself — the skill’s citations make that take minutes instead of hours.