token-discipline: an Honest Benchmark of Token Savings

We spend our days measuring other people’s Claude skills, and one pattern kept showing up in the telemetry: the sessions that got expensive didn’t get expensive because Claude wrote long answers. They got expensive because of what Claude read. Whole files opened to find one function. Files re-read right after a successful edit, “just to verify.” A 16,000-record JSON pulled into context like it was a blog post.

So we built a skill that attacks that side of the bill, published it under an MIT license, and did what we ask of every skill we list: benchmarked it against a baseline with controlled runs and published the numbers — including the ones that don’t flatter it.

The skill is called token-discipline. It’s free: github.com/Skillproofdev/token-discipline.

The gap in every token skill we surveyed

Before writing a line, we surveyed eight published token-saving skills and protocols — from the 5,800-star output-verbosity ruleset that dominates the niche, to “emergency” skills that trigger at 40% context, to protocols claiming 76–93% savings with no mechanism attached and no telemetry we could find.

Almost all of them share one blind spot: they optimize output. Kill the preamble, shorten the answers, stop restating the plan. That’s real, but it’s the smaller half of the bill. In agent sessions — Claude Code doing actual work in a repository — input dominates: every file read, every search result, every tool output lands in context and gets re-sent with every subsequent request.

Three input-side techniques appeared in none of the eight as enforceable rules:

1. Search before you read. Locate with grep/glob first, then read only the matching region. Never open a whole file to find something.
2. Batch independent tool calls. If three reads don’t depend on each other, they belong in one message. Every extra turn re-pays thinking and narration tokens between calls.
3. Trust state you already have. After your own successful edit, don’t re-read the file to verify — the edit tool errors loudly when it fails. Re-reads are for external changes, not your own.

Those three became the core of the skill. Around them sit six more rules: read slices instead of files, query big JSON/logs with python or jq instead of opening them, summarize-then-drop long outputs, delegate broad exploration to subagents, keep the context cache-friendly (stable prefix untouched — Anthropic’s own engineering blog calls prompt caching the single highest-leverage cost factor), and, yes, terse output too. Nine rules, each phrased as a hard rule with a stated alternative, so following it never costs correctness.

How we benchmarked it

Same protocol as our Skill Bench series: same model, same prompt, one variable. Each task ran twice — one Claude Sonnet agent with no skill, one that read SKILL.md first and was told to follow it strictly. The skill agent’s totals include the cost of reading the skill itself. The testbed was our own codebase: an Astro site with about 9,000 lines of templates plus a 16,682-record JSON dataset — big enough that undisciplined reading actually hurts.

Five task pairs, chosen to cover the spectrum from trivial to genuinely multi-step:

Task	Baseline	With skill	Δ
Codebase Q&A — 8 questions across subsystems	88,419	71,636	−19.0%
Multi-file edit — 4 coordinated changes	68,219	54,637	−19.9%
Code trace — one-shot	53,986	52,850	−2.1%
Audit sweep — one-shot	44,691	44,705	+0.0%
Big-JSON digest — one-shot	42,726	45,164	+5.7%

What the numbers actually say

On real multi-step work, the skill saves about 20%. The eight-question codebase Q&A is the closest thing to a normal working session — trace the auth flow, explain the webhook, find the sort logic. The baseline agent answered correctly, but along the way it read generously and verified redundantly. The disciplined agent answered the same questions, with the same accuracy (we checked every file:line reference from both), for 16,783 fewer tokens.

The multi-file edit is the more interesting result. Four coordinated changes across four files. The baseline agent finished and then — quoting its own report — “verified by re-reading the edited regions.” That habit alone is a fifth of the bill. The skill agent applied the same four changes, skipped the ceremony re-reads, and reported done. We diffed both working trees: the edits were equivalent and correct in both. Nothing was lost by trusting the tool to error on failure, because that’s what tools do.

On trivial one-shots, it’s a wash — and we’re publishing that anyway. The JSON-digest task actually cost 5.7% more with the skill. Two reasons, both instructive. First, the skill itself costs about 1,400 tokens to read, and a task that finishes in two tool calls gives that overhead nowhere to amortize. Second, a current-generation model already behaves well on easy tasks: the baseline agent took one look at a 16,682-record JSON and reached for python instead of opening it. No skill needed. Discipline pays where indiscipline is possible — long sessions, many files, compounding context — not where there’s nothing to waste.

If your sessions are mostly one-liners, the skill ships its own answer: a 60-token distilled block you paste into CLAUDE.md instead of installing the full skill. Always-on, costs about as much as this sentence, and a single avoided whole-file read repays it many times over. (We built a calculator for exactly this always-on tax — the math on trigger descriptions applies to rule blocks too.)

The nine rules, briefly

The full SKILL.md is ~1,400 tokens and readable in one screen; here’s the shape:

1. Search before you read. Grep first; then read the hit ±30 lines. Reading more than ~200 lines requires a reason you can state in one sentence.
2. Read the slice, not the file. Offset+limit for anything long; python/jq for anything structured. A big JSON is a database, not a document.
3. Batch independent tool calls. One message, several calls, whenever outputs don’t feed inputs.
4. Trust state you already have. No verify-re-reads after your own edits; no re-running searches whose results are already in context.
5. Don’t quote code back. Reference file:line. Paste only what the human must see to decide.
6. Summarize, then drop. After a long tool output, keep the 2–5 facts, never re-request the log.
7. Delegate heavy exploration. Broad sweeps go to a subagent; the heavy reads die in its context, not yours.
8. Keep the context cache-friendly. Stable prefix untouched; append, don’t rewrite; mind the cache TTL.
9. Output discipline. Answer first, no preamble, no diff essays.

Plus a self-audit checkpoint (three questions before any big re-read) and an explicit do-not list, because a token skill that makes an agent skip necessary verification isn’t saving you anything — it’s deferring rework. If a real bug hunt needs the whole file, the rule says read it, with the reason stated.

Where this sits in the bigger cost picture

Rules like these are the variable half of your bill. The fixed half — the 30,000-token preamble of MCP servers, bloated CLAUDE.md files, and always-on skill descriptions that gets re-sent with literally every request — is a different problem with different fixes, and usually the bigger win for heavy users. We wrote that audit up in How to cut Claude Code token costs; the two pieces are complementary: fix the fixed costs once, then let token-discipline keep the variable costs flat all session.

And if you’re wondering whether skills in general save tokens: our Skill Bench series measured four popular skills and found they mostly don’t — they cost extra tokens and buy quality and discipline instead. token-discipline was built specifically to be the exception, and even so, it only earns its keep on multi-step work. That’s the honest boundary, and we’d rather draw it ourselves than have you find it on your bill.

Install

git clone https://github.com/Skillproofdev/token-discipline ~/.claude/skills/token-discipline

Restart Claude Code. It triggers on “reduce token usage”, “work token-efficiently”, context-limit complaints, and cost questions — or apply it session-wide via the CLAUDE.md block inside. There’s also a sequel: research-discipline, which applies the same enforceable-rules approach to research hallucinations — measured 59% fewer wrong claims.

FAQ

Does this replace the output-verbosity skills? No — it includes their core (Rule 9) and adds the input side they skip. If you already run a terse-output ruleset, token-discipline overlaps it on one rule out of nine.

Will it make Claude sloppier? The benchmark says no: the multi-file edit came out equivalent and correct on both sides, and the skill’s do-not list explicitly protects verification that’s actually required. The rules replace ceremony reads, not necessary ones.

Why is the skill itself so short? Because a token-saving skill that costs thousands of tokens per session to load would be self-parody. The SKILL.md is ~1,400 tokens loaded on demand; the always-on CLAUDE.md block is ~60.

I’m on a Pro/Max subscription, not the API. Does this matter? Yes, in limit-currency: the same tokens that bill API users consume your usage caps. Twenty percent fewer tokens on working sessions means proportionally more work before you hit the ceiling.

Was the benchmark cherry-picked? The five tasks were designed before either variant ran, and we published the two results where the skill lost. Raw telemetry — token counts and tool-call counts per run — is in the repo README.