codebase-memory-mcp: 127% More Tokens, Not 99% Fewer

codebase-memory-mcp, from DeusData, went viral on TikTok this week. It’s a single static binary that indexes a repository into a knowledge graph — 158 languages, the Linux kernel indexed in three minutes — and exposes it to Claude Code as an MCP server. The pitch is aggressive: the GitHub description says “99% fewer tokens,” the README says “120x fewer tokens — 5 structural queries: ~3,400 tokens vs ~412,000,” and their arXiv preprint (arXiv:2603.27277) reports “10× fewer tokens, 83% answer quality” averaged over 31 repos.

We run every tool we catalog through a controlled A/B before we score it. So today (2026-07-11) we built an 8-task structural benchmark, pre-registered the questions and ground truth before either arm ran, and pointed both a plain-grep Sonnet agent and an MCP-equipped Sonnet agent at the same repo. Correctness came out tied, 8/8 both. The flagship token claim inverted: the MCP arm used 172,319 tokens against the baseline’s 75,817 — 127% more, not 99% fewer.

That’s not the whole story, and we don’t think it should be. The engineering underneath is genuinely good. This is what we found and why the number moved the way it did.

What the tool does well

codebase-memory-mcp installs as a single Homebrew-able binary, indexes with one command, and produced clean JSON on the first try. Two of our eight tasks were clear wins for the graph:

• T5 (call-chain trace) — tracing from a POST handler down to a private helper three hops deep, through two possible paths. trace_path(direction=outbound) returned the exact DAG in one call, matching manual verification exactly.
• T7 (definition lookup) — “where is formatDate defined.” get_code_snippet returned the file and the precise start/end line in one call, no ambiguity.

Both of those are exactly the shape of query a graph should dominate: pure TypeScript, static structure, no framework boundary in the way. If your repo is pure TS/JS and your questions look like these, the tool is fast and precise.

Where it broke down

Our testbed was our own repo: 157 files, Astro + TypeScript — a mixed-framework codebase, which is close to a worst case for a graph indexer built primarily around a CALLS/IMPORTS edge model. Four of the eight tasks exposed real blind spots:

• T1 (who calls getSession) — trace_path(inbound) found only 5 of 7 callers. The CALLS graph doesn’t capture calls made from .astro frontmatter, so two legitimate callers (admin.astro, account.astro) were invisible to it.
• T2 (imports of purchases.ts) — the IMPORTS graph returned 2 of 3 import statements. The missing one was a type-only import (import type { APIContext } from 'astro') — untracked as an edge.
• T3 (find the one dead export) — the tool’s own dead-code query (max_degree=0) returned four to five false positives: functions that are live, called from .astro frontmatter the graph can’t see. The graph’s is_exported flag was also wrong on several functions (b64url, hmac, devStore, nameToSlug — all flagged exported, none of them actually has an export keyword in source). Getting the right answer required pulling full source for all four files and manually verifying every declaration.
• T8 (impact of deleting store.ts) — the IMPORTS graph found the 4 static importers cleanly, but missed all 3 dynamic import() call sites entirely: one inside a devStore() helper in auth.ts, one inside submit.ts, one inside account.astro frontmatter. Dynamic imports simply aren’t modeled as edges.

The per-task scoreboard

Task	Question	BASE	MCP	What happened
T1	who calls getSession	1.0	1.0	MCP graph found 5/7; recovered 2 .astro callers via its own text-search fallback
T2	imports of purchases.ts	1.0	1.0	MCP graph found 2/3; missed the type-only import
T3	one dead export in src/lib/	1.0	1.0	MCP’s dead-code query returned 4–5 false positives; needed full manual source verification
T4	exported function count	1.0	1.0	MCP counted from raw source because is_exported metadata was unreliable
T5	call chain, POST → b64url	1.0	1.0	Clean win for the graph — pure .ts, no framework boundary
T6	highest import-statement file	1.0	1.0	MCP’s edge count conflated per-symbol with per-statement; needed a regex cross-check
T7	where is formatDate defined	1.0	1.0	Clean win — exact line number in one call
T8	impact of deleting store.ts	1.0	1.0	MCP graph found 4/7; missed all 3 dynamic import() sites
Total		8/8	8/8	MCP fell back to its own text search on 4 of 8 tasks

Both arms landed on identical, correct answers to every question. But that parity is doing a lot of work to look reassuring: the MCP agent hit a wrong or incomplete graph answer on half the tasks and had to notice it, then escape into search_code — the tool’s built-in grep-equivalent — to re-derive the correct answer from source. An agent that trusted the graph’s first answer without that self-correction would have scored roughly 4/8, wrong or unsupported on T1, T3, T4, and T8.

The token math

Arm	Total tokens	vs. claim
BASE (plain Read/Grep)	75,817	—
MCP (codebase-memory-mcp)	172,319	+127% more, not 99% fewer

measured_savings = 1 − 172,319 / 75,817 = −127.3%. The mechanism is visible in the per-task breakdown above: the agent paid for the graph query, discovered it was wrong or incomplete, then paid again for the grep-equivalent fallback to get the real answer. Graph plus grep costs more than grep alone, every time the graph needs correcting — and on this repo it needed correcting on half the tasks.

Worth sitting with: the tool’s own marketing cites “~500 tokens vs ~80K for grep” per query. Our entire 8-task grep-only run cost 75,817 tokens — less than the claimed cost of one grep query in their materials. One-time indexing (seconds, done once at setup) is excluded from the MCP total above, which is generous to the tool.

Not the memory skill you’re thinking of

If “codebase-memory-mcp” and skills like memory-management or claude-mem sound like they solve the same problem, they don’t. Session-memory skills persist facts and decisions across conversations — what you told Claude last week, project context that would otherwise vanish between sessions. codebase-memory-mcp indexes code structure within a repo — call graphs, import edges, definitions — inside a single session. Both get called “memory.” One is conversation continuity, the other is static analysis. Don’t install this expecting it to remember your last session.

Caveats, stated plainly

We’d rather draw the boundaries ourselves than have you find them the hard way:

• n=8 tasks, one repo. 157 files, Astro/TypeScript — a mixed-framework setup close to the worst case for a CALLS/IMPORTS model that has to reason about template frontmatter alongside script files.
• Their arXiv preprint tested 31 repos; we tested one. It’s entirely plausible their average holds up better across a broader, more pure-TS/JS sample than our single mixed-framework repo represents.
• Large pure-TS/JS monorepos could look different. Where a naive grep genuinely returns huge output and the graph has clean edges to work with (no .astro frontmatter, no dynamic imports), the token math could plausibly favor the graph. T5 and T7 — pure TypeScript, structurally simple — are exactly that shape, and the graph wins cleanly there too.
• Indexing time is excluded from the MCP total, which favors the tool, not the baseline.

Reproducibility note: the 8 tasks were written and locked before either arm ran. Ground truth for every answer was independently grep-verified against the repo, not derived from either arm’s output. Both arms ran on the same Sonnet model, same repo checkout, fresh context each time.

Install with your eyes open

Use it as a supplementary tracer on pure-TS/JS repos where you need fast call-chain tracing or definition lookups and your codebase doesn’t lean on dynamic import(), type-only imports, or a templating layer the graph can’t parse. T5 and T7 show exactly what it’s good at.

Don’t treat its graph as source of truth on mixed-framework codebases. If your repo mixes Astro, Vue, Svelte, or similar templating with TypeScript, or leans on dynamic imports, budget for the tool to be wrong on “who calls this” and “what’s dead code” queries specifically — and budget the token cost of the fallback grep it will need to correct itself.

FAQ

Is the tool bad? No. Installation is clean, indexing is fast, and call-chain tracing plus definition lookup in plain TypeScript are genuinely excellent. The problem is narrower than “the tool is bad”: the flagship token-savings claim doesn’t hold on a mixed-framework repo, because graph blind spots force the agent to pay for both the graph and the grep.

Why do your numbers differ from the arXiv paper? Different sample. Their preprint averages over 31 repos; we tested one 157-file Astro/TypeScript repo, close to a worst case for their edge model (frontmatter calls, dynamic imports, type-only imports all fall outside CALLS/IMPORTS). Both results can be true at once — theirs a broader average, ours a specific, reproducible worst-case data point anyone on a mixed-framework repo should weigh before trusting the headline number.

Should I uninstall it? Not necessarily. If you work primarily in pure TypeScript or JavaScript without heavy dynamic imports, our T5/T7 data says it’s a solid tracer. If your repo mixes frameworks the way ours does, keep it installed but don’t let it replace grep — treat its answers as a hypothesis to verify, the way our MCP agent ended up doing on half the tasks anyway.

What about very large repos? We didn’t test one, and it’s a real gap in our data. The theoretical case for the tool is strongest there: a single grep across a huge monorepo can return an enormous result set, while a graph query stays small regardless of repo size. If you run a large pure-TS/JS monorepo, the arXiv paper’s 31-repo average is more relevant to you than our single small-repo result — but run your own A/B before trusting either number blind.