Walk-Forward Leakage Auditor
Audits time-series backtests for leakage before you trust the Sharpe
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 30, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Jul 30, 2026 · 5489d45
⚠ This skill is no longer available upstream. Our re-check on Aug 10, 2026 couldn't find it any more (repo unreachable/deleted). The test below is what we measured on Jul 30, 2026 and we're leaving it up as a record — but there is nothing left to install, so we've removed the command.
Ran the audit head to head on a 140-line walk-forward equity backtest built on a seeded random walk, so the honest answer was known in advance: there is no signal in the data. The unaudited script reported out-of-sample AUC 0.636 and Sharpe 2.00 and concluded ship it; a normal no-checklist review caught the forward-return column in the feature matrix, the full-sample z-score and the scaler fitted before the split, which brought it to AUC 0.531 and Sharpe 1.05 — still a ship decision. Working the SKILL.md checklist section by section additionally caught the missing five-day embargo between train and test, a merge_asof with direction=forward on a macro series published three weeks after its period end, an entry threshold tuned on the test Sharpe, and a benchmark computed over a different date range and capital base; after those fixes AUC fell to 0.511 and the strategy lost to buy-and-hold on the matched window, flipping the go/no-go conclusion. A failable guard written per the skill's step 4 fails on both the original and the baseline-fixed file and passes only on the fully audited one. The repo's own worked examples reproduce exactly — 428.6 vs 873.7 MAE, matching the numbers printed in EXAMPLE.md.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 9/10
- Docs & honesty 5/5
What Walk-Forward Leakage Auditor does
A structured audit checklist for time-based machine learning evaluations: walk-forward backtests, rolling retraining, forecasting pipelines and any chronological train/test split. It makes the agent state the prediction-time contract, trace data from raw inputs through features, labels, splits and fitted transforms, then work named red-flag lists for split logic, feature timing, transform scope, label construction, simulation realism and experiment comparability. Triggers when you ask to review or change backtest, rolling-CV or forecasting code.
How to install Walk-Forward Leakage Auditor
Nothing to install: the source repository no longer has this skill. If the author brings it back, our daily re-check will pick it up and the command will reappear here.
Commands — how to trigger Walk-Forward Leakage Auditor
-
/walk-forward-leakage-auditorAudits time-series backtests for leakage before you trust the Sharpe
It also activates on plain-language prompts like these:
-
Review my walk-forward backtest before I trust this Sharpe ratio -
My forecasting model scores 0.95 AUC in backtest but fails live, what's wrong -
I'm adding rolling retraining to this pipeline, check the split logic
Frequently asked questions
- Is the Walk-Forward Leakage Auditor skill free?
- Yes. The skill itself is free from mospira/walkforward-audit. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Walk-Forward Leakage Auditor work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 30, 2026. Verdict: Tested · Works. Ran the audit head to head on a 140-line walk-forward equity backtest built on a seeded random walk, so the honest answer was known in advance: there is no signal in the data. The unaudited script reported out-of-sample AUC 0.636 and Sharpe 2.00 and concluded ship it; a normal no-checklist review caught the forward-return column in the feature matrix, the full-sample z-score and the scaler fitted before the split, which brought it to AUC 0.531 and Sharpe 1.05 — still a ship decision. Working the SKILL.md checklist section by section additionally caught the missing five-day embargo between train and test, a merge_asof with direction=forward on a macro series published three weeks after its period end, an entry threshold tuned on the test Sharpe, and a benchmark computed over a different date range and capital base; after those fixes AUC fell to 0.511 and the strategy lost to buy-and-hold on the matched window, flipping the go/no-go conclusion. A failable guard written per the skill's step 4 fails on both the original and the baseline-fixed file and passes only on the fully audited one. The repo's own worked examples reproduce exactly — 428.6 vs 873.7 MAE, matching the numbers printed in EXAMPLE.md.
- What is the Walk-Forward Leakage Auditor SkillProof Score?
- 9.6/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 9/10, docs & honesty 5/5.
- How do I install Walk-Forward Leakage Auditor?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Walk-Forward Leakage Auditor with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.