Academic Experiments

Audits ML experiments, blocking unverified numbers before a paper draft cites them.

Works with setup

Test report

Verdict
Works with setup
Score
8.8/10
Tested
Jul 13, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Aug 10, 2026 · 451431b

Ran it against a toy repo whose README claims '92% accuracy' with a results.json but a train.py that only prints a placeholder and never actually trains anything; the skill's stance/red-lines rules correctly refuse to write that number as fact and instead force a [RESULT_UNVERIFIED] placeholder plus a Protocol Risks note about missing provenance, where a naive baseline just wrote the 92% into the results paragraph as given.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 4/5
  • Triggers reliably 5/5
  • Output vs. baseline 8/10
  • Docs & honesty 5/5

What Academic Experiments does

Acts as an 'experiment forensics agent' for CS/AI/ML papers: it inventories existing run artifacts, classifies each result as newly_run, preexisting_artifact, or user_claim, attempts a minimal reproducible command before suggesting full retraining, and writes up Protocol Risks (data leakage, missing baselines, single-run results). Trigger it when someone asks to verify reproducibility, audit experiment artifacts, or build an evidence inventory before writing results into a paper; it explicitly refuses to launder an unverified user claim into a confident paper sentence. Not for literature review or prose writing tasks.

How to install Academic Experiments

git clone https://github.com/joshua-zyy/academic-paper-writer
cd academic-paper-writer
mkdir -p ~/.claude/skills
cp -r skills/academic-experiments ~/.claude/skills/academic-experiments

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger Academic Experiments

  • /academic-experiments Audits ML experiments, blocking unverified numbers before a paper draft cites them.

It also activates on plain-language prompts like these:

  • Audit my ML experiment results before I cite them
  • Verify these accuracy numbers are reproducible
  • Inventory my experiment artifacts for the paper

Frequently asked questions

Is the Academic Experiments skill free?
Yes. The skill itself is free from joshua-zyy/academic-paper-writer. SkillProof publishes the install command and an independent test verdict at no cost.
Does Academic Experiments work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 13, 2026. Verdict: Works with setup. Ran it against a toy repo whose README claims '92% accuracy' with a results.json but a train.py that only prints a placeholder and never actually trains anything; the skill's stance/red-lines rules correctly refuse to write that number as fact and instead force a [RESULT_UNVERIFIED] placeholder plus a Protocol Risks note about missing provenance, where a naive baseline just wrote the 92% into the results paragraph as given.
What is the Academic Experiments SkillProof Score?
8.8/10 — installs cleanly 4/5, triggers reliably 5/5, output vs. baseline 8/10, docs & honesty 5/5.
How do I install Academic Experiments?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use Academic Experiments with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.