Academic Experiments
Audits ML experiments, blocking unverified numbers before a paper draft cites them.
Test report
- Verdict
- Works with setup
- Score
- Tested
- Jul 13, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Aug 10, 2026 · 451431b
Ran it against a toy repo whose README claims '92% accuracy' with a results.json but a train.py that only prints a placeholder and never actually trains anything; the skill's stance/red-lines rules correctly refuse to write that number as fact and instead force a [RESULT_UNVERIFIED] placeholder plus a Protocol Risks note about missing provenance, where a naive baseline just wrote the 92% into the results paragraph as given.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 4/5
- Triggers reliably 5/5
- Output vs. baseline 8/10
- Docs & honesty 5/5
What Academic Experiments does
Acts as an 'experiment forensics agent' for CS/AI/ML papers: it inventories existing run artifacts, classifies each result as newly_run, preexisting_artifact, or user_claim, attempts a minimal reproducible command before suggesting full retraining, and writes up Protocol Risks (data leakage, missing baselines, single-run results). Trigger it when someone asks to verify reproducibility, audit experiment artifacts, or build an evidence inventory before writing results into a paper; it explicitly refuses to launder an unverified user claim into a confident paper sentence. Not for literature review or prose writing tasks.
How to install Academic Experiments
git clone https://github.com/joshua-zyy/academic-paper-writer
cd academic-paper-writer
mkdir -p ~/.claude/skills
cp -r skills/academic-experiments ~/.claude/skills/academic-experiments
Skills live in ~/.claude/skills/ (global) or .claude/skills/
(per-project). Restart Claude Code after installing.
Commands — how to trigger Academic Experiments
-
/academic-experimentsAudits ML experiments, blocking unverified numbers before a paper draft cites them.
It also activates on plain-language prompts like these:
-
Audit my ML experiment results before I cite them -
Verify these accuracy numbers are reproducible -
Inventory my experiment artifacts for the paper
Frequently asked questions
- Is the Academic Experiments skill free?
- Yes. The skill itself is free from joshua-zyy/academic-paper-writer. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Academic Experiments work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 13, 2026. Verdict: Works with setup. Ran it against a toy repo whose README claims '92% accuracy' with a results.json but a train.py that only prints a placeholder and never actually trains anything; the skill's stance/red-lines rules correctly refuse to write that number as fact and instead force a [RESULT_UNVERIFIED] placeholder plus a Protocol Risks note about missing provenance, where a naive baseline just wrote the 92% into the results paragraph as given.
- What is the Academic Experiments SkillProof Score?
- 8.8/10 — installs cleanly 4/5, triggers reliably 5/5, output vs. baseline 8/10, docs & honesty 5/5.
- How do I install Academic Experiments?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Academic Experiments with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.