Arxiv Preflight

Grounded pre-submission check that catches fabricated references and LLM residue before arXiv upload.

Tested · Works

Test report

Verdict
Tested · Works
Score
9.6/10
Tested
Jul 20, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Aug 10, 2026 · 9a09aa7

Fetched SKILL.md + all 4 scripts + 4 reference docs via sparse checkout and ran the real pipeline on the repo's own `hallucinated_ref` fixture. verify_references.py hit live OpenAlex/Crossref and correctly flagged both fabricated cites (liang2023quantum, nguyen2024recursive) as BLOCKER with counter-evidence (they resolve to unrelated real papers), matched the 3 genuine refs, and caught two year-metadata errors; generate_preflight_report.py produced a HOLD report with file:line locations. Security: no curl|sh, no eval/base64; network limited to Crossref/arXiv/OpenAlex/Semantic Scholar via urllib; one subprocess call is `pdftotext` only. Minor rough edges: a SyntaxWarning and datetime.utcnow deprecation warnings, and the report script uses `--refs` (its own --help is correct).

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 5/5
  • Triggers reliably 5/5
  • Output vs. baseline 9/10
  • Docs & honesty 5/5

What Arxiv Preflight does

Runs a submission-readiness preflight on a manuscript before arXiv: extracts LaTeX/PDF text, scans for LLM meta-comments and placeholder data, and verifies every reference against Crossref/arXiv/OpenAlex/Semantic Scholar, then emits a Markdown report with a HOLD/PASS decision. Triggers when preparing an arXiv submission, verifying references, or checking a paper for hallucinated citations or leftover AI artifacts.

How to install Arxiv Preflight

Copy the `arxiv-preflight/` directory into your skills folder (e.g. `~/.claude/skills/`). Pure Python 3.9+ stdlib, no pip installs; `pdftotext` (poppler) only needed for the optional PDF input path. Reference verification needs outbound network to the scholarly APIs.

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger Arxiv Preflight

  • /arxiv-preflight Grounded pre-submission check that catches fabricated references and LLM residue before arXiv upload.

It also activates on plain-language prompts like these:

  • Check my paper before uploading to arXiv
  • Verify all the references in this manuscript
  • Scan this draft for leftover AI meta-comments

Frequently asked questions

Is the Arxiv Preflight skill free?
Yes. The skill itself is free from aipoch/medical-research-skills. SkillProof publishes the install command and an independent test verdict at no cost.
Does Arxiv Preflight work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 20, 2026. Verdict: Tested · Works. Fetched SKILL.md + all 4 scripts + 4 reference docs via sparse checkout and ran the real pipeline on the repo's own `hallucinated_ref` fixture. verify_references.py hit live OpenAlex/Crossref and correctly flagged both fabricated cites (liang2023quantum, nguyen2024recursive) as BLOCKER with counter-evidence (they resolve to unrelated real papers), matched the 3 genuine refs, and caught two year-metadata errors; generate_preflight_report.py produced a HOLD report with file:line locations. Security: no curl|sh, no eval/base64; network limited to Crossref/arXiv/OpenAlex/Semantic Scholar via urllib; one subprocess call is `pdftotext` only. Minor rough edges: a SyntaxWarning and datetime.utcnow deprecation warnings, and the report script uses `--refs` (its own --help is correct).
What is the Arxiv Preflight SkillProof Score?
9.6/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 9/10, docs & honesty 5/5.
How do I install Arxiv Preflight?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use Arxiv Preflight with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.