Content Refinement Agent
Iterative paper.tex peer-review refinement loop with deterministic accept/revert halt gates
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 21, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Aug 10, 2026 · 27701b2
Cloned the repo; all 7 scripts, 7 references, and 4 shared files exist. Ran the deterministic core on crafted score JSONs: score_delta.py returned exit 0 ACCEPT (58.0->67.3, band Major->Minor), exit 1 REVERT on regression, exit 5 HALT_TARGET_MET at 81.0; decision_band.py mapped 67.3->Minor Revision; concession_guard.py returned exit 1 for a standing CRITICAL, exit 0 for a valid concession (rebuttal_score>=4) — matching the documented exit codes. For output vs baseline I revised a short paper.tex against reviewer feedback requesting a new ImageNet baseline plus flagging an overclaim: the no-skill baseline wrote "we acknowledge as a limitation" (fails the skill's Rule 2 limitation grep) and fabricated a 76% ImageNet number absent from experimental_log.md (fails Rule 3), while the skill-following revision reframed the overclaim, invented no number, and passed both gates. Frontmatter parses (name+description present); no security smells. Verdict pass; the full multi-iteration latexmk loop needs sibling skills plus a workspace scaffold I did not build. Category hint "sales" is wrong — this is academic writing tooling.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 7/10
- Docs & honesty 4/5
What Content Refinement Agent does
Refines an academic LaTeX draft (paper.tex) by simulating peer review and applying targeted revisions across up to three iterations, using Python scripts that deterministically compute 0-100 decision bands, accept/revert/plateau/target-met halts, and a Devil's Advocate concession guard. Triggers as Step 5 of the PaperOrchestra pipeline or when a user asks to refine a draft, iterate on a paper, or run peer review on a paper. Enforces anti-reward-hack rules: no fabricated experiments, no new numeric claims outside experimental_log.md, no defensive limitation-listing.
How to install Content Refinement Agent
git clone --depth 1 https://github.com/Ar9av/PaperOrchestra.git /tmp/content-refinement-agent-src
mkdir -p ~/.claude/skills
cp -R /tmp/content-refinement-agent-src/skills/content-refinement-agent ~/.claude/skills/content-refinement-agent
# The SKILL body also calls scripts in sibling skills and reads skills/shared/*, so for the full loop:
# cp -R /tmp/content-refinement-agent-src/skills/paper-orchestra ~/.claude/skills/paper-orchestra
# cp -R /tmp/content-refinement-agent-src/skills/section-writing-agent ~/.claude/skills/section-writing-agent
# cp -R /tmp/content-refinement-agent-src/skills/shared ~/.claude/skills/shared
# Decision scripts (score_delta.py, decision_band.py, concession_guard.py, snapshot.py) are pure Python 3 stdlib.
# Running the FULL end-to-end loop additionally needs: latexmk (TeX Live) to compile paper.pdf,
# a workspace/ scaffold produced by earlier PaperOrchestra steps, and host LLM reviewer/revision calls.
Skills live in ~/.claude/skills/ (global) or .claude/skills/
(per-project). Restart Claude Code after installing.
Commands — how to trigger Content Refinement Agent
-
/content-refinement-agentIterative paper.tex peer-review refinement loop with deterministic accept/revert halt gates
It also activates on plain-language prompts like these:
-
Refine the draft using simulated peer review -
Iterate on the paper until it hits the target score -
Run the Devil's Advocate check on this paper section
Frequently asked questions
- Is the Content Refinement Agent skill free?
- Yes. The skill itself is free from Ar9av/PaperOrchestra. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Content Refinement Agent work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 21, 2026. Verdict: Tested · Works. Cloned the repo; all 7 scripts, 7 references, and 4 shared files exist. Ran the deterministic core on crafted score JSONs: score_delta.py returned exit 0 ACCEPT (58.0->67.3, band Major->Minor), exit 1 REVERT on regression, exit 5 HALT_TARGET_MET at 81.0; decision_band.py mapped 67.3->Minor Revision; concession_guard.py returned exit 1 for a standing CRITICAL, exit 0 for a valid concession (rebuttal_score>=4) — matching the documented exit codes. For output vs baseline I revised a short paper.tex against reviewer feedback requesting a new ImageNet baseline plus flagging an overclaim: the no-skill baseline wrote "we acknowledge as a limitation" (fails the skill's Rule 2 limitation grep) and fabricated a 76% ImageNet number absent from experimental_log.md (fails Rule 3), while the skill-following revision reframed the overclaim, invented no number, and passed both gates. Frontmatter parses (name+description present); no security smells. Verdict pass; the full multi-iteration latexmk loop needs sibling skills plus a workspace scaffold I did not build. Category hint "sales" is wrong — this is academic writing tooling.
- What is the Content Refinement Agent SkillProof Score?
- 8.4/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 7/10, docs & honesty 4/5.
- How do I install Content Refinement Agent?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Content Refinement Agent with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.