SkillProof tested Claude skills

Optimise Skill

Vendors Microsoft's SkillOpt/OPRO engine to retrain one skill's judge-shaped rubric against a labelled outcome corpus, gated by a held-out set.

Door yungbose · yungbose/upskill

Getest · Werkt ★ 9.2/10

Optimise Skill — Vendors Microsoft's SkillOpt/OPRO engine to retrain one skill's judge-shaped rubric against a labelled outcome corpus, gated by a held-out set.

Wat het doet

A decision-policy optimiser for Claude skills: given a manifest, a single tunable rubric document, and a labelled markdown corpus of past decisions, it runs SkillOpt's rollout/reflect/edit/gate loop and only promotes an edited rubric when it strictly beats baseline on an adversarial held-out set. Triggers on 'train/optimise the <X> rubric' or the /optimise-skill <consumer> command; explicitly out of scope for tuning a skill's triggering description, which stays with skill-creator. Live tuning shells out to the claude CLI against a Claude Max subscription (or an OpenAI key as an alternate provider).

Testrapport

Actually ran it: scripts/setup.py --apply scaffolded a real 9-file optimisation/ contract (manifest, rubric, train/val/test corpus dirs, holdout incident/reversal fixtures, ledger) for a throwaway consumer, validate_manifest.py passed it clean, and the full 206-test suite passed instantly with stub judges and zero LLM calls, exactly matching the README's own verified claim. Did not run the live LLM training loop itself since that shells out to a real claude -p subscription call.

Getest op: 2026-07-15 · Claude Code 2.x (agent harness)

Installatie

git clone https://github.com/yungbose/upskill.git
cd upskill
mkdir -p ~/.claude/skills
cp -r optimise-skill ~/.claude/skills/optimise-skill
pip install -r optimise-skill/requirements.txt  # PyYAML only; live tuning additionally needs the claude CLI + a Claude Max subscription

Commando's en voorbeeldprompts

  • /optimise-skillVendors Microsoft's SkillOpt/OPRO engine to retrain one skill's judge-shaped rubric against a labelled outcome corpus, gated by a held-out set.

Skills reageren op gewone verzoeken — geen commando's om te onthouden. Na installatie activeren prompts zoals deze de skill (in het Engels):

  • train this skill's rubric against our labeled decision corpus
  • optimize the judge rubric and validate it on a held-out test set
  • scaffold a new optimisation consumer and run its test suite

Volledig rapport (Engels) →