A/B Testing
Designs statistically valid experiments and runs a full ICE-scored experimentation program.
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 14, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Aug 10, 2026 · e3f9997
Follows its own peeking rule to the letter: given a fake 3-day result with 95% confidence, it refused to call a winner and cited regression-to-the-mean and day-of-week effects rather than declaring victory, something the baseline answer didn't even think to flag.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 9/10
- Docs & honesty 5/5
What A/B Testing does
Builds hypothesis-first A/B/A-B-n test plans with sample-size guidance, primary/secondary/guardrail metrics, and a documented peeking-problem warning. Also covers running experimentation as an ongoing program: ICE-scored backlog, experiment velocity targets, and a reusable winner playbook. No code or platform integration — it's pure planning and analysis guidance, not a stats engine.
How to install A/B Testing
git clone https://github.com/coreyhaines31/marketingskills
cd marketingskills
mkdir -p ~/.claude/skills
cp -r skills/ab-testing ~/.claude/skills/ab-testing
Skills live in ~/.claude/skills/ (global) or .claude/skills/
(per-project). Restart Claude Code after installing.
Commands — how to trigger A/B Testing
-
/ab-testingDesigns statistically valid experiments and runs a full ICE-scored experimentation program.
It also activates on plain-language prompts like these:
-
Design an A/B test and calculate the needed sample size -
Check if my test duration is long enough to be valid -
Frame a hypothesis and stats plan for this experiment
Appears in
Frequently asked questions
- Is the A/B Testing skill free?
- Yes. The skill itself is free from coreyhaines31/marketingskills. SkillProof publishes the install command and an independent test verdict at no cost.
- Does A/B Testing work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 14, 2026. Verdict: Tested · Works. Follows its own peeking rule to the letter: given a fake 3-day result with 95% confidence, it refused to call a winner and cited regression-to-the-mean and day-of-week effects rather than declaring victory, something the baseline answer didn't even think to flag.
- What is the A/B Testing SkillProof Score?
- 9.6/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 9/10, docs & honesty 5/5.
- How do I install A/B Testing?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use A/B Testing with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.