A/B Testing

Designs statistically valid experiments and runs a full ICE-scored experimentation program.

Tested · Works

Test report

Verdict
Tested · Works
Score
9.6/10
Tested
Jul 14, 2026
Environment
Claude Code 2.x (agent harness)
Upstream re-checked
Aug 10, 2026 · e3f9997

Follows its own peeking rule to the letter: given a fake 3-day result with 95% confidence, it refused to call a winner and cited regression-to-the-mean and day-of-week effects rather than declaring victory, something the baseline answer didn't even think to flag.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

  • Installs cleanly 5/5
  • Triggers reliably 5/5
  • Output vs. baseline 9/10
  • Docs & honesty 5/5

What A/B Testing does

Builds hypothesis-first A/B/A-B-n test plans with sample-size guidance, primary/secondary/guardrail metrics, and a documented peeking-problem warning. Also covers running experimentation as an ongoing program: ICE-scored backlog, experiment velocity targets, and a reusable winner playbook. No code or platform integration — it's pure planning and analysis guidance, not a stats engine.

How to install A/B Testing

git clone https://github.com/coreyhaines31/marketingskills
cd marketingskills
mkdir -p ~/.claude/skills
cp -r skills/ab-testing ~/.claude/skills/ab-testing

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger A/B Testing

  • /ab-testing Designs statistically valid experiments and runs a full ICE-scored experimentation program.

It also activates on plain-language prompts like these:

  • Design an A/B test and calculate the needed sample size
  • Check if my test duration is long enough to be valid
  • Frame a hypothesis and stats plan for this experiment

Appears in

Frequently asked questions

Is the A/B Testing skill free?
Yes. The skill itself is free from coreyhaines31/marketingskills. SkillProof publishes the install command and an independent test verdict at no cost.
Does A/B Testing work with Claude Code?
We tested it with Claude Code 2.x (agent harness) on Jul 14, 2026. Verdict: Tested · Works. Follows its own peeking rule to the letter: given a fake 3-day result with 95% confidence, it refused to call a winner and cited regression-to-the-mean and day-of-week effects rather than declaring victory, something the baseline answer didn't even think to flag.
What is the A/B Testing SkillProof Score?
9.6/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 9/10, docs & honesty 5/5.
How do I install A/B Testing?
Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use A/B Testing with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.