Is the A/B Test Analysis skill free?

Yes. The skill itself is free from phuryn/pm-skills. SkillProof publishes the install command and an independent test verdict at no cost.

What is the A/B Test Analysis SkillProof Score?

6.8/10 — installs cleanly 5/5, triggers reliably 4/5, output vs. baseline 5/10, docs & honesty 3/5.

How do I install A/B Test Analysis?

Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.

Can I use A/B Test Analysis with Cursor, Copilot, Gemini CLI, Codex or other AI tools?

The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.

A/B Test Analysis

Evaluates A/B test results for significance, sample size, and ship/stop recommendations.

by phuryn · phuryn/pm-skills

Works with setup

Install — copy the command Download source ↗

Test report

Verdict: Works with setup
Score: 6.8/10
Tested: Jul 11, 2026
Environment: Claude Code 2.x (agent harness)
Upstream re-checked: Aug 10, 2026 · 58c2397

Verbatim install is clean and the trigger scope is tight, but the SKILL.md sample-size formula n=(Z^2*2*p*(1-p))/MDE^2 omits the z-beta power term, so it understates required n by roughly 2x (~4,780 vs ~10,350 visitors/arm here) and would rubber-stamp a genuinely underpowered test. On the checkout-CTA A/B (8.20% vs 9.30%, +13.4% relative), both the skill and a plain base agent independently landed the same correct read: p=0.0516 (borderline, 95% CI -0.01pp to +2.21pp barely crossing zero), no SRM, test underpowered for its own effect, and a guardrail AOV dip of -2.66% -> EXTEND, do not ship yet. The skill added no statistical accuracy or decision quality over the un-skilled baseline.

Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works

Installs cleanly 5/5
Triggers reliably 4/5
Output vs. baseline 5/10
Docs & honesty 3/5

What A/B Test Analysis does

Analyze A/B test results with statistical significance, sample size validation, confidence intervals, and ship/extend/stop recommendations. Use when evaluating experiment results, checking if a test reached significance, interpreting split test data, or deciding whether to ship a variant.

How to install A/B Test Analysis

git clone https://github.com/phuryn/pm-skills
cd pm-skills
mkdir -p ~/.claude/skills
cp -r pm-data-analytics/skills/ab-test-analysis ~/.claude/skills/ab-test-analysis

Skills live in ~/.claude/skills/ (global) or .claude/skills/ (per-project). Restart Claude Code after installing.

Commands — how to trigger A/B Test Analysis

/ab-test-analysis Evaluates A/B test results for significance, sample size, and ship/stop recommendations.

It also activates on plain-language prompts like these:

Check if this A/B test reached statistical significance
Analyze these split test results and recommend ship or stop
Validate the sample size on our pricing experiment

Frequently asked questions

Is the A/B Test Analysis skill free?: Yes. The skill itself is free from phuryn/pm-skills. SkillProof publishes the install command and an independent test verdict at no cost.
Does A/B Test Analysis work with Claude Code?: We tested it with Claude Code 2.x (agent harness) on Jul 11, 2026. Verdict: Works with setup. Verbatim install is clean and the trigger scope is tight, but the SKILL.md sample-size formula n=(Z^2*2*p*(1-p))/MDE^2 omits the z-beta power term, so it understates required n by roughly 2x (~4,780 vs ~10,350 visitors/arm here) and would rubber-stamp a genuinely underpowered test. On the checkout-CTA A/B (8.20% vs 9.30%, +13.4% relative), both the skill and a plain base agent independently landed the same correct read: p=0.0516 (borderline, 95% CI -0.01pp to +2.21pp barely crossing zero), no SRM, test underpowered for its own effect, and a guardrail AOV dip of -2.66% -> EXTEND, do not ship yet. The skill added no statistical accuracy or decision quality over the un-skilled baseline.
What is the A/B Test Analysis SkillProof Score?: 6.8/10 — installs cleanly 5/5, triggers reliably 4/5, output vs. baseline 5/10, docs & honesty 3/5.
How do I install A/B Test Analysis?: Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
Can I use A/B Test Analysis with Cursor, Copilot, Gemini CLI, Codex or other AI tools?: The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.