A/B Test Designer
Lover A/B teststatistik; teksten er en 230-ords generisk skabelon
Tested · Didn't pass
Hvad det gør
Annonceres som en skill for designing and analysing A/B tests - sample sizes, significance testing, multiple comparisons and results interpretation. I praksis indeholder SKILL.md body ingen statistical procedure, formula, or default, only boilerplate shared verbatim with the other 182 skills in the same repository. Installeres cleanly but adds nothing to an A/B test question.
Testrapport
Gave it a real experiment - 603/12,050 versus 683/11,980 - and asked whether it was significant and what sample size a 10% lift would need. Without the skill the answer came out complete: z = 2.400, p = 0.0164, 95% CI [+0.13pp, +1.27pp], about 31,207 users per arm for 80% power, plus SRM, peeking and winner's-curse caveats. With the skill the numbers were identical, because the body supplies no test, no formula and no alpha or power default - its actual instructions are 'understand the full context', 'apply best practices' and 'validate inputs before processing'. All 183 SKILL.md files in this repo share that same 230-word template, emitted by a generate_skills.py script still pointing at the author's Windows desktop. The words 'p-value', 'power', 'confidence' and 'Bonferroni' appear nowhere in the file.
Testet: 2026-07-30 · Claude Code 2.x (agent harness)
Installation
git clone https://github.com/inbharatai/claude-skills.git mkdir -p ~/.claude/skills cd claude-skills && cp -r skills/ab-test-designer ~/.claude/skills/ab-test-designer
Kommandoer og eksempelprompter
/ab-test-designerLover A/B teststatistik; teksten er en 230-ords generisk skabelon
Skills udløses af almindelige forespørgsler — ingen kommandoer at huske. Efter installationen aktiverer prompter som disse skillen (på engelsk):
Is my A/B test result significant? Control 603/12050, variant 683/11980How many users per variant do I need for 80% power at a 10% lift?We ran 5 variants, how do I correct the p-values for multiple comparisons?