A/B Test Designer
Obiecuje statystyki testów A/B; treść to 230-słowowy ogólny szablon
Tested · Didn't pass
Co robi ten skill
Advertised as a skill for designing and analysing A/B tests - sample sizes, significance testing, multiple comparisons and results interpretation. In practice the SKILL.md body contains no statistical procedure, formula, or default, only boilerplate shared verbatim with the other 182 skills in the same repository. Installs cleanly but adds nothing to an A/B test question.
Raport z testu
Gave it a real experiment - 603/12,050 versus 683/11,980 - and asked whether it was significant and what sample size a 10% lift would need. Without the skill the answer came out complete: z = 2.400, p = 0.0164, 95% CI [+0.13pp, +1.27pp], about 31,207 users per arm for 80% power, plus SRM, peeking and winner's-curse caveats. With the skill the numbers were identical, because the body supplies no test, no formula and no alpha or power default - its actual instructions are 'understand the full context', 'apply best practices' and 'validate inputs before processing'. All 183 SKILL.md files in this repo share that same 230-word template, emitted by a generate_skills.py script still pointing at the author's Windows desktop. The words 'p-value', 'power', 'confidence' and 'Bonferroni' appear nowhere in the file.
Testowano: 2026-07-30 · Claude Code 2.x (agent harness)
Instalacja
git clone https://github.com/inbharatai/claude-skills.git mkdir -p ~/.claude/skills cd claude-skills && cp -r skills/ab-test-designer ~/.claude/skills/ab-test-designer
Komendy i przykładowe prompty
/ab-test-designerObiecuje statystyki testów A/B; treść to 230-słowowy ogólny szablon
Skille uruchamiają się na zwykłe polecenia — bez komend do zapamiętania. Po instalacji aktywują go prompty takie jak te (po angielsku):
Is my A/B test result significant? Control 603/12050, variant 683/11980How many users per variant do I need for 80% power at a 10% lift?We ran 5 variants, how do I correct the p-values for multiple comparisons?