Analyzing Experiment Session Replays
Compares PostHog session replays across A/B experiment variants for qualitative insight.
Test report
- Verdict
- Tested · Works
- Score
- Tested
- Jul 14, 2026
- Environment
- Claude Code 2.x (agent harness)
- Upstream re-checked
- Aug 10, 2026 · cecca7c
Documents a real PostHog footgun: the type:flag recording filter silently returns unfiltered results, so it directs you to scope variants via the $feature/<flag_key> event property instead — a gotcha a generic replay-analysis answer would never surface.
Scored on four weighted criteria — install, triggering, output vs. baseline, docs. How scoring works
- Installs cleanly 5/5
- Triggers reliably 5/5
- Output vs. baseline 9/10
- Docs & honesty 4/5
What Analyzing Experiment Session Replays does
Pulls session recordings per experiment variant using PostHog's HogQL queries and query-session-recordings-list tool, then compares quantitative activity metrics (duration, clicks, console errors) and qualitative behavior patterns between control and test groups. Triggers when a user wants to see how users behave differently across experiment variants, identify usability issues from replays, or get qualitative color to complement quantitative A/B test results. Documents a specific PostHog footgun: the type:flag recording filter is silently ignored by the API, so variant scoping must go through the $feature/<flag_key> event property instead.
How to install Analyzing Experiment Session Replays
git clone https://github.com/PostHog/ai-plugin
cd ai-plugin
mkdir -p ~/.claude/skills
cp -r skills/analyzing-experiment-session-replays ~/.claude/skills/analyzing-experiment-session-replays
Skills live in ~/.claude/skills/ (global) or .claude/skills/
(per-project). Restart Claude Code after installing.
Commands — how to trigger Analyzing Experiment Session Replays
-
/analyzing-experiment-session-replaysCompares PostHog session replays across A/B experiment variants for qualitative insight.
It also activates on plain-language prompts like these:
-
Compare session recordings between our control and test experiment groups -
Pull PostHog replays for each variant and tell me how users behaved differently -
I need qualitative insight from replays to explain our A/B test results
Frequently asked questions
- Is the Analyzing Experiment Session Replays skill free?
- Yes. The skill itself is free from PostHog/ai-plugin. SkillProof publishes the install command and an independent test verdict at no cost.
- Does Analyzing Experiment Session Replays work with Claude Code?
- We tested it with Claude Code 2.x (agent harness) on Jul 14, 2026. Verdict: Tested · Works. Documents a real PostHog footgun: the type:flag recording filter silently returns unfiltered results, so it directs you to scope variants via the $feature/<flag_key> event property instead — a gotcha a generic replay-analysis answer would never surface.
- What is the Analyzing Experiment Session Replays SkillProof Score?
- 9.2/10 — installs cleanly 5/5, triggers reliably 5/5, output vs. baseline 9/10, docs & honesty 4/5.
- How do I install Analyzing Experiment Session Replays?
- Copy the install command from this page, run it in your terminal, and restart Claude Code. Skills live in ~/.claude/skills/ (global) or .claude/skills/ inside a project.
- Can I use Analyzing Experiment Session Replays with Cursor, Copilot, Gemini CLI, Codex or other AI tools?
- The SKILL.md format is native to Claude (Claude Code, Desktop, claude.ai). The instructions inside adapt to other assistants: Cursor rules, GitHub Copilot instructions, Windsurf rules, Custom GPTs, AGENTS.md for OpenAI Codex, and GEMINI.md for Google Gemini CLI — our conversion guides cover each, and the free converter on the tools page does the wrapping for you.