A/B Test Results Analysis

Analyse the results of an A/B test and produce a statistically rigorous interpretation with a clear recommendation, caveats, and suggested next steps.

Data AnalystClaudeCo-PilotChatGPTGeminiMidUpdated Mar-26

427·

Prompt

I need help interpreting the results of an A/B test. Here are the details: What was tested: Hypothesis: Primary metric: Secondary metrics tracked: Results: - Control: - Variant: Test duration: Statistical framework used: Please: 1. Calculate statistical significance and confidence intervals for the primary metric 2. Assess whether the result is practically significant, not just statistically significant 3. Check for any validity concerns — peeking, sample ratio mismatch, novelty effect, seasonality 4. Interpret the secondary metrics and flag any contradictions 5. Give a clear, unambiguous recommendation: ship, kill, or iterate — with reasoning 6. Suggest follow-up experiments if the result is inconclusive If any information is missing that would change your interpretation, ask me before proceeding.