Experiment Lab
A/B testing results and optimization analytics.
Campaign:
Completed30,000 recipients
Style Impact
Testing writing styles (content held constant)
Control
Neutral Style
n = 5,000
28.1%open rate
Variant+21.7%
Urgent Style
n = 5,000
34.2%open rate
Variant+12.1%
Premium Style
n = 5,000
31.5%open rate
Winner: Urgent Style
95% statistical confidence
Content Focus
Testing content themes (style held constant)
Control
Generic Content
n = 5,000
26.3%open rate
Variant+47.9%
Discount Focus
n = 5,000
38.9%open rate
Variant+12.9%
Seasonal Focus
n = 5,000
29.7%open rate
Winner: Discount Focus
95% statistical confidence
Thompson Sampling
Confidence intervals converge as data is collected
Variant A
Variant B
Variant C
Variant A
28%±6.0
Variant BWINNER
30%±6.0
Variant C
26%±6.0
Thompson Sampling balances exploration with exploitation, automatically shifting traffic to winning variants as confidence increases.
Key Insights
Best Style
Urgent Style
+21.7% vs control
Best Content
Discount Focus
+47.9% vs control
Optimal Combo
Urgent + Discount
+58.2% predicted