Sample-size honest

Indie SaaS experiment recipes.

Honest A/B test recipes for indie SaaS — pricing, headlines, CTAs, onboarding, email, checkout friction, trial length, social proof. Each page covers the hypothesis structure, the variant design, the decision metric, the honest sample-size band, the procedure, and the self-deceptions most indie SaaS founders make running these tests.

Pricing

Pricing test
Min sample: 200+ paying customers per variant for a 25%+ revenue-per-visitor lift detection. Below this, the math is too noisy. Indie SaaS below 500 customers should rarely run pricing tests; qualitative + price-anchor research is more honest.
Annual-vs-monthly discount test
Min sample: 200+ paying customers per variant. Annual decisions are higher-stakes; sample sizes need to be larger than monthly tests.

Headlines

Above-the-fold headline test
Min sample: 1,000 visitors per variant for a 50%+ relative lift detection. 2,000+ for 25% lift. 5,000+ for 10% lift. Most indie SaaS sites cannot run honest headline tests below 1,000 visits per variant.

CTAs

CTA copy test
Min sample: 500-1,000 visitors per variant for a 25%+ lift detection on click-rate.

Email

Onboarding email sequence test
Min sample: 300+ trial signups per variant for a 25%+ lift detection on activation.

Checkout

Checkout friction test
Min sample: 300+ checkout starts per variant for a 20%+ lift detection.

Trial mechanics

Trial-length test
Min sample: 300+ trial signups per variant for a 25%+ conversion-rate lift detection. Below 300, qualitative beats quantitative.

Social proof placement test
Min sample: 1,000+ visitors per variant for 25% lift detection.

Before running tests, diagnose the page

A/B tests on a fundamentally broken page produce two losing variants. Run the free diagnostic first to label what is breaking, fix that, then test the variations.

Get the free diagnostic Funnel metric benchmarks