Sequential Confidence Intervals for Comparing Two Proportions with Applications in A/B Testing - Jun Hu; Lijia Zheng; Ibtihal Alanazi (Mathematics)
This article addresses the use of fixed-width confidence intervals (FWCIs) for comparing two independent Bernoulli populations in A/B testing scenarios. Two sequential estimation procedures are proposed: one for estimating the difference in log probabilities of success and the other for log odds ratios. Both methods showcase great efficiency, as established via theoretical analysis and Monte Carlo simulations. The practical utility of these methods is demonstrated through two real-world applications: analyzing retention rates in mobile game Cookie Cats and evaluating the effectiveness of online advertising.