An automated platform trial framework for A/B testing
This paper proposes a platform trial for conducting A/B tests with multiple arms and interim monitoring to investigate the impact of several factors on the expected sample size and probability of early stopping. We examined the performance of three stopping boundaries: O’Brien Fleming (OBF) stopping...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | Contemporary Clinical Trials Communications |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2451865424001352 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850113836995051520 |
|---|---|
| author | Wenru Zhou Miranda Kroehl Maxene Meier Alexander Kaizer |
| author_facet | Wenru Zhou Miranda Kroehl Maxene Meier Alexander Kaizer |
| author_sort | Wenru Zhou |
| collection | DOAJ |
| description | This paper proposes a platform trial for conducting A/B tests with multiple arms and interim monitoring to investigate the impact of several factors on the expected sample size and probability of early stopping. We examined the performance of three stopping boundaries: O’Brien Fleming (OBF) stopping for either futility or difference (both), Pocock stopping for futility only, and fixed sample size design. We simulated twelve scenarios of different orders of arms based on various effect sizes, as well as considered 1 or 3 interim looks. The overall findings are summarizing in a flowchart to provide intuitive guidance for the design of the platform based on the simulation. We found that it is better to use OBF stopping for both if there is any effective variant and the trial is sufficiently powered to detect the expected effect size. If the study is underpowered to detect a difference, we recommend fixed sample size design to gather as much information as possible, however if the expected sample size is important to minimize, we recommend using Pocock boundaries with futility monitoring. Our results aimed at helping high-tech companies conduct their own studies without requiring extensive knowledge of clinical trial design and statistical methodology. |
| format | Article |
| id | doaj-art-01e77cfafb3c463896700ec89b5491b3 |
| institution | OA Journals |
| issn | 2451-8654 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Contemporary Clinical Trials Communications |
| spelling | doaj-art-01e77cfafb3c463896700ec89b5491b32025-08-20T02:37:03ZengElsevierContemporary Clinical Trials Communications2451-86542024-12-014210138810.1016/j.conctc.2024.101388An automated platform trial framework for A/B testingWenru Zhou0Miranda Kroehl1Maxene Meier2Alexander Kaizer3Department of Biostatistics and Informatics University of Colorado, United States of America; Corresponding author.Charter Communication, United States of AmericaCharter Communication, United States of AmericaDepartment of Biostatistics and Informatics University of Colorado, United States of AmericaThis paper proposes a platform trial for conducting A/B tests with multiple arms and interim monitoring to investigate the impact of several factors on the expected sample size and probability of early stopping. We examined the performance of three stopping boundaries: O’Brien Fleming (OBF) stopping for either futility or difference (both), Pocock stopping for futility only, and fixed sample size design. We simulated twelve scenarios of different orders of arms based on various effect sizes, as well as considered 1 or 3 interim looks. The overall findings are summarizing in a flowchart to provide intuitive guidance for the design of the platform based on the simulation. We found that it is better to use OBF stopping for both if there is any effective variant and the trial is sufficiently powered to detect the expected effect size. If the study is underpowered to detect a difference, we recommend fixed sample size design to gather as much information as possible, however if the expected sample size is important to minimize, we recommend using Pocock boundaries with futility monitoring. Our results aimed at helping high-tech companies conduct their own studies without requiring extensive knowledge of clinical trial design and statistical methodology.http://www.sciencedirect.com/science/article/pii/S2451865424001352Interim monitoringA/B testingError spending functionStopping rule |
| spellingShingle | Wenru Zhou Miranda Kroehl Maxene Meier Alexander Kaizer An automated platform trial framework for A/B testing Contemporary Clinical Trials Communications Interim monitoring A/B testing Error spending function Stopping rule |
| title | An automated platform trial framework for A/B testing |
| title_full | An automated platform trial framework for A/B testing |
| title_fullStr | An automated platform trial framework for A/B testing |
| title_full_unstemmed | An automated platform trial framework for A/B testing |
| title_short | An automated platform trial framework for A/B testing |
| title_sort | automated platform trial framework for a b testing |
| topic | Interim monitoring A/B testing Error spending function Stopping rule |
| url | http://www.sciencedirect.com/science/article/pii/S2451865424001352 |
| work_keys_str_mv | AT wenruzhou anautomatedplatformtrialframeworkforabtesting AT mirandakroehl anautomatedplatformtrialframeworkforabtesting AT maxenemeier anautomatedplatformtrialframeworkforabtesting AT alexanderkaizer anautomatedplatformtrialframeworkforabtesting AT wenruzhou automatedplatformtrialframeworkforabtesting AT mirandakroehl automatedplatformtrialframeworkforabtesting AT maxenemeier automatedplatformtrialframeworkforabtesting AT alexanderkaizer automatedplatformtrialframeworkforabtesting |