A/B Testing in R with Elea McDonnell Feit
R-Ladies Philly was delighted to host Elea McDonnell Feit, who led a virtual workshop on the topic of A/B Testing in R.
Our presenter
Elea McDonnell Feit is an Assistant Professor of Marketing at the LeBow College of Business at Drexel University. She develops data analysis methods to inform marketing decisions, including designing new products as well as planning advertising campaigns.
A/B testing
A/B testing (also termed split testing or bucket testing) is a type of experiment conducted to compare two variants of a marketing or promotional effort (e.g. webpage design, email, or app interface). These variants are assigned to users at random and statistical analyses are used to determine which variant performs better for a given outcome (e.g. purchases or clicks).
The virtual workshop
The virtual workshop covered parts 1-3 of Elea’s Advanced A/B Testing Workshop.
- Slides are available at https://eleafeit.github.io/ab_test/
- RMarkdown code for the slides is available at https://github.com/eleafeit/ab_test/
- Introduction
- A/B Test Basics
- When your sample size is big (Note: we stopped at slide 27)
R-Ladies Philly recorded the meetup. The video is available on our newly launched R-Ladies Philly youtube channel!
A few takeaways
A/B test plan questions
If you can answer these questions, you have an A/B test plan! (A/B Test Basics, slide 34)
What is/are the…
- Business question?
- Test setting (lab v. field)?
- Unit of analysis (visit, customer, store)?
- Treatments?
- Response variable(s)?
- Selection of units?
- Assignment to treatments?
- Sample size?
Randomization
Elea was careful to highlight the importance of randomization, with the mantra “1, 2, 3. Repeat with me. Randomization will set you free” and this reminder (A/B Test Basics, slide 12):
What is the first question you should ask about an A/B test?
Did the treatment affect the response?
Was the randomization done correctly?
Communicating results
As an analyst in a business or marketing context, don’t be afraid to provide recommendations to the client! If you are more direct about what the data shows, your recommendation can help the client understand your analysis better.
Resources
There are some great resources for learning more about marketing research using R, such as:
Thank yous
- Many thanks to our fantastic presenter, Elea McDonnell Feit, and to R-Ladies Global for making the virtual event possible via Zoom.
We are delighted to have @eleafeit share a workshop on A/B testing in R!
— R-Ladies Philly (@RLadiesPhilly) May 14, 2020
Thank you to @RLadiesGlobal for sponsoring the zoom! pic.twitter.com/YTGC0DoWx8
This post was authored by Amy Goodwin Davies. For more information contact philly@rladies.org
