A/B Testing in R with Elea McDonnell Feit

R-Ladies Philly was delighted to host Elea McDonnell Feit, who led a virtual workshop on the topic of A/B Testing in R.

Our presenter

Elea McDonnell Feit is an Assistant Professor of Marketing at the LeBow College of Business at Drexel University. She develops data analysis methods to inform marketing decisions, including designing new products as well as planning advertising campaigns.

A/B testing

A/B testing (also termed split testing or bucket testing) is a type of experiment conducted to compare two variants of a marketing or promotional effort (e.g. webpage design, email, or app interface). These variants are assigned to users at random and statistical analyses are used to determine which variant performs better for a given outcome (e.g. purchases or clicks).

The virtual workshop

The virtual workshop covered parts 1-3 of Elea’s Advanced A/B Testing Workshop.

  1. Introduction
  2. A/B Test Basics
  3. When your sample size is big (Note: we stopped at slide 27)

R-Ladies Philly recorded the meetup. The video is available on our newly launched R-Ladies Philly youtube channel!

A few takeaways

A/B test plan questions

If you can answer these questions, you have an A/B test plan! (A/B Test Basics, slide 34)

What is/are the…

  1. Business question?
  2. Test setting (lab v. field)?
  3. Unit of analysis (visit, customer, store)?
  4. Treatments?
  5. Response variable(s)?
  6. Selection of units?
  7. Assignment to treatments?
  8. Sample size?

Randomization

Elea was careful to highlight the importance of randomization, with the mantra “1, 2, 3. Repeat with me. Randomization will set you free” and this reminder (A/B Test Basics, slide 12):

What is the first question you should ask about an A/B test?
Did the treatment affect the response?
Was the randomization done correctly?

Communicating results

As an analyst in a business or marketing context, don’t be afraid to provide recommendations to the client! If you are more direct about what the data shows, your recommendation can help the client understand your analysis better.

Resources

There are some great resources for learning more about marketing research using R, such as:

Thank yous

  • Many thanks to our fantastic presenter, Elea McDonnell Feit, and to R-Ladies Global for making the virtual event possible via Zoom.

This post was authored by Amy Goodwin Davies. For more information contact