October 16, 2023

Avoid the most common pitfalls when running experiments in ecommerce



Event Date: Monday 10/16/2023 at 2:00pm PT / 5:00pm ET

Event Location: Zoom link will be shared with confirmed attendees

Registration: Click Here

Every ecommerce company with growth targets has the same goal: Make sure that customers stay with you longer and spend more than they do today. 

The question: How can we uncover the right strategy to maximize customer value? For most data-driven companies, the answer is experimentation, but experimentation is hard. 

This event will cover experimentation in ecommerce, and specifically address the four serious pitfalls we see repeated in the space: 

Pitfall #1 – Focusing on indirect effects. You don’t know whether more clicks, or even higher AOV reflect in the bottom line – for example when the treatment group showing higher AOV is more likely to churn earlier

Pitfall #2 – Which interventions to prioritize. You have some intuition, but every shot is costly, and multiple hypotheses testing and their adjustments are hard when running multiple experiments in parallel. 

Pitfall #3 – Experimenting over a long time-frame, against rare events (e.g. churn, next order, etc.). You have to hope that both your treatment and control group purchase again, in sufficient numbers, ideally over a short period of time. It is a waiting game, but also a power game – only a fraction of treatment and control group will engage in the observed behavior, e.g. next purchase, leaving your experiment potentially severely underpowered

Pitfall #4 – Making the business case. If multiple experiments happen within your org in parallel, measured against different KPIs, it is hard to understand prioritization for the business.


Recorded panel discussion with extensive Q/A:

  • 30 min panel discussion
  • 30 min Q&A
Join us on Monday October 16th, 2023 at 2:00pm PT / 5:00pm ET