Bayesian A/B Testing - Marc Garcia

[Fatha27]

Video: https://www.youtube.com/watch?v=UxEFAkDlkiA

0:00 Introduction
1:23 What is an A/B Test?
5:41 Choosing the performance metric for our test
7:20 Empirical conversion rate vs Real conversion rate
9:34 How conversion rate varies with sample size
11:25 The problem of uncertainty
12:18 3 schools of Statistics
12:35 Classic school
13:52 Frequentist school
14:44 Bayesian school
15:29 Example outcomes
16:15 Frequentist A/B Testing
18:28 Determining if the conversion rate is statistically significant
21:40 Formulating Null hyothesis
23:18 Why do we transform our problem to Null hypothesis?
25:12 The problems with the Frequentist approach
27:32 Simulation of A/B Test
28:55 Bayesian A/B Testing
32:46 Dealing with Uncertainty through Beta distribution
33:13 Sampling
33:52 Value at risk
34:10 Credible interval
34:31 Exploration and Exploitation
35:07 Book recommendation- "Bayesian methods for Hackers"