The Joy of A/B Testing, Part II: Advanced Topics | by Samuel Flender | Aug, 2022
Cookies and privacy, interleaving experiments, clean dial-ups, and test metricsPhoto by NASA on UnsplashA/B testing is one of the most critical steps in Machine Learning production: we only want to roll out a new ML model if it can be proven to be better in production. In Part I of this series we covered how to set up an A/B experiment with a population split, how to interpret the test results with statistical significance, and what kind of errors are to be expected. In this Part II we’ll go deeper into some practical…