Ever wonder how Netflix serves a great streaming experience with high-quality video and minimal playback interruptions? Or how we are confident that major UI redesigns across thousands of devices are well received?
It’s because new ideas are constantly being explored across all aspects of the Netflix product, including our UI experience, recommendation/search algorithms, sign-up flows, messaging, adaptive streaming algorithms, etc. To ensure that these ideas deliver experiences our subscribers love, we diligently test and measure the impact of all these proposed enhancements using ABlaze, Netflix’s Experimentation Platform.
Given the size and complexity of our datasets, this analysis is no trivial task. Our platform must process data for A/B test analysis across 100M+ subscribers and over 100M+ hours of video streamed every day. Our current infrastructure has served us well, but now that Netflix is in almost every country around the world, it’s time to evolve and scale.
Consequently, we’re re-architecting our data pipelines to provide our users with more flexible self-serve experimentation analysis capabilities. This means allowing them to specify their own data sources, metrics, dimensions, and statistical tests, and finally choose from a variety of data visualization options in order to construct reusable reports for each experimentation area.
Since almost every product decision at Netflix depends on A/B test analysis, your efforts on this team will impact improvements throughout our product and therefore the experience of the millions of our users across the globe.