Do It In Production - Testing Where It Counts - Seth Eliot

Talk Description

Bing analyzes Petabytes of data per day. Facebook instruments everything. Amazon “unplugs” entire data centers on a regular basis. Why? To improve quality! And it’s all done in production.

Running test cases in a lab has value but also diminishing returns as you try to make your lab emulate the complexities of the production environment and your test cases simulate the unpredictable nature of real-world usage. Go where the data is – go to production – and use this data as a new signal for quality assessment alongside your pre-production testing.

I will show you some techniques for testing in production (TiP) ranging from synthetics that resemble your current test cases to big data techniques leveraging the diversity of real production scale, illustrated with examples from Microsoft, Netflix, Google, and more. And if you think the dangers of TiP make it not right for you, I will also cover the risks and their mitigations, so there won’t be any reason you can’t also do it in production.

What you’ll learn

By the end of this talk, you'll be able to: