This post is also available in: English

This talk was held on 2013年 4月 24日 (水曜日) 19時00分

Advertising pioneer John Wanamaker famously said, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” At Indeed, our experience shows that more than half of product features are not only wasted but actually have a negative impact on user experience. Thanks to rigorous A/B testing we know precisely which half, so we can ensure that every feature we add delivers measurable value.

With 100 million monthly unique visitors and 3 billion monthly searches, our data provides empirical answers to (nearly) all of our product questions. In this talk, we will cover:

• a motivation for why we A/B test everything,
• the tools and systems we have developed to measure and analyze test results and the key metrics we focus on, and
• multiple case studies illustrating how years of testing has shaped Indeed’s design and product thinking.

We will include specific examples of how small, overlooked changes have impacted our metrics in significant and unexpected ways, and how implementing simple “no-brainer” updates on our site decreased key business metrics and caused us to re-evaluate our solutions.

This talk is for designers, product managers, and engineers from companies of any size who want to understand how data-driven design makes better products.