As I was looking around for resources and links for this post, I stumbled upon a couple of interesting things. Take for example, the curious case of the $300 million button in which the simple redesign of a form and one particular button caused a gargantuan jump in revenue. Then there's the famous 41 shades of blue story that reveals how heavily Google relies on data to influence its design decisions.
Yet the most interesting example I found came straight from the source. YouTube's blog features a post called "Look Inside a 1,024 Recipe Multivariate Experiment" about some changes it recently made to its pages and the process by which they went about doing it.
Using Google Website Optimzer to help them identify and test different versions of various icons, color combinations, and positioning, they were able to quickly test the conversion rate of each option, graph them against each other, and find out what the most optimal combination was:
"While we could have hypothesized which elements result in greater conversions (for example, the color red is more eye-catching), multivariate testing reveals and proves the combinatorial impact of different configurations. Running tests like this also help guide our design process: instead of relying on our own ideas and intuition, you have a big part in steering us in the right direction. In fact, we plan on incorporating many of these elements in future evolutions of our homepage."
Being able to do this kind of testing without huge budgets is a great, great thing, and Google Website Optimizer is a fantastic product that lets anyone test multiple combinations for any pages to optimize the rate of user-determined conversion. That's cool stuff, and it eliminates the need for lengthy discussions about this graphic or that graphic. The one problem I see is that it takes a relatively large amount of traffic to gather a large enough sample size to make truly data-driven decisions. For example, while it's possible for YouTube to gather enough impressions for each of the 1,024 variants to make a legitimate decision in a reasonable amount of time, we might not all have the same luxuries. Obviously, scale of testing should be equivalent to scale of traffic. But the bottom line is that ability to test in this fashion exists - so why not?
Luke Stevens, an author and speaker on the subject, agrees:
"On the web, you can measure what people do. You can measure what they click, how long they stay for, if they scroll, how many pages they view, if they ‘bounce’, if they return, and so on. You can ask if they were successful or not. For perhaps the first time in history we can accurately measure all interactions with a piece of design.
If it can be measured, it can be improved. And each of those improvements represents helping someone do something a little more successfully."
I encourage everyone to go and read the original post on Lukes's blog. It's full of great insight on this and I share a lot of the sentiments he expresses there. But this one excerpt is the nut of the story and where I'd like to leave this discussion for now.
Data and design have a symbiotic relationship. They work together to create meaningful and enjoyable experiences for people on the web. When we think about design in the context of an internet that allows and expects large-scale social interaction, multiple actions per session, and rich applications with a multitude of features and interfaces, it becomes quickly apparent that designer has no better friend than the data set.
To attempt to render all the differences in aesthetic tendency, design ethos, and user behavior into a cohesive design is an impossible task, and trying to do so will lead to frustration, lost productivity, and ultimately low conversions. But as Luke articulates so well, "In web design, people click or they don’t. They stay or they go. That’s good to know."
Indeed it is. Let's measure what matters, and may the best design win.
