Data-Driven Design Is Killing Our Instincts
Valuing data over design instinct puts metrics over users
Listen to an audio version of this story
Design is no longer subjective. Data rules our world now.
We’re told all design decisions must be validated by user feedback or business success metrics. Analytics are measuring the design effectiveness of every tweak and change we make. If it can’t be proven to work in a prototype, A/B test, or MVP, it’s not worth trying at all.
In this cutthroat world of data-driven design, we’re starting to lose sight of something we once cherished: the designer’s instinct. “Trusting your gut” now means “lazy, entitled designer.” When we can ask users what they want directly, there’s no room for instinct and guesswork.
Or is there?
Overreliance on data to drive design decisions can be just as harmful as ignoring it. Data only tells one kind of story. But your project goals are often more complex than that. Goals can’t always be objectively measured.
If the false god of data overshadows other methods of design decision making, we risk optimizing for easily measurable goals at the expense of totally flunking the more subjective experience.
I’d rather satisfy all the goals. Even if means sometimes ignoring the data.
What is data-driven design?
Simply put, data-driven design means making design decisions based on data you collect about how users interact with your product. According to InVision:
Data-driven design is about using information gleaned from both quantitative and qualitative sources to inform how you make decisions for a set of users.
Some common tools used to collect data include user surveys, A/B testing, site usage and analytics, consumer research, support logs, and discovery calls. By crafting your products in a way that cater to your users’ goals, preferences, and behaviors, it makes your products far more engaging — and successful.
While most data is quantitative and very objective, you can also collect qualitative data about your users’ behaviour, feelings, and personal impressions.
A designer’s instinct
Back in the days of Mad Men, a designer’s gut instinct was glorified because it was difficult to measure the success of a design in progress. You often had to wait until it shipped to know if your idea was any good. Designers justified their value through their innate talent for creative ideas and artistic execution. Those whose instincts reliably produced success became rock stars.
In today’s data-driven world, that instinct is less necessary and holds less power. But make no mistake, there’s still a place for it.
Design instinct is a lot more than innate creative ability and cultural guesswork. It’s your wealth of experience. It’s familiarity with industry standards and best practices. You develop that instinct from trial and error — learning from mistakes.
Instinct is recognizing pitfalls before they manifest into problems, recognizing winning solutions without having to explore and test endless options. It’s seeing balance, observing inconsistencies, and honing your design eye. It’s having good aesthetic taste, but knowing how to adapt your style on a whim.
Design instinct is the sum of all the tools you need to make great design decisions in the absence of meaningful data.
Clicks and conversions aren’t your only goals
Not everything that can be counted counts. Not everything that counts can be counted.
Data is good at measuring things that are easy to measure. Some goals are less tangible, but that doesn’t make them less important.
While you’re chasing a 2% increase in conversion rate you may be suffering a 10% decrease in brand trustworthiness. You’ve optimized for something that’s objectively measured, at the cost of goals that aren’t so easily codified.
This point is perfectly illustrated by a story by Braden Kowitz, a design partner at Google Ventures (via Wired):
One of my first projects at Google was to design the “Google Checkout” button. With each wave of design feedback I was asked to make the button bolder, larger, more eye catching, and even “clicky” (whatever that means). The proposed design slowly became more garish and eventually, downright ugly.
To make a point, a colleague of mine stepped in with an unexpected move: He designed the most attention-grabbing button he could possibly muster: flames shooting out the side, a massive chiseled 3-D bevel, an all-caps label (“FREE iPOD”) with a minuscule “Checkout for a chance to win”.
This move reset the entire conversation. It became clear to the team in that moment that we cared about more than just clicks. We had other goals for this design: It needed to set expectations about what happens next, it needed to communicate quality, and we wanted it to build familiarity and trust in our brand.
We could have easily measured how many customers clicked one button versus another, and used that data to pick an optimal button. But that approach would have ignored the big picture and other important goals.
It’s easy to make data-driven design decisions, but relying on data alone ignores that some goals are difficult to measure. Data is very useful for incremental, tactical changes, but only if it’s checked and balanced by our instincts and common sense.
When data-driven design gets fugly
Ever used Booking.com?
Search for a hotel and you’ll see every listing plastered with a handful of conversion triggers and manufactured urgency/scarcity indicators. Amongst all that crap, it’s difficult to find the real info you’re looking for. It’s a terrible user experience for me. I’m sure many others feel the same.
But they must have reliable data that says it works. Conversion rates must go up with each chintzy trigger they cram in. Data says: add more urgency messages, add more upsells, more, more, more. User experience says: less, less, less, just show me what I’m looking for.
What’s going on here?
Data has become an authoritarian who has fired the other advisors who may have tempered his ill will. A designer’s instinct would ask, “Do people actually enjoy using this?” or “How do these tactics reflect on our reputation and brand?”
Booking.com’s brand is cheap deals, so they’re not worried about cheap tactics. If those tacky labels stoke enough FOMO to get a few more bookings, they’ve won. It doesn’t critically damage other goals if they’re perceived as a little gaudy in the process.
But not every business has the luxury of caring only about clicks and conversions. You may need to convey quality and trust. Or exclusivity and class. Does cramming in data-driven conversion triggers serve those goals too? Or would building a more focused and delightful user experience better speak to your user’s needs?
Data-driven sameness
Digital interface design is going through a bland period of sameness. I see it in my own work, and I worry it’s becoming hard to escape from.
You could blame Apple and Google for publishing good design systems, and then everyone else trying to look the same. You could blame WordPress for the proliferation of content-agnostic templates — pulling apart the age-old marriage of content and designer. Or you could blame platforms like Dribbble that amplify trends and superficial eye-candy.
I’d argue that data-driven design also plays a role in why all websites look the same.
We’re all scared to experiment and reinvent the wheel, because data’s already proven that the wheels we’ve got work well enough. When our Agile processes are geared toward efficiency, it’s too costly to prototype and test innovative solutions. So we blindly churn out the same tried and true solutions over and over again.
Design “process” has replaced instinct as the new skill to fetishize. Some say that everyone is a designer if they can only follow the same processes we do. While that’s not true, it still leads to design decisions being made without the temperance of a professional designer’s instincts and experience. It creates more generic-looking interfaces that may perform well in numbers but fall short of appealing to our senses.
We’re all scared to experiment and reinvent the wheel, because data’s already proven that the wheels we’ve got work well enough.
Data is only as good as the questions you ask
What makes data so dangerous is that your input grossly colours your output. If you ask the wrong questions at the wrong time, or to the wrong people, you draw bad conclusions.
First adapters and eager user-testers don’t necessarily behave the same as your average user, so even when you are asking the right questions you can get tainted data.
The most empathetic designers — who are convinced they see the product the same way as their users — don’t behave the same. They know the product too intimately. They can’t see it objectively anymore. They can’t become naive.
Beware of misleading data. It’s only one source of info, and it’s only ever as good as your collection methods. Rather than blindly following the conclusions of big data, back them up with other sources (or at least common sense) before charging ahead with your shiny validation in hand.
When to use data vs. when to use instinct
Deciding between two or three options? This is where data shines. Nothing is more decisive than an A/B test to compare potential solutions and see which one actually performs better. Make sure you’re measuring long-term value metrics and not just views and clicks.
Sweating product quality and aesthetics? Turn to your instinct. The overall feeling of quality is a collection of hundreds of micro-decisions, maintained consistency, and execution with accuracy. Each one of those decisions isn’t worth validating on its own. Your users aren’t design experts, so their feedback will be too subjective and variable. Trust your design senses when finessing the details.
Unsure about user behaviour? Use data rather than asking for opinions. When asked what they’ll do, customers will do what they think you want them to. Instead, trust what they actually do when they think nobody’s looking.
Building brand and reputation? Data can’t easily measure this. But we all know trustworthiness is as important as clicks (and sometimes they’re opposing goals). When building long-term reputation, trust your instinct to guide you to what’s appealing, even if it sometimes contradicts short-term data trends. You have to play the long game here.
Data needn’t be the enemy of instinct. They can work hand in hand to balance the design process. After all, instinct is built upon years of observing the world — it’s nothing but data that’s so deeply integrated that it no longer feels external. Trusting our instinct means obeying long-term trends. Pair it with short-term data and you’ve got the recipe for faultless design decision making.
Want to comment?
See this story on Medium
Hi, I'm Benek Lisefski. Since 2001 I've run my own independent design business. Join me as I unfold 20 years of freelance business knowledge: honest advice and practical tips to help you take your indie career from good to great.
MediumTop writer in Design, Business, Creativity, and Entrepreneurship.
