“If we have data, let’s look at data. If all we have are opinions, let’s go with mine.”
Jim Barksdale, Former CEO of Netscape
The best part of my job is that I get to chat with leaders in Product and UX at a lot of great companies. So when I hear a common theme, I blog about it.
Lately, I have heard a lot of frustration around the often painful (and inefficient) process of feature prioritization. I have heard many complaints about a highly political process that is largely driven by anecdotes and debatable estimations.
“Our customer needs this feature to renew!” – Sales
“Our competitor just launched this feature and it is getting a lot of buzz!” – Marketing
“I have this great idea, it’s going to be huge!” – Sr. Exec
“This feature will have a bigger financial impact.” – Finance
When I hear this, I think of the great Jim Barksdale quote that I used to open this post.
We all have amazing customer demand analytics about the live product, yet the pre-build prioritization process is often void of customer demand data.
The good news is this is easy to fix.
I Already Know the Customers Want This! Let’s Just Build It
What customers say they will do, what customers say they did and what customers actually do are three different things.
This inconvenient truth is what makes the feature prioritization process so difficult. This fact about customers leads to three common problems:
- An over-reliance upon estimates of value
- Assuming that a customer’s description of the problem is truly what they mean. Often customers more clearly define their pain point only after seeing the designed solution. It is not uncommon to hear, “oh..that’s not what I meant..I actually meant..”
- Confusing validation of the Problem Space with validation of the Solution Space. Just because a customer validates her pain point, doesn’t mean that using your designed solution is compelling relative to alternatives.
Bring Zen to Your Prioritization Process:
The point of Lean is to reduce waste. While all of the stakeholders are well meaning and I am sure truly believe their arguments, the simple fact is that the source of their anecdotes hasn’t been validated. Until you validate the potential solution with customers, all you really have is a hypothesis about demand.
Here is how you bring Customer Demand Data into the prioritization process:
- Create prototypes of the top features in the backlog. The ones that clearly stick out among the group.
- Run Demand Validation Tests against each prototype
- Benchmark the test respondents’ answers to the Cost Question for all tests.
- Be sure to ask the same Cost Question for all of your tests
- If certain features under-perform, but are highly strategic, then feel free to iterate and test again. But if the response is consistent, then build it at your own risk.
- Prioritize based on actual solution demand from customers, making slight adjustments based on development effort (i.e. if the #3 most valued feature is easier to build, then feel free to crank that one out first and ship it).
The Key to This Exercise:
The key to the Demand Validation test is the Cost Question. That is, it has to cost the customer something important to say, “yes, I want that feature.” The customer has to make a trade-off between something of value to them and this feature. That is when they truly evaluate its significance. Without the Cost Question, a customer’s input is counterproductive.
We have stated before that there are only three things that mean something (in this context) to a customer:
- Their Time
- Their Reputation
- Their Money
If you want to understand this point deeper, read the post on Demand Validation Tests.
Common objection: But I Don’t Have Time to do This!
Setting up a Demand Validation Test is very quick. You can set one up using Validately’s template within 60 seconds. We even help you recruit test respondents. There is a time lag in getting data back, so you need to start the process at least a week before the next sprint to allow for time to get the data back and interpret it. That means no more just in time delivery of designs.
So is it worth it? Should you alter your process to allow for this type of testing before the sprint planning meeting? Well, consider these stats:
- A typical team spends 30% of their time on re-work. 30%!
- 50% of a typical product’s features are unused.
- Oh..and this ignores the opportunity cost of working on the highest impact features.
Still not convinced? Think about all of the time and brain power your team is wasting debating or jockeying for certain features. Moving to a data driven prioritization approach will free up the wasted time and energy in prioritization debates.
How to Get Started
I am not suggesting major organizational change. A good way to get started is to start two sprints from the one that you are in right now. Set up Demand Validation Tests in Validately (which takes a minute to do) for the top 5 features that you are considering for that sprint. If you want help, just email me. I am happy to help work with your team for free to set up the first test.