Many years ago, I spent an entire year working on improvements to LinkedIn’s ad matching product without moving the needle on KPIs. As disheartening as that year was, it taught me a fundamental lesson about product development. Most of our ideas do not work.
Pareto Suboptimal: The 80% Rule
Take a step back and see how this idea plays out in the wider industry:
Pendo says that 80% of features in the average software product are rarely or never used.
At Google & Bing, 80%-90% of features have a negative or neutral impact on the metrics they were designed to improve.
At Slack 70% of features created by the monetization team do not yield positive results.
So no matter what company you work at, chances are 80% of your features are not going to hit the mark with your customers.
This creates an impact gap.
Development teams pour their heart and soul into solving and issue, but their impact falls short of what a customer wants or needs.
Three Approaches to Closing the Impact Gap
Given this insight, how should product engineering teams adapt.
First, don’t lose heart. As Jeff Bezos says,
“success can come through iteration: invent, launch, reinvent, relaunch, start over, rinse, repeat, again and again.”
You are looking for a new global maxima. Learn how to wander.
“In contrast, wandering in business is not efficient … but it’s also not random. It’s guided – by hunch, gut, intuition, curiosity, and powered by a deep conviction that the prize for customers is big enough that it’s worth being a little messy and tangential to find our way there. Wandering is an essential counterbalance to efficiency. You need to employ both. The outsized discoveries – the “non-linear” ones – are highly likely to require wandering.”
Second, start moving faster. If the key to success is iteration, then you should pack as many iterations in a month as possible. According to DORA, 48% of companies are releasing software multiple times a week. This is the minimum bar and it will keep on getting faster. Invest in your product release cadence.
Lastly, invest in experimentation. Underlying this 80/20 principle is the idea that product and engineering teams at Microsoft and Slack know the metrics a feature was designed to improve and whether it had the desired impact. Experimentation platforms power these insights.
As Bezos said in his 2014 shareholder letter,
“We have our own internal experimentation platform called ‘Weblab’ that we use to evaluate improvements to our websites and products. In 2013, we ran 1,976 Weblabs worldwide, up from 1,092 in 2012, and 546 in 2011.”
You can get started by identifying key product metrics, tracking impact, and experimenting with features as you develop them. Check out Understanding Experimentation Platforms for more tips!
Stay up to date
Don’t miss out! Subscribe to our digest to get the latest about feature flags, continuous delivery, experimentation, and more.
At Split, we “dogfood” our own product in so many ways. Our engineering and product teams are using Split nearly every day. It’s how we make Split better.
A/B testing is a powerful tool for learning about your users, understanding your features’ impact, and making informed business decisions. To ensure you make the best decisions and are extracting the most insights from your experiments, some experimental design guidelines are essential. These guidelines can be cumbersome or confusing at…
Feature flags provide so much for software organizations: they allow teams to separate code deployment from feature release, test in production, run experiments, and more. However, some rules apply to the feature flagging process that are easy for teams to overlook. I’ve gathered the best practices of feature flags from…