Last Thursday night I was fortunate enough to see Marty Cagan of the Silicon Valley Product Group speak at an event just around the corner from Atlassian in SOMA. I thoroughly enjoyed reading his book – Inspired: How To Create Products Customers Love
On Thursday Marty spoke about the importance of building a strong product culture within a company. He explains that a strong product culture is one where the company is constantly developing and delivering new revenue streams.
Marty went on to describe a number of areas that strong product cultures excel at, I thought of these as similar to design patterns – ie, formalised best practices that you should implement in your organisation. That of course means there are also anti-patterns. Lets start with one of those:
the single biggest reason for product failure is gathering requirements. don’t confuse what the customer asks for with what their problem is.
A product managers role, and the role of the whole team, is to invent a solution on behalf of the customers. A successful solution is one where you have a number of happy reference customers for the MVP.
Now we will turn our attention to some patterns of strong product cultures.
Define Success Upfront
How does a product manager ensure that they are going to have happy reference customers for the MVP? Define success upfront. Build a hypothesis that you want to test and specify the success criteria prior to writing any code. Let’s take a look at a simple example of a login page for a consumer SaaS:
We have a user story:
As a user I would like to log in with my Facebook account so that I only have to remember one username and password
This user story will also have acceptance criteria:
- display the Facebook logo on the login page,
- return to the login page if incorrect username & password are supplied,
- provide error message
- etc
The acceptance criteria is there to ensure the team is clear about what they are delivering to the customer.
Now, let’s add success criteria to that user story:
- ‘new user signups increase by 30% due to the simplified signup/login process’
- ‘requests to the forgot my password page cease’
- etc
Thus our user story has acceptance criteria and success criteria.
Instead of rolling out this change to all customers at once we will extend our definition of done to include meeting the success criteria. The team will then A/B test the user story with our customer base to ensure that the success criteria is met by the cohort that sees the new login page.
Assuming the cohort that sees the new login page leads to 30% increase in conversion and no password requests we would then consider the user story ‘Done’ as it meets both the acceptance and success criteria. We would roll the user story out to all customers. Of course, if our success criteria was not met then we will reject that story – perhaps leading to a subsequent hypothesis in another story.
To visualise that these user stories have not met the definition of done yet you may have a board like the following, with a column for ‘In A/B’:
This approach to defining the success criteria upfront is a great method to ensuring that features which are released to customers are actually worthwhile, actually used, that they provide value to the customer.
Product Discovery
Product Discovery is an approach to quickly testing and validating our ideas. Every product manager I have ever spoken with has more ideas that they want to build and provide to the customer than they could ever possibly deliver. While this is intertwined with Defining Success Upfront (above) it is really in the pre-user-story stage such as when we are aiming for an MVP. We don’t want to write production code for all of the ideas that we want to test as that is costly and leads to a lot of waste. Instead, we can do one of the following:
- User-clickable or paper prototypes
- Live data prototypes
I won’t got into detail on the paper prototypes here as I’ve linked to a great intro video above. You can also read a follow up post on usability testing with paper prototypes which gives some examples.
The live data prototype is similar to the A/B test that we put the user story through above. The key difference is that we are not writing production quality code – no unit or functional tests, for instance. There are two questions that the live data prototype will help us answer:
- Could the customer use the functionality?
- Would they use the functionality? If not, what would it take to get them to use it?
It is key that you are asking the customer whether they would use the functionality – for instance, would they pay for it? If not, why not?
Product Managers have a tonne of ideas they want to try out. This Product Discovery process is essential to finding out what resonates with your customers, what they can use and what they will pay for.
Product Discovery Team
Finding new product opportunities takes time and energy. Having a Product Discovery Team dedicated to this task will enable existing product teams to focus on optimising the value to their customers while still allowing the company to approach new ideas for testing and validation. Of course, this depends on the size of the company – perhaps you’ve got time to cover these alongside existing products or perhaps you’re a new company trying to find the MVP for your product.
In either case the Product Discovery Team will require three roles:
- Product Manager
- Lead Designer
- Lead Developer
Together these three come up with a solution to the customers problem. It is rare to find two of these roles in one person – hard too as they are probably out doing their own thing already.
One takeaway for me, from Marty’s talk, was that you should lead with UX as it is likely more important than engineering. I’m starting to see the constraint on UX, there are not enough good people and they are the bottleneck for many teams.
Random Tidbits
- Bring at least three customers into the office each week, watch them use the product and use this as an opportunity for product discovery. It can’t help to bring customers closer to the team either.
- You want a Founder / CEO that demands excellence. Better that than someone who doesn’t care about the product or customer and is only interested in the bottom line. Demanding pixel perfection is excellence – a real eye opener for me.
- Really, there is no reason not to test something these days – you can run a test over a few days and have the metrics handy when you have a discussion with the Founder / CEO.
Recommended Reading
- Lean UX: Applying Lean Principles to Improve User Experience (available for pre-order)
– Jeff Gothelf
- MVP vs Minimum Product – SVPG
- Inspired: How To Create Products Customers Love – Marty Cagan
- The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity
– Alan Cooper
- Social Contract for Product Managers – SVPG
Conclusion
How can we come up with an experience worth building with the evidence to validate this?
At the end of the day the Product Managers role is twofold – customer discovery and product discovery. A product managers job is not to be right, it is to find out how to get to right, fast. Getting a new product into the hands of customers isn’t simply a matter of building something. It is much more than that, as we’ve explored above.
Once the product has launched the product manager will switch to an optimisation role – where the process is more iterative and defining success for new user stories is vital.
“think in leaps, iterate in steps” – Google
Build and deploy incrementally and iteratively, but make sure you have a compelling product vision. You don’t want to go faster and get nowhere – so make sure you have a vision.
