post

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Customer Research

Hardware

Digital Transformation

Our Privacy Policy is changing on April 5, 2022 - for more information and a description of the updates, <a href="https://mtpdevwpe.wpengine.com/privacy/" style="color:#FFFFFF;text-decoration:underline;">see the policy here.</a>

Switching back the focus from ‘Build only’ to ‘Build-measure-learn’

Product career special part 2: Making a change

Build what matters with vision-led product management

Lessons in fearless leadership

Building a product organisation from scratch by Kevin Mansfield

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

The executive product leader persona

Test blog post

Hamidreza is an associate product manager at Basalam.com, a C2C social marketplace where people chat, buy and sell goods. He has 2 years of experience in C2C and B2B2C startups.

Hamidreza Mojtabavi

<p>Product Managers are bombarded with feature requests and ideas every day. Even in companies with profound product knowledge, they’re consistently asked to test those ideas (sometimes carelessly called designs) in their daily routine. Here is why you shouldn’t do so as a product manager.</p>
<h2>Preface</h2>
<p>The following story is based on true events, only the names have been ignored.</p>
<p>You get to work early in the morning to have an unplanned meeting with a key stakeholder. He texted you last night to talk about <span style="font-style: italic;">a revolutionary idea </span>he found during his experience with another product. After half an hour or so, he finishes his sentences waiting for you to nod but you are puzzled and unsure what to say as you can’t relate the idea to any of your customers&#8217; problems. For the next five minutes, you try to explain the risks involved but he interrupts you: “Undoubtedly, it would move the needle. At least, it is worth testing. Okay? Build an MVP or something.”</p>
<p>In the following days, you discuss the idea with your teammates, turn the wireframes into a prototype with the product designer’s help and prepare some documents (or a few user stories if you follow Scrum practices) to communicate the exact requirements to developers. Traditionally, you choose the A/B test approach to have statistical significance on your results. Hence, a handful of new backend endpoints and a bunch of client-side codes must be developed. The team estimation is 5-7 workdays. After exactly two weeks from your initial meeting with the stakeholder, Google Optimize results appear on your screen, stating a 3% change in conversion rate. Everything seems fine except that it’s a down lift instead of an uplift! You wait for another two weeks (making a total of 28 days) according to Google Optimize instructions, but the situation is worsening.</p>
<p>And here begins the nightmare.</p>
<p>The stakeholder shows up saying “It would have worked if you just ….” and proposes an incremental development. The product designer starts iterating on her design to make another prototype. And the engineers ask you what to do now.</p>
<h2>What happened?</h2>
<p>The product manager in the story above made several (unfortunately common) mistakes which resulted in a major hassle. We discuss them one by one, so hopefully, at the end of this article, you won’t get into similar trouble.</p>
<p>We can group the mistakes into the following categories:</p>
<h3>Falling in love with the solution, instead of the problem</h3>
<p>Joshua Seiden in “Outcomes Over Outputs”, states that “features can be finished and delivered and worked perfectly but still not deliver any value”. When we fall in love with our solutions (a lot of times indeed), we selfishly forget the customer and focus on our desires instead. Therefore, it’s not a surprise that they don’t care and do the same to you.</p>
<h3>Mistaking the MVP concept</h3>
<p>Eric Ries introduced MVP as “a version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort”. Marty Cagan, the product guru, has also defined the MVP as a prototype used in the discovery phase to learn from your customers. In the story above, the product manager wasted a lot of resources and time by mistaking MVP for a messy product (in need of serious refactoring) and didn’t learn anything helpful after a month. Remember the dropbox fake video MVP? It was not a product at all and yet did its sole mission of validating desirability risk perfectly.</p>
<h3>Testing Ideas instead of assumptions</h3>
<p>Teresa Torres has devoted a complete chapter of her wonderful book, Continuous Discovery Habits, to this pitfall so I would keep this short. When you test an idea, you are testing a set of underlying assumptions simultaneously. Take video recorded product reviews on Amazon for example. Let’s say the outcome was to increase the amount of data gathered by each review. If you test the whole idea right away, you won’t know why it did or did not work. Was it because the users couldn’t upload videos from their devices? Did they find it weird? Was their product digital or physical? And the list goes on. When you test assumptions instead, you learn from every move.</p>
<h3>Making a yes/no choice</h3>
<p>Instead of debating on whether the proposed solution is right or wrong, the product manager could ask the stakeholder what problem he thinks it solves and what are the other possible solutions. By turning a yes/no choice into a compare and contrast one, iterating on solutions based on invalidating assumptions becomes possible.</p>
<h2>Conclusion</h2>
<p>Next time you face a stakeholder trying to sell you an idea, remember the story of a product team stuck in a negative loop of incremental development and A/B testing hoping for a one-time success that can’t be learned from or copied elsewhere. I’ll cover more stuff on stakeholder management and other things in future articles.</p>
<p>Good luck building great products!</p>
<h2>Bonus</h2>
<p>As we reach the end of this article, I want to show you one of the most underrated types of MVP to test your next assumptions, and how to use it. ‘Piecemeal MVP’ is when you embed or utilize another product or service in your product in a new way. Why do I think it’s underrated? If you can put an ‘iframe’ on your website, you can easily tackle usability (and desirability) risks upfront with virtually no code.</p>
<p>Let’s say you’ve made a prototype in Figma but you’re uncertain of its usability. There are tools to test it with random users but your product may have a special customer segment. Besides, your users might behave differently in experiment environments, from their unconscious daily usage. With piecemeal MVP, you can export the Figma prototype as an iframe and embed it in your customer journey and watch the users’ interactions using tools like MS Clarity or Hotjar.</p>
<p>Be careful when you do so, as you just need a small percent of visitors in the test group (I use the Cochran formula for a rough estimate)</p>
<p>Another usage is when for instance you want to test whether people would like to provide helpful audio responses alongside their texts in product reviews or not (desirability risk). You may content yourself with a fake button but it won’t let you check helpfulness. Alternatively, you can embed a customized audio survey iframe into your website. There can be countless uses of this MVP that you can design with your creativity and I’d be happy to discuss this further in future articles if you are interested.</p>
<h2>What to read next&#8230;</h2>
<ul>
<li><a href="https://mtpdevwpe.wpengine.com/the-complete-guide-to-building-mvps/" target="_blank" rel="noopener">The Complete Guide to Building MVPs</a></li>
<li><a href="https://mtpdevwpe.wpengine.com/testing-business-ideas-by-david-bland/" target="_blank" rel="noopener">Testing Business Ideas, by David Bland</a></li>
<li><a href="https://mtpdevwpe.wpengine.com/testing-mindset-find-product-market-fit/" target="_blank" rel="noopener">The Testing Mindset – How to Find Product/Market fit</a></li>
</ul>
<div class="g g-21"><div class="g-single a-95"><a class="gofollow" data-track="OTUsMjEsMSw2MA==" href="https://mtpdevwpe.wpengine.com/product-management-training/public-workshops/product-metrics-workshop/"><img decoding="async" src="https://mtpdevwpe.wpengine.com/wp-content/uploads/2022/04/Training-Metrics-what-should-a-PM-know_-FOOTER-scaled.jpg" /></a></div></div>

core/freeform

Business Strategy

How to navigate tricky work relationships

Organizing around complex products at the intersection of physical and digital by Matthew Cornford

Scaling Empowered Teams: Advice From Nilan Peiris

Bottom-Up Product Strategy – The State of Product, by Susana Lopes

Testing Business Ideas, by David Bland

How to Connect Your Product Work to Your Top Business Metrics, By Pratima Arora

Atomic Product Strategy, By Rob Hayes

Setting Up an Internal Product Function by Prag Patel

Socially Preferable by Nathan Kinch

Driving Growth vs Building Core Value by Roan Lavery

Why it’s never a good idea to test your idea

ProductTank Richmond

ProductTank Charlotte

ProductTank Reston

ProductTank Washington

ProductTank Sofia

ProductTank Raleigh

ProductTank Ljubljana

ProductTank Kraków

ProductTank Athens

ProductTank Barcelona

ProductTank Dublin

ProductTank Montréal

ProductTank Seattle

ProductTank Madison

ProductTank Singapore

ProductTank Portland

ProductTank Budapest

ProductTank Valencia

ProductTank Porto

ProductTank Los Angeles

ProductTank Austin

ProductTank Norwich

ProductTank Des Moines

ProductTank San Francisco

ProductTank Helsinki

ProductTank Edinburgh

ProductTank Reading

ProductTank Brighton

ProductTank Montevideo

ProductTank Vienna

ProductTank Buenos Aires

ProductTank Bielefeld

ProductTank Charleston

ProductTank Bogotá

ProductTank Newcastle Upon Tyne

ProductTank Tokyo

ProductTank Columbus

ProductTank Dallas

ProductTank Köln

ProductTank Malmö

ProductTank Canberra

ProductTank Paris

ProductTank Manchester

ProductTank Madrid

ProductTank Zürich

ProductTank Krakow

ProductTank Heidelberg

ProductTank NYC

Product Leadership Course

ProductTank Recife

ProductTank London

ProductTank Birmingham

Member AMA: Unpacking User Research with Laura Klein

ProductTank Copenhagen

Trainer-Led Workshops

ProductTank Tauranga

World Product Day

Member AMA with Ken Norton

#mtpcon London+EMEA 2022 Workshops

#mtpcon London+EMEA 2022 Conference

Why it’s never a good idea to test your idea

Hamidreza Mojtabavi

Comments

Read next