post

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Customer Research

Hardware

Digital Transformation

Our Privacy Policy is changing on April 5, 2022 - for more information and a description of the updates, <a href="https://mtpdevwpe.wpengine.com/privacy/" style="color:#FFFFFF;text-decoration:underline;">see the policy here.</a>

Switching back the focus from ‘Build only’ to ‘Build-measure-learn’

Product career special part 2: Making a change

Build what matters with vision-led product management

Lessons in fearless leadership

Building a product organisation from scratch by Kevin Mansfield

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

The executive product leader persona

Test blog post

Andres is a freelance writer and digital marketer with a background in engineering and a keen interest in the latest technology advancements. He currently helps tech companies and startups tell their stories by turning their complex ideas into easy to understand material. You can reach out to him on LinkedIn

Andres Phillips

<p>In this ProductTank New York talk, <a href="https://www.linkedin.com/in/tomersharon/" target="_blank" rel="noopener noreferrer">Tomer Sharon</a>, Managing Director, Head of User Research &amp; Metrics at <a href="https://www.goldmansachs.com" target="_blank" rel="noopener noreferrer">Goldman Sachs</a>, introduces us to Key Experience Indicators (KEIs).</p>
<p>The key points of his talk include:</p>
<ul>
<li>Framing</li>
<li>Principles</li>
<li>The Google HEART framework</li>
<li>Actions to take</li>
</ul>
<p>Watch the video to see Tomer’s talk in full. Or read on for an overview of his key points.</p>
<h2>Framing</h2>
<p>KEIs are quantitative scores for specific, important, and actionable phenomena related to using products or services. They are a type of user research method used after a product has launched and can be useful in helping teams move from launching products to landing products. In landing products, teams can find out if the product was successful, if users liked it and how they can go about getting repeat users.</p>
<h2>Principles</h2>
<p>The principles of using KEIs are as follows:</p>
<ul>
<li>Specific vs overall. Measure very specific things vs the overall satisfaction or engagement of a product. Try to hone in on things related to the critical user journey, especially when starting out.</li>
<li>In context vs out of context. Focus on finding the right people at the right time and in the right place.</li>
<li>Core vs perimeter. Measure only your core experiences and not everything. In many cases, this would be three but for other companies, there could be a lot more.</li>
</ul>
<h2>Google HEART Framework</h2>
<p>The Google HEART framework focuses on the areas of happiness, engagement, adoption, retention and task success. Tomer uses this framework to provide some key metrics that product people should focus on and some mistakes that are made in measuring them.</p>
<h3>Happiness</h3>
<p><span style="font-style: italic;">Metrics to measure: </span>the average satisfaction score and actual net promoter score. Learn how satisfied customers are with a product or if they recommended the product to a person in the last week, as opposed to at all like the NPS metric typically asks.</p>
<p><span style="font-style: italic;">Some of the mistakes made:</span> are not being specific, being out of context or overcomplicating things. Happiness cannot be explained using data, it only gives some guidelines. To get further clarification, product people need to go out and talk to customers.</p>
<h3>Engagement</h3>
<p><span style="font-style: italic;">Metrics to measure</span>: percentage of people to use the product at least once in a particular time frame, actions per user and time between action. Product people need to choose the right event to determine how engaged people are with a product.</p>
<p><span style="font-style: italic;">Some mistakes made are: </span>measuring everything and not being specific enough or assuming that engagement is different for B2B compared to B2C customers.</p>
<h3>Adoption</h3>
<p><span style="font-style: italic;">Metrics to measure:</span> adoption rate and time to first customer. Adoption can tell you how fast people are adopting a product or feature. It can provide insight into marketing efforts.</p>
<p><span style="font-style: italic;">Mistakes typically made:</span> are only measuring an overall adoption rate or confusing it with engagement.</p>
<h3>Retention</h3>
<p><span style="font-style: italic;">Metrics to measure</span>: retention rate, churn rate. Understand how many users are being retained.</p>
<p><span style="font-style: italic;">Mistakes made include:</span> choosing the wrong time frame to measure or the wrong event.</p>
<h3>Task Success</h3>
<p><span style="font-style: italic;">Metrics to measure</span>: success rate, time on task, lostness. Anything within a product that has a clear start and endpoint you can measure task success.</p>
<p><span style="font-style: italic;">Mistakes made include: </span>ignoring this measurement completely or asking customers about the success rate rather than measuring it.</p>
<h2>Actions</h2>
<p>Actions that product teams can take are to set goals for KEIs and use them to help evaluate planned changes. The key takeaway from this talk is that product managers need to measure experience to truly understand how a product is doing.</p>
<div class="g g-21"><div class="g-single a-93"><a class="gofollow" data-track="OTMsMjEsMSw2MA==" href="https://mtpdevwpe.wpengine.com/product-management-training/public-workshops/communication-alignment-workshop/"><img decoding="async" src="https://mtpdevwpe.wpengine.com/wp-content/uploads/2022/04/Training-CA-layer-of-strategy-FOOTER-scaled.jpg" /></a></div></div>

core/freeform

Product Management Metrics and KPIs

Metrics don’t have all the answers: a human-centered approach to business strategy by Samihah Azim

Your ticket to product analytics expertise by Ross Walker and Matt Smith

MTP panel: How to define the metrics that matter

Bets, Boards, Missions and North Stars by John Cutler

How to Connect Your Product Work to Your Top Business Metrics, By Pratima Arora

Key Experience Indicators for Product Managers by Tomer Sharon

The only product metric that matters by Josh Elman

ProductTank Tokyo

ProductTank Belgrade

ProductTank Madrid

ProductTank Leeds

ProductTank Zürich

ProductTank Perth

ProductTank Chicago

ProductTank Auckland

ProductTank Richmond

ProductTank Charlotte

ProductTank Reston

ProductTank Washington

ProductTank Sofia

ProductTank Raleigh

ProductTank Ljubljana

ProductTank Kraków

ProductTank Athens

ProductTank Barcelona

ProductTank Dublin

ProductTank Montréal

ProductTank Bogotá

ProductTank Seattle

ProductTank Madison

ProductTank Singapore

ProductTank Portland

ProductTank Budapest

ProductTank Valencia

ProductTank Porto

ProductTank Los Angeles

ProductTank Austin

ProductTank Norwich

ProductTank Des Moines

ProductTank San Francisco

ProductTank Helsinki

ProductTank Edinburgh

ProductTank Reading

ProductTank Brighton

ProductTank Bucharest

ProductTank Bielefeld

ProductTank Malmö

ProductTank Canberra

ProductTank Paris

ProductTank Manchester

ProductTank Krakow

ProductTank Heidelberg

ProductTank NYC

Product Leadership Course

ProductTank Recife

ProductTank London

ProductTank Birmingham

Member AMA: Unpacking User Research with Laura Klein

ProductTank Copenhagen

Trainer-Led Workshops

ProductTank Tauranga

World Product Day

Member AMA with Ken Norton

#mtpcon London+EMEA 2022 Workshops

#mtpcon London+EMEA 2022 Conference