Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Product Management by Numbers:
Using Metrics to Optimize your Product
Dan Olsen
CEO & Founder, YourVersion
Dec 17, 2008   ...
Using Metrics to Optimize your Product
 Product Management’s Role
 Integrating metrics & optimization
 Using metrics to op...
My Background
Education
  BS, Electrical Engineering, Northwestern
  MS, Industrial Engineering, Virginia Tech
  MBA, Stan...
Product Management’s 
         Role



              Copyright © 2008 Olsen Solutions LLC
A Product Manager by any Other 
   Name Would Smell as Sweet
Product managers are sometimes called
  Product marketing man...
Product Management is
    Critical Link in Value Creation




Market            Product          Development 
• Current   ...
A Process View of Product Management
                                    “Inbound”
                                     Pr...
PM’s Job: A Successful Product
Know market and customer better than anyone else
Translate business objectives and customer...
Integrating Metrics and 
 Optimization into your 
    Product Process



                Copyright © 2008 Olsen Solutions ...
Adding Metrics and Optimization to 
             your Product Process
                                                    ...
Optimization through Iteration:
  Continuous Improvement
                   Measure
                  the metric


       ...
Optimizing
Business Value Creation 
     with Metrics



                Copyright © 2008 Olsen Solutions LLC
Approaching Business as an 
    Optimization Exercise


Given reality as it exists today,
optimize our business results
su...
Define the Equation of your Business
          “Peeling the Onion”
Profit = Revenue ‐ Cost

   Unique Visitors x  Ad Reven...
How to Track Your Metrics
Track each metric as daily time series
           Unique     Page       Ad      New User 
      ...
Sample Signup Page Yield Data
                                                Daily Signup Page Yield vs. Time
           ...
Using Metrics to Optimize the 
           Equation of your Business
  What are the metrics for your business?
  Where is c...
Case Study:
                Optimizing Viral Loop Metrics
   % of users
    sending = 15%     Invites per                 ...
Maximum Upside Potential of a Metric
                                                                           ?




    ...
Doubling Number of Invitations Sent per Sender by 
         Adding Address Book Importer




                             ...
Optimizing
Customer Value Creation 
     with Metrics



               Copyright © 2008 Olsen Solutions LLC
The Fuzzy Math of Customer Value
 You create customer value with a product that
   Satisfies customers’ needs
   Is easy t...
Olsen’s Hierarchy of Web User Needs
       (adapted from Maslow)
    Customer’s Perspective                               ...
Kano Model: User Needs & Satisfaction
              User Satisfaction
                                  Delighter (wow)


...
Importance vs. Satisfaction
       Ask Users to Rate for Each Feature
              100                                   ...
What is Your Value Proposition?
  Which user benefits are most important?
  How well does your product deliver?
          ...
Return (Value Created)   Analyzing Product Ideas by ROI

                                        ?
                       ...
Return (Value Created)   Prioritizing Product Ideas by ROI

                                                 Idea C
      ...
The UI Design Iceberg
  What most
  people see
  and react to                       Visual
                               ...
Approaching UI Design Analytically
 Typical UI design question:
 “When using web pages, do users scroll down?”
   ‐ Yes
  ...
Put Key Conversion Actions Above The Fold
          Landing Page A                 Landing Page B




                    ...
The Fold: “Chrome” Takes Away Pixels




                              Only
                                        768
  ...
The Fold Isn’t Binary Either

                                               768 px

                                     ...
Analyze User Screen Height Distribution
 to Select Design Height for Key Pages
                                           ...
Case Study: Account Signup Process Redesign

                    100%
             100%
                             Bigge...
Analysis of Sign In/Registration Flow
 Open
Account
                                                           55%
    44%...
Redesigned User Flow Improved 
                                                           Registration Conversion Rate
   ...
Product Management by Numbers
          Cheat Sheet
Define what success means
  Customer value proposition
  Equation of y...
YourVersion proactively discovers relevant
web content for you based on your interests.




 Discover      Bookmark       ...
Questions?    www.yourversion.com
              dan@yourversion.com
         40          Copyright © 2008 Olsen Solutions ...
Upcoming SlideShare
Loading in …5
×

of

Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 1 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 2 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 3 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 4 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 5 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 6 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 7 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 8 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 9 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 10 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 11 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 12 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 13 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 14 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 15 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 16 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 17 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 18 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 19 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 20 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 21 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 22 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 23 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 24 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 25 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 26 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 27 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 28 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 29 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 30 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 31 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 32 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 33 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 34 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 35 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 36 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 37 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 38 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 39 Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen Slide 40
Upcoming SlideShare
Finding Product Market Fit
Next
Download to read offline and view in fullscreen.

192 Likes

Share

Download to read offline

Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen

Download to read offline

Best practices in using metrics to optimize your web product. I gave this webinar on Dec 17, 2008, as part of FeaturePlan's series "The Product Management View".

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Product Management by Numbers: Using Metrics To Optimize Your Product by Dan Olsen

  1. Product Management by Numbers: Using Metrics to Optimize your Product Dan Olsen CEO & Founder, YourVersion Dec 17, 2008 1 Copyright © 2008 Olsen Solutions LLC Copyright 
  2. Using Metrics to Optimize your Product Product Management’s Role Integrating metrics & optimization Using metrics to optimize value creation Business Value Customer Value UI Design Illustrate with real world examples Copyright © 2008 Olsen Solutions LLC
  3. My Background Education BS, Electrical Engineering, Northwestern MS, Industrial Engineering, Virginia Tech MBA, Stanford PHP, MySQL, JavaScript, XHTML, CSS, UI design 17 years of Product Management Experience Managed submarine design for 5 years 5 years at Intuit, led Quicken Product Management Led Product Management at Friendster Olsen Solutions LLC, PM consultant for start‐ups CEO & Founder of YourVersion, Social Discovery start‐up
  4. Product Management’s  Role Copyright © 2008 Olsen Solutions LLC
  5. A Product Manager by any Other  Name Would Smell as Sweet Product managers are sometimes called Product marketing manager Program manager Project manager Label and definition of role can vary Based on industry or company Based on B2C (consumer) vs. B2B (enterprise) Based on stage of company Can be area of responsibility vs. actual position Copyright © 2008 Olsen Solutions LLC
  6. Product Management is Critical Link in Value Creation Market Product  Development  • Current  Management Team customers • Prospective customers • Competitors Copyright © 2008 Olsen Solutions LLC
  7. A Process View of Product Management “Inbound” Product “Outbound” Management Product Long Business Product Management Term Strategy Strategy Market/ Sell Short Business Product Product Term Objectives Objectives Development Service/ Support Copyright © 2008 Olsen Solutions LLC
  8. PM’s Job: A Successful Product Know market and customer better than anyone else Translate business objectives and customer needs  into product requirements Be the clearinghouse for all product ideas  Identify, plan & prioritize product ideas to maximize  ROI on engineering resources Work with team to design and build a great product Evangelize product within and outside the company Keep the product team motivated Copyright © 2008 Olsen Solutions LLC
  9. Integrating Metrics and  Optimization into your  Product Process Copyright © 2008 Olsen Solutions LLC
  10. Adding Metrics and Optimization to  your Product Process Site Level Business Product Prioritized  Plan Objectives Objectives Feature List Scoping Feature  Level Requirements  Design & Design Code Test Launch Develop Metrics & User  Optimize Feedback Copyright © 2008 Olsen Solutions LLC
  11. Optimization through Iteration: Continuous Improvement Measure the metric Analyze Learning the metric Gaining knowledge: • Market Identify top  • Customer opportunities to improve • Domain • Usability Design & develop   the enhancement Launch the enhancement Copyright © 2008 Olsen Solutions LLC
  12. Optimizing Business Value Creation  with Metrics Copyright © 2008 Olsen Solutions LLC
  13. Approaching Business as an  Optimization Exercise Given reality as it exists today, optimize our business results subject to our resource constraints. Copyright © 2008 Olsen Solutions LLC
  14. Define the Equation of your Business “Peeling the Onion” Profit = Revenue ‐ Cost Unique Visitors x  Ad Revenue per Visitor Impressions/Visitor x  Effective CPM / 1000 Visits/Visitor  x  Pageviews/Visit  x  Impressions/PV New Visitors + Returning Visitors Invited Visitors + Uninvited Visitors # of Users Sending Invites  x  Invites Sent/User  x  Invite Click‐through Rate Copyright © 2008 Olsen Solutions LLC
  15. How to Track Your Metrics Track each metric as daily time series Unique  Page  Ad  New User  … Date Visitors views Revenue Sign‐ups 4/24/08 10,100 29,600 25 490 4/25/08 10,500 27,100 24 480 … Create ratios from primary metrics:  X / Y Example: How good is your registration page? Okay: # of registered users per day Better: registration conversion rate = # registered users / # uniques to reg page Copyright © 2008 Olsen Solutions LLC
  16. Sample Signup Page Yield Data Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page 100% 90% 80% Daily Signup Page Yield 70% 60% 50% 40% 30% Started requiring registration 20% Changed Added questions messaging to signup page 10% 0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0 Copyright © 2008 Olsen Solutions LLC
  17. Using Metrics to Optimize the  Equation of your Business What are the metrics for your business? Where is current value for each metric?  How many resources to “move” each metric? Developer‐hours, time, money Which metrics have highest ROI opportunities? Metric A Metric B Metric C Good ROI Bad ROI Great ROI Return Return Return Investment Investment Investment Copyright © 2008 Olsen Solutions LLC
  18. Case Study: Optimizing Viral Loop Metrics % of users sending = 15% Invites per Invite invites sender = 2.3 click-through rate Active Invite Prospective Click Registration Fail Users Users Process % of users who are Succeed Don’t active Click Conversion rate = 85% Users • Multiplied together, these metrics determine your viral ratio • Which metric has highest ROI opportunity? Copyright © 2008 Olsen Solutions LLC
  19. Maximum Upside Potential of a Metric ? 100% 100% 85% 15% 2.3 0 0 0 Registration % of users sending Avg # of invites conversion rate invitations sent per sender Max possible 0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?% improvement Copyright © 2008 Olsen Solutions LLC
  20. Doubling Number of Invitations Sent per Sender by  Adding Address Book Importer Copyright © 2008 Olsen Solutions LLC
  21. Optimizing Customer Value Creation  with Metrics Copyright © 2008 Olsen Solutions LLC
  22. The Fuzzy Math of Customer Value You create customer value with a product that Satisfies customers’ needs Is easy to use Has a good price Is better than other alternatives Not fuzzy: # of users and frequency of use Applying metrics approach to customer value Need framework of user needs & satisfaction Quantitative: site analytics, usage metrics, surveys Qualitative: 1‐on‐1 user feedback sessions  are best Copyright © 2008 Olsen Solutions LLC
  23. Olsen’s Hierarchy of Web User Needs (adapted from Maslow) Customer’s Perspective What does it mean to us? How easy to use is it? Usability & Design Satisfaction Increasing Does the functionality Feature Set meet my needs? Does the functionality work? Absence of Bugs Dissatisfaction Decreasing Is the site fast enough? Page Load Time Is the site up when I want to use it? Uptime Copyright © 2008 Olsen Solutions LLC
  24. Kano Model: User Needs & Satisfaction User Satisfaction Delighter (wow) Performance  (more is better) Need Need not met fully met Must Have Needs & features  migrate over time User Dissatisfaction Copyright © 2008 Olsen Solutions LLC
  25. Importance vs. Satisfaction Ask Users to Rate for Each Feature 100 98 Great 95 84 87 90 Bad 86 85 79 84 55 70 80 Importance 80 75 72 80 70 75 65 60 55 41 50 40 50 60 70 80 90 100 Satisfaction Recommended reading: “What  Customers Want” by Anthony Ulwick Copyright © 2008 Olsen Solutions LLC
  26. What is Your Value Proposition? Which user benefits are most important? How well does your product deliver? Importance Current User Upside User Benefit to User Satisfaction Potential Low High – + Benefit 1 High Benefit 2 High Benefit 3 Low ? opportunity Benefit 4 Med Benefit 5 Low ? opportunity Benefit 6 Low Doesn’t matter Copyright © 2008 Olsen Solutions LLC
  27. Return (Value Created) Analyzing Product Ideas by ROI ? 4 Idea D 3 Idea A Idea B 2 Idea C 1 Idea F 1 2 3 4 Investment (developer-weeks) Copyright © 2008 Olsen Solutions LLC
  28. Return (Value Created) Prioritizing Product Ideas by ROI Idea C 8 7 Idea B 6 5 4 Idea A 3 2 1 1 2 3 4 5 Investment (developer-weeks) Copyright © 2008 Olsen Solutions LLC
  29. The UI Design Iceberg What most people see and react to Visual Design What good PMs and Designers Interaction think about Design Information Architecture Conceptual Design Recommended reading: Jesse James Garrett’s “Elements of User Experience” chart, free at www.jjg.net Copyright © 2008 Olsen Solutions LLC
  30. Approaching UI Design Analytically Typical UI design question: “When using web pages, do users scroll down?” ‐ Yes ‐ No UI questions are never yes/no! (not binary) Should ask: “What percentage of users …?” UI changes impact your metrics Impact can be positive, negative, small, large Seek high‐ROI UI changes Copyright © 2008 Olsen Solutions LLC
  31. Put Key Conversion Actions Above The Fold Landing Page A Landing Page B The Fold Key conversion action is above the fold Key conversion action is below the fold Copyright © 2008 Olsen Solutions LLC
  32. The Fold: “Chrome” Takes Away Pixels Only 768 600 pixels pixels 1024 pixels
  33. The Fold Isn’t Binary Either 768 px The chrome steals  % of Users about 170 pixels 600 px 1024 px Data courtesy of ClickTale Free trial at www.clicktale.com Copyright © 2008 Olsen Solutions LLC
  34. Analyze User Screen Height Distribution to Select Design Height for Key Pages 100% 600, 100% 768, 92% 90% % of Users with Given (or Higher) Screen Height 80% 70% Going above 60% 768 px = big drop  50% 40% 772, 38% 864, 29% 30% 807, 29% 1024, 20% 866, 25% 20% 1000, 20% 10% 1030, 5% 0% 600 650 700 750 800 850 900 950 1000 1050 1100 Page Design Height (pixels) Monitor Screen Height (pixels)
  35. Case Study: Account Signup Process Redesign 100% 100% Biggest  80% drop % of Users 62.3% 58.8% 60% 50.9% 40% 34.4% 32.7% 20% 0% Sign in / Account  Cash vs. 5 Partner 3 Partner Registration Type Margin Pages Pages Copyright © 2008 Olsen Solutions LLC
  36. Analysis of Sign In/Registration Flow Open Account 55% 44% Register Registration  (24% of Total) Process 45% drop off 64% (20% of total) of Total Account  36% overall  Selection 83% 30% drop off for  56% (46% of Total) (14% of Total) this step Sign in Forget  70% Change  80% Password (32% of Total) Password (26% of Total) 17% drop off  20% drop off (10% of total) (6% of total) Copyright © 2008 Olsen Solutions LLC
  37. Redesigned User Flow Improved  Registration Conversion Rate Abandonment Rate (7 Day Moving Average) Steps 1-2 80% 70% Abandonment Rate (7 Day Moving Average) 60% 50% 40% 30% Released 37% improvement New Design in conversion rate 20% 10% 0% 10/14/02 10/21/02 10/28/02 11/11/02 11/18/02 11/25/02 12/16/02 12/23/02 12/30/02 10/7/02 11/4/02 12/2/02 12/9/02 1/13/03 1/20/03 1/6/03 Copyright © 2008 Olsen Solutions LLC
  38. Product Management by Numbers Cheat Sheet Define what success means Customer value proposition Equation of your business Instrument your site and track key metrics Identify opportunities and prioritize by ROI Launch, learn, and iterate Copyright © 2008 Olsen Solutions LLC
  39. YourVersion proactively discovers relevant web content for you based on your interests. Discover Bookmark Share
  40. Questions? www.yourversion.com dan@yourversion.com 40 Copyright © 2008 Olsen Solutions LLC Copyright 
  • FalconsHamilton

    Dec. 5, 2021
  • DominicCorbinPMIACP

    Jan. 14, 2021
  • AndhikaRioAndriyanto

    Dec. 17, 2020
  • iiiiraviiii

    Jun. 24, 2020
  • DamionRobinson1

    Jun. 23, 2020
  • lukehan

    Mar. 21, 2020
  • AmitAgarwal297

    Jan. 10, 2020
  • JamesWoodward23

    Nov. 9, 2019
  • AndyKim43

    Apr. 16, 2019
  • jerthorne

    Feb. 21, 2019
  • SuparatWirattanaporn

    Jun. 4, 2018
  • BenyaminNajafi

    Mar. 28, 2018
  • martinusk

    Mar. 12, 2018
  • HerbertKnibiehly

    Jan. 30, 2018
  • GursahibSinghSuri

    Dec. 17, 2017
  • MabelTan2

    Dec. 13, 2017
  • MarieMiller18

    Dec. 3, 2017
  • alaalotfy96

    Dec. 2, 2017
  • SyarifahRiefandania

    Nov. 10, 2017
  • Pine-tree-consultants

    Sep. 2, 2017

Best practices in using metrics to optimize your web product. I gave this webinar on Dec 17, 2008, as part of FeaturePlan's series "The Product Management View".

Views

Total views

100,625

On Slideshare

0

From embeds

0

Number of embeds

35,389

Actions

Downloads

3,790

Shares

0

Comments

0

Likes

192

×