Thinking Like an Optimizer, by Jennifer Ruffner
•
11 j'aime•21,864 vues
This is a talk given to my class on User Experience by Jen Ruffner, a Product Manager on the art of optimization. It is critical for modern designers, product managers and start-up folks ot understand how to think about designing and executing tests.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (19)
Similaire à Thinking Like an Optimizer, by Jennifer Ruffner
Similaire à Thinking Like an Optimizer, by Jennifer Ruffner (20)
Plus de Christina Wodtke
Plus de Christina Wodtke (20)
Dernier
Dernier (20)
Thinking Like an Optimizer, by Jennifer Ruffner
- 1. Thinking Like an Optimizer Jennifer Ruffner
- 3. Dining Out Do you know the tricks of the trade?
- 5. Which section of the menu do people look at first? 2 3 1 4
- 6. Placement 4
- 7. Which way do a users eyes flow when reading a menu? 2 3 1 4
- 8. What items to people pay attention to within a list? Entrees Entrees
- 9. Pricing 1. $11.99 2. 12 3. twelve Increases the spending by 8.15% per person
- 10. Avoid Embarassment Market Price
- 11. What’s the lowest margin wine on the menu? 2nd cheapest bottle of wine has a high margin. http://online.wsj.com/article/SB123638925101858707.html
- 12. Planning It’s Easier Said Than Done
- 13. Features and your KPIs/OKRs Create Company and Team Wide OKR’s (Objective and Key Results) Specific, measureable objectives that every feature in your roadmap should fall under. Identify Key Performance Metrics that support the company OKRs Example OKRs Increase Daily Active User; N DAU per day Increase Revenue; $N Revenue per day Maintain Net Promoter Score (NPS); >25 Increase social actions; N Social Interactions pp/day
- 14. Roadmap Planning Given our 2-3 KPI’s; what features will support them? Start with sub metrics that support those features Build features that support them Which features do you prioritize? How many dev days should it get? It’s about diligence. Brainstorm What features do we want this quarter? Hypothesize What metrics will those features move? Expected Outcomes vs Dev Days What are the goals of the feature? What metrics do we all agree no that we will use to measure the success of this feature? How many dev days is the feature? How much revenue, engagement, and/or social actions will we get out of it? Every 12-15 dev days = 1 PM day estimating Build Analyze and Discuss Weekly team Company wide reviews discussing learnings Every 4 dev days = 1 PM day analyzing
- 15. Optimization on the Web
- 16. Design, Estimate, and Measure in FUNNELS
- 18. Estimate and Expected Day Measure in Funnels When figuring out how to track a feature, start by building out the Energy pinch was an unnecessary gate on virals funnel. Use comparable data to fill in the blanks for your forecast
- 19. Birds and the Bees Where do users come from?
- 20. User Acquisition: Where does traffic come from? Email Mainly invitations from friends, or users sharing content Organic The best kind! User’s coming directly to your site SEO 25% of LinkedIn’s new users came from search Ads/Partnerships Social Feeds Facebook, LinkedIn, Twitter, Pinterest Wherever possible, use user generated content as messaging to increase the response to your outreach. What feature within your site prompt generating user generated content? It’s likely those have your highest conversions and are a place you could start to double down. http://blog.adamnash.com/2012/03/28/product-leaders-user-acquisition-series/
- 21. Measuring Virality You are viral when every 1 user brings in more than 1 new user K-Factor (wikipedia) i = number of invites sent by each customer c = percent conversion of each invite K-Factor = i * c Z-Factor (Mixpanel/Dave McClure) x = % of users who invite y = avg number of people they invited z= % of users who accepted an invitation Z-Factor =x*y*z
- 22. The often overlooked piece of virality Cycle Time “Given that I get a new customer today, how many new customers will they bring in over the next N days?” Let’s assume a viral factor of 0.5, and an N of 7. User Acquisition will look like the below which can be used for projections: 1 + 0.5 + 0.25 + 0.125 ….
- 23. Optimizing Emails and Registration Pages
- 24. Get to & complete registration Registered users is a key metric for success Once a user is registered: You have a way to reach out to them You have some key demographics to target advertising
- 25. Sample Invitation Funnel Data Notes Receipts Receipts/Recipient Are you sending too many per day? Causing spam to increase? Recipients % getting inboxed IP Sender Score, Domain Reputation Inboxed emails % Opened Sender Name, Subject, Time Sending etc. Opened Emails % Clicked On Messaging for CTAs, color of CTA, # of CTAs, Placement of CTA (above the fold) Unique Clickers
- 26. Email Test Checklist Delivery Theme Tests Sender Friend name (Jen, Jen Ruffner, Jen R.), Unique ID, Company Subject Under 60 characters, test different messaging tactics below Body Messaging tactics, CTA’s above the fold, # of CTA’s, Time Highest open rate given the hour in users time zone, Other performance issues associated with email blasts? Frequency Digest information, Aggregate messages, Test opt-out positions vs spam rates Messaging Theme Tests Statement Type Declarative, Imperative, Interrogative, Conditional Sentence Tense Past, Present, Future Persuasiveness Light vs. Heavy Wildcards !! … ;) >>
- 27. Emails
- 28. First Impressions – Subject lines Goal 1: Get people to open the email - Optimize the sender - Optimize the subject
- 29. Email Body Content Goal 2: Get people to click on the email - Optimize the body (language, layout, images) - Optimize the call to action
- 31. Email Body Content (w/out images downloaded) 50% of user’s don’t download images.
- 32. Email Body Content (w/images)
- 34. Body Content
- 35. Great that it’s an HTML button, but I would change the text to “See 15 more >>” Messaging is all about me and my profile, playing to my vanity.
- 36. Subject line is compelling that there is a specific number of things waiting for me to see. Personally I wouldn’t put the main message on a grey background, but the faces help to offset that. Unsub info is grey text on a grey background which helps you ignore it.
- 38. Registration Goal 3: Get the user to complete registration - Reduce Distraction -Minimize user effort -Hold their Hand
- 40. Registration
- 41. Registration
- 44. The header is context specific. This is fantastic! Add an image of the album cover of the song I just liked to remind me why I’m registering.
- 45. Appendix
- 46. Increasing Conversion Users Users Entering Completing Flow Flow - Make it dead simple - Increase Entry Points - Hold their hand - Overlay social on everything - Reward them - Test everything Introduce social learning throughout these flows to teach the user how to use the site
Notes de l'éditeur
- Hi My name is Jen Ruffner. As Christina mentioned I’m going to talk about Site Flow OptimizationBefore before I jump into a website --- I want to visit an experience that we are all familiar with – eating out. I’ll talk about some of the industry secrets that restaurants use to improve the check’s bottom line. What’s interesting is that some of them translate very nicely to what you can do on a website, and some just seem to go against what we would believe to be common sense.
- High margin items in the upper right hand corner of the menu “1978 issue of The Cornell Hotel and Restaurant Administration Quarterly”High margin items in the upper right hand corner of the menu
- And then counter clockwise to the right bottom ---- Which is why you see appetizers in the top left“People tend to remember the top two items on a list and the bottom item” Steve Miller This lesson here is that every site is different. Some things work everywhere and other sites have communities that go against the norm. You should always be iterating by learning from your users and testing new approaches.
- Much like search results
- Reduce any hurdle that you can see between the user and what you want them to do. In this case, not only does the user have to remember to ask the waiter what the market price is before ordering, but they also risk embarassment
- Things I told CW – Game Mechanics (pinch, unlock, expiration, handholding, tell users what to do)FunnelsData every dayDesigners are part of the triangle
- Best practice from Zynga and also something that mixpanel heavily preaches is to think about everything in funnels. When you’re planning
- If your feature isn’t simple enough to fit in a Google drawing, adoption will be difficult. Every major step that you add has 10-25% drop off. Increasing handholding (like arrows, familiar faces, additional marketing and reminders) reduces drop off between steps. You can literally tell your users want to do.
- Most important part of feature development is expected outcomes. - start by agreeing on goals how you would measure success. Often times what you think is a simple If you don’t have great comps, guess
- Email - Drives growth, engagement, and retentionOrganic – the best kind which usually yields the highest retention. Cold joins, word of mouth, Seo Search – 25% of LI’s traffic comes from hereAds - $250m/year spend by ebay. Under $2 conversion rate on FB for an app install is pretty good. Social Feeds – Works really well when people take actions frequently (ex. Listened to a song)
- Avg Viral coefficients Tagged went viral – but mostly avg’d above 0.4LinkedIn hovered around 0.3
- The first flow that I’m going to walk through is registration. Registration is obviously an important flow for just about any web site.As I walk through the flow, I’m going to pointing out very small details that can make a big difference to the bottom line. While we’re walking through these slides, I’d encourage you to think of a few flows on your site where some optimization would be beneficial. Looking at the same site day in and day out, it’s sometimes easy to overlook some obvious changes. Hopefully this presentation will help you get in the mindset for optimization and look at things with a fresh eye.
- Subject lines are easy to test for a lot of reasonsLook at who’s hot right now. Try to guess their hypothesis behind the test and see if you should test it out yourself.
- One of the most popular ways to hear about a social site is through a friend Objective is to get people to open the email. How do you do that?Who is it sent from?Should use the inviter’s name – not the siteTwitter doesn’t Users are more likely to open an email when it’s from a friend instead of a website. Twitter doesn’t even mention my name in the subject line. People won’t recognize JenG24Is the subject line short, but compellingDoes it make you feel wanted and special?Twitter doesn’t even use my name in the subjectFacebook – first off when I sent this email it wasn’t to share photos. It was just to invite my friend to join the site. Second, it doesn’t make me feel special.Tagged makes me feel good because something is waiting there and it’s specifically for meSuggestions1. Make the reader feel like they need to respond, while still keeping it light.2. Wildcards – try jazzing up the subject line to see if it does anything [SMILEY FACE]
- GoodPicture is great for conversation AND it clicks to registerThe stats about me tell the story of what you can do on Facebook (social learning)
- BadToo many links (make one clear call to action and lock down the flow)Note that one link is in case you already have an account. Normally I’d say to remove it, but I’d imagine that FB gets a lot of CS requests to merge duplicate accounts. If that is costing a lot of money it may be worth keeping that link here).Too much text – don’t make the user read a story here – get them to join the site and let them learn from their friends.Call to action is to sign up for facebook – not look at Jen’s photosSuggestions – Images can be scary to put in the email because the user may not download it. In that event add text about where I live or where I went to school etc.
- Good Buttons instead of a link is a very clear call to action.Both buttons click to reg – if you click no you register without becoming friends with the inviter.Posed a question which envokes the user to respond.Bad- The messaging is a little harsh
- GoodNo navProgress barGood header - it’s inviting and rewardingHeld the users hand with info on the inviter and allows you to get more information inlineHandholding text as to why you should do certain things (rewarding the user)Country is prepopulatedThe button is a good color and has a good message. Buttons are important not only because they are the main call to action, but it’s the last thing that a user looks at before they decide whether or not to continue.BadGet rid of the marketing textAllow the user to skip things that aren’t optional
- You’ve got to get it to one page. Why? Because once you have the user’s email, you can reach out to them and try to get them to come back. If you bog them down with excess requested information, you’re going to lose them. Email addressGoodHolding the users hand by keeping the inviter at the top of the page w/ good messaging like “Join your friends on MS”Mentions the word “Free”Absolute minimal requirements. - Prepopulated the email addressOrder!Submit button is above the fold and it’s a good clear call to askingIT’s a good idea that any page in this flow, the submit button needs to be above the fold. That doesn’t mean that you should take what was 3 really long pages and turn them into 6 short pages, because there is a drop off rate of about 20% per page. Make sure you get as much as you can and need before you’ve lost the opportunity to land them on the home page and let them do what they came there to do.Got rid of the checkbox that they agree to the TOSKept the captcha until later
- BadGet rid of the navCirsor not defaultedShould have default selected one of the gendersFull looks like first
- For a social website -- your goals are probably include growth, engagement, and revenue. So key flows and conversions you should be focussed on include: - getting a user registered- getting them to submit content or share something- or getting them to buy something….