SlideShare une entreprise Scribd logo
1  sur  52
Télécharger pour lire hors ligne
The researcher’s blind spot:
6 cognitive biases we shouldn’t ignore
in research
UX Australia 2016, Melbourne
Ruth Ellison, Principal User Researcher, PwC’s Experience Centre
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
$7$3 $6.50
@RuthEllison from PwC’s Digital Services
$7$3
@RuthEllison from PwC’s Digital Services
$7$3 $6.50
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
The Decoy Effect is a Cognitive Bias
@RuthEllison from PwC’s Digital Services
What is a cognitive bias?
Mental shortcut
@RuthEllison from PwC’s Digital Services
We are all human
@RuthEllison from PwC’s Digital Services Image source: http://www.businessinsider.com.au/what-new-coke-looked-like-in-1985-2013-3
@RuthEllison from PwC’s Digital Services
Applying it to design research
Preparing
Running
Analysing
@RuthEllison from PwC’s Digital Services
Preparing for research
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
Selection bias
@RuthEllison from PwC’s Digital Services 15
@RuthEllison from PwC’s Digital Services
Multiple channels - mix of
recruitment companies, social
media, trusted networks
Avoid professional respondents
Careful screeners
Behavioural based recruitment
@RuthEllison from PwC’s Digital Services
Running research sessions
Photo available under a CC by 2.0 licence: https://www.flickr.com/photos/gdsteam/20649386153
@RuthEllison from PwC’s Digital Services
Observer-Expectancy effect
@RuthEllison from PwC’s Digital Services
The way we frame our questions matter
@RuthEllison from PwC’s Digital Services
Loftus, E. F., & Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and
Verbal Behavior, 1974, 13, 585-589.
McLeod, S. A. (2014). Loftus and Palmer. Retrieved from www.simplypsychology.org/loftus-palmer.html
About how fast were the cars going
when they smashed into each
other?
@RuthEllison from PwC’s Digital Services
About how fast were the cars going when
they collided into each other?
About how fast were the cars going when
they bumped into each other?
About how fast were the cars going when
they contacted into each other?
@RuthEllison from PwC’s Digital Services
Triangulate research
Use observational methods
Keeping positive-neutral body
language, watch the tone of your
voice
Avoid leading questions
@RuthEllison from PwC’s Digital Services 23
I need a volunteer please
24
2
4
8
PwC’s Digital Services
What’s going on?
@RuthEllison from PwC’s Digital Services
Confirmation bias
27
Take the opposing view
28
Don’t just confirm your hypothesis, see if you can prove it wrong
Analysis of Competing Hypotheses (ACH)
29
Use open ended questions
Some people think that soft drinks are bad for you.
What do you think?
What’s your opinion about soft drinks?
@RuthEllison from PwC’s Digital Services
List assumptions
Be skeptical, especially if everyone
agrees with you
Remain open
Consider all evidence equally
Multiple user researchers
Leave your ego by the door.
@RuthEllison from PwC’s Digital Services
@RuthEllison from PwC’s Digital Services
Groupthink & the Bandwagon effect
@RuthEllison from PwC’s Digital Services 32
https://youtu.be/MDD4IkVZWTM?t=52s
@RuthEllison from PwC’s Digital Services
Triangulate with other research
methods (e.g. observational)
Use a mixture of individual and
group exercises
Avoid stating preferences and
expectations at start
Give someone the devil’s advocate
role to question assumptions
@RuthEllison from PwC’s Digital Services
Anchoring bias
@RuthEllison from PwC’s Digital Services 35
36https://www.flickr.com/photos/naotakem/17105160150/
@RuthEllison from PwC’s Digital Services
Consider the order of questions and
designs carefully
Open ended questions
Alternate order in which participants
are shown concept or design versions
Show version A first
1 3 5
2 4 6 Show version B first
@RuthEllison from PwC’s Digital Services
Analysing
@RuthEllison from PwC’s Digital Services 39
@RuthEllison from PwC’s Digital Services
Apophenia
40Source: https://au.pinterest.com/pin/249738741807276910/
@RuthEllison from PwC’s Digital Services
Clustering illusion & reporting bias
@RuthEllison from PwC’s Digital Services
Breadcrumbs
@RuthEllison from PwC’s Digital Services
Sample sizes – it’s about the WHY
Consider evidence equally – not just
the ones that confirm your
belief/assumption
Collaborative analysis sessions
Strive for objectivity
@RuthEllison from PwC’s Digital Services 44
How vulnerable are you to
these biases?
@RuthEllison from PwC’s Digital Services
What percentage of the
population will be more biased
than the average person?
@RuthEllison from PwC’s Digital Services
Are you more biased than the
average person?
@RuthEllison from PwC’s Digital Services
Blind spot bias
@RuthEllison from PwC’s Digital Services
Listen with an open
mind
Become more rational, but less
rationalising
Continuous learning
48
Always assess your method,
your analysis and yourself for
bias
@RuthEllison from PwC’s Digital Services 49
We are all human
@RuthEllison from PwC’s Digital ServicesImage source: https://en.wikipedia.org/wiki/Twenty_Questions#/media/File:20_questions_1954.JPG
Thank you
:)
Say hello @RuthEllison
@RuthEllison from PwC’s Digital Services
Further reading
List of cognitive biases
You are not so smart: a celebration
of self delusion
9 Biases In Usability Testing
52

Contenu connexe

En vedette

How to start a customer-centred approach
How to start a customer-centred approachHow to start a customer-centred approach
How to start a customer-centred approachLaura Lorenzo
 
8 Requirements To Be An International Designer 成為設計師的八個要件
8 Requirements To Be An International Designer 成為設計師的八個要件8 Requirements To Be An International Designer 成為設計師的八個要件
8 Requirements To Be An International Designer 成為設計師的八個要件Amy Huang
 
Getting emotional: MOO's first steps with affective interaction
Getting emotional: MOO's first steps with affective interactionGetting emotional: MOO's first steps with affective interaction
Getting emotional: MOO's first steps with affective interactionTerri Herbert
 
Everything You Ever Wanted to Know About Content Strategy, But Afraid to Ask
Everything You Ever Wanted to Know About Content Strategy, But Afraid to AskEverything You Ever Wanted to Know About Content Strategy, But Afraid to Ask
Everything You Ever Wanted to Know About Content Strategy, But Afraid to AskIxDA Chicago
 
Daniela Petrillo | design researcher
Daniela Petrillo | design researcherDaniela Petrillo | design researcher
Daniela Petrillo | design researcherDaniela Petrillo
 
A 45 minute talk about designing a single button
A 45 minute talk about designing a single buttonA 45 minute talk about designing a single button
A 45 minute talk about designing a single buttonolliecampbell
 
The Uncanny Valley - BarCamp Canberra
The Uncanny Valley - BarCamp CanberraThe Uncanny Valley - BarCamp Canberra
The Uncanny Valley - BarCamp CanberraRuth Ellison
 
UX Bootcamp - August 2016
UX Bootcamp - August 2016UX Bootcamp - August 2016
UX Bootcamp - August 2016Jacklyn Burgan
 
Designing for Financial Inclusion - Sending Money Home
Designing for Financial Inclusion - Sending Money HomeDesigning for Financial Inclusion - Sending Money Home
Designing for Financial Inclusion - Sending Money HomeGabriel White
 
Biases in epidemiology
Biases in epidemiologyBiases in epidemiology
Biases in epidemiologySubraham Pany
 
Designing for real world participation and social interaction
Designing for real world participation and social interactionDesigning for real world participation and social interaction
Designing for real world participation and social interactionAndrew Barrie
 
Ux Australia 2016
Ux Australia 2016Ux Australia 2016
Ux Australia 2016Amir Ansari
 
The Social Life of the Internet of Things
The Social Life of the Internet of ThingsThe Social Life of the Internet of Things
The Social Life of the Internet of ThingsStephen Cox
 
Thinking Fast & Slow presentation
Thinking Fast & Slow presentationThinking Fast & Slow presentation
Thinking Fast & Slow presentationLaure Parsons
 
Inspiring Leaders to Make a Difference: Leadership Quotes & Insights
Inspiring Leaders to Make a Difference: Leadership Quotes & InsightsInspiring Leaders to Make a Difference: Leadership Quotes & Insights
Inspiring Leaders to Make a Difference: Leadership Quotes & InsightsO.C. Tanner
 
ThinkAR Augmented Reality Game Concept
ThinkAR Augmented Reality Game ConceptThinkAR Augmented Reality Game Concept
ThinkAR Augmented Reality Game ConceptSusan Oldham
 
SXSW 2016: Confirmation Bias: Innovation's Blind Eye
SXSW 2016: Confirmation Bias: Innovation's Blind EyeSXSW 2016: Confirmation Bias: Innovation's Blind Eye
SXSW 2016: Confirmation Bias: Innovation's Blind EyeAllison Abbott
 
Francia Sandoval UX Portfolio
Francia Sandoval UX PortfolioFrancia Sandoval UX Portfolio
Francia Sandoval UX PortfolioFrancia Sandoval
 

En vedette (19)

Fastlane
FastlaneFastlane
Fastlane
 
How to start a customer-centred approach
How to start a customer-centred approachHow to start a customer-centred approach
How to start a customer-centred approach
 
8 Requirements To Be An International Designer 成為設計師的八個要件
8 Requirements To Be An International Designer 成為設計師的八個要件8 Requirements To Be An International Designer 成為設計師的八個要件
8 Requirements To Be An International Designer 成為設計師的八個要件
 
Getting emotional: MOO's first steps with affective interaction
Getting emotional: MOO's first steps with affective interactionGetting emotional: MOO's first steps with affective interaction
Getting emotional: MOO's first steps with affective interaction
 
Everything You Ever Wanted to Know About Content Strategy, But Afraid to Ask
Everything You Ever Wanted to Know About Content Strategy, But Afraid to AskEverything You Ever Wanted to Know About Content Strategy, But Afraid to Ask
Everything You Ever Wanted to Know About Content Strategy, But Afraid to Ask
 
Daniela Petrillo | design researcher
Daniela Petrillo | design researcherDaniela Petrillo | design researcher
Daniela Petrillo | design researcher
 
A 45 minute talk about designing a single button
A 45 minute talk about designing a single buttonA 45 minute talk about designing a single button
A 45 minute talk about designing a single button
 
The Uncanny Valley - BarCamp Canberra
The Uncanny Valley - BarCamp CanberraThe Uncanny Valley - BarCamp Canberra
The Uncanny Valley - BarCamp Canberra
 
UX Bootcamp - August 2016
UX Bootcamp - August 2016UX Bootcamp - August 2016
UX Bootcamp - August 2016
 
Designing for Financial Inclusion - Sending Money Home
Designing for Financial Inclusion - Sending Money HomeDesigning for Financial Inclusion - Sending Money Home
Designing for Financial Inclusion - Sending Money Home
 
Biases in epidemiology
Biases in epidemiologyBiases in epidemiology
Biases in epidemiology
 
Designing for real world participation and social interaction
Designing for real world participation and social interactionDesigning for real world participation and social interaction
Designing for real world participation and social interaction
 
Ux Australia 2016
Ux Australia 2016Ux Australia 2016
Ux Australia 2016
 
The Social Life of the Internet of Things
The Social Life of the Internet of ThingsThe Social Life of the Internet of Things
The Social Life of the Internet of Things
 
Thinking Fast & Slow presentation
Thinking Fast & Slow presentationThinking Fast & Slow presentation
Thinking Fast & Slow presentation
 
Inspiring Leaders to Make a Difference: Leadership Quotes & Insights
Inspiring Leaders to Make a Difference: Leadership Quotes & InsightsInspiring Leaders to Make a Difference: Leadership Quotes & Insights
Inspiring Leaders to Make a Difference: Leadership Quotes & Insights
 
ThinkAR Augmented Reality Game Concept
ThinkAR Augmented Reality Game ConceptThinkAR Augmented Reality Game Concept
ThinkAR Augmented Reality Game Concept
 
SXSW 2016: Confirmation Bias: Innovation's Blind Eye
SXSW 2016: Confirmation Bias: Innovation's Blind EyeSXSW 2016: Confirmation Bias: Innovation's Blind Eye
SXSW 2016: Confirmation Bias: Innovation's Blind Eye
 
Francia Sandoval UX Portfolio
Francia Sandoval UX PortfolioFrancia Sandoval UX Portfolio
Francia Sandoval UX Portfolio
 

Similaire à The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research

Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...
Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...
Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...ux singapore
 
Evolution of assessment 2017 final
Evolution of assessment   2017 finalEvolution of assessment   2017 final
Evolution of assessment 2017 finalLucy Standing
 
Evolution of assessment 2017 final
Evolution of assessment   2017 finalEvolution of assessment   2017 final
Evolution of assessment 2017 finalLucy Standing
 
Freshly squeezed beyondave
Freshly squeezed beyondaveFreshly squeezed beyondave
Freshly squeezed beyondaveMichael Blowers
 
LOMA - How actuaries can use advanced analytical techniques to modernize thei...
LOMA - How actuaries can use advanced analytical techniques to modernize thei...LOMA - How actuaries can use advanced analytical techniques to modernize thei...
LOMA - How actuaries can use advanced analytical techniques to modernize thei...AnandRaoPwC
 
Title of Your Research Proposal Student Name Walde
Title of Your Research Proposal Student Name WaldeTitle of Your Research Proposal Student Name Walde
Title of Your Research Proposal Student Name WaldeTakishaPeck109
 
Web Conference at Penn State 2015
Web Conference at Penn State 2015Web Conference at Penn State 2015
Web Conference at Penn State 2015laurentgc
 
Promise and Perils of Predictive Analytics
Promise and Perils of Predictive Analytics Promise and Perils of Predictive Analytics
Promise and Perils of Predictive Analytics Kevin Wheeler
 
State of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRState of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRRay Poynter
 
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360Socioversity - India's Most Exhaustive Social Media Ranking by Careers360
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360Saurabh Pareek
 
Socioversity - India's most exhaustive social media ranking for universities ...
Socioversity - India's most exhaustive social media ranking for universities ...Socioversity - India's most exhaustive social media ranking for universities ...
Socioversity - India's most exhaustive social media ranking for universities ...Naveen goyal
 
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...Saurabh Pareek
 
Master PR Measurement
Master PR MeasurementMaster PR Measurement
Master PR MeasurementCision
 
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and ExcellenceTesting in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and ExcellenceTechWell
 

Similaire à The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research (20)

Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...
Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...
Lightning Talk #12:7 cognitive biases we shouldn’t ignore in research by Ruth...
 
Evolution of assessment 2017 final
Evolution of assessment   2017 finalEvolution of assessment   2017 final
Evolution of assessment 2017 final
 
Evolution of assessment 2017 final
Evolution of assessment   2017 finalEvolution of assessment   2017 final
Evolution of assessment 2017 final
 
Freshly squeezed beyondave
Freshly squeezed beyondaveFreshly squeezed beyondave
Freshly squeezed beyondave
 
LOMA - How actuaries can use advanced analytical techniques to modernize thei...
LOMA - How actuaries can use advanced analytical techniques to modernize thei...LOMA - How actuaries can use advanced analytical techniques to modernize thei...
LOMA - How actuaries can use advanced analytical techniques to modernize thei...
 
How to measure: Beyond the AVE
How to measure: Beyond the AVEHow to measure: Beyond the AVE
How to measure: Beyond the AVE
 
Title of Your Research Proposal Student Name Walde
Title of Your Research Proposal Student Name WaldeTitle of Your Research Proposal Student Name Walde
Title of Your Research Proposal Student Name Walde
 
The Evidence-Based Organization: A Platform for Innovation
The Evidence-Based Organization: A Platform for InnovationThe Evidence-Based Organization: A Platform for Innovation
The Evidence-Based Organization: A Platform for Innovation
 
Web Conference at Penn State 2015
Web Conference at Penn State 2015Web Conference at Penn State 2015
Web Conference at Penn State 2015
 
Zontee Hou - Bionic Consumers
Zontee Hou - Bionic ConsumersZontee Hou - Bionic Consumers
Zontee Hou - Bionic Consumers
 
Formation and Learning Analytics?
Formation and Learning Analytics?Formation and Learning Analytics?
Formation and Learning Analytics?
 
Promise and Perils of Predictive Analytics
Promise and Perils of Predictive Analytics Promise and Perils of Predictive Analytics
Promise and Perils of Predictive Analytics
 
Swachh Bharat Abhiyaan buzz angles report till 7th oct'14
Swachh Bharat Abhiyaan buzz angles report  till 7th oct'14Swachh Bharat Abhiyaan buzz angles report  till 7th oct'14
Swachh Bharat Abhiyaan buzz angles report till 7th oct'14
 
State of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMRState of Research Insights in Q1, 2024 from NewMR
State of Research Insights in Q1, 2024 from NewMR
 
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360Socioversity - India's Most Exhaustive Social Media Ranking by Careers360
Socioversity - India's Most Exhaustive Social Media Ranking by Careers360
 
Socioversity - India's most exhaustive social media ranking for universities ...
Socioversity - India's most exhaustive social media ranking for universities ...Socioversity - India's most exhaustive social media ranking for universities ...
Socioversity - India's most exhaustive social media ranking for universities ...
 
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...
Socioversity - India's First and Most Exhaustive Social Media Ranking for Uni...
 
Master PR Measurement
Master PR MeasurementMaster PR Measurement
Master PR Measurement
 
Building an Online Profile Using Social Networking Tools
Building an Online Profile Using Social Networking ToolsBuilding an Online Profile Using Social Networking Tools
Building an Online Profile Using Social Networking Tools
 
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and ExcellenceTesting in the Year 2020: The Erosion of Governance, Management, and Excellence
Testing in the Year 2020: The Erosion of Governance, Management, and Excellence
 

Plus de Ruth Ellison

Rise of the makers web directions
Rise of the makers   web directionsRise of the makers   web directions
Rise of the makers web directionsRuth Ellison
 
Getting older? Tips & tricks for designing inclusive open source user interfaces
Getting older? Tips & tricks for designing inclusive open source user interfacesGetting older? Tips & tricks for designing inclusive open source user interfaces
Getting older? Tips & tricks for designing inclusive open source user interfacesRuth Ellison
 
Climate vis jewellery
Climate vis jewelleryClimate vis jewellery
Climate vis jewelleryRuth Ellison
 
Designing for cognitive disabilities
Designing for cognitive disabilitiesDesigning for cognitive disabilities
Designing for cognitive disabilitiesRuth Ellison
 
Graffiti: art or vandalism
Graffiti: art or vandalismGraffiti: art or vandalism
Graffiti: art or vandalismRuth Ellison
 
An intro to critical thinking, skepticism & quirkology
An intro to critical thinking, skepticism & quirkologyAn intro to critical thinking, skepticism & quirkology
An intro to critical thinking, skepticism & quirkologyRuth Ellison
 
Guerrilla User and Design Research
Guerrilla User and Design ResearchGuerrilla User and Design Research
Guerrilla User and Design ResearchRuth Ellison
 
Twitter: more than a microblogging tool
Twitter: more than a microblogging toolTwitter: more than a microblogging tool
Twitter: more than a microblogging toolRuth Ellison
 
The Uncanny Valley - UX Australia
The Uncanny Valley - UX AustraliaThe Uncanny Valley - UX Australia
The Uncanny Valley - UX AustraliaRuth Ellison
 
From Apples to eye surgery: Designing useful user experiences
From Apples to eye surgery: Designing useful user experiencesFrom Apples to eye surgery: Designing useful user experiences
From Apples to eye surgery: Designing useful user experiencesRuth Ellison
 
User experience in Government: Tales From Oz
User experience in Government: Tales From OzUser experience in Government: Tales From Oz
User experience in Government: Tales From OzRuth Ellison
 
Integrating Accessibility Into The Design Process
Integrating Accessibility Into The Design ProcessIntegrating Accessibility Into The Design Process
Integrating Accessibility Into The Design ProcessRuth Ellison
 
Usability for Government: improving service delivery
Usability for Government: improving service deliveryUsability for Government: improving service delivery
Usability for Government: improving service deliveryRuth Ellison
 
Usability for Government: improving service delivery
Usability for Government: improving service deliveryUsability for Government: improving service delivery
Usability for Government: improving service deliveryRuth Ellison
 
Ruth's User Experience Nightmares
Ruth's User Experience NightmaresRuth's User Experience Nightmares
Ruth's User Experience NightmaresRuth Ellison
 

Plus de Ruth Ellison (15)

Rise of the makers web directions
Rise of the makers   web directionsRise of the makers   web directions
Rise of the makers web directions
 
Getting older? Tips & tricks for designing inclusive open source user interfaces
Getting older? Tips & tricks for designing inclusive open source user interfacesGetting older? Tips & tricks for designing inclusive open source user interfaces
Getting older? Tips & tricks for designing inclusive open source user interfaces
 
Climate vis jewellery
Climate vis jewelleryClimate vis jewellery
Climate vis jewellery
 
Designing for cognitive disabilities
Designing for cognitive disabilitiesDesigning for cognitive disabilities
Designing for cognitive disabilities
 
Graffiti: art or vandalism
Graffiti: art or vandalismGraffiti: art or vandalism
Graffiti: art or vandalism
 
An intro to critical thinking, skepticism & quirkology
An intro to critical thinking, skepticism & quirkologyAn intro to critical thinking, skepticism & quirkology
An intro to critical thinking, skepticism & quirkology
 
Guerrilla User and Design Research
Guerrilla User and Design ResearchGuerrilla User and Design Research
Guerrilla User and Design Research
 
Twitter: more than a microblogging tool
Twitter: more than a microblogging toolTwitter: more than a microblogging tool
Twitter: more than a microblogging tool
 
The Uncanny Valley - UX Australia
The Uncanny Valley - UX AustraliaThe Uncanny Valley - UX Australia
The Uncanny Valley - UX Australia
 
From Apples to eye surgery: Designing useful user experiences
From Apples to eye surgery: Designing useful user experiencesFrom Apples to eye surgery: Designing useful user experiences
From Apples to eye surgery: Designing useful user experiences
 
User experience in Government: Tales From Oz
User experience in Government: Tales From OzUser experience in Government: Tales From Oz
User experience in Government: Tales From Oz
 
Integrating Accessibility Into The Design Process
Integrating Accessibility Into The Design ProcessIntegrating Accessibility Into The Design Process
Integrating Accessibility Into The Design Process
 
Usability for Government: improving service delivery
Usability for Government: improving service deliveryUsability for Government: improving service delivery
Usability for Government: improving service delivery
 
Usability for Government: improving service delivery
Usability for Government: improving service deliveryUsability for Government: improving service delivery
Usability for Government: improving service delivery
 
Ruth's User Experience Nightmares
Ruth's User Experience NightmaresRuth's User Experience Nightmares
Ruth's User Experience Nightmares
 

Dernier

Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 

Dernier (19)

Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 

The researcher’s blind spot: 6 cognitive biases we shouldn’t ignore in research

Notes de l'éditeur

  1. I was at the movies recently to watch Star Trek (don’t judge!). If you’re ever been tempted to buy popcorn (like I have, many times), it’s sometimes tricky trying to work out which size popcorn to buy.
  2. Sit down if you would buy the small popcorn for $3 Sit down if you would buy the medium popcorn for $6.50? Remain standing if you would buy the large popcorn for $7? Ok, you can now all sit down – thanks!
  3. When popcorn is presented with two options, most people will pick the cheaper option. Look, they just spent a fortune buying movie tickets!
  4. When presented in this way, more people purchased the large popcorn. So why is this the case? This is an example the decoy effect - where people will change their preferences when a third option is presented. The third option services as a decoy to increase a preference for a dominating option. The idea is simple - price one item in such a way that it makes the other price seems very reasonable.
  5. Here’s an example of a decoy effect from Telstra’s website - they’re clearly using the Medium and Large pack as the decoy.
  6. A cognitive bias is a mental shortcut. Cognitive biases are psychological tendencies that cause us to use a number of simplifying strategies and rules of thumb to ease the burden of mentally processing information to make judgements and decisions. These rules of thumb (or heuristics) are often really useful in helping us to deal with complexity and ambiguity. But in many instances, cognitive biases can lead to faulty judgements.
  7. There are cognitive biases lurking everywhere in the research process. But we are ALL HUMAN. We all want our research to provide reliable input into our projects and most of us wouldn’t deliberately distort data. Yet, we’re human, and we’re all susceptible to many cognitive biases that can affect the outcomes at any stage of our projects. Biases is unavoidable, but being a good researcher is about understanding our inherent biases and how we can minimise the effects. Failing to account for our cognitive biases in research activities can lead to participants being unintentionally and unknowingly influenced in to producing biased responses. Distorted or misleading results can be very detrimental to a project. It can misinform the direction of a project, or provide false confidence about decisions. What happens when research provides false confidence about a decision?
  8. Here’s a story about coke Coca-Cola Company lost millions of dollars due to a research mistake. In the mid-1980s, the Coca-Cola Company made a decision to introduce a new beverage product (Hartley, 1995, pp. 129–145). The company had evidence that taste was the single most important cause of Coke’s decline in the market share in the late 1970s and early 1980s. A new product dubbed “New Coke” was developed that was sweeter than the original-formula Coke. Almost 200,000 blind product taste tests were conducted in the United States, and more than one-half of the participants favoured New Coke over both the original formula and Pepsi. But participants were never told that this was to replace the original coke formula. The new product was introduced and the original formula was withdrawn from the market. This turned out to be a big mistake! Eventually, the company reintroduced the original formula as Coke Classic and tried to market the two products simultaneously. Ultimately, New Coke was withdrawn from the market.
  9. We’re going to be looking at a number of biases that affect both us at each major stage of design research
  10. Quite a number of years ago, I was helping to organise a research session for a famous institution in Canberra. We wanted to evaluate the effectiveness of the online catalog that was used to find items in the collections. The team had a budget constraint (sounds familiar?) and couldn’t use a recruiter to find research participants. They also had a short amount of time. We ended up running a survey on the website and asking for research participants from the survey. We got quite a number of participants from the survey and we use this to organise research sessions in Canberra where we learned a lot of interesting things about the website. But recruiting in this way is an example of a cognitive bias known as selection bias.
  11. There are a number of selection bias that can happen - one way is sampling bias, which is where a non-random sample of a population may be selected which can result in a biased sample. In my previous slide, the sampling bias occurred because we were relying on one method of recruiting - we were using an online survey and asking for people who would be happy to take part in our research. This resulted in getting people who were already familiar with using the online channel and also people who self-selected to take part in our research. While we still got interesting insights from our research about what was working and not working with our website, we had a lost opportunity to find out how the website served the needs of users who may not be using the web as a primary channel.
  12. Another example of selection bias to consider is time – when we do a research study at a point in time can affect the results, or may support a desired or expected outcome. I’m currently working on a project that is focussed on simplifying attendance processes for families, providers of child care services and government. We were running a discovery research piece earlier this year. At the start of the discovery phase, our initial families that we talked to were sharing about a big pain point for them – their child care rebate running out before the end of the financial year. This theme kept coming up in the first two weeks of the research but we discovered that this was a time related issue – this happens to certain families in situations where their children are in care for a significant part of each week, which can cause their rebate to run about from around February to June. As our research was conducted in that time frame, this was at the forefront of people’s minds. We had to be careful that results from people sampled during this period had to considered over the course of a year – not just what was at the forefront of their mind.
  13. Whilst these bias may not necessarily mean the results and analysis are wrong, it is important that these bias are recognised in analysis. Particular demographics may have different opinions that do not necessarily represent that of the general population. Ways to deal with selection biases include: Multiple channels - mix of recruitment companies, social media Avoid professional respondents – I once ran a workshop where the participants walked in and knew each other by name! These tend to be professional respondents with a goal to earn a part-time salary from focus groups and survey incentives. Careful screeners – use screeners to help weed out professional respondents and participants that are just not appropriate. Behavioural based recruitment – when I started doing research years ago, I would recruit based on a set of well researched and well established demographics that various marketing and communications teams would provide. But years ago, an article featuring the lovely Dana Chisnell, discussed why behaviour/motivation based research.
  14. It was a number of years ago at the start of research career. I was listening to this parent tell me about the trials and tribulations of parenting (she wasn’t selling parenting to me!). We were doing a one on one interview behind a one way mirror to explore her interactions with a particular parenting paper based claim. This mum had never been in a research session before so wasn’t sure what to expect. Anyhow, she was telling me about the time she was trying to filling in the form and I had the form in front of us. We knew that there was one particular part of the form that was quite problematic (but another team wasn’t convinced that that area of the form was the problem). As she started moving her way down the form, to the bit that we were unsure about, I started to subconsciously lean in towards her. She started to slow down and started talking about this particular area of the form. I kept nodding my head and smiling. I occasionally injected, ‘oh that’s interesting’.
  15. The observer expectancy effect, also called the experimenter expectancy effect, is a phenomenon that can occur when a researcher’s beliefs or expectations cause him or her to uncon­sciously influ­ence the research par­tic­i­pants. Think subconscious “uh huh”, head nods and smiling when the participant is going down the “right path”. Or scribbling down notes and having the participants say “Have I done something wrong?”
  16. In another study (Loftus & Palmer, 1974), subjects saw films of automobile accidents and then answered questions about the accidents. The wording of a question was shown to affect a numerical estimate.
  17. In particular, the question, “About how fast were the cars going when they smashed into each other?” consistently elicited a higher estimate of speed than when “smashed” was replaced by “collided,” “bumped,” “contacted,” or “hit.”
  18. I’m going to give you a three number sequence. I have a rule in mind that these 3 numbers obey. I want you to try to figure out this rule. You can find this out by suggesting your own three numbers and I’ll say yes it follows the rule, or no it doesn’t. Keeping suggesting your own three numbers until you’re sure of the rule and then tell me the rule. Are we ready?
  19. This experiment is part of the Wason’s Rule Discovery Test. When most people try this experiment, we form a hypothesis and then try a number of sequences that confirm the hypothesis. Very few people tried a number sequence that might DISPROVE their hypothesis. The Wason’s Rule Discovery Test proves that most people do not try to test their hypotheses critically, but rather to confirm their hypothesis.
  20. This mental shortcut is known as confirmation bias. Confirmation bias is considered one of the most dangerous biases. This is the tendency to search for or interpret information in a way that confirms your beliefs. Common factors such as internal politics, personal goals or simply lack of knowledge can turn into a cherry picking exercise, where researchers or our stakeholders, may consider some results and ignore others.
  21. Leaves you open to new evidence It’s not about proving your hypothesis – it’s about using null hypothesis. The null hypothesis (H0) is a hypothesis which the researcher tries to disprove, reject or nullify. #YayScience
  22. Analysis of Competing Hypotheses (ACH). Developed by Richards Heuer, it consists of a matrix with possible hypotheses (or scenarios) across the top, with each individual piece of evidence or information going down the side. This can be done with 10 pieces of information or hundreds. The most important point is that each piece of evidence must be evaluated individually against each hypothesis, and market as “consistent,” “neutral,” or “inconsistent.” By going “across” rather than “down” the matrix, it helps analysts to think critically about each piece of evidence, as opposed to thinking about each hypothesis. This method can generate a numeric score by tallying “inconsistent” ratings, but the analyst needs to be able to make a judgement call regarding the importance of each piece of information. Although it can be applied to quantitative and qualitative research, this method itself is NOT a quantitative method. Ultimately, each hypothesis is evaluated by how much “inconsistent” evidence is in its column, and this is compared to the analyst’s prior judgments. This method is best used to identify hypotheses which are problematic, not to diagnose a hypothesis which is most likely. Source: https://en.wikipedia.org/wiki/Analysis_of_competing_hypotheses
  23. Accept that we all have assumptions
  24. Groupthink and the bandwagon effect refers to the tendency to do (or believe) things because many other people do (or believe) the same. With Groupthink, group members try to minimise conflict and reach a consensus decision without critical evaluation of alternative viewpoints by actively suppressing dissenting viewpoints Source: https://en.wikipedia.org/wiki/Groupthink, https://en.wikipedia.org/wiki/Bandwagon_effect
  25. https://youtu.be/MDD4IkVZWTM?t=52s
  26. Focus groups feel like such a quick easy way to engage with large number of participants in a short amount of time (which is why it’s a favourite of so many consultancies), but as design researchers, if you’re going to use focus groups, make sure you use a mixture of research methods. My favourite is observational methods. If you’re having to run a focus group or any kind of group exercise, use a mix of individual and group exercises. Particularly start with the individual exercises, then do a playback with the bigger group.
  27. The anchoring bias refers to the tendency to rely too heavily, or "anchor", on one trait or piece of information when making decisions (usually the first piece of information acquired on that subject).
  28. Leather jacket – price is $1000 Sales person says that it’s on sale for $400!
  29. Anchoring particularly comes into effect in UX research when more than one version of a product or concept is shown to research participants, such as in A/B testing. Participants may often be more inclined to prefer the first version they are shown, which can skew the research results significantly if efforts are not made to prevent this bias. To further complicate the issue, we are often inclined to select a ‘middle’ option, or have a central bias, if there are many options presented to us in any vertical or horizontal order. Read more at: https://youarenotsosmart.com/2010/07/27/anchoring-effect/
  30. Ask general questions before specific questions unaided before aided questions positive questions before negative questions behaviour questions before attitude questions Ordering your topics, questions and activities needs some judgment. Ask yourself if the order sequence causes bias. Change the sequence. See what makes sense. Typically, anchoring can be prevented by alternating the order in which participants are shown versions.
  31. As humans, we’re pretty good at identifying patterns. Finding patterns helps us make sense out of the world. This tendency is so automatic that sometimes, our brains are just too good and they find patterns that aren’t there. In cognitive science, this tendency to perceive meaningful patterns with random data is called apophenia. One from of apophenia is finding images or sounds in random stimuli. This is known as pareidolia. Read more: http://bias123.com/clustering_illusion
  32. Does anyone see a pattern in these pieces of toast? On a side note, the ‘Virgin Mary’ toast on the right sold for $28,000 back in 2004!
  33. This is known as the clustering illusion which is the tendency to wrongly interpret clusters or streaks of data in small sample size to be statistically significant
  34. Collaborative analysis sessions – also with people who wasn’t involved in the research – fresh eyes, still have some understanding of the user
  35. So at this point, you’re probably thinking, ok, we’re had a taster of some of the cognitive biases that exist. We should be ok as we know what kind of biases we have so we don’t need to counter for them. But we’re not any less prone to bias than anyone else.
  36. The answer is fairly obvious – 50% of people are more biased than the average, and as you expect, 50% are less biased than the average.
  37. In a study conducted by three social psychologists at Stanford University, they asked more than 600 residents of the US this question. More than 85% of the sample believed that they were less biased than the average American. Only one participant believed that they were more biased than an average American.
  38. This is known as the blind spot bias. Our ability to perceive bias in others is actually pretty good. Our ability to perceive bias in ourselves is generally pretty bad. Society teaches us that bias is a bad thing. The word has negative connotations. Most of us would prefer not to see ourselves as the kind of people who do bad things. This leads us to believe that we must be rational. That our actions and judgements are accurate and therefore without bias. However, it doesn’t stop us from seeing the flaws in others because of their biases.
  39. Our biases are part of what makes us human.