SlideShare une entreprise Scribd logo
1  sur  160
Télécharger pour lire hors ligne
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
Intro to Focus Groups
Emily Geisen
Amanda Wilmot
3/25/2014
RTI International
Ground Rules
Cell phones turned to vibrate or off
Refreshments and restrooms any time
Several bio-breaks throughout
Questions are encouraged at end of each section
RTI International
Course Outline
1. The role and use of focus groups
2. Focus group demonstration
3. Developing the protocol guide
4. Moderating focus groups
5. Recruiting and scheduling
6. Participant selection
7. Data management/analysis
8. Reporting findings
9. Drawing wider inferences
10. Qualitative Research Ethics
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
1. The Role and Use of Focus Groups
RTI International
What is a Focus Group?
“Focus group interviews typically have five characteristics
or features: (a) people, who (b) possess certain
characteristics, (c) provide data (d) of a qualitative nature
(e) in a focused discussion.”
-Focus Groups: A Practical Guide for Applied Research (Krueger)
RTI International
Examples
A group of coal miners sit around a small conference
room discussing mine safety. They all agree that safety
is important, but some argue that existing safety
regulations are too unrealistic or impractical to be
implemented.
A group of employees sit in a conference room
discussing access to email and other work functions
from mobile devices. One participant says answering
emails on his phone will save him time. Another
participant worries that her supervisor will expect her to
reply to emails at all hours if she has access on her
phone.
RTI International
What is a Focus Group, again?
Researchers have a topic they want to know about:
– Why aren’t coal miners following safety precautions?
– What concerns do employees have about mobile email access?
These topics are communicated to a group of
participants to discuss
– Pre-scripted questions to generate discussion
– Targeted moderation to focus the discussion
Researchers summarize and interpret what they have
learned
RTI International
Uses of Focus Groups
Exploration and Discovery
– Collect info that will be used to guide development of a survey
– Collect info for a needs assessment
– Test a new product before development
Context and Depth
– Adding new topics or population groups to a survey
– Delving deeper into a survey topic
– Testing a product with a new audience
Interpretation
– Deciding how to use survey results
– Discuss applications with end users
-Focus Group Guidebook (Morgan)
RTI International
Uses of Focus Groups*, Continued
Stand-alone data collection method, or
Supplement other qualitative or quantitative
research methods
– In-depth personal interviews
– Survey development
– Needs assessment
– Product development
RTI International
Focus Groups for Survey Research
Identify topics/themes that are of interest to population
– Use this to add questions on these topic/themes
– Cut questions that are not relevant, not applicable, or show no
variation
Get more detail on a topic so that you don’t have to ask
open-ended questions
– Which of the following are reasons why you do not have health
insurance? Too expensive, No health problems, …
Get reaction to survey materials such as advanced
letters, envelopes, etc.
Explore methods to boost cooperation: what type of
incentives would people prefer?
RTI International
Strengths of Focus Groups (over other
qual./quan. methods)
If you’re unfamiliar with a topic, you can learn a lot
quickly (often used as the first step in research)
Can be used to observe interactions or group dynamics
Generate hypotheses (that can be tested using
quantitative methods)
Cheaper/quicker than other methods
RTI International
Limitations of Focus Groups
Some topics are too sensitive or controversial to be
discussed in a group setting (social desirability)
Discussions can get easily sidetracked/derailed
Limited generalizability of findings/results
Cannot quantify findings or determine statistical
significance
RTI International
Thinking about your research…
How are you going to use focus groups?
– Exploration and discovery? How so?
– Context and depth? How so?
– Interpretation? How so?
Will your focus groups be stand-alone or will you use
them with other research methods?
What are the strengths/weaknesses of using focus
groups for your research?
RTI International
Questions/Discussion
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
2. Focus Group Demonstration
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
3. Developing the Protocol Guide
RTI International
The What, Why, Who, How, Where
What Review research objectives
Who Determine number and type of participants
Recruit and schedule participants
Where Determine test location and equipment
Why Identify major topic areas of interest
How Script focus group questions
Develop focus group guide
RTI International
The What, Why, Who, How, Where
What Review research objectives
Who Determine number and type of participants
Recruit and schedule participants
Where Determine test location and equipment
Why Identify major topic areas of interest
How Script focus group questions
Develop protocol guide
RTI International
What’s the Focus Group About: Review
Research Objectives
Start with the overall research objectives / study goals
Identify what do you already know about the topic
Determine what new information you want to learn
Determine what information you want to explore in more
detail
Decide how are you going to use this information
RTI International
Review Research Objectives: Example
Step Personal Health Records Example
Study goals: Learn about healthcare consumers
understanding, experiences, needs, and
concerns about the ways technology can
be used within the healthcare system
What do you already
know about the topic:
Research has shown that use of electronic
PHRs can save consumers money and
improve health, yet they are not being used
extensively.
What do we want to learn
through focus groups:
Why aren’t more consumers using
electronic PHRs to manage their health
information?
How will we use this info: Provide recommendations for improving
PHRs and promoting awareness.
RTI International
Review Research Objectives: Your Turn
Step Your Study
Study goals:
What do you already know
about the topic:
What do we want to learn
through focus groups:
How will we use this info:
RTI International
Why: Identify Major Discussion Topics
Once you identify what you want to learn, identify major
discussion topics (brainstorm)
Why are we doing these focus groups:
– What is the most important question your research must answer?
– What’s the next most important question?
– And so on …
Consult with project team members as needed
Consider how topic areas may need to differ by
participant selection
RTI International
Identify Major Discussion Topics: Example
Why aren’t more consumers using electronic PHRs to
manage their health information?
Consumers who have not used electronic PHRs
How familiar are consumers with electronic PHRs?
How do consumers currently manage their health
information?
What are the most important factors consumers consider
when deciding how to manage their health information
What are their biggest concerns/fears about using PHRs?
Narrow list of discussion topics to about 4 major topic
questions per hour (more if questions are more specific)
RTI International
How: Scripting Questions
Avoid yes/no or short-answer questions, unless:
Open-ended follow up is planned (e.g., why/why not?)
Used to tally group (e.g., those in favor/opposed)
Open-ended questions generate discussion, however:
– Keep questions focused, one-dimensional
– Be careful of “Why?” questions
– Use scripted probes to focus or narrow topics
Questions should be easily comprehensible: use familiar
words/terms
Questions should be reflective, not hypothetical
Move from general to more specific
RTI International
Scripting Questions: Example 1
Topic: How do consumers currently manage their health
information?
Bad: How do you manage your family’s health
information?
Good: How do you currently maintain or store your and
your family’s medical records, such as records of
vaccinations or prescriptions?
– Do you keep paper or electronic records?
– How does this differ for you compared to your children (or elderly
parents)?
RTI International
Scripting Questions: Example 2
Bad: Why do you use the mobile device you
have?
Better: What kind of mobile device do you use
at work?
Best: How did you decide what kind of mobile
device to use at work?
RTI International
Other aspects of Protocol Guide
Welcome/Introduction
Icebreaker
Opening (or Engagement) questions:
– Easy and inviting, comfortable to discuss
– Introduces the topic of discussion, provides context/background
Exploration questions (main study questions)
Exit questions:
– Used to wrap-up the discussion
– Check to see if there is anything else people would like to
mention that they did not get a chance to
Pre/Post Questionnaires
RTI International
Welcome/Introduction
Welcome
– Introduce yourself, note-taker AND people behind the glass
– Make participants feel comfortable
Informed consent (if required)
– Read or summarize consent, ask participants to sign
– Note audio-recording if applicable
– Explain confidentiality
Go over ground rules
– Refreshments, restrooms, cell phones off or on vibrate
– No right or wrong answers
– Like to hear from everyone
RTI International
Icebreaker
Sometimes it’s helpful to start with an ice-breaker
Ask participant to provide their name (first name only)
Have them answer an easy question
– Participants should be able to answer it briefly, quickly
– Should not be sensitive
– Can be related to focus group topic or unrelated
Example
– Employee focus group: How long at company?
Department/Division?
– Focus group of moms: How many children and their ages
RTI International
Intro Questions
Introduce Topic
– Goal/Purpose of the study
– Why participants were selected (if not obvious)
Provide any needed context
– For focus group on Personal Health Records, we had to explain
what PHRs were
– Show participants related materials or products
– Provide background
Opening Question
– Should be easy/simple to answer
– Non-sensitive
RTI International
Pre/Post Questionnaires
Pre questionnaire:
– Use to collect information about your participants without wasting
valuable time during the focus group
– Use to determine how familiar/knowledgeable participants are
with topic at start of focus group
Post questionnaire:
– Can be same as pre-questionnaire to monitor changes in opinion
or knowledge
– Can be a handy way to summarize participants’ opinions on
discussion topics
Keep questionnaires short (5 minutes)
RTI International
Putting it all together
Allow time for welcome, explaining the purpose of the
study, and consent procedures (if required)
– At least 5 minutes
Allow time for introductory or ice-breaker question
– At least 5 minutes
Time for your main discussion questions
– About 4 question topics (per hour)
– 10-15 minutes per question
Exit/Wrap-up
– At least 5 minutes
RTI International
Determining Question Flow
What order would you put these questions in?
What kind of device would you prefer to use and why?
What factors did you consider when determining what kind of
device to use?
What are the biggest limitations or barriers to using a mobile
device for the work you do?
What are the benefits of using a mobile device for the work
you do?
Tell me about the mobile device you use at work. What kind of
device is it?
What are the main reasons that you use a mobile device for
work?
RTI International
Question Flow
Tell me about the mobile device you use at work. What
kind of device is it?
– What factors did you consider when determining what kind of
device to use?
– What kind of device would you prefer to use and why?
What are the main reasons that you use a mobile device
for work?
– What are the benefits of using a mobile device for the work you
do?
– What are the biggest limitations or barriers to using a mobile
device for the work you do?
RTI International
Protocol Guide Examples: Handouts
Handout 1: Mine Contractors Protocol Guide
Handout 2: Healthcare Consumers Protocol Guide
Handout 3: Tobacco Panel Protocol Guide
Handout 4: Post questionnaire
RTI International
Individual Practice (15 minutes)
Use your study or class example
Identify 1-2 major discussion topics for your study
Prepare an introductory question
Prepare 1-2 discussion questions with probes as needed
RTI International
Questions/Discussion
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
4. Moderating Focus Groups
RTI International
Understanding the group Process
Model of group phases
(Ritchie and Lewis 2003)
Forming
Testing and dependence
Dependence on the leader
Storming
Intragroup conflict
Criticism
Norming
Development of group cohesion
Optimism
Performing
Functional role relatedness
Cohesiveness
Adjourning (Mourning)
Termination, Separation
RTI International
The Ideal Moderator:
Has adequate knowledge of topic
Has good communication skills
Is similar to / can build a rapport with participants
Can maintain balanced viewpoint
Is prepared for surprises
Reports accurately, even if it’s uncomfortable
Has Experience with:
– group dynamics
– neutral probing/feedback
RTI International
What the Moderator Does
Welcome/Introductions
Leads discussion
Concludes discussion
RTI International
Welcome/Introduction
Make participants feel comfortable
Read or paraphrase this welcome/introduction in
protocol guide
Make sure to collect informed consent
Notify participants of observers and/or audio recording
Go over any ground rules
Let participants get refreshments and get situated before
starting actual focus group discussion
RTI International
Leads Discussion
Provides unobtrusive control
Manages group dynamics
Asks spontaneous probes as needed to gather more
information
Provides unbiased feedback
Summarize/verify information
Manages time
RTI International
Provide Unobtrusive Control
Know when to stop talking
– Don’t constantly interject, let participants discuss back and forth
Learn to focus the discussion
– You said X, can you tell me more about that?
– Note topics said, that you want to circle around to
Manage the flow of the discussion
– Don’t have to follow script, allow natural segues
– If a topic you plan to cover later is brought up, indicate that you’ll
talk more about that later.
Encourage different point of views
– Has anyone had a different experience?
– Does anyone view it differently?
RTI International
Tips for Managing Group Dynamics
Dominant talkers:
– Don’t look at them when you ask a question.
– (Politely) cut them off: “Thank you Amanda. Does anyone else
have an experience they’d like to share?”
Ramblers/Off-topic:
– Avoid eye contact after so long off topic, look down or at clock
– Be prepared to interject at next pause
Shy participants:
– Maximize eye contact
– Call by name: I’ve heard from some of you on this and I want to
hear from the rest of you. Sarah, what is your experience with X?
RTI International
Tips on Spontaneous Probing
Leading probes:
– So you had a hard time with that then?
– You didn’t want to learn more about that?
– Was that difficult for you?
Neutral probing:
– Can you tell me more about that?
– Would you explain that further?
– Can you give me an example?
– Is there anything else?
RTI International
When you hear yourself asking a leading
question, balance it
47
Leading
question
“So you think
that’s difficult
then?”
Balanced
question
“...or was it
easy?”
RTI International
Tips on Neutral Feedback
Monitor your reaction to participants/discussion
Be careful of nonverbal cues (leaning back, crossed
arms)
Limit head nodding: shows you’re listening, but also
implies agreement
Provide quick verbal responses periodically (uh-huh,
yes, mmm, okay, go on)
Avoid loaded feedback (which implies agreement)
– Loaded feedback: “That’s good!” “Excellent.”
– Neutral feedback: “That’s helpful” “That’s interesting.”
RTI International
Summarize/Verify Information
As needed, summarize/paraphrase what has been said
– I understand that work-life balance is your most important
concern.
– Safety is critical, but some of the precautions you’re asked to
follow are pointless or unrealistic. Is that correct?
See if anything has been missed?
– Did I miss anything?
– Does anyone have anything else to add?
Probe on any subtopics you didn’t cover
– We talked about X, but I’m also interested in Y
Move on to next topic
RTI International
Conclusion / Wrap-up
Be sure to leave time for wrap-up, especially for
sensitive topics or intense discussions
Ask participants to identify most important issues
– Of everything we have discussed, what is the most important?
– If you could make one request to X, what would it be?
Summarize major themes (ask note-taker for help)
– Ask if its an adequate summary
– Ask if anything has been missed, anything to add
Check with observers to see if they have any
questions/clarifications
Pay participants!
RTI International
Dealing with Difficult Situations
Too many people show up
Participants bring their children
Only a few attend
Group is reluctant to talk
Can’t get them to stop talking
RTI International
Plan Your Response…
20 minutes into the focus group, and Sarah has not said
anything…
You can’t get Joe to stop talking…
John mentions something interesting, but you don’t want
to interrupt because there’s a good discussion going…
RTI International
Role of Note-Taker
Can Handle logistics & refreshments
Collects signed informed consent (if required)
Takes careful notes
Does not participate in discussion
Can recap major themes at end of discussion (used
before wrap-up question)
Monitors recording equipment
Liaison between moderator and observers/clients
Debriefs with moderator after session
Assist with analysis and reports
RTI International
Role of Co-Moderator
Not required, but can be useful in some situations
Balance out strengths/weaknesses in moderator
Use to match moderator (without being obvious)
Switch leading focus group (good for long or intense
focus groups)
Support leader by keeping on track, recapping major
themes, etc.
RTI International
Moderating style
Different styles for different people
– Spontaneous flow
– Follow guide strictly
Adapt to your style, but plan ahead
– Spontaneous: Be sure to note what you’ve covered and what you
haven’t so that you don’t forget anything.
– Strict: “That’s a very interesting point, and we’re going to get to
that in a minute, but right now I want to focus on X:
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
Group Practice Moderating (30 min)
RTI International
Questions/Discussion
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
5. Recruiting and Scheduling
(review on your own)
RTI International
At Least a Month Before Focus Group
Decide on what you’re testing
Decide on the participants (see Section 6) :
– Who you want to recruit
– How many
– How you will find them
Plan the test date
– Find a date when your stakeholders can observe
– Decide on your location
RTI International
How to Find Your Participants
Frame/list if available
Advertisements: Craigslist, flyers, newspapers
– Pros: quick, easy, cheap
– Cons: yields younger, higher educated users
Participant databases/recruiting orgs
– Pros: quick and easy, pay per user recruited
– Cons: can lead to “professional respondents”
Other sources: word of mouth; sample lists; clubs,
churches, and societies; snowball recruiting
RTI International
What to Pay Participants?
Are participants allowed to accept monetary incentives?
– Federal employees
– Participants who are being paid by their employer
Is it appropriate for your organization to pay participants?
Is topic interesting/important enough that incentives are
not necessary
How much should you pay them?
– Trade-off between incentive amount and recruitment effort
– Tailor amount to population and location
– Will participants need to pay for parking?
RTI International
Recruiting Tips
Recruit extras due to no-shows or cancellations
– Example: Recruit 12 to ensure at least 8 show up
Schedule sessions about 3-4 weeks ahead
– Any earlier and they may forget or make new plans
– Any later and you may not be able to recruit enough
participants
Send them an email or letter confirmation
Remind them the day before (by email or phone)
RTI International
Testing Location
At a minimum, use a large conference room (at your
organization’s offices, hotel conference room)
For numerous focus groups: formal observation rooms
are nice (see picture on next slide)
– One-way mirror so you can see them, but they can’t see you
– Microphones in ceiling to pick up voices, but not other sounds
(paper shuffling)
– Video-taping capability
Rent a focus group facility
– Usually very nice but can be expensive ($1,000 for 2-hour
session)
RTI International
Observation Room Example
RTI International
Create a Virtual Observation Room using
screen-sharing software
Examples: Skype, Go to Meeting
Fosters collaboration
– Can accommodate observers from any location
– Facilitate discussions in conference setting
Improved schedule
– Stakeholders get information immediately
– No waiting for recorded videos or report
Cheaper: Inexpensive compared to travel costs
RTI International
Note-Taking
Recommend having a note-taker and audio or video-
recording the session
Note-takers should be in the room or observation room
For audio-recording, consider getting additional
microphones so that you can hear everyone
Video-recording can be helpful, but makes some
participants uncomfortable
Participants usually forget they’re being recorded/
observed after 5-10 minutes
RTI International
At Least Three Weeks Before Focus Group
Get started on recruitment
Organize your incentives
Develop your focus group guide
RTI International
One Week Before the Focus Group
Final your focus group guide
Organize roles in the test:
– Meet and greet
– Observers/Stakeholders
– Moderator
– Note-takers
Do a practice run on any equipment
Arrange any refreshments
– At a minimum provide beverages
– Tailor food to participants / time of day
– Nothing messy or loud
RTI International
Prepare your materials
Develop consent forms, screeners
Instructions/directions for participants
Prepare any visual materials for participants
Pretest/posttest questionnaires
RTI International
The Day Before / Morning Of
Send out reminders:
– Phone or email to respondents
– Email to observers, stakeholders, note-takers
Equipment/Facility
– Make sure the room you’ll use is tidy
– Make sure your meet/greet person
has the final list of participants’ names
– Incentives are available
– Check any equipment (video/audio recording)
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
6. Participant Selection
RTI International
Purpose of the session
To understand the principles of participant selection for
focus group research
To consider the practicalities of participant selection for
focus group research
RTI International
Focus groups: a qualitative data collection method
Focus groups are a qualitative data collection method
When reporting the findings the rationale should be
provided for the research along with the methods used
Is the research…
– Justified
– Rigorous
– Systematic
– Transparent
RTI International
Sampling for qualitative research
The design of a sampling strategy for qualitative
research is as important as that for quantitative research
Qualitative research uses non-probability sampling
RTI International
Informing the design of a qualitative sampling strategy
What are the research objectives?
What is the scope of the research?
Who is out of scope and should be excluded from the sample?
Who is in scope and should be included in the sample?
What is the budget?
What is the reporting time period?
What sampling technique will be employed?
How are the data to be analyzed?
What data collection methods should be employed?
What are the sample criteria?
What size should the sample be?
What will be used as the sampling frame?
How are potential respondents/participants recruited?
RTI International
Sampling technique
Quantitative sampling
Probability sampling
Members of the research population are chosen at random and have a
known probability of selection
The aim is to produce a statistically representative sample
Qualitative sampling
Purposive non-probability sampling
The number of people interviewed is less important that the criteria used to
select them
Members of the research population are chosen on the basis of their
characteristics to reflect breadth and diversity of the research population
We do not aim to produce a statistically representative sample or draw
statistical inference
RTI International
Sample criteria
What characteristics will need to be reflected in the
sample population to ensure breadth and diversity?
Criteria used may be based on demographic
characteristics or behaviours or attitudes
Some criteria may be considered more important than
others
RTI International
Sample size
Small sample sizes for qualitative research
There is no need for scale because there is no need for
statistical inference
Sample size determinants:
Heterogeneous or homogenous nature of sample population
Number of selected criterion
Scale
10 to 50 for one-to-one investigation
40 to 100 for group interview
RTI International
Sampling frames
A sampling frame is a list that identifies units within the
target population
Frame evaluation:
Comprehensive
Sufficient numbers
Geographical dispersion
Respondent contact details correct
RTI International
Sampling frames
Existing frames
– Administrative sources
– Survey samples
Constructed frames
– Direct and/or indirect methods
RTI International
Constructed frames
Focussed enumeration
Snowballing
Screening questionnaires
Organisations
Advertisements
RTI International
Study about attitudes and behaviors surrounding
dental attendance
Selection criteria initially considered
Age to ensure demographic balance
Gender because patterns of attendance differ between men and women
Family Unit composition because attendance by others in the family might
influence the respondents attitudes or behaviours
Employment activity because attendance might be affected by time constraints
during working hours
Income as a known factor affecting dental attendance patterns
Regional location as dental attendance varies across the country
Ethnic origin as may influence attitudes or behaviours
Type of area as urban/rural location may affect attitudes and behaviours
Dental health to explore how attitudes vary among people with different dental
health
Current pattern of dental attendance (regular/irregular/occasional) for comparative
analysis
Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2003)
RTI International
Prioritised selection criteria
Primary criteria Secondary criteria
Dental attendance pattern Dental Health
Age Ethnic Origin
Gender Type of Area
Region
Family Unit
Employment activity
Income
Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2006)
RTI International
Sample Matrix
Area 1 of 6
Group 1 Group 2 Group 3
18-29 30-44 45+
4 men 4 men 4 men
1-2 regular attenders 1 regular attender 1-2 regular attenders
1-2 irregular attenders 1-2 irregular attenders 1 irregular attender
1 occasional attender 1-2 occasional attenders 1-2 occasional attenders
4 women 4 women 4 women
1-2 regular attenders 1 regular attender 1-2 regular attenders
1-2 irregular attenders 1-2 irregular attenders 1 irregular attender
1 occasional attender 1-2 occasional attenders 1-2 occasional attenders
Spread of family unit type and employment activity
Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2006)
RTI International
Any Questions?
RTI International
Excercise
Groups
Designing a sampling strategy for focus groups on the
use of electronic personal health records
15 mins
Feedback
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
7. Data Management and analysis
RTI International
Purpose of the session
To understand the principles of good qualitative focus
group analysis
To consider the practicalities of analysing qualitative
data
RTI International
Acknowledgement
Some of the slides contributed by the UK National
Centre for Social Research (NatCen).
The Framework methodology was developed by
Jane Ritchie and Liz Spencer in the 1980s at NatCen.
http://www.natcen.ac.uk/events-and-training/our-
training
RTI International
Qualitative data
Focus group recording
Focus group transcription
Notes
Admin data
Photographs, videos etc
RTI International
Transcription
Transcription is time consuming
The focus group transcription should capture the
discussion in it’s entirety
Focus groups are more difficult to transcribe than one-to-
one interviews as there are multiple participants to
distinguish between
It may be important for the analysis for the transcriber to
be able to identify each participant’s words. The
transcriber will use the notetaker’s notes to help
determine who is speaking
Check on quality of the transcription
Anonymize transcript
RTI International
Different approaches to data management, analysis and
reporting
Summary report
Qualitative analysis
RTI International
Nature of qualitative analysis
Quantitative research
– Fixed categories aiming to
enumerate
– Requires enumeration in
order to be meaningful
– Test hypotheses and
provide explanation
Qualitative research
- Creating categories
- Developing explanations
- Does not need enumerating
- Range and diversity key
RTI International
Aim of analysis - analytical outputs
Categories of things (thematic analysis)
Reasons for gambling
Sources of debt advice
Categories of people or processes (typologies)
Types of parents of children with learning difficulties
Types of welfare benefit claimants
Explanations of attitudes, choices or impacts
(explanatory analysis)
Factors influencing how people save for retirement
What helps achieve positive outcomes from an employment
programme
RTI International
The nature of qualitative data
The data are:
Voluminous
Unwieldy
Multi-stranded with overlapping categories and theme
RTI International
7.1 Data management
“Chaos is merely order waiting to be deciphered”
Jose Saramago
Introduction to Framework © NatCen Learning
RTI International
What is Framework?
Developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen
Social Research
Key characteristics
Approach to data management which facilitates case and theme
based analysis of qualitative data
Case and theme based approach
Matrix display
Reduces data through summarisation and synthesis
Retains links to original data
Output allows comprehensive and transparent data analysis
Introduction to Framework © NatCen Learning
RTI International
Objectives of data management
Primary objective
Re-order
Make data accessible
Secondary objective
Reduce, prioritize
Introduction to Framework © NatCen Learning
RTI International
Data management
• Data management involves ‘cutting’ up data
• Aim is to create useful ‘piles’ of data
• Three ways you can ‘cut’ data
Thematically
By case
By case and theme
Introduction to Framework © NatCen Learning
RTI International
Theme-based approach
Crime
Contact with
CJS
Impact
Introduction to Framework © NatCen Learning
RTI International
Case-based approach
Jim
Bob
Wendy
Robbery Contact
with police
Financial
impact
Introduction to Framework © NatCen Learning
RTI International
Framework is a case and theme based approach
Name Crime Contact with
CJS
Impacts
Jim
Bob
Wendy
Introduction to Framework © NatCen Learning
RTI International
The Framework approach
Introduction to Framework © NatCen Learning
RTI International
Stage 1: Data familiarization process
• Familiarize yourself with the data by reading
through transcripts and other documentation
• Immerse yourself in the data
• Begin to construct your analytical framework
Introduction to Framework © NatCen Learning
RTI International
Stage 2: Creating the ‘framework’
Multitude of potential descriptive categories
different kinds and levels of category
Choice of categories depends on
interview guide
research questions
emergent themes
recurrence across cases
Data management ≠≠≠≠ interpretation
RTI International
Frame construction: individual data
One chart represents one main theme
Chart each individual case
– Each individual as one row
– Subthemes in each column
– Give cases same space on each thematic chart
RTI International
Frame construction: group data
Chart as a whole group
– Each group as one case
– Several groups on each thematic chart
One group per page, tracing individual cases
– Order cases in the same way
– Give cases same space on each thematic chart
– Don’t be alarmed if every cell is not filled for each case
Guidelines for entering data
– May be useful to keep a record of group process
– Levels of participation, context in which comments made, how
views evolve/develop/change
Introduction to Framework © NatCen Learning
RTI International
Respondent
Employment
situation/
activity at time
of contact
Employment
history/work
activity in past
Hopes & aims
regarding
work before
contact
Barriers
perceived
in
achieving
aims
Efforts made to
overcome
barriers before
contacting
NDLP
Perception
of needs
before
contact
Overview of
financial
situation &
views about
at time
Other/Misc
# 1
# 2
# 3
# 4
A chart with sub-categories
Before contact with employment scheme
RTI International
Stage 3: Testing the analytic framework
Testing framework is important
– multi-stranded & overlapping categories
– different experiences
Test by coding or indexing
– apply numerical code to categories and sub-categories
– code using software or write codes in margin of
transcript
Test by piloting
– enter data from small number of transcripts
Checking for: overlap / gaps
RTI International
Data management coding / indexing
Example of analytical framework index for sexual identity project
1. Demographic details
2. Defining sexual identity
2.1 Conceptualization
2.2 Salience
2.3 Self categorization
2.4 Changes over time
3. Language used
3.1 Terminology used
3.2 Categories used
4. Acceptability
4.1 Acceptance by society
4.2 Understanding purpose of questioning
4.3 Trust in data security
4.4 Trust in data collector
5. Other
RTI International
Developing an analytical framework exercise
Groups to discuss analytic framework that could be used
for the morning focus group session
Create main themes and subthemes
Post-it notes
15 mins
Feedback – group spokesperson
Introduction to Framework © NatCen Learning
RTI International
Stage 4: Summarising data (‘charting’)
• Condensing data
• retains richness and flavour of data
• Process
• theme by theme (if indexed) or by
transcript
• Takes practice – an art
RTI International
RTI International
Introduction to Framework © NatCen Learning
RTI International
Introduction to Framework © NatCen Learning
RTI International
Introduction to Framework © NatCen Learning
RTI International
Entering data (‘charting’)
Process
theme by theme (only when indexed)
by transcript
Guidelines for entering data
summarise material from transcripts/other data sources
retain language
mark but don’t recite quotations
note page references / create links between summaries and
the original transcript within CAQDAS software
use agreed abbreviations/conventions
avoid repetition by cross referencing data
Introduction to Framework © NatCen Learning
RTI International
Data Management: Good practice
Reading the transcript beforehand
structure of account and nature of the data
repetition, clarification, contradiction
Reading chart afterwards
clarity
balance and emphasis
check blank cells
Use ‘other’ column for interpretative notes
note ‘flavor’ of discussion
note overarching issues
alert others to important issues
RTI International
7.2 Interpretation
“All meanings, we know, depend on the key of interpretation”
George Elliot
RTI International
Descriptive or explanatory analysis
Descriptive accounts
Explanatory accounts
Introduction to Framework © NatCen Learning
RTI International
There are three key steps…
Detection
familiarisation
extraction (highlight/ summarise)
Categorisation
creating meaningful conceptual boxes
assigning data
Classification
creating higher order categories
assessing relationships between categories
Introduction to Framework © NatCen Learning
RTI International
Categorization and classification
Lone parents orientation to the Labor Market
• Relevant dimensions
Attitudes to work
Barriers to work
Stage of job search activity
• Typology
Work not currently an option
Beginning to think about work
Personal issue barriers
Labor market barriers
Close to work
RTI International
Categorization and classification (2)
Classification of sexual identity
Relevant dimensions
– Conceptualization
– Salience
– Self-categorization
– Changes over time
Typology
– Latent identifiers
– Conscious identifiers
– Reluctant identifiers
RTI International
Associations and explanations
Linkages between:
– Two attitudes
– Attitude and behavior
– Circumstances and need
Verified through explanation
– Explicit respondent accounts
– Implicit identified by the researcher
RTI International
Making explanations
Informed by
– Hunches and hypothesis
– Reflections during field work and analysis
– Other research or theories
Involves
– Detailed within case analysis
– Comparison between cases
– Repeated interrogation of data
Comprehensive
Expect multiplicity
RTI International
Levels of classification and interpretation
In summary:
Descriptive categories
– Factors, reasons, impacts
Classifications or typologies
Explained associations
Meaning
Theory/Strategy generation
RTI International
Any Questions?
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
8. Reporting the Findings
RTI International
Purpose of session
To provide a basic overview of the principles involved in
reporting qualitative findings and constructing a final
report.
RTI International
Aims of the final report
To complete and document the research process
To inform key stakeholders of the findings
To communicate the findings in a clear and coherent
way
To guide readers in the interpretation of findings (avoid
misinterpretations)
RTI International
Basic principles for writing
Tell the story
Structure the report
Know your reader
Write in plain English
Use visual representation
RTI International
Reporting the Findings
Revisit the objectives (ensure the report covers these)
Consider the audience
Check report format
Consider time available
Consider length of the report (focus on relevant points
only)
Report findings in past tense
RTI International
Structuring the report
Title
Contents
Executive summary
Introduction / Background
Methodology
Findings
Conclusions
Recommendations
Appendices e.g. sample matrix, interview guide
RTI International
Nature of the reporting
Descriptive
Explanatory
RTI International
Defining the boundaries
Research rationale
Methodology
– Sampling strategy
– Data collection
– Method of analysis
– What can or cannot be inferred form the findings
Examples of relevant documentation
– Interview guide
– Contact letter
– Screening questionnaire
– Sample matrix
Substantiate conclusions by grounding the data in the findings
– Examples
– Quotations
– Case illustrations
RTI International
Using Quotations
Avoid over-use
Avoid under-use
Present range and balance
Edit sparingly
Should provide illustration only
Amplify but do not repeat a point
Avoid reliance on a few articulate respondents
Avoid very long quotations
Identify relevant characteristics
Preserve participant confidentiality
RTI International
Quotation examples
“I don’t feel you have to be out in all contexts…” [Gay
man, 35-44]
“The Government uses statistics to show what it wants to
show, for example unemployment, they’ve used all sorts
of different measures for employment to show it’s going
down…rather than it necessarily going down itself.”
[MHE Wales]
“They don’t XXX care about us. It’s everyone for
themselves” [Male, 18-24, Central London]
RTI International
Quotation examples
“I think it depends quite a lot on the way you live and
how old you are. For example, I was born down [South
West county], and my parents being extremely religious,
have very definite views on sexuality.” [Bisexual woman,
aged 35-44]
“I’ve lived in [Muslim country], nobody would ever admit
that [being gay], so they all choose to be heterosexual.
Somebody’s not been here that long, it’s even more of
an issue.” [Heterosexual woman, aged 35-44]
RTI International
Is a quotation necessary?
The actual presence of the question was considered to
have an important purpose in its own right. It was
thought that as the questioning became more
commonplace it would demonstrate to the wider public
that non-heterosexuality was unremarkable.
“It’s having it on there, it’s beginning to usualise the
question, it’s a cultural shift, it’s enabling people to
gradually recognise it’s no bloody big deal. But if we
don’t have it there, we are setting up this whole process
of we must be quiet about these people and all the rest
of it.” [Gay/lesbian woman, 55+]
RTI International
Avoiding quantitative language
Avoid tendency to use numbers
Do not discuss qualitative results in terms of proportions,
percentages or statistics
Indicate strength of finding by discussing in terms of:
– A common finding, A recurring problem, Problems were
observed…. etc
Use diagrams for illustration
RTI International
Principles to remember
Present balanced findings
Present grounded conclusions (avoid assumptions)
Distinguish between findings and your own conclusions
Provide commentary alongside the analysis (do not
leave the reader to do this for you)
Support with evidence from other research findings (if
appropriate)
RTI International
Editing and reviewing
Allow some time before editing (if possible)
Review the content
Check accuracy of statements
Peer Review
Proof read
RTI International
Any Questions?
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
9. Drawing Wider Inferences
RTI International
Purpose of the session
To understand whether wider inference can be made
when using qualitative data
RTI International
Concerns
Small samples
Not statistically representative
Interviews/discussions not standardized
Proximity of researcher / too personal
Biased reporting
RTI International
Generalization
Can qualitative findings be generalized beyond the
sample and context of the study research?
Different definitions of ‘generalization’.
– Representational generalization.
RTI International
Representational generalization
Meaning
Inference to parent or sampled population
Whether range and diversity (experiences, explanations
etc) can be matched with the sampled population
Whether list (of experiences, explanations etc) is
inclusive of what would be found in the sampled
population
Basis
Depends on validity and reliability of findings
RTI International
Validity
Validity: Accuracy of representation
Has the researcher understood the issues
from the respondent’s perspective?
Have the issues been fully articulated and explained
Are interpretations underpinned by the data
Alternative terms: Credibility and Plausibility
RTI International
Reliability
Reliability: Replicability
Internal reliability
– Extent to which assessments and judgments are replicated
between researchers
External reliability
– Extent to which findings would be replicated
if the study were repeated with the same
or different sample
Alternative terms: Confirmability, Consistency,
Dependability
RTI International
Drawing wider inference
Depends on:
Quality of sampling
– Reflects diversity of sample
population
– Includes all key constituents
– Includes outliers and a-typical
cases
Quality of data collection
– Free from interference: neutral
and objective
– Probing of meaning
– Exploration of all relevant
explanations
Quality of analysis
– Systematic, comprehensive and
inclusive
– Within and between case
analysis
– Displays diversity
Quality of interpretation and reporting
– Comprehensive
– Clear
– Underpinned by data
– Displays multiplicity of accounts
and explanations
– Non-quantitative
Documentation
– Transparency
– Research process
RTI International
Validation of the inference
Validation
– Comparison and check of fit
– Deviant cases analysis
– Member or respondent validation
– Triangulation
RTI International
In summary
Validity: do the data and findings accurately reflect the
behavior , views etc of respondents?
– Content validity: the respondent’s expression or articulation of the
issue
– Validity of interpretation: the researchers interpretation of the
issue
Reliability: would the data and findings be replicated if
the study were repeated?
Representational generalization: are the data and
findings applicable to the wider population from which
the sample is drawn?
RTI International
Any Questions?
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
10. Qualitative Research Ethics
RTI International
Qualitative Research Ethics
Check the legal regulations for the State / Country
Check your organisations Institutional Review Board
(IRB) policies
Understanding “informed consent” (verbal/written)
Understanding “confidentiality”
Basic principles of good social research
– Participant safety (Physical/Emotional/Psychological)
– Researcher safety
Recompense
RTI International
RTI International is a trade name of Research Triangle Institute. www.rti.org
References and Reading List
RTI International
Reading List
Quality in Qualitative Evaluation: A Framework for assessing research evidence. A
quality framework. Produced on behalf of the Cabinet Office by Liz Spencer, Jane
Ritchie, Jane Lewis and Lucy Dillon, National Centre for Social Research. ISBN:
07715 04465 8. August 2003. Government Chief Social Researcher’s Office Crown
Copyright 2003.
Assessing the Quality of Qualitative Research. Patient Education and Counseling 90
(2013) 1–3.
Qualitative Research Practice: A guide for Social Science Students and Researchers.
Edited by Jane Ritchie, Jane Lewis, Carol NcNaughton Nicholls and Rachel Ormston.
SAGE 2014.
Focus Groups: Theory and Practice. D. W. Stewart and P.N. Shamdasani. Applied
Social Research Methods, Volume 20.
Focus Groups as Qualitative Research. David Morgan. Sage 1989.
Focus Groups: A Practical Guide for Applied Research. Richard Krueger. Sage 1989.
Sampling and Choosing Cases in Qualitative Research: a realistic approach. Nick
Emmel. SAGE 2013.
What Are Qualitative Research Ethics? Rose Wiles. Bloomsbury Academic 2013.
RTI International
Other Resources
http://www.eiu.edu/~ihec/Krueger-
FocusGroupInterviews.pdf
http://www.tgci.com/magazine/How%20to%20Conduct%
20a%20Focus%20Group.pdf
http://managementhelp.org/businessresearch/focus-
groups.htm
http://assessment.aas.duke.edu/documents/How_to_Co
nduct_a_Focus_Group.pdf
http://www.rowan.edu/colleges/chss/facultystaff/focusgro
uptoolkit.pdf
RTI International
Final Questions and Discussion

Contenu connexe

Similaire à Odum focus groupcourse_mar252014

Introduction to Focus Groups, Odum Institute, October 30
Introduction to Focus Groups, Odum Institute, October 30Introduction to Focus Groups, Odum Institute, October 30
Introduction to Focus Groups, Odum Institute, October 30egeisen
 
Marketing research exploratory research using qualitative and observation me...
Marketing research  exploratory research using qualitative and observation me...Marketing research  exploratory research using qualitative and observation me...
Marketing research exploratory research using qualitative and observation me...Stevanus Handoko, S.Kom., MM
 
Formulation of the research probleme Dr. SC sharma
Formulation of the research probleme Dr. SC sharmaFormulation of the research probleme Dr. SC sharma
Formulation of the research probleme Dr. SC sharmaDr Kirpa Ram Jangra
 
Questionnaire and its Types
Questionnaire and its Types Questionnaire and its Types
Questionnaire and its Types Mumbai University
 
John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...Sax Institute
 
Chapter 4&5 mr
Chapter 4&5   mrChapter 4&5   mr
Chapter 4&5 mrr2sa
 
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...sreenath T.V
 
Introduction and Summary In NU you developed your proposal.docx
Introduction and Summary In NU you developed your proposal.docxIntroduction and Summary In NU you developed your proposal.docx
Introduction and Summary In NU you developed your proposal.docxstudywriters
 
QUESTIONNAIRE METHOD.pptx
QUESTIONNAIRE METHOD.pptxQUESTIONNAIRE METHOD.pptx
QUESTIONNAIRE METHOD.pptxGarimaBhati5
 
Integrating impact into your UKRI case for support
Integrating impact into your UKRI case for supportIntegrating impact into your UKRI case for support
Integrating impact into your UKRI case for supportMark Reed
 
Understanding Why, When, and What it Will Take to do Operations and/or Implem...
Understanding Why, When, and What it Will Take to do Operations and/or Implem...Understanding Why, When, and What it Will Take to do Operations and/or Implem...
Understanding Why, When, and What it Will Take to do Operations and/or Implem...CORE Group
 
Writing research proposal
Writing research proposalWriting research proposal
Writing research proposalabween1
 
Research Methodology UNIT 1.pptx
Research Methodology UNIT 1.pptxResearch Methodology UNIT 1.pptx
Research Methodology UNIT 1.pptxPallawiBulakh1
 
Class 6 research quality in qualitative methods 3 2-17
Class 6 research quality in qualitative methods 3 2-17Class 6 research quality in qualitative methods 3 2-17
Class 6 research quality in qualitative methods 3 2-17tjcarter
 

Similaire à Odum focus groupcourse_mar252014 (20)

Introduction to Focus Groups, Odum Institute, October 30
Introduction to Focus Groups, Odum Institute, October 30Introduction to Focus Groups, Odum Institute, October 30
Introduction to Focus Groups, Odum Institute, October 30
 
Marketing research exploratory research using qualitative and observation me...
Marketing research  exploratory research using qualitative and observation me...Marketing research  exploratory research using qualitative and observation me...
Marketing research exploratory research using qualitative and observation me...
 
Focus group
Focus groupFocus group
Focus group
 
unit 2.3.ppt
unit 2.3.pptunit 2.3.ppt
unit 2.3.ppt
 
Formulation of the research probleme Dr. SC sharma
Formulation of the research probleme Dr. SC sharmaFormulation of the research probleme Dr. SC sharma
Formulation of the research probleme Dr. SC sharma
 
Questionnaire and its Types
Questionnaire and its Types Questionnaire and its Types
Questionnaire and its Types
 
John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...
 
Chapter 4&5 mr
Chapter 4&5   mrChapter 4&5   mr
Chapter 4&5 mr
 
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...
Preparing a-case-study-a-guide-for-designing-and-conducting-a-case-study-for-...
 
Introduction and Summary In NU you developed your proposal.docx
Introduction and Summary In NU you developed your proposal.docxIntroduction and Summary In NU you developed your proposal.docx
Introduction and Summary In NU you developed your proposal.docx
 
pr 1 week 2.pptx
pr 1 week 2.pptxpr 1 week 2.pptx
pr 1 week 2.pptx
 
QUESTIONNAIRE METHOD.pptx
QUESTIONNAIRE METHOD.pptxQUESTIONNAIRE METHOD.pptx
QUESTIONNAIRE METHOD.pptx
 
Integrating impact into your UKRI case for support
Integrating impact into your UKRI case for supportIntegrating impact into your UKRI case for support
Integrating impact into your UKRI case for support
 
Research idea generation
Research idea generationResearch idea generation
Research idea generation
 
Understanding Why, When, and What it Will Take to do Operations and/or Implem...
Understanding Why, When, and What it Will Take to do Operations and/or Implem...Understanding Why, When, and What it Will Take to do Operations and/or Implem...
Understanding Why, When, and What it Will Take to do Operations and/or Implem...
 
Writing research proposal
Writing research proposalWriting research proposal
Writing research proposal
 
Research Methodology UNIT 1.pptx
Research Methodology UNIT 1.pptxResearch Methodology UNIT 1.pptx
Research Methodology UNIT 1.pptx
 
Coletando dados qualitativos
Coletando dados qualitativosColetando dados qualitativos
Coletando dados qualitativos
 
IRP for Dummies
IRP for DummiesIRP for Dummies
IRP for Dummies
 
Class 6 research quality in qualitative methods 3 2-17
Class 6 research quality in qualitative methods 3 2-17Class 6 research quality in qualitative methods 3 2-17
Class 6 research quality in qualitative methods 3 2-17
 

Dernier

Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxiammrhaywood
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxKatherine Villaluna
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxEduSkills OECD
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 

Dernier (20)

Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptxAUDIENCE THEORY -- FANDOM -- JENKINS.pptx
AUDIENCE THEORY -- FANDOM -- JENKINS.pptx
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 
Finals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quizFinals of Kant get Marx 2.0 : a general politics quiz
Finals of Kant get Marx 2.0 : a general politics quiz
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 

Odum focus groupcourse_mar252014

  • 1. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org Intro to Focus Groups Emily Geisen Amanda Wilmot 3/25/2014
  • 2. RTI International Ground Rules Cell phones turned to vibrate or off Refreshments and restrooms any time Several bio-breaks throughout Questions are encouraged at end of each section
  • 3. RTI International Course Outline 1. The role and use of focus groups 2. Focus group demonstration 3. Developing the protocol guide 4. Moderating focus groups 5. Recruiting and scheduling 6. Participant selection 7. Data management/analysis 8. Reporting findings 9. Drawing wider inferences 10. Qualitative Research Ethics
  • 4. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 1. The Role and Use of Focus Groups
  • 5. RTI International What is a Focus Group? “Focus group interviews typically have five characteristics or features: (a) people, who (b) possess certain characteristics, (c) provide data (d) of a qualitative nature (e) in a focused discussion.” -Focus Groups: A Practical Guide for Applied Research (Krueger)
  • 6. RTI International Examples A group of coal miners sit around a small conference room discussing mine safety. They all agree that safety is important, but some argue that existing safety regulations are too unrealistic or impractical to be implemented. A group of employees sit in a conference room discussing access to email and other work functions from mobile devices. One participant says answering emails on his phone will save him time. Another participant worries that her supervisor will expect her to reply to emails at all hours if she has access on her phone.
  • 7. RTI International What is a Focus Group, again? Researchers have a topic they want to know about: – Why aren’t coal miners following safety precautions? – What concerns do employees have about mobile email access? These topics are communicated to a group of participants to discuss – Pre-scripted questions to generate discussion – Targeted moderation to focus the discussion Researchers summarize and interpret what they have learned
  • 8. RTI International Uses of Focus Groups Exploration and Discovery – Collect info that will be used to guide development of a survey – Collect info for a needs assessment – Test a new product before development Context and Depth – Adding new topics or population groups to a survey – Delving deeper into a survey topic – Testing a product with a new audience Interpretation – Deciding how to use survey results – Discuss applications with end users -Focus Group Guidebook (Morgan)
  • 9. RTI International Uses of Focus Groups*, Continued Stand-alone data collection method, or Supplement other qualitative or quantitative research methods – In-depth personal interviews – Survey development – Needs assessment – Product development
  • 10. RTI International Focus Groups for Survey Research Identify topics/themes that are of interest to population – Use this to add questions on these topic/themes – Cut questions that are not relevant, not applicable, or show no variation Get more detail on a topic so that you don’t have to ask open-ended questions – Which of the following are reasons why you do not have health insurance? Too expensive, No health problems, … Get reaction to survey materials such as advanced letters, envelopes, etc. Explore methods to boost cooperation: what type of incentives would people prefer?
  • 11. RTI International Strengths of Focus Groups (over other qual./quan. methods) If you’re unfamiliar with a topic, you can learn a lot quickly (often used as the first step in research) Can be used to observe interactions or group dynamics Generate hypotheses (that can be tested using quantitative methods) Cheaper/quicker than other methods
  • 12. RTI International Limitations of Focus Groups Some topics are too sensitive or controversial to be discussed in a group setting (social desirability) Discussions can get easily sidetracked/derailed Limited generalizability of findings/results Cannot quantify findings or determine statistical significance
  • 13. RTI International Thinking about your research… How are you going to use focus groups? – Exploration and discovery? How so? – Context and depth? How so? – Interpretation? How so? Will your focus groups be stand-alone or will you use them with other research methods? What are the strengths/weaknesses of using focus groups for your research?
  • 15. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 2. Focus Group Demonstration
  • 16. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 3. Developing the Protocol Guide
  • 17. RTI International The What, Why, Who, How, Where What Review research objectives Who Determine number and type of participants Recruit and schedule participants Where Determine test location and equipment Why Identify major topic areas of interest How Script focus group questions Develop focus group guide
  • 18. RTI International The What, Why, Who, How, Where What Review research objectives Who Determine number and type of participants Recruit and schedule participants Where Determine test location and equipment Why Identify major topic areas of interest How Script focus group questions Develop protocol guide
  • 19. RTI International What’s the Focus Group About: Review Research Objectives Start with the overall research objectives / study goals Identify what do you already know about the topic Determine what new information you want to learn Determine what information you want to explore in more detail Decide how are you going to use this information
  • 20. RTI International Review Research Objectives: Example Step Personal Health Records Example Study goals: Learn about healthcare consumers understanding, experiences, needs, and concerns about the ways technology can be used within the healthcare system What do you already know about the topic: Research has shown that use of electronic PHRs can save consumers money and improve health, yet they are not being used extensively. What do we want to learn through focus groups: Why aren’t more consumers using electronic PHRs to manage their health information? How will we use this info: Provide recommendations for improving PHRs and promoting awareness.
  • 21. RTI International Review Research Objectives: Your Turn Step Your Study Study goals: What do you already know about the topic: What do we want to learn through focus groups: How will we use this info:
  • 22. RTI International Why: Identify Major Discussion Topics Once you identify what you want to learn, identify major discussion topics (brainstorm) Why are we doing these focus groups: – What is the most important question your research must answer? – What’s the next most important question? – And so on … Consult with project team members as needed Consider how topic areas may need to differ by participant selection
  • 23. RTI International Identify Major Discussion Topics: Example Why aren’t more consumers using electronic PHRs to manage their health information? Consumers who have not used electronic PHRs How familiar are consumers with electronic PHRs? How do consumers currently manage their health information? What are the most important factors consumers consider when deciding how to manage their health information What are their biggest concerns/fears about using PHRs? Narrow list of discussion topics to about 4 major topic questions per hour (more if questions are more specific)
  • 24. RTI International How: Scripting Questions Avoid yes/no or short-answer questions, unless: Open-ended follow up is planned (e.g., why/why not?) Used to tally group (e.g., those in favor/opposed) Open-ended questions generate discussion, however: – Keep questions focused, one-dimensional – Be careful of “Why?” questions – Use scripted probes to focus or narrow topics Questions should be easily comprehensible: use familiar words/terms Questions should be reflective, not hypothetical Move from general to more specific
  • 25. RTI International Scripting Questions: Example 1 Topic: How do consumers currently manage their health information? Bad: How do you manage your family’s health information? Good: How do you currently maintain or store your and your family’s medical records, such as records of vaccinations or prescriptions? – Do you keep paper or electronic records? – How does this differ for you compared to your children (or elderly parents)?
  • 26. RTI International Scripting Questions: Example 2 Bad: Why do you use the mobile device you have? Better: What kind of mobile device do you use at work? Best: How did you decide what kind of mobile device to use at work?
  • 27. RTI International Other aspects of Protocol Guide Welcome/Introduction Icebreaker Opening (or Engagement) questions: – Easy and inviting, comfortable to discuss – Introduces the topic of discussion, provides context/background Exploration questions (main study questions) Exit questions: – Used to wrap-up the discussion – Check to see if there is anything else people would like to mention that they did not get a chance to Pre/Post Questionnaires
  • 28. RTI International Welcome/Introduction Welcome – Introduce yourself, note-taker AND people behind the glass – Make participants feel comfortable Informed consent (if required) – Read or summarize consent, ask participants to sign – Note audio-recording if applicable – Explain confidentiality Go over ground rules – Refreshments, restrooms, cell phones off or on vibrate – No right or wrong answers – Like to hear from everyone
  • 29. RTI International Icebreaker Sometimes it’s helpful to start with an ice-breaker Ask participant to provide their name (first name only) Have them answer an easy question – Participants should be able to answer it briefly, quickly – Should not be sensitive – Can be related to focus group topic or unrelated Example – Employee focus group: How long at company? Department/Division? – Focus group of moms: How many children and their ages
  • 30. RTI International Intro Questions Introduce Topic – Goal/Purpose of the study – Why participants were selected (if not obvious) Provide any needed context – For focus group on Personal Health Records, we had to explain what PHRs were – Show participants related materials or products – Provide background Opening Question – Should be easy/simple to answer – Non-sensitive
  • 31. RTI International Pre/Post Questionnaires Pre questionnaire: – Use to collect information about your participants without wasting valuable time during the focus group – Use to determine how familiar/knowledgeable participants are with topic at start of focus group Post questionnaire: – Can be same as pre-questionnaire to monitor changes in opinion or knowledge – Can be a handy way to summarize participants’ opinions on discussion topics Keep questionnaires short (5 minutes)
  • 32. RTI International Putting it all together Allow time for welcome, explaining the purpose of the study, and consent procedures (if required) – At least 5 minutes Allow time for introductory or ice-breaker question – At least 5 minutes Time for your main discussion questions – About 4 question topics (per hour) – 10-15 minutes per question Exit/Wrap-up – At least 5 minutes
  • 33. RTI International Determining Question Flow What order would you put these questions in? What kind of device would you prefer to use and why? What factors did you consider when determining what kind of device to use? What are the biggest limitations or barriers to using a mobile device for the work you do? What are the benefits of using a mobile device for the work you do? Tell me about the mobile device you use at work. What kind of device is it? What are the main reasons that you use a mobile device for work?
  • 34. RTI International Question Flow Tell me about the mobile device you use at work. What kind of device is it? – What factors did you consider when determining what kind of device to use? – What kind of device would you prefer to use and why? What are the main reasons that you use a mobile device for work? – What are the benefits of using a mobile device for the work you do? – What are the biggest limitations or barriers to using a mobile device for the work you do?
  • 35. RTI International Protocol Guide Examples: Handouts Handout 1: Mine Contractors Protocol Guide Handout 2: Healthcare Consumers Protocol Guide Handout 3: Tobacco Panel Protocol Guide Handout 4: Post questionnaire
  • 36. RTI International Individual Practice (15 minutes) Use your study or class example Identify 1-2 major discussion topics for your study Prepare an introductory question Prepare 1-2 discussion questions with probes as needed
  • 38. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 4. Moderating Focus Groups
  • 39. RTI International Understanding the group Process Model of group phases (Ritchie and Lewis 2003) Forming Testing and dependence Dependence on the leader Storming Intragroup conflict Criticism Norming Development of group cohesion Optimism Performing Functional role relatedness Cohesiveness Adjourning (Mourning) Termination, Separation
  • 40. RTI International The Ideal Moderator: Has adequate knowledge of topic Has good communication skills Is similar to / can build a rapport with participants Can maintain balanced viewpoint Is prepared for surprises Reports accurately, even if it’s uncomfortable Has Experience with: – group dynamics – neutral probing/feedback
  • 41. RTI International What the Moderator Does Welcome/Introductions Leads discussion Concludes discussion
  • 42. RTI International Welcome/Introduction Make participants feel comfortable Read or paraphrase this welcome/introduction in protocol guide Make sure to collect informed consent Notify participants of observers and/or audio recording Go over any ground rules Let participants get refreshments and get situated before starting actual focus group discussion
  • 43. RTI International Leads Discussion Provides unobtrusive control Manages group dynamics Asks spontaneous probes as needed to gather more information Provides unbiased feedback Summarize/verify information Manages time
  • 44. RTI International Provide Unobtrusive Control Know when to stop talking – Don’t constantly interject, let participants discuss back and forth Learn to focus the discussion – You said X, can you tell me more about that? – Note topics said, that you want to circle around to Manage the flow of the discussion – Don’t have to follow script, allow natural segues – If a topic you plan to cover later is brought up, indicate that you’ll talk more about that later. Encourage different point of views – Has anyone had a different experience? – Does anyone view it differently?
  • 45. RTI International Tips for Managing Group Dynamics Dominant talkers: – Don’t look at them when you ask a question. – (Politely) cut them off: “Thank you Amanda. Does anyone else have an experience they’d like to share?” Ramblers/Off-topic: – Avoid eye contact after so long off topic, look down or at clock – Be prepared to interject at next pause Shy participants: – Maximize eye contact – Call by name: I’ve heard from some of you on this and I want to hear from the rest of you. Sarah, what is your experience with X?
  • 46. RTI International Tips on Spontaneous Probing Leading probes: – So you had a hard time with that then? – You didn’t want to learn more about that? – Was that difficult for you? Neutral probing: – Can you tell me more about that? – Would you explain that further? – Can you give me an example? – Is there anything else?
  • 47. RTI International When you hear yourself asking a leading question, balance it 47 Leading question “So you think that’s difficult then?” Balanced question “...or was it easy?”
  • 48. RTI International Tips on Neutral Feedback Monitor your reaction to participants/discussion Be careful of nonverbal cues (leaning back, crossed arms) Limit head nodding: shows you’re listening, but also implies agreement Provide quick verbal responses periodically (uh-huh, yes, mmm, okay, go on) Avoid loaded feedback (which implies agreement) – Loaded feedback: “That’s good!” “Excellent.” – Neutral feedback: “That’s helpful” “That’s interesting.”
  • 49. RTI International Summarize/Verify Information As needed, summarize/paraphrase what has been said – I understand that work-life balance is your most important concern. – Safety is critical, but some of the precautions you’re asked to follow are pointless or unrealistic. Is that correct? See if anything has been missed? – Did I miss anything? – Does anyone have anything else to add? Probe on any subtopics you didn’t cover – We talked about X, but I’m also interested in Y Move on to next topic
  • 50. RTI International Conclusion / Wrap-up Be sure to leave time for wrap-up, especially for sensitive topics or intense discussions Ask participants to identify most important issues – Of everything we have discussed, what is the most important? – If you could make one request to X, what would it be? Summarize major themes (ask note-taker for help) – Ask if its an adequate summary – Ask if anything has been missed, anything to add Check with observers to see if they have any questions/clarifications Pay participants!
  • 51. RTI International Dealing with Difficult Situations Too many people show up Participants bring their children Only a few attend Group is reluctant to talk Can’t get them to stop talking
  • 52. RTI International Plan Your Response… 20 minutes into the focus group, and Sarah has not said anything… You can’t get Joe to stop talking… John mentions something interesting, but you don’t want to interrupt because there’s a good discussion going…
  • 53. RTI International Role of Note-Taker Can Handle logistics & refreshments Collects signed informed consent (if required) Takes careful notes Does not participate in discussion Can recap major themes at end of discussion (used before wrap-up question) Monitors recording equipment Liaison between moderator and observers/clients Debriefs with moderator after session Assist with analysis and reports
  • 54. RTI International Role of Co-Moderator Not required, but can be useful in some situations Balance out strengths/weaknesses in moderator Use to match moderator (without being obvious) Switch leading focus group (good for long or intense focus groups) Support leader by keeping on track, recapping major themes, etc.
  • 55. RTI International Moderating style Different styles for different people – Spontaneous flow – Follow guide strictly Adapt to your style, but plan ahead – Spontaneous: Be sure to note what you’ve covered and what you haven’t so that you don’t forget anything. – Strict: “That’s a very interesting point, and we’re going to get to that in a minute, but right now I want to focus on X:
  • 56. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org Group Practice Moderating (30 min)
  • 58. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 5. Recruiting and Scheduling (review on your own)
  • 59. RTI International At Least a Month Before Focus Group Decide on what you’re testing Decide on the participants (see Section 6) : – Who you want to recruit – How many – How you will find them Plan the test date – Find a date when your stakeholders can observe – Decide on your location
  • 60. RTI International How to Find Your Participants Frame/list if available Advertisements: Craigslist, flyers, newspapers – Pros: quick, easy, cheap – Cons: yields younger, higher educated users Participant databases/recruiting orgs – Pros: quick and easy, pay per user recruited – Cons: can lead to “professional respondents” Other sources: word of mouth; sample lists; clubs, churches, and societies; snowball recruiting
  • 61. RTI International What to Pay Participants? Are participants allowed to accept monetary incentives? – Federal employees – Participants who are being paid by their employer Is it appropriate for your organization to pay participants? Is topic interesting/important enough that incentives are not necessary How much should you pay them? – Trade-off between incentive amount and recruitment effort – Tailor amount to population and location – Will participants need to pay for parking?
  • 62. RTI International Recruiting Tips Recruit extras due to no-shows or cancellations – Example: Recruit 12 to ensure at least 8 show up Schedule sessions about 3-4 weeks ahead – Any earlier and they may forget or make new plans – Any later and you may not be able to recruit enough participants Send them an email or letter confirmation Remind them the day before (by email or phone)
  • 63. RTI International Testing Location At a minimum, use a large conference room (at your organization’s offices, hotel conference room) For numerous focus groups: formal observation rooms are nice (see picture on next slide) – One-way mirror so you can see them, but they can’t see you – Microphones in ceiling to pick up voices, but not other sounds (paper shuffling) – Video-taping capability Rent a focus group facility – Usually very nice but can be expensive ($1,000 for 2-hour session)
  • 65. RTI International Create a Virtual Observation Room using screen-sharing software Examples: Skype, Go to Meeting Fosters collaboration – Can accommodate observers from any location – Facilitate discussions in conference setting Improved schedule – Stakeholders get information immediately – No waiting for recorded videos or report Cheaper: Inexpensive compared to travel costs
  • 66. RTI International Note-Taking Recommend having a note-taker and audio or video- recording the session Note-takers should be in the room or observation room For audio-recording, consider getting additional microphones so that you can hear everyone Video-recording can be helpful, but makes some participants uncomfortable Participants usually forget they’re being recorded/ observed after 5-10 minutes
  • 67. RTI International At Least Three Weeks Before Focus Group Get started on recruitment Organize your incentives Develop your focus group guide
  • 68. RTI International One Week Before the Focus Group Final your focus group guide Organize roles in the test: – Meet and greet – Observers/Stakeholders – Moderator – Note-takers Do a practice run on any equipment Arrange any refreshments – At a minimum provide beverages – Tailor food to participants / time of day – Nothing messy or loud
  • 69. RTI International Prepare your materials Develop consent forms, screeners Instructions/directions for participants Prepare any visual materials for participants Pretest/posttest questionnaires
  • 70. RTI International The Day Before / Morning Of Send out reminders: – Phone or email to respondents – Email to observers, stakeholders, note-takers Equipment/Facility – Make sure the room you’ll use is tidy – Make sure your meet/greet person has the final list of participants’ names – Incentives are available – Check any equipment (video/audio recording)
  • 71. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 6. Participant Selection
  • 72. RTI International Purpose of the session To understand the principles of participant selection for focus group research To consider the practicalities of participant selection for focus group research
  • 73. RTI International Focus groups: a qualitative data collection method Focus groups are a qualitative data collection method When reporting the findings the rationale should be provided for the research along with the methods used Is the research… – Justified – Rigorous – Systematic – Transparent
  • 74. RTI International Sampling for qualitative research The design of a sampling strategy for qualitative research is as important as that for quantitative research Qualitative research uses non-probability sampling
  • 75. RTI International Informing the design of a qualitative sampling strategy What are the research objectives? What is the scope of the research? Who is out of scope and should be excluded from the sample? Who is in scope and should be included in the sample? What is the budget? What is the reporting time period? What sampling technique will be employed? How are the data to be analyzed? What data collection methods should be employed? What are the sample criteria? What size should the sample be? What will be used as the sampling frame? How are potential respondents/participants recruited?
  • 76. RTI International Sampling technique Quantitative sampling Probability sampling Members of the research population are chosen at random and have a known probability of selection The aim is to produce a statistically representative sample Qualitative sampling Purposive non-probability sampling The number of people interviewed is less important that the criteria used to select them Members of the research population are chosen on the basis of their characteristics to reflect breadth and diversity of the research population We do not aim to produce a statistically representative sample or draw statistical inference
  • 77. RTI International Sample criteria What characteristics will need to be reflected in the sample population to ensure breadth and diversity? Criteria used may be based on demographic characteristics or behaviours or attitudes Some criteria may be considered more important than others
  • 78. RTI International Sample size Small sample sizes for qualitative research There is no need for scale because there is no need for statistical inference Sample size determinants: Heterogeneous or homogenous nature of sample population Number of selected criterion Scale 10 to 50 for one-to-one investigation 40 to 100 for group interview
  • 79. RTI International Sampling frames A sampling frame is a list that identifies units within the target population Frame evaluation: Comprehensive Sufficient numbers Geographical dispersion Respondent contact details correct
  • 80. RTI International Sampling frames Existing frames – Administrative sources – Survey samples Constructed frames – Direct and/or indirect methods
  • 81. RTI International Constructed frames Focussed enumeration Snowballing Screening questionnaires Organisations Advertisements
  • 82. RTI International Study about attitudes and behaviors surrounding dental attendance Selection criteria initially considered Age to ensure demographic balance Gender because patterns of attendance differ between men and women Family Unit composition because attendance by others in the family might influence the respondents attitudes or behaviours Employment activity because attendance might be affected by time constraints during working hours Income as a known factor affecting dental attendance patterns Regional location as dental attendance varies across the country Ethnic origin as may influence attitudes or behaviours Type of area as urban/rural location may affect attitudes and behaviours Dental health to explore how attitudes vary among people with different dental health Current pattern of dental attendance (regular/irregular/occasional) for comparative analysis Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2003)
  • 83. RTI International Prioritised selection criteria Primary criteria Secondary criteria Dental attendance pattern Dental Health Age Ethnic Origin Gender Type of Area Region Family Unit Employment activity Income Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2006)
  • 84. RTI International Sample Matrix Area 1 of 6 Group 1 Group 2 Group 3 18-29 30-44 45+ 4 men 4 men 4 men 1-2 regular attenders 1 regular attender 1-2 regular attenders 1-2 irregular attenders 1-2 irregular attenders 1 irregular attender 1 occasional attender 1-2 occasional attenders 1-2 occasional attenders 4 women 4 women 4 women 1-2 regular attenders 1 regular attender 1-2 regular attenders 1-2 irregular attenders 1-2 irregular attenders 1 irregular attender 1 occasional attender 1-2 occasional attenders 1-2 occasional attenders Spread of family unit type and employment activity Illustration from Ritchie J and Lewis J ‘Qualitative Research Practice’ (2006)
  • 86. RTI International Excercise Groups Designing a sampling strategy for focus groups on the use of electronic personal health records 15 mins Feedback
  • 87. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 7. Data Management and analysis
  • 88. RTI International Purpose of the session To understand the principles of good qualitative focus group analysis To consider the practicalities of analysing qualitative data
  • 89. RTI International Acknowledgement Some of the slides contributed by the UK National Centre for Social Research (NatCen). The Framework methodology was developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen. http://www.natcen.ac.uk/events-and-training/our- training
  • 90. RTI International Qualitative data Focus group recording Focus group transcription Notes Admin data Photographs, videos etc
  • 91. RTI International Transcription Transcription is time consuming The focus group transcription should capture the discussion in it’s entirety Focus groups are more difficult to transcribe than one-to- one interviews as there are multiple participants to distinguish between It may be important for the analysis for the transcriber to be able to identify each participant’s words. The transcriber will use the notetaker’s notes to help determine who is speaking Check on quality of the transcription Anonymize transcript
  • 92. RTI International Different approaches to data management, analysis and reporting Summary report Qualitative analysis
  • 93. RTI International Nature of qualitative analysis Quantitative research – Fixed categories aiming to enumerate – Requires enumeration in order to be meaningful – Test hypotheses and provide explanation Qualitative research - Creating categories - Developing explanations - Does not need enumerating - Range and diversity key
  • 94. RTI International Aim of analysis - analytical outputs Categories of things (thematic analysis) Reasons for gambling Sources of debt advice Categories of people or processes (typologies) Types of parents of children with learning difficulties Types of welfare benefit claimants Explanations of attitudes, choices or impacts (explanatory analysis) Factors influencing how people save for retirement What helps achieve positive outcomes from an employment programme
  • 95. RTI International The nature of qualitative data The data are: Voluminous Unwieldy Multi-stranded with overlapping categories and theme
  • 96. RTI International 7.1 Data management “Chaos is merely order waiting to be deciphered” Jose Saramago
  • 97. Introduction to Framework © NatCen Learning RTI International What is Framework? Developed by Jane Ritchie and Liz Spencer in the 1980s at NatCen Social Research Key characteristics Approach to data management which facilitates case and theme based analysis of qualitative data Case and theme based approach Matrix display Reduces data through summarisation and synthesis Retains links to original data Output allows comprehensive and transparent data analysis
  • 98. Introduction to Framework © NatCen Learning RTI International Objectives of data management Primary objective Re-order Make data accessible Secondary objective Reduce, prioritize
  • 99. Introduction to Framework © NatCen Learning RTI International Data management • Data management involves ‘cutting’ up data • Aim is to create useful ‘piles’ of data • Three ways you can ‘cut’ data Thematically By case By case and theme
  • 100. Introduction to Framework © NatCen Learning RTI International Theme-based approach Crime Contact with CJS Impact
  • 101. Introduction to Framework © NatCen Learning RTI International Case-based approach Jim Bob Wendy Robbery Contact with police Financial impact
  • 102. Introduction to Framework © NatCen Learning RTI International Framework is a case and theme based approach Name Crime Contact with CJS Impacts Jim Bob Wendy
  • 103. Introduction to Framework © NatCen Learning RTI International The Framework approach
  • 104. Introduction to Framework © NatCen Learning RTI International Stage 1: Data familiarization process • Familiarize yourself with the data by reading through transcripts and other documentation • Immerse yourself in the data • Begin to construct your analytical framework
  • 105. Introduction to Framework © NatCen Learning RTI International Stage 2: Creating the ‘framework’ Multitude of potential descriptive categories different kinds and levels of category Choice of categories depends on interview guide research questions emergent themes recurrence across cases Data management ≠≠≠≠ interpretation
  • 106. RTI International Frame construction: individual data One chart represents one main theme Chart each individual case – Each individual as one row – Subthemes in each column – Give cases same space on each thematic chart
  • 107. RTI International Frame construction: group data Chart as a whole group – Each group as one case – Several groups on each thematic chart One group per page, tracing individual cases – Order cases in the same way – Give cases same space on each thematic chart – Don’t be alarmed if every cell is not filled for each case Guidelines for entering data – May be useful to keep a record of group process – Levels of participation, context in which comments made, how views evolve/develop/change
  • 108. Introduction to Framework © NatCen Learning RTI International Respondent Employment situation/ activity at time of contact Employment history/work activity in past Hopes & aims regarding work before contact Barriers perceived in achieving aims Efforts made to overcome barriers before contacting NDLP Perception of needs before contact Overview of financial situation & views about at time Other/Misc # 1 # 2 # 3 # 4 A chart with sub-categories Before contact with employment scheme
  • 109. RTI International Stage 3: Testing the analytic framework Testing framework is important – multi-stranded & overlapping categories – different experiences Test by coding or indexing – apply numerical code to categories and sub-categories – code using software or write codes in margin of transcript Test by piloting – enter data from small number of transcripts Checking for: overlap / gaps
  • 110. RTI International Data management coding / indexing Example of analytical framework index for sexual identity project 1. Demographic details 2. Defining sexual identity 2.1 Conceptualization 2.2 Salience 2.3 Self categorization 2.4 Changes over time 3. Language used 3.1 Terminology used 3.2 Categories used 4. Acceptability 4.1 Acceptance by society 4.2 Understanding purpose of questioning 4.3 Trust in data security 4.4 Trust in data collector 5. Other
  • 111. RTI International Developing an analytical framework exercise Groups to discuss analytic framework that could be used for the morning focus group session Create main themes and subthemes Post-it notes 15 mins Feedback – group spokesperson
  • 112. Introduction to Framework © NatCen Learning RTI International Stage 4: Summarising data (‘charting’) • Condensing data • retains richness and flavour of data • Process • theme by theme (if indexed) or by transcript • Takes practice – an art
  • 115. Introduction to Framework © NatCen Learning RTI International
  • 116. Introduction to Framework © NatCen Learning RTI International
  • 117. Introduction to Framework © NatCen Learning RTI International Entering data (‘charting’) Process theme by theme (only when indexed) by transcript Guidelines for entering data summarise material from transcripts/other data sources retain language mark but don’t recite quotations note page references / create links between summaries and the original transcript within CAQDAS software use agreed abbreviations/conventions avoid repetition by cross referencing data
  • 118. Introduction to Framework © NatCen Learning RTI International Data Management: Good practice Reading the transcript beforehand structure of account and nature of the data repetition, clarification, contradiction Reading chart afterwards clarity balance and emphasis check blank cells Use ‘other’ column for interpretative notes note ‘flavor’ of discussion note overarching issues alert others to important issues
  • 119. RTI International 7.2 Interpretation “All meanings, we know, depend on the key of interpretation” George Elliot
  • 120. RTI International Descriptive or explanatory analysis Descriptive accounts Explanatory accounts
  • 121. Introduction to Framework © NatCen Learning RTI International There are three key steps… Detection familiarisation extraction (highlight/ summarise) Categorisation creating meaningful conceptual boxes assigning data Classification creating higher order categories assessing relationships between categories
  • 122. Introduction to Framework © NatCen Learning RTI International Categorization and classification Lone parents orientation to the Labor Market • Relevant dimensions Attitudes to work Barriers to work Stage of job search activity • Typology Work not currently an option Beginning to think about work Personal issue barriers Labor market barriers Close to work
  • 123. RTI International Categorization and classification (2) Classification of sexual identity Relevant dimensions – Conceptualization – Salience – Self-categorization – Changes over time Typology – Latent identifiers – Conscious identifiers – Reluctant identifiers
  • 124. RTI International Associations and explanations Linkages between: – Two attitudes – Attitude and behavior – Circumstances and need Verified through explanation – Explicit respondent accounts – Implicit identified by the researcher
  • 125. RTI International Making explanations Informed by – Hunches and hypothesis – Reflections during field work and analysis – Other research or theories Involves – Detailed within case analysis – Comparison between cases – Repeated interrogation of data Comprehensive Expect multiplicity
  • 126. RTI International Levels of classification and interpretation In summary: Descriptive categories – Factors, reasons, impacts Classifications or typologies Explained associations Meaning Theory/Strategy generation
  • 128. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 8. Reporting the Findings
  • 129. RTI International Purpose of session To provide a basic overview of the principles involved in reporting qualitative findings and constructing a final report.
  • 130. RTI International Aims of the final report To complete and document the research process To inform key stakeholders of the findings To communicate the findings in a clear and coherent way To guide readers in the interpretation of findings (avoid misinterpretations)
  • 131. RTI International Basic principles for writing Tell the story Structure the report Know your reader Write in plain English Use visual representation
  • 132. RTI International Reporting the Findings Revisit the objectives (ensure the report covers these) Consider the audience Check report format Consider time available Consider length of the report (focus on relevant points only) Report findings in past tense
  • 133. RTI International Structuring the report Title Contents Executive summary Introduction / Background Methodology Findings Conclusions Recommendations Appendices e.g. sample matrix, interview guide
  • 134. RTI International Nature of the reporting Descriptive Explanatory
  • 135. RTI International Defining the boundaries Research rationale Methodology – Sampling strategy – Data collection – Method of analysis – What can or cannot be inferred form the findings Examples of relevant documentation – Interview guide – Contact letter – Screening questionnaire – Sample matrix Substantiate conclusions by grounding the data in the findings – Examples – Quotations – Case illustrations
  • 136. RTI International Using Quotations Avoid over-use Avoid under-use Present range and balance Edit sparingly Should provide illustration only Amplify but do not repeat a point Avoid reliance on a few articulate respondents Avoid very long quotations Identify relevant characteristics Preserve participant confidentiality
  • 137. RTI International Quotation examples “I don’t feel you have to be out in all contexts…” [Gay man, 35-44] “The Government uses statistics to show what it wants to show, for example unemployment, they’ve used all sorts of different measures for employment to show it’s going down…rather than it necessarily going down itself.” [MHE Wales] “They don’t XXX care about us. It’s everyone for themselves” [Male, 18-24, Central London]
  • 138. RTI International Quotation examples “I think it depends quite a lot on the way you live and how old you are. For example, I was born down [South West county], and my parents being extremely religious, have very definite views on sexuality.” [Bisexual woman, aged 35-44] “I’ve lived in [Muslim country], nobody would ever admit that [being gay], so they all choose to be heterosexual. Somebody’s not been here that long, it’s even more of an issue.” [Heterosexual woman, aged 35-44]
  • 139. RTI International Is a quotation necessary? The actual presence of the question was considered to have an important purpose in its own right. It was thought that as the questioning became more commonplace it would demonstrate to the wider public that non-heterosexuality was unremarkable. “It’s having it on there, it’s beginning to usualise the question, it’s a cultural shift, it’s enabling people to gradually recognise it’s no bloody big deal. But if we don’t have it there, we are setting up this whole process of we must be quiet about these people and all the rest of it.” [Gay/lesbian woman, 55+]
  • 140. RTI International Avoiding quantitative language Avoid tendency to use numbers Do not discuss qualitative results in terms of proportions, percentages or statistics Indicate strength of finding by discussing in terms of: – A common finding, A recurring problem, Problems were observed…. etc Use diagrams for illustration
  • 141. RTI International Principles to remember Present balanced findings Present grounded conclusions (avoid assumptions) Distinguish between findings and your own conclusions Provide commentary alongside the analysis (do not leave the reader to do this for you) Support with evidence from other research findings (if appropriate)
  • 142. RTI International Editing and reviewing Allow some time before editing (if possible) Review the content Check accuracy of statements Peer Review Proof read
  • 144. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 9. Drawing Wider Inferences
  • 145. RTI International Purpose of the session To understand whether wider inference can be made when using qualitative data
  • 146. RTI International Concerns Small samples Not statistically representative Interviews/discussions not standardized Proximity of researcher / too personal Biased reporting
  • 147. RTI International Generalization Can qualitative findings be generalized beyond the sample and context of the study research? Different definitions of ‘generalization’. – Representational generalization.
  • 148. RTI International Representational generalization Meaning Inference to parent or sampled population Whether range and diversity (experiences, explanations etc) can be matched with the sampled population Whether list (of experiences, explanations etc) is inclusive of what would be found in the sampled population Basis Depends on validity and reliability of findings
  • 149. RTI International Validity Validity: Accuracy of representation Has the researcher understood the issues from the respondent’s perspective? Have the issues been fully articulated and explained Are interpretations underpinned by the data Alternative terms: Credibility and Plausibility
  • 150. RTI International Reliability Reliability: Replicability Internal reliability – Extent to which assessments and judgments are replicated between researchers External reliability – Extent to which findings would be replicated if the study were repeated with the same or different sample Alternative terms: Confirmability, Consistency, Dependability
  • 151. RTI International Drawing wider inference Depends on: Quality of sampling – Reflects diversity of sample population – Includes all key constituents – Includes outliers and a-typical cases Quality of data collection – Free from interference: neutral and objective – Probing of meaning – Exploration of all relevant explanations Quality of analysis – Systematic, comprehensive and inclusive – Within and between case analysis – Displays diversity Quality of interpretation and reporting – Comprehensive – Clear – Underpinned by data – Displays multiplicity of accounts and explanations – Non-quantitative Documentation – Transparency – Research process
  • 152. RTI International Validation of the inference Validation – Comparison and check of fit – Deviant cases analysis – Member or respondent validation – Triangulation
  • 153. RTI International In summary Validity: do the data and findings accurately reflect the behavior , views etc of respondents? – Content validity: the respondent’s expression or articulation of the issue – Validity of interpretation: the researchers interpretation of the issue Reliability: would the data and findings be replicated if the study were repeated? Representational generalization: are the data and findings applicable to the wider population from which the sample is drawn?
  • 155. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org 10. Qualitative Research Ethics
  • 156. RTI International Qualitative Research Ethics Check the legal regulations for the State / Country Check your organisations Institutional Review Board (IRB) policies Understanding “informed consent” (verbal/written) Understanding “confidentiality” Basic principles of good social research – Participant safety (Physical/Emotional/Psychological) – Researcher safety Recompense
  • 157. RTI International RTI International is a trade name of Research Triangle Institute. www.rti.org References and Reading List
  • 158. RTI International Reading List Quality in Qualitative Evaluation: A Framework for assessing research evidence. A quality framework. Produced on behalf of the Cabinet Office by Liz Spencer, Jane Ritchie, Jane Lewis and Lucy Dillon, National Centre for Social Research. ISBN: 07715 04465 8. August 2003. Government Chief Social Researcher’s Office Crown Copyright 2003. Assessing the Quality of Qualitative Research. Patient Education and Counseling 90 (2013) 1–3. Qualitative Research Practice: A guide for Social Science Students and Researchers. Edited by Jane Ritchie, Jane Lewis, Carol NcNaughton Nicholls and Rachel Ormston. SAGE 2014. Focus Groups: Theory and Practice. D. W. Stewart and P.N. Shamdasani. Applied Social Research Methods, Volume 20. Focus Groups as Qualitative Research. David Morgan. Sage 1989. Focus Groups: A Practical Guide for Applied Research. Richard Krueger. Sage 1989. Sampling and Choosing Cases in Qualitative Research: a realistic approach. Nick Emmel. SAGE 2013. What Are Qualitative Research Ethics? Rose Wiles. Bloomsbury Academic 2013.