Critical Incident Technique in Information Literacy Research in the XXI Century

1. CriticalIncidentTechnique
in InformationLiteracyResearch
in theXXICentury
SabinaCisek
Institute of Information and Library Science, Jagiellonian
University in Kraków, Poland
EuropeanConference on Information Literacy 2016,
Prague, 10-13 October 2016

2. Contents
Critical IncidentTechnique (CIT) –
what is it?
My research goal, questions and
method
Findings
Conclusions
2

3. Critical Incident
Technique (CIT) –
what is it?
3

4. Creator
of the
method
John C. Flanagan
Flanagan, John C. (1954).The
critical incident technique.
Psychological Bulletin, vol. 51,
no. 4, p. 327-358.
4

5. Scholarly
disciplines
in which
CIT is used
 Business,
 Consumer studies,
 Education,
 LIS, in particular – information behavior
research
 Management,
 Marketing,
 Medicine,
 Military studies,
 Nursing,
 Political science,
 Psychology,
 Sociology
5

6. Definition
ofCIT (1)
 Critical IncidentTechnique (CIT) is
 a set of procedures
 used to systematically identify and
analyze
 human behaviors
 that have critical importance and
contribute to
 success or failure
 of individuals or organizations
 in specific situations (contexts).
6

7. Definition
ofCIT (2)
CIT is used to identify
effective (or ineffective)
behaviours relating to a
particular activity
7

8. What is a
critical
incident ?
(1)
 A critical incident
 is a single, „separable” event (fact,
factor),
 that significantly influences an
activity, phenomenon, task,
 and has an impact on someone’s
achievements, attitudes, success or
failure.
 Critical incidents need not to be
dramatic or unusual.
8

9. What is a
critical
incident ?
(2)
„Critical incidents are not ‘things’ which exist
independently of an observer and are awaiting
discovery like gold nuggets or desert islands
but like all data, critical incidents are created.
Incidents happen, but critical incidents are
produced by the way we look at a situation: a
critical incident is an interpretation of the
significance of an event.To take something as
a critical incident is a value judgment we
make, and the basis of that judgment is the
significance we attach to the meaning of the
incident.” [Tripp, 1993, p. 29]
9

10. Types of
critical
incidents
„A critical incident is considered
positive (effective) if it helps to
solve a problem and negative
(ineffective) if it fails to solve a
problem, creates new problems
or facilitates the need for further
actions.” [Cordes, 2012, p. 358]
10

11. Data
gathering
techniques
withinCIT
(1)
„The CIT encourages participants to
tell their story.” [Urquhart et al.,
2003]
The details and context of a single
event are collected – in order to
avoid generalizations (by the study
participants) from similar incidents.
11

12. Data
gathering
techniques
withinCIT
(2)
Empirical data for CIT come from:
 semi-structured interviews,
 observation,
 questionnaires,
 written accounts (essays, reports),
 other sources.
12

13. Features
ofCIT
Critical IncidentTechnique is a
qualitative approach,
particularly useful for exploratory
studies.
CIT is a flexible method, may be
modified and adapted to meet
the specific situation at hand.
13

14. My research
goal, questions
and method
14

15. My
general
goal
To check if, how and what
for the Critical Incident
Technique was used in
information literacy (IL)
research in years 2001-2015
15

16. My
research
questions
1. How often CIT was used in
empirical research on IL in years
2001-2015?
2. What IL aspects and problems
were studied using CIT in years
2001-15?
3. How that research was
conducted?
16

17. My
method
(1)
Systematic review (critical
literature review)
Qualitative content analysis
17

18. My
method
(2)
Six databases were searched
with query „critical incident”
AND „information literacy” (or
similar), with limiters
(publication dates 2001-2015)
18

19. My
method
(3)
The recall appeared to be very
limited → only few records in each
database
 Emerald Management Extra (5 results)
 Francis (3 results)
 LISTA (5 results)
 SAGE Journals (12 results)
 Scopus (6 results)
 Web of Science – SSCI (4 results)
19

20. My
method
(4)
After elimination od duplicates etc.
only 10 articles occurred to be
relevant for the present study –
and these were read in depth to
answer the research questions
(using a priori analytical categories)
20

21. Analytical
categories
related to
question 3
HowtheCIT-
basedresearch
onILwas
conducted?
 How critical incidents have been
defined/understood in the analyzed papers?
 What data gathering techniques (empirical data
sources) were employed?
 What types of sampling (convenience, probability,
purposive, self-selecting, snowballing, theoretical)
were used, and what were the sample sizes (the
number of respondents)?
 How the analyses of empirical data were
conducted?
 What conceptual or theoretical frameworks were
used?
 Were the reported research enterprises purely
qualitative or mixed methodology?
21

22. Findings
22

23. Research
question 1
HowoftenCIT
was usedinthe
empirical research
onILinyears
2001-15?
CIT was not common in the IL
research in years 2001-2015
(only 10 relevant works found)
23

24. Research
question 2
WhatILaspects
andproblems
were studied
usingCITinyears
2001-15?
 Critical reflection and critical thinking during library
instruction programs [Gilstrap, Dupree, 2008a; 2008b]
 Developing information seeking profiles of nursing
students to support development of IL programmes
[Stokes, Urquhart, 2015]
 Health information literacy of public library patrons
and librarians [Yi,You 2015;Yi, 2015]
 International students using online information
sources to learn (for study purposes) [Hughes, 2012;
2013]
 Library impact on student learning [Rodriguez, 2012]
 Relationship between critical thinking ability (in its
both cognitive and affective dimensions) and library
anxiety [Kwon, 2008]
 Technological (ICT), information seeking and
information gathering competencies of
undergraduate students [Cordes, 2012]
24

25. Research
question 3
(a)
HowtheCIT-
basedresearch on
ILwas conducted
inyears 2001-15?
 How critical incidents have been
defined/understood in the analyzed papers?
 Assignments based on online information resources,
recently completed by students [Hughes, 2013]
 Critical incidents related to a research project in history
[Rodriguez, 2012]
 „Most recent or most memorable experience of using
the library and its resources to write a research paper”
[Kwon, 2008, p. 120]
 Most rewarding and most surprising experiences in a
class; Moments of understanding and moments of
confusion about the instructional content [Gilstrap,
Dupree, 2008a]
 Negative critical incidents [Cordes, 2012]
 Specific health information activities, including health
information needs, situations causing health
information seeking, and obstacles/difficulties
encountered [Yi, 2015]
25

26. Research
question 3
(a)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
How critical incidents have been
defined/understood in the analyzed papers?
An example:
„1) At what moment in the class today did you feel
like you most understood the instructional
content?
2) At what moment in the class today did you feel
most confused about the instructional content?
3)What was your most rewarding experience in
today’s class?
4)What was your most surprising experience in
today’s class?” [Gilstrap, Dupree, 2008b, p. 424]
26

27. Research
question 3
(b)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
 What data gathering techniques (empirical
data sources) occured in the analyzed set of
10 papers?
 An activity log page for a class assignment (creating
a set of wiki pages) [Cordes, 2012]
 Critical incident type interviews [Stokes, Urquhart,
2015]
 Essays (written by students to report their critical
incidents from the past) [Kwon, 2008]
 Observation, observed task [Hughes, 2013]
 Questionnaires – on paper [Gilstrap, Dupree, 2008a;
2008b] and web-based [Rodriguez, 012]
 Semi-structured interviews [Hughes 2012; 2013;Yi,
2015]
27

28. Research
question 3
(c)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
 What types of sampling were used?
 Convenience sample, purposive sample
(usually students) [e.g., Cordes, 2012]
 Purposive sample [Yi, 2015]
 Random selection [Stokes, Urquhart, 2015]
 Self-selecting sample [Hughes 2012; 2013]
 What were the sample sizes (the number
of respondents)?
 11, 20, 22, 25, 127, 137, 321, 348
28

29. Research
question 3
(d)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
 How the analyses of empirical data were
done?
 Constant comparative method [Cordes,
2012]
 Content analysis [Cordes, 2012; Kwon,
2008;Yi, 2015]
 Inductive analysis [Hughes, 2013]
 Qualitative interpretative categorisation
QIC [Stokes, Urquhart, 2015]
 Statistical analysis [e.g., Gilstrap, Dupree,
2008a; 2008b]
29

30. Research
question 3
(e)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
 What conceptual or theoretical frameworks
were used?
 ACRL Information Literacy Competency Standards
for Higher Education [Cordes, 2012; Gilstrap, Dupree,
2008a; 2008b]
 Brookfield’s Critical Incident Questionnaire CIQ
[Gilstrap, Dupree, 2008a; 2008b]
 CaliforniaCriticalThinking Disposition Inventory
CCTDI [Kwon, 2008]
 Expanded Critical IncidentApproach ECIA [Hughes
2012; 2013]
 Phenomenography [Yi,You, 2015]
 Reflective learning and teaching [Gilstrap, Dupree,
2008a; 2008b]
 Self-perception theory of Daryl Bem [Yi, 2015]
30

31. Research
question 3
(f)
HowtheCIT-
basedresearch
onILwas
conductedin
years 2001-15?
Were the reported research
enterprises purely qualitative or
mixed methodology?
 Both, „pure” qualitative [e.g.,
Hughes, 2012; 2013] as well as
mixed methodology [e.g., Kwon,
2008;Yi, 2015]
31

32. Conclusions
32

33. The
advantage
ofCIT
Critical incident technique is based
on real-life experiences, and sees
the incidents (e.g., „instances” of
databases searching or information
seeking asignments) exactly as they
are perceived by their actual
partcicipants.
So, the true meaning and
importance level attached to them
by subjects may be discovered.
33

34. Analyzed
works,
found in
the six
databases
1. Cordes, S.: Student Technology Use in the Information-Seeking and Information-
Gathering Process: A Critical Incident Approach for Benchmarking Performance. E-
Learning and Digital Media 9 (4), 356–367 (2012)
2. Gilstrap, D.L., Dupree, J.:A Regression Model of PredictorVariables on Critical Reflection
in the Classroom: Integration of the Critical Incident Questionnaire and the Framework
for ReflectiveThinking. Journal of Academic Librarianship 34 (6), 469–481 (2008)
3. Gilstrap, D.L., Dupree, J.:Assessing Learning, Critical Reflection, and Quality Educational
Outcomes:The Critical Incident Questionnaire. College and Research Libraries 69 (5),
407–426 (2008)
4. Hughes, H.: An Expanded Critical Incident Approach for Exploring Information Use and
Learning. Library and Information Research 36 (112), 72–95 (2012)
5. Hughes, H.: International Students Using Online Information ResourcesTo Learn:
Complex Experience and Learning Needs. Journal of Further and Higher Education 37 (1),
126–146 (2013)
6. Kwon, N.: A Mixed-Methods Investigation of the Relationship between CriticalThinking
and Library Anxiety among Undergraduate Students inTheir Information Search
Process. College and Research Libraries, 69 (2), 117–131 (2008)
7. Rodriguez, D.: Answering Questions About Library Impact on Student Learning. In the
Library with the Lead Pipe, 1–14 (2012),
http://www.inthelibrarywiththeleadpipe.org/2012/answering-questions-about-library-
impact-on-student-learning/
8. Stokes, P., Urquhart, C.: Profiling Information Behaviour of Nursing Students: Part 2:
Derivation of Profiles. Journal of Documentation 71 (1), 52–79 (2015)
9. Yi,Y.J.,You, S.: Understanding the Librarian/UserGap in Perception of Health
Information Services:A Phenomenographic Approach. Journal of Librarianship and
Information Science 47 (4), 356–367 (2015)
10.Yi,Y.J.: Health Literacy and Health Information Behavior of Florida Public Library Users:
A Mixed Methods Study. Journal of Librarianship and Information Science 47 (1), 17–29
(2015)
34

35. Selected
bibliography
on the
Critical
Incident
Technique
 Butterflield, Lee D. et al. (2005). Fifty years of the critical incident
technique: 1954–2004 and beyond. Qualitative Research,Vol. 5, No.
4, p. 475-497.
 Fivars, Grace (2001). Critical Incident Technique Bibliography – 2001.
Introduction. http://www.apa.org/pubs/databases/psycinfo/cit-
intro.pdf
 Fivars, Grace; Fitzpatrick, Robert (2001). Critical Incident Technique
Bibliography – 2001.
http://www.apa.org/pubs/databases/psycinfo/cit-full.pdf
 Flanagan, John C. (1954).The critical incident technique.
Psychological Bulletin,Vol. 51, No. 4, p. 327-358.
 Hughes, Hilary (2007). Critical Incident Technique. In: Lipu, S.,
Williamson, K., Lloyd, A. eds. Exploring Methods in Information
Literacy Research. Wagga Wagga, N.S.W. : Centre for Information
Studies, Charles Sturt University, pp. 49-66.
 Tripp, David (1993).Critical Incidents inTeaching: Developing
Professional Judgment. London: Routledge.
 Urquhart, Christine et al. (2003). Critical incident technique and
explicitation interviewing in studies of information behavior. Library
and Information Science Research,Vol. 25, Issue 1, p. 63–88.
35