The word “analytics” has become a buzzword in current educational technology conversations, applied to everything from analysis of student work to LMS usage reporting to institutional analysis of ERP data. Broadly speaking, Learner Analytics refers to the analysis of student data using statistical techniques to improve decision-making. In the context of educational technology, Learner Analytics promises to improve our understanding of effective (and ineffective) student learning and technology usage. What progress have we seen in realizing this promise? This session offers a discussion of the promise of Learner Analytics, current research findings and tools, and explores examples from CSU Chico and the CSU Office of the Chancellor.

1. Learner Analytics
Realizing the “Big Data” Promise in the CSU
John Whitmer, CSU Office of the Chancellor & CSU Chico
Download slides at:
http://bit.ly/HqaHBF

2. Outline
1. Big Data & Analytics Promise(s)
2. National Examples of Tools & Systems
3. Learner Analytics @ Chico State
4. Q&A

3. 1. BIG DATA & ANALYTICS PROMISE(S)

4. Steve Lohr, NY Times, August 5, 2009

5. Draft DOE Report
released April 12
http://1.usa.gov/GDFpnI

6. Economist. (2010, 11/4/2010). Augmented business: Smart systems will disrupt lots of industries, and perhaps the entire
economy. The Economist.

7. Current GPA: 3.3
First in family to
attend college
SAT Score: 877
Hasn’t taken college-
level math
No declared major
Source: jisc_infonet @ Flickr.com
7
Source: jisc_infonet @ Flickr.com

8. What’s different with Big Data?
4 V’s:
1. Volume
2. Variety
3. Velocity
4. Variability
(IBM & Brian Hopkins, Forrester)
8

9. Academic Analytics
“Academic Analytics marries large data sets with
statistical techniques and predictive modeling to
improve decision making”
(Campbell and Oblinger 2007, p. 3)

10. Academic Analytics
1. Term adopted in 2005 ELI research
report (Goldstein & Katz, 2005)
– Response to widespread adoption ERP
systems, desire to use data collected
for improved decision making
– 380 respondents; 65% planned to
increase capacity in near future
2. Call to move from
transactional/operational
reporting to what-if analysis,
predictive modeling, and alerts
3. LMS identified as potential domain
for future growth 10

11. DD Screenshot

12. Learner Analytics:
“ ... measurement, collection, analysis and
reporting of data about learners and their
contexts, for purposes of understanding and
optimizing learning and the environments in
which it occurs.” (Siemens, 2011)

13. or said plainly:
 What are students doing?
 Does it matter?

14. Learner Analytics
1. Analyze combinations of data including:
– Frequency of ed tech usage (e.g. clickstream analysis)
– Student learning “outputs” (e.g. quiz scores, text answers)
– Student background characteristics (e.g. race/ethnicity)
– Academic achievement (e.g. grades, retention, graduation)
2. Current rsch: mostly data mining, not hypothesis-driven
3. More complex than Academic Analytics, considering:
– Immaturity of ed tech reporting functionality
– Translation of usage into meaningful activity
– No significant difference: not what technology used, it’s how
it’s used, who uses it, and for what purpose

15. A few promises of analytics for faculty
and students …
1. Provide behavioral data to investigate student
performance
2. Inform faculty about students succeeding or at
risk of failing a course
3. Warn students that they are likely to fail a
course – before it’s too late
4. Help faculty evaluate the effectiveness of
practices and course designs
5. Customize content and learning activities
(e.g. adaptive learning materials)

16. What’s the promise of analytics for
academic technologists?
1. Decision-making based on actual practices (not
just perceptions) and student outcomes
2. Support movement of A.T. into strategic role re:
teaching and learning by:
– demonstrating the link between technology
and learning
– distinguishing our role from a technology
infrastructure provider

17. Our 2 biggest barriers
Image Source: http://bit.ly/Hq9Cdg

18. Image Source: Utopian Inc http://bit.ly/Hq9sCq

19. Image Source: Privacy in the Cloud: http://bit.ly/HrF6zk

20. 2. NATIONAL EXAMPLES OF TOOLS &
SYSTEMS

21. SIGNALS
Purdue Signals Project http://www.itap.purdue.edu/studio/signals/

22. SNAPP
SNAPP (Social Networks Adapting Pedagogical Practice) http://www.snappvis.org/

23. KHAN
Khan Academy http://www.khanacademy.org/

24. OLI
CM Open Learning Initiative http://oli.web.cmu.edu/openlearning/initiative/process

25. PARCHMENT
Parchment http://www.parchment.com/c/my-chances/

26. 3. LEARNER ANALYTICS @ CHICO
STATE
26

27. LMS Learner Analytics @ Chico State
Campus-wide
– How are faculty & students using the LMS?
– What meaningful activities are being conducted?
– How does that usage vary by student background, by college, by
department?
Course level
– What is the relationship between LMS actions, student
background characteristics and student academic achievement?
(6 million dollar question)
– Intro to Religious Studies: redesigned in Academy eLearning,
increased enrollment from 80 to 327 students first semester
Ultimate goal: provide faculty and administrators with what-if
modeling tools to identify promising practices and early alerts
27

28. 28
Chart from Scott Kodai, Chico State

29. CSU Practice

30. INTRODUCTION TO RELIGIOUS
STUDIES (RELS 180)

34. CLOSING THOUGHTS
34

35. Call to Action
1. Metrics reporting is the foundation for Analytics
2. Don’t need to wait for student performance
data; good metrics can inspire access to
performance data
3. You’re *not* behind the curve, this is a rapidly
emerging area that we can (should) lead ...
4. If there’s any ed tech software folks in the
audience, please help us with better reporting!

36. Want more? Resources on Analytics
 Googledoc: http://bit.ly/HrG6Dm

37. Q&A and Contact Info
Resources Googledoc: http://bit.ly/HrG6Dm
Contact Info:
• John Whitmer (jwhitmer@csuchico.edu)
• Hillary C Kaplowitz (hillary.kaplowitz@csun.edu)
• Berggren, Kate E (kate.berggren@csun.edu)
Download presentation at:
http://bit.ly/HqaHBF
37