Cherie Lucas School of Pharmacy, Graduate School of Health

1. UTS CRICOS 00099F
24/7 Instant Feedback on Writing:
Integrating AcaWriter into your Teaching
Simon Buckingham Shum (Connected Intelligence Centre)
Cherie Lucas (School of Pharmacy, Graduate School of Health)
Monthly briefing: 3rd Thursday, 3.30pm

2. WELCOME &
INTRODUCTION!
UTS CIC 2

3. AGENDA
• Introduction to AcaWriter (10 min)
• Learning Context 1 (Law Essay & Business Report) (10 min)
• Learning Context 2 (Research Abstract) (5 min)
• Learning Context 3 (Critical Reflection on practice) (10 min)
• Hands-on activity (10 min)
• Sharing and Discussion (10 min)
• Follow up with us to take the next step…
UTS CIC 3

4. AcaWriter now a UTS-wide tool
May 2020 announcement to students:
UTS CIC 4
https://www.uts.edu.au/current-students/news/get-instant-feedback-your-academic-writing

5. Academic Writing Analytics: a long-term R&D project in CIC
https://cic.uts.edu.au/tools/awa
UTS CIC 5

6. UTS CIC 6

7. AcaWriter aims to help…
• Students need precise, timely feedback on academic writing
to learn how to ‘make their thinking visible’
• Humanly impossible to provide that to all students, 24/7
• AcaWriter is a website providing instant formative feedback
(not grading) on “rhetorical moves” in writing
• Two types of writing genres are supported
1. Analytical Writing
e.g. Essay, Research Paper Introduction or Abstract
2. Reflective Writing
e.g. Learning Journal, Reflective Essay, Project Review
UTS CIC 7

8. Rhetorical moves in analytical writing
A rhetorical move is “a discoursal or rhetorical unit that performs a coherent
communicative function in a written or spoken discourse” (Swales, 2004).
UTS CIC 8

9. How to integrate AcaWriter into your course?
UTS CIC 9

10. UTS CIC 10
Shibani, A., Knight, S. and Buckingham Shum, S. (2019). Contextualizable Learning Analytics Design: A Generic Model, and Writing Analytics Evaluations. Proc. 9th
International Conference on
Learning Analytics & Knowledge (LAK19). ACM Press, NY, pp. 210-219. DOI: https://doi.org/10.1145/3303772.3303785. Eprint: https://tinyurl.com/lak19clad
How to integrate AcaWriter into your course?

11. LEARNING CONTEXT 1
Law essay writing
UTS CIC 11

12. UTS CIC 12

13. Law essay context – Feedback tuning
UTS CIC 13

14. UTS CIC 14
Law essay context – Feedback tuning

15. Student Writing Activity (a learning design pattern you can adopt/adapt)
UTS CIC 15

16. Matching
exercise
Understanding
instructor’s rubric by
matching sample
sentences to rubric
elements
UTS CIC 16
A version of this
interactive is in the
UTS Open module
(see end)
Students
drag + drop rubric
elements next to
paragraphs

17. Viewing sample revisions
UTS CIC 17

18. Grading a sample text prior to using AcaWriter
UTS CIC 18

19. Self-assessment task to promote deeper reflection on revisions
UTS CIC 19

20. Peer-discussion and revision activity to promote deeper
reflection on the feedback
UTS CIC 20

21. End of activity feedback survey
UTS CIC 21

22. Accounting students – Tips tab tuned to the assignment
UTS CIC 22

23. Accounting students – Tips tab tuned to the assignment
UTS CIC 23

24. Promoting deeper engagement with the feedback:
annotate the PDF report
24
Prompts for self-evaluation

25. 25
Promoting deeper engagement with the feedback
5) Spend some time working on the soft copy of your
report in the AcaWriter tool, adjusting your report and
re-run the AcaWriter analysis. What effect did your
changes have in on the feedback from AcaWriter?
6) After using AcaWriter what changes did you/will you
make to your report?

26. Accounting Writing Activity
UTS CIC 26
Week Activity
Week 1 Homework Online activity
Week 2 Class exercise – Peer discussion
Week 3 Submitting their draft report
Week 4 Self-evaluation of draft report (with AcaWriter
feedback)
Week 5, Week 6 Peer feedback (optional) and working on final report
Week 7 Report submission

27. LEARNING CONTEXT 2
HDR research writing
UTS CIC 27

28. UTS CIC 28
Adapting AcaWriter to the “CARS” model
Create A Research Space

29. Example AcaWriter feedback on an abstract
UTS CIC 29

30. UTS CIC 30
Embedding AcaWriter in an HDR activity

31. UTS CIC 31
Embedding AcaWriter in an HDR activity

32. UTS CIC 32
Embedding AcaWriter in an HDR activity

33. UTS Online Open Course: Writing an Abstract
https://open.uts.edu.au/uts-open/faculty/connected-intelligence-centre/writing-an-abstract/
33

34. LEARNING CONTEXT 3
Reflective Writing
UTS CIC 34

35. Learning reflective writing:
Distillation of theory and pedagogy into a framework

36. Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proceedings of 7th
International Conference on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver, BC, Canada. (ACM Press). DOI: http://dx.doi.org/10.1145/3027385.3027436
Learning reflective writing:
Distillation of theory and pedagogy into a framework

37. Learning reflective writing:
Simplification of framework à a visual language
Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proceedings of 7th
International Conference on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver, BC, Canada. (ACM Press). DOI: http://dx.doi.org/10.1145/3027385.3027436

38. Information design + Interface design
The key to automated annotations of the reflection

39. AcaWriter feedback on reflective writing
UTS CIC 39

40. UTS CIC 40
Rubric for a Pharmacy Masters Critical Reflection Essay

41. Elements of Reflection – How Summative Rubric is related to
Formative Feedback prompted by AcaWriter?
Stages of Reflection1
Stage 1. Returning to the event (Describe incident) CONTEXT
Stage 2. Attending to Feelings (AcaWriter: Blue Square prompt, words associated with strong feelings, also includes
reactions and thoughts) CONTEXT
Stage 3. Association (AcaWriter: Pink Circle prompt Challenges in New Knowledge learned) CHALLENGE
Stage 4. Integration (Bridging the gap; AcaWriter Expressions of self-critique) CHALLENGE
Stage 5. Validation (AcaWriter: Beliefs, learning knowledge, Depth: internalisation) CHALLENGE
Stage 6. Appropriation Why you have these beliefs? (AcaWriter: Bold writing personally applied) CHALLENGE
Stage 7. Outcomes of reflection (AcaWriter: Triangle prompt and Green Writing for self-critique: Knowledge leads to
change in perspective, eg new strategy, tool CHANGE
1. Boud D, Keough R, Walker D. Promoting reflection in learning: a model. Reflection: Turning Experience into Learning. London Kogan Page : NY, Nicols Pub: 1985:18-40
41

42. UTS CIC 42
Co-designing feedback with an academic:
Pharmacy placement reflection report

43. Co-design feedback (Contd..)
• The feedback template consists of a set of rules,
which will trigger feedback device to students given
the presence or absence of particular writing
analytics features.
• An example of co-designed feedback rules in
pharmacy. For a given feature (col.1), we
considered what should be said to the student if it
appeared (col.2) or was missing (col.3). More
complex patterns can be defined.
43

44. Example student feedback
“It prompts you into developing those reflective ideas. ..you can only go so far on your own” 1
“It makes you think of things that you normally would disregard” 1
“because it (the feedback) was in front of me, I could see what I was thinking” 1
“Building self-confidence because it seems to provide “clarity” of your thoughts and reasoning” and “a
better understanding of yourself “ 1
“makes you discover things not only about things that you’re doing but things about yourself that you really
never thought of” 1
Utilising AW makes students more confident with writing submissions (pretest 3.2, posttest 3.7, p=0.05) 2
“Prompted me to follow through with the reflection to the last step of the process - I had written about my
thoughts and feelings, discussed challenges, but had not followed through with reflecting on how this
can lead to change” 3
44
1 Lucas C, Buckingham Shum S, Liu M, Bebawy M. Implementing AcaWriter as a novel strategy to support pharmacy students’ reflective practice
in scientific research. Am J Pharm Edu (Accepted Manuscript)
2
Lucas, C, Gibson, Buckingham Shum S. Pharmacy students’ utilization of an online tool for immediate formative feedback on reflective writing
tasks. Am J Pharm Edu, 2019. https://doi.org/10.5688/ajpe6800
3
Gibson et al. 2017 Reflective writing analytics for actionable feedback. Proceedings of the LAK’17 Conference, Canada (Best Full Paper Award)
https://dl.acm.org/doi/proceedings/10.1145/3027385

45. SUMMARY:
3 WAYS OF ADAPTING ACAWRITER
MAPPING TO THE 3 TABS
UTS CIC 45

46. Levels of customisation for a specific subject and students
UTS CIC 46
AcaWriter comes loaded with several
different ‘genres’ of writing, which
may match your requirements fine.
But we can provide you with a
template to help you think through
what (if anything) you might change
in AcaWriter’s feedback
There are different depths of
adaptation possible — requiring
different levels of effort from both you
and the tech team

47. ACAWRITER
HANDS-ON
& QA
UTS CIC 47

48. Hands-on / Q&A
• Go to https://acawriter.uts.edu.au and login with your staff ID.
• Create your own assignment using a genre type.
• If you have a sample text to try, paste it into the editor and click on
Get Feedback.
• Revise the text to see if it changes the feedback.
• Or explore later, and ask us questions now…
• What are your thoughts on the potential of AcaWriter for your
teaching?
• Do you have any concerns?
UTS CIC 48

49. Educator resources: AcaWriter Educators portal
https://www.uts.edu.au/research-and-teaching/teaching-and-research-integration/acawriter/educators
UTS CIC 49

50. UTS Open with interactive exercises to learn “the moves”
https://open.uts.edu.au/uts-open/study-area/communication--media/writing-an-abstract
UTS CIC 50
Note: While focused on Abstracts, since this uses all
the Analytical moves, this is potentially relevant for
many genres of student writing

51. UTS CRICOS 00099F
This was just the
conversation starter.
Do get in touch...
Enquiries:
cic@uts.edu.au