Presentation given by Mark Billinghurst on August 26th 2015 about Augmented Reality and industrial training applications.

1. GETTING REAL ABOUT AR
Mark Billinghurst
mark.billinghurst@unisa.edu.au
August 26th 2015

2. 50 Years of Progress
• From lab to living room
• AR devices in every pocket

3. 1998 vs. 2008
CPU: 300 Mhz
HDD; 9GB
RAM: 512 mb
Camera: VGA 30fps
Graphics: 500K poly/sec
1998: SGI O2 2008: Nokia N95
CPU: 332 Mhz
HDD; 8GB
RAM: 128 mb
Camera: VGA 30 fps
Graphics: 2m poly/sec

4. AR Business Today
• Around $600 Million USD in 2014 (>$2B 2015)
• > 80% Games and Marketing applications

5. Market Projections
cf. 2014 computer game market = $84 Billion

6. Crossing the Chasm
http://www.gartner.com/newsroom/id/3114217

7. Getting from Here to There
• New markets
• Medical, Education, Industry
• New types of applications
• Training, Information Presentation, Collaboration
• New devices/platforms
• Head mounted displays, projected AR
• New Interaction Techniques
• Gesture based, empathic interfaces, multimodal

8. 2012 – Iron Man 2

9. AR for Industrial Training
• Potential Benefits
• Training on target equipment
• Showing spatial information in place
• Allow worker to remain focused on work site
• Able to view content from any angle

10. Case Study 1: Vehicle Maintenance
• Using AR for armoured vehicle maintence
Henderson, S. J., & Feiner, S. (2009, October). Evaluating the benefits of augmented reality for task localization
in maintenance of an armored personnel carrier turret. In ISMAR 2009. 8th IEEE International Symposium on
(pp. 135-144). IEEE.

11. Current System
• Electronic Manual on laptop computer
• 2D images, pdf, fixed views

12. Interfaces
• Three display conditions
• AR, HUD, LCD panel
• Using stereo video-see through AR HMD system
AR HUD LCD

13. Experiment
• 6 Mechanics as subjects
• Subjects completed 18 common tasks
• Installing fasteners, connecting cables, etc
• Measured
• Performance time, localisation time
• Errors made
• Head movement, exertion
• Subjective feedback (satisfaction, intuitiveness)

14. Key Results
• No difference in times between AR & LCD
• But LCD (34.5s) faster than HUD (55.2s)
• AR able to locate task areas faster
• 4.9s in AR, vs 9.2s in LCD, vs. 11.1s in HUD
• Fewer head motions in AR
• Typically 50% less motions than other conditions
• AR rated as good as LCD in satisfaction
• AR rated as most intuitive of conditions

15. Case Study 2: Boeing & Iowa State
• Work instructions presented in 3 conditions
• Desktop interface
• Tablet interface
• Tablet AR interface
T. Richardson, S. Gilbert, J. Holub, F. Thompson, A. MacAllister, R. Radkowski, E. Winer, P. Davies, and S.
Terry. "Fusing Self-Reported and Sensor Data from Mixed-Reality Training."
Desktop Interface Tablet AR Interface

16. Experiment Design
• Reproduced industrial work cell
• Task
• Subjects completed a five-step physical assembly process
• Data Collected
• User motion, activity analysis
• Task performance (accuracy, completion time)
• Time looking at instructions, moving between work areas
• Subjective feedback

17. Results
• Users liked AR condition the best
• Fewer errors in AR condition, faster performance time
• AR has 30% faster performance, 90% few errors
Performance Time Errors

18. Commercial Systems
• Ngrain
• http://www.ngrain.com/
• Training authoring tool
• Model based AR tracking
• ScopeAR
• http://www.scopear.com/
• Remote assistance
• Image based tracking

19. Ngrain

20. IntelligentTraining
• Most AR systems stupid
• Don’t recognize user behaviour
• Don’t provide feedback
• Don’t adapt to user
• Especially important for training
• Scaffolded learning
• Moving beyond check-lists of actions

21. Intelligent Interfaces
• AR interface + intelligent tutoring system
• ASPIRE constraint based system (from UC)
• Constraints
• relevance cond., satisfaction cond., feedback
Westerfield, G., Mitrovic, A., & Billinghurst, M. (2013). Intelligent Augmented
Reality Training for Assembly Tasks. In Artificial Intelligence in Education (pp.
542-551). Springer Berlin Heidelberg.

22. Domain Ontology

23. Intelligent Feedback
• Actively monitors user behaviour
• Implicit vs. explicit interaction
• Provides corrective feedback

25. Evaluation Results
• 16 subjects, with and without ITS
• Improved task completion
• Improved learning

26. IntelligentAgents
• AR characters
• Virtual embodiment of system
• Multimodal input/output
• Examples
• AR Lego,Welbo, etc
• MrVirtuoso
• AR character more real, more fun
• On-screen 3D and AR similar in usefulness
Wagner, D., Billinghurst, M., & Schmalstieg, D. (2006). How real should virtual
characters be?. In Proceedings of the 2006 ACM SIGCHI international
conference on Advances in computer entertainment technology (p. 57). ACM.

27. Conclusions
• AR is becoming commonly available
• In order to achieve significant growth AR needs to
• Expand into new markets
• Move onto new platforms
• Create new types of applications
• AR for Training is a particularly promising area
• Spatial skills, intelligent interfaces

28. www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au