Course on Designing Wearable interfaces taught by Mark Billinghurst at Siggraph 2014. Presented on August 10th, 2014 from 10:45am - 12:15pm. The course focuses mainly on design guidelines and tools for rapid prototyping for Google Glass.

2. The Glass Class: Designing Wearable Interfaces
Mark Billinghurst
The HIT Lab NZ, University of Canterbury
The 41st International
Conference and Exhibition
on Computer Graphics and
Interactive Techniques

3. INTRODUCTION

4. Mark Billinghurst
▪ Director of The HIT Lab NZ, University of
Canterbury
▪ PhD Univ. Washington
▪ Research on AR, mobile HCI,
Collaborative Interfaces, Wearables
▪ Joined Glass team at Google [x] in 2013

7. How do you Design for this?

8. Course Goals
In this course you will learn
▪ Introduction to head mounted wearable computers
▪ Understanding of current wearable technology
▪ Key design principles/interface metaphors
▪ Rapid prototyping tools
▪ Areas for future research

9. What You Won’t Learn
▪ Who are the companies/universities in this space
▪ See the Siggraph exhibit floor
▪ Designing for non-HMD based interfaces
▪ Watches, fitness bands, etc
▪ How to develop wearable hardware
▪ optics, sensor assembly, etc
▪ Evaluation methods
▪ Experimental design, statistics, etc

10. Schedule
• 10:45 am Introduction
• 10:55 am Technology Overview
• 11:05 am Design Guidelines
• 11:25 am Prototyping Tools
• 11:55 am Example Applications
• 12:05 am Research Directions/Resources

11. A Brief History of Computing
Trend
▪ Smaller, cheaper, faster, more intimate
▪ Moving from fixed to handheld and onto body
1950’s
1980’s
1990’s

12. Wearable Computing
▪ Computer on the body that is:
▪ Always on
▪ Always accessible
▪ Always connected
▪ Other attributes
▪ Augmenting user actions
▪ Aware of user and surroundings

13. Desk Lap Hand Head

14. The Ideal Wearable
▪ Persists and Provides Constant Access: Designed for
everyday and continuous use over a lifetime.
▪ Senses and Models Context: Observes and models the
users environment, mental state, it’s own state.
▪ Augments and Mediates: Information support for the user in
both the physical and virtual realities.
▪ Interacts Seamlessly: Adapts its input and output modalities
to those most appropriate at the time.
Starner, T. E. (1999). Wearable computing and contextual awareness
(Doctoral dissertation, Massachusetts Institute of Technology).

15. History of Wearables
▪ 1960-90: Early Exploration
▪ Gamblers and Custom build devices
▪ 1990 - 2000: Academic, Military Research
▪ MIT, CMU, Georgia Tech, EPFL, etc
▪ 1997: ISWC conference starts
▪ 1995 – 2005+: First Commercial Uses
▪ Niche industry applications, Military
▪ 2010 - : Second Wave of Wearables

16. Origins - The Gamblers
• Thorp and Shannon (1961)
– Wearable timing device for roulette prediction
• Keith Taft (1972)
– Wearable computer for blackjack card counting
Belt computer Shoe Input Glasses Display

17. Steve Mann (1980s - )
http://wearcomp.org/

18. Thad Starner (1993 - )

19. MIT Wearable Computing (1993-)
http://www.media.mit.edu/wearables/

20. CMU Wearables (1991–2000)
▪ Industry focused wearables
▪ Maintenance, repair
▪ Custom designed interface
▪ Dial/button input
▪ Rapid prototyping approach
▪ Industrial designed, ergonomic
http://www.cs.cmu.edu/afs/cs/project/vuman/www/frontpage.html

21. Prototype Applications
▪ Remembrance Agent
▪ Rhodes (97)
▪ Augmented Reality
▪ Feiner (97), Thomas (98)
▪ Remote Collaboration
▪ Garner (97), Kraut (96)
■ Maintenance
■ Feiner (93), Caudell (92)

22. Mobile AR: Touring Machine (1997)
▪ University of Columbia
▪ Feiner, MacIntyre, Höllerer, Webster
▪ Combined
▪ See through head mounted display
▪ GPS tracking, Orientation sensor
▪ Backpack PC (custom)
▪ Tablet input
Feiner, S., MacIntyre, B., Höllerer, T., & Webster, A. (1997). A touring machine: Prototyping 3D mobile
augmented reality systems for exploring the urban environment. Personal Technologies, 1(4), 208-217.

23. Touring Machine View
▪ Virtual tags overlaid on the real world
▪ “Information in place”

24. Early Commercial Systems
▪ Xybernaut (1996 - 2007)
▪ Belt worn, HMD, 200 MHz
▪ ViA (1996 – 2001)
▪ Belt worn, Audio Interface
▪ 700 MHz Crusoe
■ Symbol (1998 – 2006)
■ Wrist worn computer
■ Finger scanner

25. Google Glass (2011 - )

26. The Second Wave of Wearables
▪ Vuzix M-100
▪ $999, professional
▪ Recon Jet
▪ $600, more sensors, sports
▪ Opinvent
▪ 500 Euro, multi-view mode
▪ Motorola Golden-i
▪ Rugged, remote assistance

27. Projected Market

29. Summary
Wearables are a new class of computing
Intimate, persistent, aware, accessible
Evolution over 50 year history
Backpack to head worn
Custom developed to consumer ready device
Enables new applications
Collaboration, memory, AR, industry, etc
Many head worn wearables are coming

30. TECHNOLOGY

31. ▪ fafds

32. Enabling Technologies (1989-)
▪ Private Eye Display (Reflection Technologies)
▪ 720 x 280 dipslay
▪ Vibrating mirror
▪ Twiddler (Handykey)
▪ Chording keypad
▪ Mouse emulation

33. Tin Lizzy (Platt, Starner, 1993)
▪ General Purpose Wearable
▪ 150 MHz Pentium CPU
▪ 32-64 Mb RAM, 6 GB HDD
▪ VGA display
▪ 2 PCMCIA slots
▪ Cellular modem
http://www.media.mit.edu/wearables/lizzy/lizzy/index.html

34. • asda

36. ▪ Hardware
▪ CPU TI OMAP 4430 – 1 Ghz
▪ 16 GB SanDisk Flash, 2 GB Ram
▪ Input
▪ 5 mp camera, 720p recording, microphone
▪ InvenSense MPU-9150 inertial sensor
▪ Output
▪ Bone conducting speaker
▪ 640x360 micro-projector display
Google Glass Specs

37. Glass Display

38. ViewThrough Google Glass
Always available peripheral information display
Combining computing, communications and content capture

39. Google Glass Demo

40. Google Glass User Interface
• dfasdf

41. Timeline Metaphor

42. User Experience
• Truly Wearable Computing
– Less than 46 ounces
• Hands-free Information Access
– Voice interaction, Ego-vision camera
• Intuitive User Interface
– Touch, Gesture, Speech, Head Motion
• Access to all Google Services
– Map, Search, Location, Messaging, Email, etc

43. Types of Head Mounted Displays
Occluded
See-thru
Multiplexed

44. Multiplexed Displays
▪ Above or below line of sight
▪ Strengths
▪ User has unobstructed view of real world
▪ Simple optics/cheap
▪ Weaknesses
▪ Direct information overlay difficult
▪ Display/camera offset from eyeline
▪ Wide FOV difficult

45. Vuzix M-100
▪ Monocular multiplexed display ($1000)
▪ 852 x 480 LCD display, 15 deg. FOV
▪ 5 MP camera, HD video
▪ GPS, gyro, accelerometer

46. Optical see-through HMD
Virtual images
from monitors
Real
World
Optical
Combiners

47. Epson Moverio BT-200
▪ Stereo see-through display ($700)
▪ 960 x 540 pixels, 23 degree FOV, 60Hz, 88g
▪ Android Powered, separate controller
▪ VGA camera, GPS, gyro, accelerometer

48. Strengths of optical see-through
▪ Simpler (cheaper)
▪ Direct view of real world
▪ Full resolution, no time delay (for real world)
▪ Safety
▪ Lower distortion
▪ No eye displacement
▪ see directly through display

49. Video see-through HMD
Video
cameras
Monitors
Graphics
Combiner
Video

50. Vuzix Wrap 1200DXAR
▪ Stereo video see-through display ($1500)
■ Twin 852 x 480 LCD displays, 35 deg. FOV
■ Stereo VGA cameras
■ 3 DOF head tracking

51. Strengths of Video See-Through
▪ True occlusion
▪ Block image of real world
▪ Digitized image of real world
▪ Flexibility in composition, match time delays
▪ More registration, calibration strategies
▪ Wide FOV is easier to support
▪ wide FOV camera

52. Input Options
▪ Physical Devices
▪ Keyboard, Pointer, Stylus
▪ Natural Input
▪ Speech, Gesture
▪ Other
▪ Physiological sensors

53. Twiddler Input
▪ Chording or multi-tap input
▪ Possible to achieve 40 - 60 wpm after 30+ hours
▪ cf 20 wpm on T9, or 60+ wpm for QWERTY
Lyons, K., Starner, T., Plaisted, D., Fusia, J., Lyons, A., Drew, A., & Looney, E. W. (2004, April).
Twiddler typing: One-handed chording text entry for mobile phones. In Proceedings of the SIGCHI
conference on Human factors in computing systems (pp. 671-678). ACM.

54. Virtual Keyboards
▪ In air text input
▪ Virtual QWERTY keyboard up to 20 wpm
▪ Word Gesture up to 28 wpm
▪ Handwriting around 20-30 wpm
A. Markussen, et. al. Vulture: A Mid-Air Word-Gesture Keyboard (CHI 2014)

55. Unobtrusive Input Devices
▪ GestureWrist
▪ Capacitive sensing, changes with hand shape
Rekimoto, J. (2001). Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In
Wearable Computers, 2001. Proceedings. Fifth International Symposium on (pp. 21-27). IEEE.

56. Unobtrusive Input Devices
▪ GesturePad
▪ Capacitive multilayered touchpads
▪ Supports interactive clothing

57. Skinput
Using EMG to detect muscle activity
Tan, D., Morris, D., & Saponas, T. S. (2010). Interfaces on the go.
XRDS: Crossroads, The ACM Magazine for Students, 16(4), 30-34.

58. Issues to Consider
▪ Fatigue
▪ “Gorrilla” Arm from free-hand input
▪ Comfort
▪ People want to do small gestures by waist
▪ Interaction on the go
▪ Can input be done while moving?

59. DESIGN GUIDELINES

60. INTERACTION DESIGN

61. Design For the Device
• Simple, relevant information
• Complement existing devices

62. Last year Last week NowForever
The Now machine
Focus on location, contextual and timely
information, and communication.

63. Don’t design an app
Glass OS is time-based model, not an app model.

64. The
world
is
the
experience
Get
the
interface
and
interac-ons
out
of
the
way.

65. It's
like
a
rear
view
mirror
Don't
overload
the
user.
S-ck
to
the
absolutely
essen-al,
avoid
long
interac-ons.
Be
explicit.

66. Micro
Interac8ons
The
posi-on
of
the
display
and
limited
input
ability
makes
longer
interac-ons
less
comfortable.
Using
it
shouldn’t
take
longer
than
taking
out
your
phone.

67. Micro-Interactions
On mobiles people split attention
between display and real world

68. Time Looking at Screen
Oulasvirta, A. (2005). The fragmentation of attention in mobile
interaction, and what to do with it. interactions, 12(6), 16-18.

69. Design for MicroInteractions
▪ Design interaction less than a few seconds
– Tiny bursts of interaction
– One task per interaction
– One input per interaction
▪ Benefits
– Use limited input
– Minimize interruptions
– Reduce attention fragmentation

70. Make it Glanceable
• Seek to rigorously reduce information density.
• Design for recognition, not reading.
Bad Good

71. Reduce the Number of Info Chunks
• You are designing for recognition, not reading.
• Reducing the total # of information chunks will
greatly increase the glanceability of your design.
• .
1
2
3
1
2
3
4
5 (6)

72. Design single interactions < 4 s
Eye movements
For 1: 1 230ms
For 2: 1 230ms
For 3: 1 230ms
For 4: 3 690ms
For 5: 2 460ms
~1,840ms
Eye movements
For 1: 1-2 460ms
For 2: 1 230ms
For 3: 1 230ms
~920ms
1
2
3
1
2
3
4
5 (6)

73. Test the glanceability of your design

74. Don’t Get in the Way
• Enhance, not replace, real world interaction

75. Design for Interruptions
▪ Gradually increase engagement and attention load
▪ Respond to user engagement
Receiving SMS on Glass
“Bing”
Tap
Swipe
Glass
Show Message Start Reply
User Look
Up
Say
Reply

76. Do one thing at a time

77. Keep it Relevant
• Information at the right time and place

78. Design for Context

79. Avoid the Unexpected
• Don’t send unexpected content at wrong times
• Make it clear to users what your application does

80. Build for People
• Use imagery, voice interaction, natural gestures
• Focus on fire and forget interaction model

81. VISUAL DESIGN

82. Transparent
displays
are
tricky
Colors
are
funny
and
inconsistent.
You
can
only
add
light
to
a
scene,
not
cover
anything
up.
Mo-on
can
be
disorien-ng.
Clarity,
contrast,
brightness,
visual
field
and
aHen-on
are
important.

84. White is your new black

85. Establish hierarchy with color
White is your <h1> and grey is your <h2> or <h3>.
Footer text - establishing time, attribution, or
distance - is the only place with smaller font size.

86. Use brand-specific typography

87. Test your design indoors + outdoors

88. EXAMPLE APPLICATIONS

89. • https://glass.google.com/glassware
Glassware Applications

93. Virtual Exercise Companion
• GlassFitGames
– http://www.glassfitgames.com

94. Vipaar Telemedicine
• Vipaar + UAB - http://www.vipaar.com
• Endoscopic view streamed remotely
• Remote expert adds hands – viewed in Glass

95. CityViewAR
• Using AR to visualize Christchurch city buildings
– 3D models of buildings, 2D images, text, panoramas
– ARView, Map view, List view
– Available on Android/iOS market

96. CityViewAR on Glass
• AR overlay of virtual buildings in Christchurch

97. PROTOTYPING TOOLS

98. How can we quickly
prototype Wearable
experiences with
little or no coding?

99. Why Prototype?
▪ Quick visual design
▪ Capture key interactions
▪ Focus on user experience
▪ Communicate design ideas
▪ “Learn by doing/experiencing”

100. Prototyping Tools
▪ Static/Low fidelity
▪ Sketching
▪ User interface templates
▪ Storyboards/Application flows
▪ Screen sharing
▪ Interactive/High fidelity
▪ Wireframing tools
▪ Mobile prototyping
▪ Native Coding

101. Important Note
▪ Most current wearables run Android OS
▪ eg Glass, Vuzix, Atheer, Epson, etc
▪ So many tools for prototyping on Android
mobile devices will work for wearables
▪ If you want to learn to code, learn
▪ Java, Android, Javascript/PHP

102. Typical Development Steps
▪ Sketching
▪ Storyboards
▪ UI Mockups
▪ Interaction Flows
▪ Video Prototypes
▪ Interactive Prototypes
▪ Final Native Application
Increased
Fidelity &
Interactivity

103. Low Fidelity Tools
• Sketching
• GlassSim
• UI Templates
• Storyboards
• GlassWare flow designer
• Android Design Preview
• Video sketches

104. High Fidelity Tools
• UXPin/Proto.io
• JustinMind
• Processing
• WearScript
• Unity3D
• Native Coding

105. Sketched Interfaces
▪ Sketch + Powerpoint/Photoshop/Illustrator

106. GlassSim – http://glasssim.com/
▪ Simulate the view through Google Glass
▪ Multiple card templates

107. GlassSim Card Builder
▪ Use HTML for card details
▪ Multiple templates
▪ Change background
▪ Own image
▪ Camera view

108. GlassSim Samples

109. Glass UI Templates
▪ Google Glass Photoshop Templates
▪ http://glass-ui.com/
▪ http://dsky9.com/glassfaq/the-google-glass-psd-template/

110. Application Storyboard
▪ http://dsky9.com/glassfaq/google-glass-
storyboard-template-download/

111. Glassware Flow Designer
• Features
– Design using common patterns and layouts
– Specify interactions and card flow
– Share with other designers
• Available from:
– https://developers.google.com/glass/tools-
downloads/glassware-flow-designer

112. Example Flow
• Blah

113. Screen Sharing
▪ Android Design Preview
– Tool for sharing screen content onto Glass
– https://github.com/romannurik/
AndroidDesignPreview/releases
Mac Screen
Glass

114. ▪ Series of still photos in a movie format.
▪ Demonstrates the experience of the product
▪ Discover where concept needs fleshing out.
▪ Communicate experience and interface
▪ You can use whatever tools, from Flash to iMovie.
Video Sketching

115. See https://vine.co/v/bgIaLHIpFTB
Example: Glass Vine UI

116. Limitations
▪ Positives
▪ Good for documenting screens
▪ Can show application flow
▪ Negatives
▪ No interactivity/transitions
▪ Can’t be used for testing
▪ Can’t deploy on wearable
▪ Can be time consuming to create

117. Interactive Wireframing
▪ Developing interactive interfaces/wireframes
▪ Transitions, user feedback, interface design
▪ Web based tools
▪ UXpin - http://www.uxpin.com/
▪ proto.io - http://www.proto.io/
▪ Native tools
▪ Justinmind - http://www.justinmind.com/
▪ Axure - http://www.axure.com/

118. UXpin - www.uxpin.com
▪ Web based wireframing tool
▪ Mobile/Desktop applications
▪ Glass templates, run in browser

119. Proto.io - http://www.proto.io/
▪ Web based mobile prototyping tool
▪ Features
▪ Prototype for multiple devices
▪ Gesture input, touch events, animations
▪ Share with collaborators
▪ Test on device

120. Proto.io - Interface

121. Demo: Building a Simple Flow

122. Gesture Flow
Scr1
Scr2 Scr3
Scr4 Scr5 Scr6
Tap
Swipe

123. Start Transitions

124. Justinmind
▪ Native wireframing tool
▪ Build mobile apps without programming
▪ drag and drop, interface templates
▪ web based simulation
▪ test on mobile devices
▪ collaborative project sharing
▪ Templates for Glass, custom templates

125. User Interface - Glass Templates

126. Web Simulation Tool

127. Wireframe Limitations
▪ Can’t deploy on Glass
▪ No access to sensor data
▪ Camera, orientation sensor
▪ No multimedia playback
▪ Audio, video
▪ Simple transitions
▪ No conditional logic

128. Processing
▪ Programming tool for Artists/Designers
▪ http://processing.org
▪ Easy to code, Free, Open source, Java based
▪ 2D, 3D, audio/video support
▪ Processing For Android
▪ http://wiki.processing.org/w/Android
▪ Strong Android support, builds .apk file

129. Basic Processing Sketch
/* Notes comment */
//set up global variables
float moveX = 50;
//Initialize the Sketch
void setup (){
}
//draw every frame
void draw(){
}

130. Importing Libraries
▪ Can add functionality by Importing Libraries
▪ java archives - .jar files
▪ Include import code
import processing.opengl.*;
▪ Popular Libraries
▪ Minim - audio library, OCD - 3D camera views
▪ bluetoothDesktop - bluetooth networking

131. Processing and Glass
▪ One of the easiest ways to build rich
interactive wearable applications
▪ focus on interactivity, not coding
▪ Collects all sensor input
▪ camera, accelerometer, touch
▪ Can build native Android .apk files
▪ Side load onto Glass

132. Hello World Image
PImage img; // Create an image variable
void setup() {
size(640, 360);
//load the ok glass home screen image
img = loadImage("okGlass.jpg"); // Load the image into
the program
}
void draw() {
// Displays the image at its actual size at point (0,0)
image(img, 0, 0);
}

133. Demo

134. Touch Pad Input
▪ Tap recognized as DPAD input
void keyPressed() {
if (key == CODED){
if (keyCode == DPAD) {
// Do something ..
▪ Java code to capture rich motion events
▪ import android.view.MotionEvent;

135. Motion Event
//Glass Touch Events - reads from touch pad
public boolean dispatchGenericMotionEvent(MotionEvent event) {
float x = event.getX(); // get x/y coords
float y = event.getY();
int action = event.getActionMasked(); // get code for action
switch (action) { // let us know which action code shows up
case MotionEvent.ACTION_MOVE:
touchEvent = "MOVE";
xpos = myScreenWidth-x*touchPadScaleX;
ypos = y*touchPadScaleY;
break;

136. Demo

137. Sensors
▪ Ketai Library for Processing
▪ https://code.google.com/p/ketai/
▪ Support all phone sensors
▪ GPS, Compass, Light, Camera, etc
▪ Include Ketai Library
▪ import ketai.sensors.*;
▪ KetaiSensor sensor;

138. Using Sensors
▪ Setup in Setup( ) function
▪ sensor = new KetaiSensor(this);
▪ sensor.start();
▪ sensor.list();
▪ Event based sensor reading
void onAccelerometerEvent(…){
accelerometer.set(x, y, z);
}

139. Sensor Demo

140. Using the Camera
▪ Import camera library
▪ import ketai.camera.*;
▪ KetaiCamera cam;
▪ Setup in Setup( ) function
cam = new KetaiCamera(this,640,480,15);
▪ Draw camera image
void draw() { //draw the camera image
image(cam, width/2, height/2);

141. Camera Demo

142. Native Coding
▪ For best performance need native coding
▪ Low level algorithms etc
▪ Most current wearables based on Android OS
▪ Need Java/Android skills
▪ Many devices have custom API/SDK
▪ Vusix M-100: Vusix SDK
▪ Glass: Mirror API, Glass Developer Kit (GDK)

143. Glassware Development
▪ Mirror API
▪ Server programming, online/web application
▪ Static cards / timeline management
▪ GDK
▪ Android programming, Java (+ C/C++)
▪ Live cards
▪ See: https://developers.google.com/glass/

144. ▪ REST API
▪ Java servlet, PHP, Go,
Python, Ruby, .NET
▪ Timeline based apps
▪ Static cards
- Text, HTML, media attachment (image & video)
▪ Manage timeline
- Subscribe to timeline notifications, contacts
- Location based services
Mirror API

145. GDK
▪ Glass Development Kit
▪ Android 4.0.3 ICS + Glass specific APIs
▪ Use standard Android Development Tools

146. ▪ GDK add-on features
▪ Timeline and cards
▪ Menu and UI
▪ Touch pad and gesture
▪ Media (sound, camera and voice input)
GDK

147. Glass Summary
▪ Use Mirror API if you need ...
▪ Use GDK if you need ...
▪ Or use both

148. Hardware Prototyping

149. Build Your Own Wearable
▪ MyVu display + phone + sensors

150. Beady-i
▪ http://www.instructables.com/id/DIY-
Google-Glasses-AKA-the-Beady-i/

151. Rasberry Pi Glasses
▪ Modify video glasses, connect to Rasberry Pi
▪ $200 - $300 in parts, simple assembly
▪ https://learn.adafruit.com/diy-wearable-pi-near-eye-kopin-video-glasses

152. Physical Input Devices
▪ Can we develop unobtrusive input devices ?
▪ Reduce need for speech, touch pad input
▪ Socially more acceptable
▪ Examples
▪ Ring, pendant,
▪ bracelet, gloves, etc

153. Prototyping Platform
Arduino Kit Bluetooth Shield Google Glass

154. Example: Glove Input
▪ Buttons on fingertips
▪ Map touches to commands

155. Example: Ring Input
▪ Touch strip, button, accelerometer
▪ Tap, swipe, flick actions

156. How it works
Bracelet
Armband
Gloves
1,2,
3,4
Values/
output

157. Other Tools
▪ Wireframing
▪ Pidoco, FluidUI
▪ Rapid Development
▪ Phone Gap, AppMachine
▪ Interactive
▪ App Inventor, Unity3D, WearScript

158. WearScript
▪ JavaScript development for Glass
▪ http://www.wearscript.com/en/
▪ Script directory
▪ http://weariverse.com/

159. WearScript Features
• Community of Developers
• Easy development of Glass Applications
– GDK card format
– Support for all sensor input
• Support for advanced features
– Augmented Reality
– Eye tracking
– Arduino input

160. WearScript Playground
• Test code and run on Glass
– https://api.wearscript.com/

161. Summary
▪ Prototyping for wearables is similar to mobiles
▪ Tools for UI design, storyboarding, wireframing
▪ Android tools to create interactive prototypes
▪ App Inventor, Processing, etc
▪ Arduino can be used for hardware prototypes
▪ Once prototyped Native Apps can be built

162. RESEARCH DIRECTIONS

163. Challenges for the Future (2001)
▪ Privacy
▪ Power use
▪ Networking
▪ Collaboration
▪ Heat dissipation
▪ Interface design
▪ Intellectual tools
▪ Augmented Reality systems
Starner, T. (2001). The challenges of wearable
computing: Part 1. IEEE Micro,21(4), 44-52.
Starner, T. (2001). The challenges of wearable
computing: Part 2. IEEE Micro,21(4), 54-67.

164. Interface Design

165. Gesture Interaction

166. Gesture Interaction With Glass
▪ 3 Gear Systems
▪ Hand tracking
▪ Hand data sent to glass
▪ Wifi networking
▪ Hand joint position
▪ AR application rendering
▪ Vuforia tracking

167. Capturing Behaviours
▪ 3 Gear Systems
▪ Kinect/Primesense Sensor
▪ Two hand tracking
▪ http://www.threegear.com

168. Performance
▪ Full 3d hand model input
▪ 10 - 15 fps tracking, 1 cm fingertip resolution

169. Meta Gesture Interaction
▪ Depth sensor + Stereo see-through
▪ https://www.spaceglasses.com/

170. Collaboration

171. Social Panoramas

172. Ego-Vision Collaboration
▪ Wearable computer
▪ camera + processing + display + connectivity

173. Current Collaboration
▪ First person remote conferencing/hangouts
▪ Limitations
- Single POV, no spatial cues, no annotations, etc

174. Social Panoramas
▪ Capture and share social spaces in real time
▪ Enable remote people to feel like they’re with you

175. Key Technology
▪ Google Glass
▪ Capture live panorama (compass + camera)
▪ Capture spatial audio, live video
▪ Remote device (desktop, tablet)
▪ Immersive viewing, live annotation

176. Awareness Cues
▪ Where is my partner looking?
▪ Enhanced radar display, Context compass

177. Interaction
▪ Glass Touchpad Input/Tablet Input
▪ Shared pointers, Shared drawing

178. Cognitive Models

179. Modeling Cognitive Processes
• Model cognitive processes
– Based on cognitive psychology
• Use model to:
– Identify opportunity for wearable
– Predict user’s cognitive load

180. Typical Cognitive Model
1. Functional Modularity: cognitive system divided
into functionally separate systems
2. Parallel Module Operation: cognitive modules
operate in parallel, independent of each other
3. Limited Capacity: cognitive modules are limited in
capacity with respect to time or content
4. Serial Central Operation: central coordination of
modules (eg monitoring) is serial

181. Cognitive Interference
▪ Structural interference
▪ Two or more tasks compete for limited
resources of a peripheral system
- eg two cognitive processes needing vision
▪ Capacity interference
▪ Total available central processing
overwhelmed by multiple concurrent tasks
- eg trying to add and count at same time

182. Example: Going to work ..
Which is the most cognitively demanding?

183. Cognitive Resources & Limitations
asdfasdf

185. Application of Cognitive Model
Busy street > Escalator > Café > Laboratory.
But if you made Wayfinding, Path Planning, Estimating
Time to Target, Collision Avoidance easier?

186. Social Perception

187. How is the User Perceived?

188. GlassHoles
• safa

189. TAT Augmented ID

192. The Future of Wearables

193. RESOURCES

194. Online Wearables Exhibit
Online at http://wcc.gatech.edu/exhibition

195. Glass Developer Resources
▪ Main Developer Website
▪ https://developers.google.com/glass/
▪ Glass Apps Developer Site
▪ http://glass-apps.org/glass-developer
▪ Google Design Guidelines Site
▪ https://developers.google.com/glass/design/
index?utm_source=tuicool

196. Other Resources
▪ AR for Glass Website
▪ http://www.arforglass.org/
▪ Vandrico Database of wearable devices
▪ http://vandrico.com/database

197. Glass UI Design Guidelines
• More guidelines
– https://developers.google.com/glass/design/index

198. Books
▪ Programming Google Glass
▪ Eric Redmond
▪ Rapid Android
Development: Build Rich,
Sensor-Based Applications
with Processing
▪ Daniel Sauter

199. • Beginning Google
Glass Development
by Jeff Tang

200. • Microinteractions: Designing
with Details
– Dan Saffer
– http://microinteractions.com/

201. Conclusions
• Wearable computing represents a fourth
generation of computing devices
• Google Glass is the first consumer wearable
– Lightweight, usable, etc
• A range of wearables will appear in 2014
– Ecosystem of devices
• Significant research opportunities exist
– User interaction, displays, social impact

202. Contact Details
Mark Billinghurst
▪ email: mark.billinghurst@hitlabnz.org
▪ twitter: @marknb00
Feedback + followup form
▪ goo.gl/6SdgzA