Experiencer, and open-source context-sensitive wearable experience sampling tool. Presented at the EAI Pervasive Health Conference 2022 by Alireza Khanshan. PhD candidate at the Eindhoven University of Technology.

1. Experiencer: An Open-source Context-Sensitive
Wearable Experience Sampling Tool
ALIREZA KHANSHAN, PIETER VAN GORP, PANOS MARKOPOULOS
EAI PERVASIVE HEALTH CONFERENCE; THESSALONIKI, GREECE; 12-13 DEC. 2022
Alireza Khanshan, PhD Candidate, a.khanshan@tue.nl
Eindhoven University of Technology
Industrial Design dept., Future Everyday group

2. Research on Behaviours, Thoughts, Feelings
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
2
• Psychology
• Healthcare
• Well-being
• Education
• Human-Computer Interaction
• …

3. Evolution of Methods
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
3
Diary
Observation
Survey
Retrospection
Reactivity
Experience sampling method

4. The Experience Sampling Method (ESM)
• Notification (beeps/prompts) are systematically spread
• During day-to-day activities
• Higher ecological validity
• Variants
• Ecological Momentary Assessment (EMA)
• Contextual Aware Experience Sampling (CAES)
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
4

5. History of ESM Digital Devices
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
5
Experience Sampling Program
Barrett and Barrett (2005)
Psymate
Myin-Germeys et al. (2011)
Electronic Mood Device
Hoeksma et al. (2000)
TEMPEST
Batalas et al. (2018)

6. ESM is not a Panacea
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
6
• Drop out (especially for studies that should last longer)
• Response rate declines
Why?
Intrusive
At inopportune times
With technical difficulties
…

7. Recent Efforts
• Personalization (timing, content, …)
• Context-sensitivity (location, calendar events, PA level, …)
• Mainly on smartphones
• Prevalence of commodity-level smartwatches
• More sensors
• Accurate sensors
• Higher reachability (wrist-worn)
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
7

8. Wearable Experience Sampling Method (wESM)
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
8
WellBeat
Park et al. (2020)
ROAMM
Kheirkhahan et al. (2019)
μEMA
Intille et al. (2016)

9. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
9

10. Requirements for a Smartwatch-based wESM Tool
• Data collection and analysis
• Scheduling
• Data entry
• Web interface
• Scalability
• Optimization
• Openness, availability, and security
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
10
Recording sensor data
Data analytics dashboard
Configurable sensors
➡Context-sensitive
➡Temporal ➡Input widgets on the smartwatch
➡Configurable widgets
➡Data visualization
➡Administration dashboard
➡Remote device management
➡Event-based data collection
➡Custom data synchronization
➡Reusability and availability
➡GDPR compliance

11. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
11
Category Feature Software
WellBeat ROAMM µEMA Physiqual Experiencer
Data collection and analysis Recording sensor data ✔ ✔ ✔ ✔ ✔
Data analytics dashboard ✔ ✔  ✔ 
Configurable sensors  ✔   ✔
Scheduling Context-sensitive     ✔
Temporal  ✔   ✔
Data entry Input widgets on the device ✔ ✔ ✔  ✔
Configurable widgets  ✔  ✔ ✔
Web interface Data visualization  ✔ ✔  
Administration dashboard  ✔  ✔ 
Scalability Remote device management  ✔   ✔
Optimization Event-based data collection     ✔
Custom data synchronization ✔ ✔ ✔  ✔
Openness, availability, and
security
Reusability and availability     ✔
GDPR compliance    ? ✔

12. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
12
Sensor data collection
(event-based)
Various interfaces
Configurable with
JSON

13. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
13
Sensor data collection
(event-based)
Various interfaces
Configurable with
JSON

14. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
14
With dashboard for
managing hundreds of
watches remotely
Data goes exclusively to
TU/e cloud
(as opposed to Fitbit,
Google Fit, …)!
Context-sensitive
prompting + classics

15. Overview of Experiencer Configuration
• Sampling policy
• Inter-notification time
• Unobtrusive sensing
• Compliance status
• Questionnaire
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
15
➡Active/resting
➡Known/unknown
➡Fixed-interval
➡Acceleration
➡Photoplethysmography (PPG)
➡Heart rate
➡Peak-to-peak interval
➡Physical activity
➡Received
➡Read
➡Submitted
➡Sequence of single choice questions

16. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
16

17. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
17

18. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
18
<<YOUR BACKEND>>

19. Case Studies
• Health intervention campaigns in 2020 and 2021
• Test and compare context sensitive policies (active/resting)
• Create predictive models for emotion
• Mental healthcare facility
• Measure and predict stress
• Persuasion profiling
• Adaptive notifications
• Affective state
• Study lifestyle behaviors
• Test and compare context sensitive policies (known/unknown)
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
19

20. Future Work and Limitations
Future work
• Branching questionnaire
• More UIs tailored to watch interface
• Easier data access/management (users & researchers)
• Data sharing
Limitations
• Battery life
• Sensor accuracy
• Tizen OS
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
20

21. Conclusions
• Commodity-level smartwatches as ESM devices
• Reachability
• Ecological validity
• Variety of sensors
• Requirements checklist for wESM
• Experiencer: wESM tool that is open-source and context-sensitive
• Data sharing
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
21

22. Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
22
experiencer.eu
github.com/khnshn/Experiencer

23. Thank You
Experiencer: An Open-source Context-Sensitive Experience Sampling Tool
23
Dr. Pieter Van Gorp
Associate Professor
Eindhoven University of Technology (TU/e)
Industrial Engineering and Innovation Sciences
Eindhoven AI Systems Institute
p.m.e.v.gorp@tue.nl
Alireza Khanshan
PhD Candidate
Eindhoven University of Technology (TU/e)
Industrial Design
Eindhoven AI Systems Institute
a.khanshan@tue.nl