Sharing the results of our recent journal article to the Dutch national congress on translational research for cardiology. Besides sharing published results, also discussing ongoing engineering and scientific challenges. The Experience Sampling Method (ESM) is gaining ground for collecting self-reported data from human participants during daily routines. An important methodological challenge is to sustain sufficient response rates, especially when studies last longer than a few days. An obvious strategy is to deliver the experiential questions on a device that study participants can access easily at different times and contexts (e.g., a smartwatch). However, responses may still be hampered if the prompts are delivered at an inconvenient moment. Advances in context sensing create new opportunities for improving the timing of ESM prompts. Specifically, we explore how physiological sensing on commodity-level smartwatches can be utilized in triggering ESM prompts. We have created Experiencer, a novel ESM smartwatch platform that allows studying different prompting strategies. We ran a controlled experiment (N=71) on Experiencer to study the strengths and weaknesses of two sampling regimes. One group (N=34) received incoming notifications while resting (e.g., sedentary), and another group (N=37) received similar notifications while being active (e.g., running). We hypothesized that response rates would be higher when experiential questions are delivered during lower levels of physical activity. Contrary to our hypothesis, the response rates were found significantly higher in the active group, which demonstrates the relevance of studying dynamic forms of experience sampling that leverage better context-sensitive sampling regimes. Future research will seek to identify more refined strategies for context-sensitive ESM using smartwatches and further develop mechanisms that will enable researchers to easily adapt their prompting strategy to different contextual factors.

1. Considerations for longitudinal monitoring on
commodity-level smartwatches
JUNE 24TH, 2022
HTTPS://CONGRESS2022.HEART-INSTITUTE.NL/
Alireza Khanshan, Pieter Van Gorp, and Panos Markopoulos

2. Disclosure of speaker’s interests
(Potential) conflicts of interest
• Sponsorship or research funding2
• Fee or other (financial) payment3
• Shareholder4
• Other relationship, i.e. ...5
None
Company relationships in connection with 6th Translational Cardiovascular Research Meeting

3. Project STRAP
“Self TRAcking for Prevention and diagnosis of heart disease”
STRAP is a multidisciplinary project involving different
PhD positions from TU/e, TUDelft, and ErasmusMC.
NWO (Dutch Research Council)
Program “Big Data & Health”
https://www.nwo.nl/en/news/four-public-private-projects-big-data-and-health-have-been-awarded-grants (2019)
3

4. STRAP goals
To make measurements at
home which would
otherwise require visiting
the clinic.
To collect longitudinal data
pertaining to daily life
activity, emotions, and other
relevant aspects.
To ensure higher adherence
in self-tracking through the
application of behavior
change techniques.
4

5. Does a dynamic form of experience sampling based on
activity monitor data help in increasing response rate?
JUNE 24TH, 2022

6. Contents
• Experience sampling method
• ESM and personalization: some example studies
• More general reflection
7

7. Experience sampling method (ESM)
In various scientific experiments with human participants, scientists rely on self-
reported data. This can be done via surveys, but they are too often coming too soon or
too late to be considered ecologically valid. Experience Sampling Methods (ESM)
address that issue by breaking down long surveys into short questions which are then
administered during the daily activities of study participants instead of before or after
such activities.
8

8. ESM Software: active research from 2001 until Today!
Towards Personalization: Machine Teaching and Experience Sampling
9

9. ESM Software: active research from 2001 until Today!
Left photo from Myin-Germeys, I., Birchwood, M., and Kwapil, T. (2011). From environment to therapy in psychosis: A real-
world Momentary Assessment approach. Schizophrenia Bulletin, 37(2):244–247.
10
Psymate (ca. 2000) Experiencer (2021)

10. Contents
• Experience sampling method
• ESM and personalization
• More general reflection
11

11. ESM and Personalization
Problem?
For long-running ESM studies the quality of responses can be compromised, and the
response rates decrease.
Possible Solution?
Personalization (time, content, and modality)
About time: absolute clock time on a given day, or relative in diurnal cycle, or relative to the current task/activity
Goal?
Increase adherence and improve response quality.
12

12. Contents, refined
• Specific Study on Personalized ESM
• Poll
• Hypothesis
• Experiencer.eu software
• Study Design
• Results
• More general reflection on longitudinal monitoring with commodity wearables
Does a dynamic form of experience sampling based on activity monitor data help in increasing response rate?
13

13. 15 Menti

14. Does a dynamic form of experience sampling based on
activity monitor data help in increasing response
rate?
16
We thought so… (and saw an opportunity for personalization)

15. Hypothesis
H1: the level of body activity of users at the time of an experience
sampling prompt (beep) affects their response rate.
Does a dynamic form of experience sampling based on activity monitor data help in increasing response rate?
17

16. Hypothesis’
H2: response rates are higher when prompts are delivered during lower
levels of physical activity
Does a dynamic form of experience sampling based on activity monitor data help in increasing response rate?
18

17. Proposed method
Active vs. Resting
Resting: prompts when not active
Active: prompts when physically active
(walking, running)
Does a dynamic form of experience sampling based on activity monitor data help in increasing response rate?
19
Content of the beep?
We asked participants to self-report
their emotion on the PANAS scale (=
detail)

18. Experiencer.eu
21
Prompting at
”active” or
“resting”
moments
With dashboard
for managing
hundreds of
watches remotely
Data goes
exclusively to
TUE cloud
(as opposed to Fitbit,
Google Fit, …)!

19. • Duration: 5 weeks
16 Nov – 20 Dec’20
• #Participants: 71 (34 for rest vs 37 for active)
• #Delivered beeps: 10,709
22

20. Results overall
active group (M = 0.227, SD = 0.222)
resting group (M = 0.085, SD = 0.069)
p = 0.001
25
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Active
Resting
Response rate (mean)
Treatment
group

21. Results Refined
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1
2
3
4
5
Response rate (mean)
Week
(time)
Active
Resting
Mann-Whitney U test, two-tailed, with alpha = 0.05, we found significant difference
in week 2 (i.e., p = 0.0.015), and week 3 (i.e., p = 0.033)
26

22. Does a dynamic form of experience sampling based on
activity monitor data help in increasing response
rate?
27
So it did! And even using a “one size fits all” strategy
(even though not the one we had assumed)

23. Other ESM Questions can be configured easily
29

24. Additional analyses beyond personalizing prompt time
• Samsung-derived activity type
• Raw Accelerometer data
• Samsung-derived HR and PPI
• Raw PPG data
• ESM responses
So, “obviously”, we also analyze in follow-up studies:
Predicting emotions (or other user reported input) via smartwatch sensors
Current work: collect more data, via collaborations (e.g., GGz Centraal)
30

25. Wild cocktail of scientific and engineering challenges
• Access to raw data, Access to screen for interaction, GDPR compliance
• Battery consumption, Operating System policies…
• Tackling low response rates (dedicated PhD project)
• Towards offline simulations
• “optimal” number of beeps per day, per target group, per type of question, etc.
• Adaptive approach (reinforcement learning without cold start issues)
• Preserving Ecological Validity
• Potential in JITAI studies
31

26. Conclusions
• Commodity-level smartwatches enable a richer variant of Experience
Sampling that is scalable to large cohorts
• Fine-tuning the moment of delivery based on context sensing has potential
• Further research is needed for specific target groups and question types
Call to Action
• Explore our open access article
• Collaborate with TUE in joint
research projects
32

27. Thank You
Pieter Van Gorp, p.m.e.v.gorp@tue.nl
33