Alexa, what is the Internet of Things? Now that technology is small enough to be embedded in everyday devices, Healthcare has an opportunity to exploit the extraordinary potential of connecting ordinary devices. In this presentation, we explain how to rapidly build an IoT system and how to drive the Cloud with your voice on an Amazon Echo. In addition to describing how to use Alexa, we explore using AWS IoT, Lambda, Amazon SNS, and DynamoDB.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
HLC304
Building IoT Applications
with AWS and Amazon Echo
November 28, 2016
Chris McCurdy - AWS Specialist SA in Healthcare and Life Sciences
Nitin Gujral – Directory of Software Engineering and Innovation at Boston Children's

2. What to Expect from the Session
• What is IoT and what are some patterns that are being
used in industry?
• What are some alternatives if I just want health sensor-
related data?
• How is KidsMD using Alexa to help improve home care?

3. What is IoT?

4. What is IoT?

5. What is AWS IoT?

6. What is AWS IoT?

7. What is AWS IoT?

8. What does AWS IoT consist of?
Security First
• All traffic to and from AWS IoT must be encrypted
over Transport Layer Security (TLS)
• AWS Managed Certificates
• Certificate Signing Request (CSR) Certificates
• Just-in-time (JIT) Registration
• Protect your certificates
• chroot
• Security Enhanced Linux (SELinux)
• One-Time Programmable (OTP) Fuses
• Trusted Platform Modules
• Smartcards
• Locks and Boxes
• FIPS-style hardware

9. What does AWS IoT consist of?
Thing Gateway

10. What does AWS IoT consist of?
Thing Gateway
Supported Protocols
HTTPS, Websockets, Secure MQTT
What is MQTT?
A lightweight pub/sub protocol, designed to minimize network bandwidth and
device resource requirements. MQTT supports TLS for encryption.

11. What does AWS IoT consist of?
Thing Gateway
var device = awsIot.device({
keyPath: 'private_key.pem',
certPath: 'certificate.pem',
caPath: 'root.pem',
clientId: clientId,
region: 'us-east-1' });
var button = new groveSensor.GroveButton(6);
function readButtonSensorValue() {
var buttonState = button.value();
if (buttonState == 1) {
device.publish(topic, JSON.stringify({
SubmitTime: (new Date()).toJSON()}));
}
}
setInterval(readButtonSensorValue, 100);

12. What does AWS IoT consist of?
IoT Rules Engine
Amazon
Kinesis
Amazon S3
bucket

13. What does AWS IoT consist of?
IoT Rule Engine
• Augment or filter data received from a device.
• Write data received from a device to an Amazon DynamoDB
database.
• Save a file to Amazon S3.
• Send a push notification to all users using Amazon SNS.
• Publish data to an Amazon SQS queue.
• Invoke a Lambda function to extract data.
• Process messages from a large number of devices using
Amazon Kinesis.
• Send data to the Amazon Elasticsearch Service.
• Capture an Amazon CloudWatch metric.
• Change an Amazon CloudWatch alarm.
• Send the data to Amazon Machine Learning to make
predictions based on an Amazon ML model.

14. What does AWS IoT consist of?
IoT Rule Engine
{
"sql": "SELECT * FROM 'Sensor/DispensingSensor'",
"ruleDisabled": false,
"actions": [
{
"kinesis": {
"roleArn": "arn:aws:iam::XX:role/sensor_status_role",
"streamName": "sensor_status_stream",
"partitionKey": "${ClientID}"
}
}]
}

15. What does AWS IoT consist of?
Thing Registry
• Associate MQTT Client IDs
• Associate Certificates
• Specify up to 3 custom attributes

16. What does AWS IoT consist of?
Thing Registry
{
"version": 3,
"thingName": "PortableDispenser",
"defaultClientId": "abc123",
"thingTypeName": "Dispenser",
"attributes": {
"model": "123",
"dispensingVolume": "75"
}
}

17. What does AWS IoT consist of?
Thing Registry
response = client.update_thing(
thingName=PortableDispenser',
attributePayload={
'attributes': {
'dispensingVolume': '150'
},
},
expectedVersion=2,
)

18. What does AWS IoT consist of?
Thing Shadow
• Store and retrieve current and future state of the
thing
• Simple JSON Document
• Get and set the state of a thing over MQTT or
HTTP
• Access the shadow regardless of whether the
thing is connected to the Internet

19. What does AWS IoT consist of?
Thing Shadow
{
"state" : {
“desired" : {
"sensor" : "ON"
},
"reported" : {
"sensor" : "OFF"
},
"delta" : {
"sensor" : "ON"
} },
"version" : 10
}

20. What does AWS IoT consist of?
Thing Shadow
data = {
"state" : {
"desired" : {
"switch" : "on"
}
}
}
response = client.update_thing_shadow(
thingName = PortableDispenser',
payload = mypayload
)

21. Other services in an scalable IoT architecture
Kinesis
•Streams are made of Shards
•Each Shard ingests data up to 1 MB/sec, and up to 1000
TPS
•Each Shard emits up to 2 MB/sec
•All data is stored for 24 hours – 7 days
•Scale Kinesis streams by splitting or merging Shards
•Replay data inside of 24Hr -7days Window

22. Other services in an scalable IoT architecture
Amazon DynamoDB
• Provisioned at the table level
• Write capacity units (WCUs) are measured in 1 KB per
second
• Read capacity units (RCUs) are measured in 4 KB per
second
• RCUs measure strictly consistent reads
• Eventually consistent reads cost ½ of constant
reads
• Read and write throughput limits are independent
• Increase as necessary, decrease at most 4 times per
UTC day

23. Other services in an scalable IoT architecture
Lambda
• AWS Lambda offers 23 "power levels"
• Higher levels offer more memory and more CPU
power
• 128 MB, lowest CPU power
• 1.5 GB, highest CPU power
• Compute price scales with the power level
• Duration ranging from 100 ms to 5 minutes

24. Example IoT architecture
IoT MQTT
protocol
IoT
certificate
IoT
rule
IoT
topic
Amazon
Kinesis
AWS
Lambda Amazon
DynamoDB
Amazon
SNS
Alexa
Medication Status
monitoring device
Medication Status Backend
Node.js
AWS
Lambda
AWS
Lambda

25. Elephant in the room
http://nos.twnsnd.co/post/104252656546/elephants-tea-party-robur-tea-room-24-march
Amazon
Kinesis
AWS
Lambda
Amazon
DynamoDB
Amazon
SNS
Alexa
AWS IoT
HIPAA Eligible Not HIPAA Eligible

26. Example IoT (PHI) Architecture
IoT MQTT
protocol
IoT
certificate
IoT
rule
IoT
topic
Amazon
Kinesis Amazon
DynamoDB
Alexa
Medication Status
monitoring device
Medication Status Backend
Node.js
AWS
Lambda
Amazon
S3
Amazon EC2
client
Amazon EC2
AWS
Lambda
SQS

27. Validic Digital Health Platform
Clinical, Fitness, Wellness and
Nutritional Devices, Wearables,
Applications, and Equipment
Healthcare Providers, Clinical
Research, Wellness Apps, Insurance,
Academia

28. Validic architecture
3rd Party Devices,
Apps, and APIs
Validic Core Platform
VitalSnap™ OCR
Image Algorithm
Calibration
Validic
Standardization
Services
Health Data
Retrieval
Services
Admin
Database
Primary
Database
Replica
Databases
Validic API
Healthcare Systems,
Clinical Trials Platforms,
Wellness Applications
Patient / Individual

29. Alexa Voice Service in Healthcare
Nitin Gujral
Director of Software Engineering, Innovation and Digital Health

30. Boston Children’s Hospital
#1 ranked pediatric hospital
nationwide (8 out of 10
specialties)
400 plus bed; children's
hospital affiliated with Harvard
Medical School
Largest recipient of NIH funds
in pediatric-focused research

31. Innovation and Digital Health Accelerator
Shaping the future of health care and extending Boston Children’s pediatric
leadership globally
BCH DataClinical Expertise HCIT Development
Products Platforms Ventures
Boston Children’s
Expertise & Capabilities:
+
Industry Partners &
Startups, like:
=
To launch disruptive
health care focused:
Learn more:
bostonchildrens.org/accelerator
GE GrandRoundsCerner
IBM Watson Health Rock Health

33. Ask the Internet? –
Parents are actively looking for guidance—traditional medical content sites lack
the trustworthy and readily accessible health guidance.
Although I will ALWAYS go with my gut about my kids
being sick, I typically ask the internet to confirm my
suspicion about what might be happening with them.
– Female, 36, Kids ages 1 & 4
Source: 3rd Party consumer research survey (n=200)

34. Decision support in consumers’ hands
Boston Children’s
Data & Expertise
Withings
Raiing

35. Kids.MD™ is delivering consumer facing digital health tools to patients and families across digital mediums.
Kids.MD™
powered by:
Boston Children’s Hospital
web
mobile
voice
device
research
platform

36. “Alexa, ask Kids.MD”

37. Kids.MD – Alexa Skill Architecture
AVS +
Kids.MD
“Alexa, ask Kids.MD about fever”
Skill identified in Alexa
voice service
Request sent to Kids.MD
clinical decision support
Sends
‘recommendations’ back
to Alexa voice service
Alexa converts ‘text’ to
spoken words and
streams audio
Or access
via mobile &
web app
Kids.MD™
Electronic Health
Record
Patient
Portal

38. Kids.MD on AWS
Kids.MD™
Amazon RDS Amazon S3
AWS Lambda API Gateway Amazon EC2
Amazon Dynamo DB

39. Kids.MD Skill: Development

40. Node Module - alexa-sdk
Handlers for intents

41. Node Module - alexa-sdk
Custom events

42. Node Module - alexa-sdk
State Machine
Context persistence using DynamoDB

43. Node Module - alexa-sdk
Setting context
Response

44. Project structure
• Structure backend code; e.g.,
handlers, model and other assets
- including tests
• Organize Alexa Skills Kit speech
assets

45. Debugging
Locally debug AWS Lambda code
Reference:
Blog by Nathan Grice (https://developer.amazon.com/public/community/post/Tx24Z2QZP5RRTG1/New-Alexa-
Technical-Tutorial-Debugging-AWS-Lambda-Code-Locally
• within any IDE
supporting
Node.JS
• simulate AVS
request /
response

46. Unit testing
Javascript testing with Jasmine (a behavior driven framework)
https://jasmine.github.io/
NPM module - jasmine or jasmine-node
Ability to mock requests and responses
Reference:
Article by Jeremy Bohrer: https://medium.com/@jjbskir/unit-testing-an-amazon-alexa-skill-with-node-js-and-
jasmine-98982544471f

47. Integration testing – developer tools

48. Lambda deployment
• NPM package
• Leverages AWS SDK
• Packaging for Lambda
• Upload to Lambda
• Supports unit testing as well
https://github.com/Tim-B/grunt-aws-lambda

49. Kids.MD Skill: technical
considerations

50. States and decision trees

51. Managing context in an Alexa skill
• short time frame for the duration of skill invocation
• access to user and application info
• persistent context for a user

52. Intent Schema Slot types
• better voice recognition
• more accuracy with user response
• supports free form responses
Custom Slot Types
Built-in Slot Types
• e.g. AMAZON.NUMBER, AMAZON.DATE,
AMAZON.DURATION

53. Speech Synthesis Markup Language (SSML)
• for more natural speech
• pauses and predefined formats (characters, digits, date, telephone,
address, etc.)
<s>Let us start again.</s>
<p>Would you like to hear this information again?</p>
<break strength='weak'/>
<say-as interpret-as="digits">101.4</say-as>

54. Speech Synthesis Markup Language (SSML)
• improved pronunciation using “phoneme”
• specifically for medical terminology
<phoneme alphabet="ipa"
ph="sɪmp.təms">symptoms</phoneme>
<phoneme alphabet="ipa" ph="moətrɪŋ">Motrin</phoneme>

55. Logging, auditing, and monitoring
• AWS Lambda CloudWatch logs and metrics
• Custom error logs
• Custom auditing of user interactions
• Monitoring of various service components

56. Account linking
• Link an Alexa user to the user of your system
• Enables contextual interactions based on user’s preferences
• Performed at the time of skill installation
• Using OAuth authorization flow
Reference:
https://developer.amazon.com/public/community/post/Tx3CX1ETRZZ2NPC/Alexa-Account-Linking-5-
Steps-to-Seamlessly-Link-Your-Alexa-Skill-with-Login-wit

57. Images and audio in Alexa Skills
• Display images in Cards
• JPG or PNG (less than 2 MB) Need CORS support for image URLS
• SSML Audio Tags <audio>
• MP3 support
• 90 sec limit
• Audio Streaming
• Leverage built-in intents for playback
Reference: https://developer.amazon.com/public/community/post/Tx1DSINBM8LUNHY/New-Alexa-Skills-Kit-ASK-
Feature-Audio-Streaming-in-Alexa-Skills

58. Voice design – lessons learned
• Maintain intuitive conversation
• provide guidance via prompts
• Balance the amount of verbal responses
• Effectively utilize Companion App(s)
• with images, audio and streaming sound

59. Some enterprise deployment challenges
Security
Connectivity

60. Voice in healthcare
A new way for patients and clinicians to access health information and
personalized care in various settings
@ home
@ inpatient
@ outpatient
by patients & families
by clinicians
Context: Users:

61. Photo by John Blanding for The Boston Globe
Example usage:
• Home Health Hubs
• At the Bedside for Patient Care
• Virtual Assistant for Clinicians
Voice in healthcare – a new medium
Streamline and ease delivery of health care services

62. “Alexa, ask Children’s Hospital”

63. Growth of “Internet of Healthcare”

64. Promise of connected healthcare
Portable
Proactive
Patient Centric

65. Thank you!

66. Remember to complete
your evaluations!