Enable text extraction, semantic search, sentiment analysis, metadata extraction, and intelligent data warehouses.

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ben Snively, Solutions Architect – Data and Analytics, AI/ML
March 27th 2019
Innovate Digital Content Management
AI/ML Week 2019

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Collect Store Enrich Deliver
GB to Petabytes of
documents, images
and video assets
Centralized
storage
& global registry
Metadata enrichment
through machine
learning and deep
learning
Enhanced value
and search experience
Content Management Systems Dataflow
Right
content to
the right
users

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Content comes in many forms:
Social media
and Blogs
Applications and
Systems
Documents Media:
Images, Audio,
and Video

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enrich Deliver
Enrichment
through machine
learning and deep
learning
Enhanced value
and search experience
ML enriches your digital content
Documents Media:
Images,
Audio, and
Video
Social
media
and
Blogs
Applications
and
Systems
Right
content to
the right
users

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Document Processing

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Extracting Text, Forms, and Tables:
…not a single corresponding pixel value in common

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Document Enrichment Flow:
Quickly turn extracted text/data into actionable insights
Input
Uploaded document,
text, tables, and forms
Amazon S3
Uploaded documents
are stored in S3
NLP
Use natural language
processing to extract
insights
from documents
Search
Easily search through
extracted data and
text insights
Output
Discover insights
Extract
Automatically extract
words and lines of
text, key/values from
forms and tables

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Text extraction
Table extraction
Form extraction
Entities
Key Phrases
Language
Sentiment
Syntax
Topic Models
Amazon Textract Amazon Comprehend

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demonstration
Processing of scanned WWII
documents and key values from a mail
form

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
of
storage
events per day
30+pb135 Billion
Up to
Monitoring 99% Equities
& 65% Options in the US
Reconstructing
Trillions
of Market Nodes & Edges
Investor
protection
Market
integrity

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FINRA challenges
High volumes of unstructured content
• ~1M documents each year from stock brokers and investors
to be reviewed
• Documents contain incredibly useful information but
mining is a challenge
Overlooking important information
• Risk-based approach itself presents risks
Numerous features of interest
• Finding information about the Who, What, Where, When
and How is labor intensive
• Source: AWS re:invent 2018

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FINRA solution: Document mining

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FINRA Architecture
Commix Command Center for People and Organizations (C3PO)
<bucket>/input
Data
Preparation
Entity
Matching
Amazon Comprehend
<bucket>/matched

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
NLP and Graph Example

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Image and Media Processing

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition
Faces Celebrities Objects
Moderation ScenesActivities Paths
Text

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demonstration
Extracting searchable metadata from WWII
images

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Transcribe
Automatic Speech Recognition
Time
stamps
Support for
both regular &
telephony audio
Punctuation
&
formatting
S3
integration
Recognize
multiple
speakers
Custom
dictionaries
? !

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Machine learning services
Language
Amazon Comprehend,
Amazon Transcribe
Vision
Amazon Rekognition Image &
Video
Media Analysis Solution starter kits
Go To aws.amazon.com > answers > media-entertainment > Media-Analysis-Solution
Automated metadata generation
Label & face detection
Celebrity detection
Face search
Person in picture tracking
Subtitling
Context
Key entity & phrase detection
Other Services
such as Amazon Translate Future starter kits
Automatically provision the services necessary for building common media use cases on AWS
Media Analysis Solution Interface

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Media Analysis Solution
Serverless
website,
built on
Amplify
Authentication
with Amazon
Cognito User
Pools
Dynamic
website
interactions
Media is
uploaded to S3
S3 event triggers
Lambda function,
which parses the
event details and
starts the state
machine
AWS Step Functions
coordinates the
processing of the media
using AI services
Processes the video and
images for visual
information (objects,
people, faces,
celebrities, etc.)
Audio is extracted from
the video files for
transcription
Audio is transcribed with
timecodes to be used for
captions
The transcribed content
is analyzed for key
phrases and entities
Results are indexed in
the Elasticsearch
cluster
Retrieves results from
ElasticSearch / S3

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AI Generated Metadata
UUID, MD5, Technical
Transcription Contextual Metadata
Object & Face
Recognition
+

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Document and Media Combined Case
Study:
Fox Entertainment Group

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fox Entertainment Group
A.I. PLATFORM
Video, Audio, Images, Logos
Documents, Contracts, Scripts,
…….
PROCESS CONSUME
Custom Experience
Standard Experience
………
……
• Content monetization
• Social Insights
• Consumer insights
• Creative insights
• Media workflows
• Rights workflows
• Experiment
• POC
• Text-in-Image
• Faces
• Celebrities
• Labels
• Moderation
• NLP
• Automatic
speech
recognition

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Detect Text in Images
Speech-to-Text
Moderation
Fox Entertainment Group Use case :
Worldwide theatrical distribution

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Collect Store Enrich Deliver
GB to Petabytes of
documents, images
and video assets
Centralized
storage
& global registry
Metadata enrichment
through machine
learning and deep
learning
Enhanced value
and search experience
Content Management Systems
Right
content to
the right
users

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Getting Started
Setup storage1
Move data2
Cleanse, prep,
and catalog data
3
Configure and enforce
security and compliance
policies
4
Make data available
for analytics and
machine learning
5
https://aws.amazon.com/solutions/media-analysis-solution/

27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you