14. Ingest Step Requirements
• Support File-based workflow only for now
• Simple upload workflow for clients
• Fast upload speeds anywhere in the world
• Durable, scalable, fast storage
• No additional disk storage required for upload
• No maintenance required
15. Ingest Step Requirements
• Support File-based workflow only for now
• Simple upload workflow for clients
• Fast upload speeds anywhere in the world
• Durable, scalable, fast storage
• No additional disk storage required for upload
• No maintenance required
16. S3 Upload using pre-signed
URL
• Allows client can upload
directly to S3 using pre-
signed URL
• AWS-SDK supports pre-
signed url generation
25. Two options to accelerate S3 transfer
• Proxy file transfer traffic through CloudFront
• Cloudfront can accelerate dynamic content
• Uses Cloudfront Edge locations to optimize route path
• Enable S3 Transfer Acceleration
• Takes advantage of CloudFront’s globally distributed
edge locations
• More transfer optimizations
31. Processing Requirements
• use fully-managed service if possible
• scalable, reliable
• adaptive bitrate streaming
• Important: compatible with various platforms, devices
• Desktop Web - IE 10+, Edge, Chrome, Firefox
• Mobile Web - Mobile Safari, Chrome Mobie
• Android - 4.1 ICS+
• iOS - 9+
• Most Important: Delivery video to users ASAP
33. Adaptive Bitrate Streaming
• From Wikipedia:
• Adaptive bitrate streaming is a technique used in
streaming multimedia over computer networks.
• It works by detecting a user's bandwidth and
CPU capacity in real time and adjusting the
quality of a video stream accordingly.
• The result: very little buffering, fast start time and a
good experience for both high-end and low-end
connections.
40. Progressive Download
Single File via HTTP
• Pros
• Easy to implement
• Cons
• Implementing Adaptive Bitrate Streaming is so hard (almost
impossible) without getting “choppy” frames, or “pop” gap
sound between streams
• Slow seeking/starting since container requires metadata reading
• Large binary can occur lower cache hit
• No Live Stream compatibility
41. MPEG-DASH
• Pros
• New standard
• Based on Fragmented MPEG4 (similar to MPEG2 Transport Stream on
HLS)
• Broader codec support
• Cons
• IMO it is not production ready
• Market, Platform, Community support is not good at this moment
• Workflow was unstable during experiment
43. First Spec - Protocol
• HTTP Live Stream (HLS)
• Apple invented standard, natively supported on macOS/iOS/tvOS
• Natively supports adaptive bitrate streaming, Live Streaming, and DRM
(Apple FairPlay)
• Based on HTTP Protocol, Play well with transitional HTTP CDN
• Trusted by many providers, Many stable implementations
• Amazon, Youtube, Facebook, Twitter, Vimeo, Twitch, Akamai, New
York Times, FOX TV, TED, VK, POOQ, SK Broadband TV …
• Android - ExoPlayer
• Web - MediaSource Extension is standard spec, HLS.js, Video.js
44. First Spec - Input
• No codec limitation of input source
… but expects most input source are H.264/AVC
+ AAC based
• Max acceptable resolution: 1080p
• Max acceptable bitrate: 10Mbps
• Max acceptable duration: 5 minutes
45. First Spec - Output
• Common
• H.264/AVC1 + AAC LC
• Many graphic chipset supports hardware accelerated decoding
• Most popular codec in market
• Apple HLS
• Three variants for adaptive bitrate streaming
• High Quality: Up to 1080p, 60p, High/4.1, 4Mbps
• Medium Quality: Up to 720p, 30p, Main/4.0, 2Mbps
• Low Quality: Up to 480p, 30p, Baseline/3.1 800kbps
• Progressive Download Single File via HTTP for HLS fallback
• Medium Quality only
48. Requirements
• use fully-managed service if possible
• scalable, reliable
• adaptive bitrate streaming
• Important: compatible with various platforms, devices
• Desktop Web - IE 10+, Edge, Chrome, Firefox
• Mobile Web - Mobile Safari, Chrome Mobie
• Android - 4.1 ICS+
• iOS - 9+
• Important: Delivery video to users ASAP
49. Requirements
• use fully-managed service if possible
• scalable, reliable
• adaptive bitrate streaming
• Important: compatible with various platforms, devices
• Desktop Web - IE 10+, Edge, Chrome, Firefox
• Mobile Web - Mobile Safari, Chrome Mobie
• Android - 4.1 ICS+
• iOS - 9+
• Important: Delivery video to users ASAP
52. Video Encoding Products
• AWS Elastic Transcoder
• Fully-managed service, pay only for what you use (per minute of output video)
• HLS Support
• Configurable Presets
• AWS Elemental MediaConvert
• Newly launched, Fully-managed service, pay only for what you use (per minute of output video)
• HLS Support
• Broadcasting-grade, high quality output, Professional Codec Support (e.g. ProRes)
• Configurable Presets
• In some scenarios, MediaConvert is cheaper than Elastic Transcoder
• New! Basic Tier for Simple Web Videos
61. Results
• Overall workflow was stable
• Quality
• The output quality of both was similar
• Fixed GOP => Consistant segment duration
• Processing Speed (sorted by faster)
• Elastic Transcoder
• MediaConvert Basic
• MediaConvert Pro
62. Problems
• Pricing is quite expensive
• Needed more faster encoding speed without losing quality
• Missing CRF (Constant Rate Factor) bitrate mode support
• Can’t control encoder options
• MediaConvert has better encoder option (e.g . quantize algorithm -
CABAC)
• Can’t control video filters
(e.g. lanczos for resizing, yadif for deinterlacing, color filters like
instagram)
• There’s no way to remux video
64. Three ways to run job
• Batch - EC2
• Fargate - ECS
• Step Functions + Lambda
65. Three ways to run job
• Batch - EC2
• Fargate - ECS
• Step Functions + Lambda - WINNER!
66. Three ways to run job
• Batch - EC2
• Requires additional instance provisioning time (extremely slow)
• Fargate - ECS
• Requires additional container provisioning time
(about 30sec~60sec)
• Setting High CPU power requires High Memory Size
• Lambda
• Computing Power is quite slower than EC2/Fargate instances
• …but now we have 3008MB Memory option, which has faster processing power
• No additional provisioning time required
71. How to get encoding
software running on AWS
Lambda?
72. Lambda-runtime compatible
build
• Linux program
• Lambda execution runtime is based on Amazon Linux AMI (amzn-ami-hvm-2017.03.1.20170812-
x86_64-gp2)
• Instead of using build instance, just make build image which is based on amazon linux docker
image
• docker pull amazonlinux:2017.03.1.20170812
• vim Dockerfile # Be careful updating openssl. it can make incompatible shared library issue
• docker build . -t my-awesome-lambda-build-env
• docker run --rm -v $PWD:/workspace my-awesome-lambda-build-env /workspace/build.sh
• Test built binaries or libraries using lambci lambda image
• docker pull lambci/lambda:nodejs8.10
• docker run --rm -v $PWD:/workspace --entrypoint bash lambci/lambda:nodejs8.10 /
workspace/test.sh
76. Step functions
• Makes workflow simple to orchestrate AWS Lambda functions
• Supports Parallel state
• built-in error handling, retry, fallback conditions
• Useful patterns
• Delay execution for X seconds or until specific date/time
• Recursive Execution using Choice State
• Scan all records from DynamoDB table
• dequeue messages from SQS until queue is empty
82. VPC Issue
• Assigning Lambda to VPC introduces long cold-start time
• about 10 sec, even used 3008MB memory size
• impacts overall video processing time
• Occasionally we had “connection timeout” issues
• AWS Support confirmed sometimes ENI may not have
internet connectivity
• we had to implement our own retry logic
84. VPC Issue
• VPC was required for S3 access through S3
Endpoint only (to authenticate s3 access using VPCE
ID)
• Removed VPC configuration
• Switched to stream data using pre-signed urls or
download object using AWS-SDK
• reduced cold-start time (<40ms)
• “connection timeout” issue has gone
87. Tiny ephemeral storage (/
tmp)
• Ephemeral storage capacity is 512MB, too small for media processing
• You can’t add block device to lambda
• NFS / EFS
• Lambda does not provide root permission
• so you cannot mount disks - you cannot use NFS in Lambda
• Ramdisk
• Ramdisk like /dev/shm is not available on Lambda
• Lambda does not provide root permission
• so you cannot create or mount ramdisk
88. Tiny ephemeral storage (/
tmp)
• S3 - only available option for now
• stream everything if possible
• use s3 as temporary storage
• Video Input - S3 supports seeking object using Range request header => can be streamed
• Video Output - may vary depending on container
• MP4 - requires seekable output => you cannot pipe output => you cannot stream outputs
• HLS - does not support writing outputs to pipes, since HLS consists of several files
• Watch file system (e.g. chokidar)
• Upload segments to s3 if generated
• Removed uploaded file
• Note: Higher memory size option provides faster Disk I/O and Network I/O
• Note: S3 Object keys are stored in UTF-8 binary ordering across multiple partitions in the index
=> UUID key prefix helps to reduce first byte latency
95. Delivery block
• Currently we don’t need any DRM or AD
insertion solution
• Simple architecture
• Use S3 as origin, Cloudfront as CDN
• Use maximum TTL to maximize cache hit
105. 2nd Version
• Improves encoding speed, output quality
• Support Parallel Encoding
• Split video streams to small chunks
• Split Video/Audio encoding process
• Encode streams per each preset, not outputs
• HLS HQ / HLS MQ / HLS LQ / MP4 MQ
=> Encode streams to HQ / MQ / LQ and assemble them
• Bypass encoding if input is compatible with our spec
• Skip HLS encoding process if source quality is not good, or duration is too short
• Support GIF input/output
109. Results
• Fast processing without losing quality
• now processing f(x) M/V (1080p, 3m 33s) just takes 60 sec
• if input source is compatible with our spec, processing can be done within <10 sec
• Great output quality
• Saving cost
• encoding is not required if input source is compatible with our spec
• Stable workflow
• There’s no resource to manage
• Acceptable max bitrate: Up to 300Mbps (Tested with ProRes source)
Acceptable max resolution: 4K
Acceptable codec: HEVC, ProRes
Acceptable max duration: 1 hour
110. Further Steps
• Support 1 hour+ video processing using Elastic Transcoder
• Support Live Streaming
• MediaLive? Fargate?
• Finding solution for WebRTC gateway to relay live video stream
• Video Advertisement
• MediaTailor?
• Metrics, Management Tools …
• NSFW filtering - rekognition video or rekognition