1. Using CloudFront with S3
at SMARTSTUDY
Amazon Web Service Korean User Group
Park Hyun-woo, a SCV of SMARTSTUDY / ez@smartstudy.co.kr
2. What is S3 and CF?
● S3 : Simple Storage Service
○ Extreme durability.
○ Multiple copies in selected region.
■ vs EBS : Multiple copies in single available zone.
● CF : CloudFront
○ Content Delivery Network(CDN) service.
○ Distribute from origin to AWS Edge Network
■ http://aws.amazon.com/cloudfront/#details
■ About 30 edges in global. (only in Earth, lol)
3. Why choose these?
● We serve large multimedia contents to
global market in mobile application era.
● We need redundant, fast and reasonable
price content delivery service aims globally.
● We have not enough human resource to
maintain various system.
5. Transcoder
Flash Gnash
FFmpeg
Package
( encrypted )
Movie qtfaststart
Thumbnails
npk
Video
S3 Bucket s3cmd
( tokyo )
6. npk
Video
S3 Bucket s3cmd
( tokyo )
{
Edge Edge Edge
CloudFront
Edge Edge
7. How it works?
● Copy to S3
○ ~100 files, just use Cyberduck or other GUI tool.
○ Or, you have to use s3fs or s3cmd.
○ If you need sync or invalidation, use s3cmd.
○ We have ~30,000 files. Thus, s3cmd.
○ We chose Tokyo bucket for faster uploading.
● s3cmd
○ Awesome!
○ Works on S3 and CloudFront.
8. How it works? (cont.)
● s3cmd
○ Can do almost everything as like as AWS console.
○ Support INI-style configuration file.
○ And, magical 'sync' command. (yeah!)
● s3cmd sync
○ Support glob-style exclude / include option.
○ ... rexclude / rinclude for regular expression.
○ Support automatic invalidation request.
○ ... with rsync-like 'dry-run' option. (-n, --dry-run)
9. How it works? (cont.)
● s3cmd sync
○ Basically, sync based on MD5 information.
○ (!) Over 15MB files, using only filesize.
○ Our 30,000 files, 60GB = 8 min to dry-run.
( env : i7 2500K + SATA HDD to bucket in tokyo )
● s3cmd sync --cf-invalidate
○ Call invalidation API automatically.
○ (!) Simultaneous invalidation is up to 3.
○ (!) Free for ~1,000 files per month.
○ Check it up by s3cmd cfinvalinfo
10. So, is it really?
FAST
{ REDUNDANT
NOT EXPENSIVE
14. Bill for Bill for
Domestic CDN CloudFront
Based on Based on
Peak traffic Total traffic
● 350Mbps/@peak ● 10TB/month
● $6.00/Mbps ● $0.209/GB
= $2,100 = $2,090*
* without transfer fee for deploying origin bucket to edges
18. Tips
● Smaller
○ Encode media with optimal options.
○ Use dedicated option for each platform.
○ Storage is cheaper than transfer fee!
● Static
○ Do not deploy volatile objects to CloudFront.
○ Invalidation needs ~10min, often.
○ We're still using Nginx for them.
● Streamable
○ Mobile networks are still slow.
○ Do not make users to wait until finishing download.