Presentation given the first AWS Startup Event 4/14/2010. Describes how Runa was using AWS for its SaaS for e-commerce sites

1. Runa on AWS
Big Data & Machine Intelligence for a SaaS
Startup

2. Runa

3. a
SaaS

4. converts
Shoppers to Buyers

5. for
Online
Commerce Sites

6. by presenting
Dynamic
Personalized
Promotions

7. on the
Merchant’s
Website

8. in
Real-Time

9. in the
Shopping Flow

10. Tech
Challenges

11. Big Data

12. JavaScript client
collects activity on
every Merchant page
for every Shopper

13. One or more
Ajax call & Event
Store to Runa per
Merchant page
view

14. Step function
increase of calls
and stores as
each new
Merchant added

15. We capture
everything we
can and store it
forever

16. Expecting to
grow to
thousands of
merchants

17. That’s a lot
of Data

18. Processing Data with
Machine
Intelligence

19. Batch Processing for
Statistical Analysis
and Reports

20. Real-Time
Rule based
inserts of
Promotions

21. Why AWS for
Runa?

22. At First
(a couple years ago)

23. Not Much
Money in
the Bank

24. Didn’t Know
exactly
what were
making

25. Or exactly
how we were
going to do it

26. Prototyped with
Ruby / Rails / MySQL

27. Then
Prototype
became
Production

28. EC2 & AWS let
us scale the
prototype to
Beta
Production

29. Flexibility to
incrementally
refine service &
infrastructure

30. Confidence
we could scale
as we added
Merchants

31. More Recently
Incrementally
added next-gen
Tech & Full
Production

32. Goal:
Everything
Horizontally
Scalable

33. Batch Processing &
Infinite Storage
Map / Reduce
& BigTable
via
Hadoop & HBase

34. Flexible Real-Time
parallel processing
via
Clojure / Swarmiji

35. Opscode Chef
Management
& Monitoring
Consumers on
Merchant Websites
Internet
Admin &
Merchant
Dashboard
(Rails)
Runtime
Rules
Merchant
Info
Merchants
Internet
AnalyticsReporting
Monitor &
Recovery
Data
Collectors
Hadoop / HBase
Map / Reduce
Petabyte Store
Load
Balancer
HTTP
Shared
Session
Memory
HTTP
Dispatchers
Redis
Mem
Cache
Redis
Mem
Cache
Redis
Mem
Cache
Redis
Mem
Cache
Redis
Mem
Cache
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
HBase
Amazon S3
Data Backup
9+ Amazon EC2 Instances
Amazon Elastic
Load Balancer
3+ Amazon EC2
Instances
Cheshire / Swarmiji
Dynamic
Runtime
Queue

36. AWS Elastic Load Balancer
Rails App Servers
Nginx / Unicorn EC2 m1.xlarge
MySQL
Master / Slave
EC2 m1.xlarge EBS
Legacy Runtime
Rails App Nginx/Unicorn
MySQL Master/Slave EC2 m1.xlarge / EBS
Merchant Dashboard
EC2 m1.xlarge
HBase / Hadoop
EC2 m1.xlargeEBS
RabbitMQ
Cheshire / Swarmiji
Redis
EC2 m1.xlarge
Clojure Based Runtime
AWS Elastic
Load Balancer
EC2 m1.large
Opscode Chef
Monitoring
EC2 m1.large
All Deployed on

37. Deployment &
Configuration
Management
via
Opscode Chef

38. Good Things

39. Able to Start Small

40. Then
GROW
BIGGER

41. Having the
flexibility to throw
“Hardware” at
our Prototype got
us to market
faster

42. Ability to
launch test
and staging
environments
almost at will

43. “Hardware”
as
“Software”

45. Living in
“interesting”
times

46. Managing
Complexity
lots of moving parts

47. Easy to launch a
few instances

48. Impossible to
manage
horizontal stacks
“by hand”

49. Must have
tool like
Opscode Chef

50. Chef automates
deployment &
puts it under
Revision Control

51. There’s going to
be some blood
when using
cutting edge tech

52. Lots of
Learning
Curves to
climb

53. Useful
Monitoring is
hard but
Critical

54. HBase on
AWS may be
dangerous
because of Hadoop namenode
SPOF

55. EC2 bill can surprise
you if you cavalierly
deploy multiple
versions of
horizontally scalable
environments

56. Could not do our
startup without
AWS or lots more
VC Funding