A fotopedia presentation made at the MongoDay 2012 in Paris at Xebia Office. Talk by Pierre Baillet and Mathieu Poumeyrol. French Article about the presentation: http://www.touilleur-express.fr/2012/02/06/mongodb-retour-sur-experience-chez-fotopedia/ Video to come.

1. MongoDB vs MySQL
A DevOps point of view.
Pierre Baillet <oct@fotopedia.com> @octplane
Mathieu Poumeyrol <kali@fotopedia.com>

2. Summary
«The question is, which is to be master» Humpty Dumpty
Who we are
Context and constraints
High availability and day-to-day operations
Scalability

3. Who we are
Fotopedia (and not photopedia or fotolia)
Created in 2006, Paris based
around 20 people, some Apple ex-employees
Pictures for humanity, cross-breed between flickr and
wikipedia

4. What we do
Website http://www.fotopedia.com (100% free website,
become member and show us your best photos)
2011 Crunchies award for Best Tablet Application

5. Some statistics
150 Millions photos views
1 MySQL database
4 MongoDB ‘clusters’ (spread on 4 servers)
Around 500GB of structured data

6. Context and Constraints
«La nuit ne peut qu'empirer mille fois» Roméo & Juliette
24/7 Website and web-services
Continuous deployment
Current Infrastructure
What we expect from our NoSQL DBMS and our
compromise

7. 24/7
Several million of users around the world. Between 300
and several thousands at once connected.
Using either the website or one of the 7 available iOS
applications. Business Critical
When the website is down at the application level,
everything starts to fail gradually
We cannot stop the website completely. Ever.

8. Overall activity on the main HTTP entry point

9. Continuous deployment
Git-based development flow with several active
branches
development branch deployed every wednesday
an average of 3 minor hot-fixes every workday
agile: any developer can push its hot fixs in production,
at any time
We cannot easily schedule migrations. They should be
as transparent as possible.

10. Our infrastructure
Software stack
Monitoring tools
Hosting platform

11. Software Stack
RoR Website
Multiple OpenSource software used in the web stack:
HAProxy, Nginx, Unicorn, mongo-resque, ...
Some well known NoSQL tools:
MySQL used to manage what was the core of our
data
MongoDB in production since September 2009, now
managing more than 70% of our data

12. Monitoring tools
Munin, Nagios
Custom log feeder build around MongoDB (cf
slideshare presentation: "mongodb as a log collector")
MongoDB is also used to store slow transactions,
exceptions and profiling traces for later inspection

13. Hosting Platform: 100% AWS
Instances are not highly reliable
but they are both abundant and disposable
Disk is abundant and disposable too
Use AWS RDS for MySQL hosting. Cheap and easy to
setup but very shaky failover process (DNS based).
We cannot rely too much on the hardware

14. What we expect from our NoSQL DBMS and our
compromise
No downtime:
High availability
No migration cost
Easy to deploy, redeploy, replicate, reconfigure
Quietly losing seconds of writes is preferable to
weekly minutes-long maintenances periods
minutes-long unscheduled downtime and manual
failover in case of hardware failure

15. High Availability,
Day to Day operations
«Au fond de la cave, Paraît qu'il y a pas de sots métiers» Le poinçonneur de lilas
Development environment
Operations cycle
Fit for the DevOps

16. Dev’ Cycle
Data locality
Data migration
Alter table
Index creation
Data backup/restoration

17. Dev’ Data Locality
In MongoDB, a collection will typically replace 2 or 3
SQL tables
The physical proximity, locality, enables faster, simpler
and more complete data retrieval from the application
point of view. Less requests, more data.

18. Dev’ Data Migration: ALTER
ALTER TABLE is nightmarish
leads to various forms of model abusing strategy:
reuse of fields
flag fields (binary encoded), blob fields (json/xml
encoded), ...
MongoDB solution: free form data storage, extensible.

19. Defensive strategy
Application code aware of possible inconsistencies:
gracefully failing view layer
self-healing data access layer
routine data checking and fixing batch

20. Dev’ Data Migration: INDICES
Indices creation leads to table-wide lock in MySQL.
Renders part of the Cluster unavailable
MongoDB solution: Background indices creation, slows
access a tiny bit, but do not lock !

21. Dev’ Backup/Restore
MongoDB ability to dump a db/collection empowers
developer
Possible to restore part of the production dataset
simply on a development box
Backup a MongoDB by collections in S3, recover on
dev’ platform in a matter of minutes

22. Ops Cycle
MongoDB, small is beautiful
Cornerstone: the Replica Set
High availability
Backup and data import/export
Hardware migration

23. Ops, MongoDB, small and beautiful
Young software, relatively compact (around 150,000 of
C++ code)
Builds out of the box on modern distributions
Distros Package made by 10gen
Drivers for most popular languages are also provided
and maintained by 10gen staff. (although quality varies)

24. Ops, Replica Set
A set of machine sharing the same data
Only one Primary, several Secondaries
All writes go to Primary, routed to secondaries.
Reads can be routed to primary or secondary at the
application choice
With the combination of AWS, Replica Set are very
powerful. MongoDB Loves the Cloud !

25. Ops, Master/Slave reloaded
Client libraries are replica set aware
connect to any node(s), the configuration and current
layout is discovered
Database semantics are preserved
Incredibly easy to setup
Priority between nodes can be dynamically changed
It’s possible to prevent a node from ever becoming
master (slow-disk server used as a «hot backup»)

26. Ops, High Availability Strengths
Primary step down can be triggered. Lead to election
of a new Primary.
a new Primary is picked when the Primary becomes
unreachable
clients will transparently connect to the new Primary
MongoDB Arbiter ensure split brain will not happen
Config. Server contains the sharding information. 1 or 3
config servers with internal failover mechanism

27. Ops, High Availability compromise
Switch over will take 20 to 25 seconds
Some queries in the interval may crash
Some writes may reach a split primary

28. Ops, Backup and exports
Stop the secondary and do whatever you need done.
Easy to backup a single collection or a whole database
As a matter of fact, we just dumbly «mongodump»
every collection of interest separately.

29. Ops, Hardware migration
Optionally possible to «preload» with a FS or block level
snapshot
Add the brand new node to the replica set
Wait for synchro
Change RS rules to get your new server primary
Remove the old hardware

30. Fit for the DevOps
In the modern sense of DevOps, MongoDB provides
the Agility and Ease of use required
It provides working tools for developers
And is much more confortable than MySQL in its daily
usage
Truly a DevOps-friendly tool.

31. Scalability
«Accroche toi au pinceau, j’enlève le shell.» Entendu @fotopedia
Cloud Limitations
Sharding and Replica Set
Performance
Reading
Writing
Storage
Scalable from the ground up

32. Cloud Limitations
Virtual Hardware
Neighbors can eat all you I/O
No precise control and overview of this situation
Largest VM cannot compare to largest Metal
Hardware issue means zero-notice before instance
retirement (Metal has same issue though). Need to be
flexible
Scaling-out is the way to scale on the Cloud

33. Sharding
Use a business key to part your data
Each shard is typically a replica set
Access is provided via the MongoS servers
Configuration is stored and managed in the Config
servers

34. Reading
Without Sharding
Reading is performed on a master by default to
perserve read-your-own-writes. Can be
programmatically allowed on a slave.
To scale up reads, add Replica Set nodes
With Sharding
Reading is performed in parallel across data nodes
To scale up reads: ensure most queries will reach
only one single shard

35. Writing
Writes are always performed on the Primary node, so
replica set does not help.
Sharding distributes the write among the cluster

36. Storage
Replica Set and shards can be moved on as many
servers as needed.
To get more space
scale up by migrating your Replica Set to bigger
hardware
scale out by sharding existing the collection

37. Scalable from the ground up
MongoDB is scalable as soon as you need it to.
No complex configuration for replication
Beautiful ability to handle replica set and shards out of
the box
MongoS / Config Server / Shards allows more complex
setup
Cloud Friendly

38. Questions ?
Oh, and btw, we hire in Paris !