MongoDB Best Practices for Developers

© All rights reserved: Moshe Kaplan Big Data – Leading Platforms© All rights reserved: Moshe Kaplan MongoDB for Developers
For Developers

MongoDB
For Developers
Moshe Kaplan
Scale Hacker
http://top-performance.blogspot.com
http://blogs.microsoft.co.il/vprnd

© All rights reserved: Moshe Kaplan Big Data – Leading Platforms
It’s all About
Scale

HELLO. MY NAME IS MONGODB
Introduction

Who is Using mongoDB?
5

Who is Behind mongoDB

Key Value Store (with benefits)
• insert
• get
• multiget
• remove
• truncate
7
<Key, Value>
://wiki.apache.org/cassandra/API

When Should I Choose NoSQL?
• Eventually Consistent
• Document Store
• Key Value
8
http://guyharrison.squarespace.com/blog/tag/nosq

What mongoDB is Made of?
9
http://www.10gen.com/products/mongodb

Why MongoDB?
What? Why?
JSON End to End
No Schema “No DBA”, Just Serialize
Write 10K Inserts/sec on virtual machine
Read Similar to MySQL
HA 10 min to setup a cluster
Sharding Out of the Box
GeoData Great for that
No Schema None: no downtime to create new columns
Buzz Trend is with NoSQL
10

NoSQL and Data Modeling
What is the Difference

Database for Software Engineers
Class
Subclass
Document
Subdocument

Same Terminology
• Database  Database
• Table  Collection
• Row  Document

A Blog Case Study in MySQL
http://www.slideshare.net/nateabele/building-apps-with-mongodb

as a SW Engineer would like it to be…

Migration from RDBMS to NoSQL
How to do that?

Data Migration
• Map the table structure
• Export the data and Import It
• Add Indexes
http://igcse-geography-lancaster.wikispaces.com/1.2+MIGRATION

Selected Migration Tool

Usage Details> Install ruby
> gem install mongify
… Modify the code to your needs
… Create configuration files
> mongify translation db.config >
translation.rb
> mongify process db.config translation.rb

Date Functions
• Year(), Month()… function included
• … buy only in the JavaScript engine
• Solution: New fields!
• [original field]
• [original field]_[year part]
• [original field]_[month part]
• [original field]_[day part]
• [original field]_[hour part]

NO SCHEMA IS A GOOD THING BUT…
Schemaless

Default Values
• No Schema
• No Default Values
• App Challenge
• Timestamps…
No single source of truth

Casting and Type Safety
• No Schema
• No …
• App Challenge

Auto Numbers
• Start using _id
{
"_id" : 0,
"health" : 1,
"stateStr" : "PRIMARY",
"uptime" : 59917
}
• Counter tables
• Dedicated database
• 1:1 Mapping
• Counter++ using findAndModify

The ORM Solution

Data Analysts
http://www.designersplayground.com/pr/internet-meme-list/data-analyst-2/

Data Analysts
• This is not SQL
• There are no joins
• No perfect tools
Pentaho
RockMongoMongoVUE RoboMongo

No Joins
• Do in the application
• Leverage the power of NoSQL

Limited Resultset
• 16MB document size
• Limit and Skip
• Adjusted WHERE
• GridFS

Bottom Line
• Powerful tool
• Embrace the Challenge
• Schema-less limitations: counters, data types
• Tools for Data Scientists
• Data design

Mastering a New Query
Language

Connect to the Database
• Connect:
• > mongo
• Show current database:
• >> db
• Show Databases
• >> show databases;
• Show Collections
• >> show collections; or show tables;

Databases Manipulation: Create & Drop
• Change Database:
• >> use <database>
• Create Database
• Just switch and create an object…
• Delete Database
• > use mydb;
• > db.dropDatabase();

Collections Manipulation
• Create Collcation
>db.createCollection(collectionName)
• Delete Collection
> db.collectionName.drop()
Or just insert to it

SELECT: No SQL, just ORM…
• Select All
• db.things.find()
• WHERE
• db.posts.find({“comments.email” : ”b@c.com”})
• Pattern Matching
• db.posts.find( {“title” : /mongo/i} )
• Sort
• db.posts.find().sort({email : 1, date : -1});
• Limit
• db.posts.find().limit(3)

Specific fields
Select All
db.users.find(
{ },
{ user_id: 1, status: 1, _id: 0 }
)
1: Show; 0: don’t show

WHERE
• != “A” { $ne: "A" }
• > 25 { $gt: 25 }
• > 25 AND <= 50 { $gt: 25, $lte: 50 }
• Like ‘bc%’ /^bc/
• < 25 OR >= 50 { $or : [ { $lt: 25 }, { $gte : 50 } ] }

Join
• Wrong Place…
• Or Map Reduce

39
 db.article.aggregate(
 { $group : {
 _id : { author : "$author“, name : “$name” },
 docsPerAuthor : { $sum : 1 },
 viewsPerAuthor : { $sum : "$pageViews" }
 }}
 );
GROUP BY
< GROUP BY author, name
< SUM(pageViews)
< SUM(1) = N

40
db.posts.update(
{“comments.email”: ”b@c.com”},
{$set : {“comments.email”: ”d@c.com”}}
}
SET age = age + 3
• db.users.update(
• { status: "A" } ,
• { $inc: { age: 3 } },
• { multi: true }
• )
UPDATE

41
j = { name : "mongo" }
k = { x : 3 }
db.things.insert( j )
db.things.insert( k )
INSERT

42
db.users.remove(
{ status: "D" }
)
DELETE

Performance Tuning
Make a Change

MONGODB TUNING

journalCommitInterval = 300:
Write to disk: 2ms <= t <= 300ms
Default 100ms, increase to 300ms to save resources
Disk
The Journal
Memory
Journal Data
1 2

RAM Optimization:
dataSize + indexSize < RAM
OS
Data Index
Journal

PROFILING AND SLOW LOG

Profiling Configuration
• Enable:
• mongod --profile=1 --slowms=15
• db.setProfilingLevel([level] , [time])
• How much:
• 0 (none)  1 (slow queries only)  2 (all)
• 100ms: default
• Where:
• system.profile collection @ local db

Profiling Results Analysis
• Last 5 >1ms: show profile
• w/o commands:
db.system.profile.find( { op: { $ne : 'command' } } ).pretty()
• Specific database:
db.system.profile.find( { ns : 'mydb.test' } ).pretty()
• Slower than:
db.system.profile.find( { millis : { $gt : 5 } } ).pretty()
• Between dates:
db.system.profile.find({ts : {
$gt : new ISODate("2012-12-09T03:00:00Z") ,
$lt : new ISODate("2012-12-09T03:40:00Z")
}}).pretty()

Explain
> db.courses.find().explain();
{ "cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 11, “nscannedObjects" : 11, "nscanned" : 11,
"nscannedObjectsAllPlans" : 11, "nscannedAllPlans" : 11,
"scanAndOrder" : false, "indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {},
"server" : "primary.domain.com:27017"
}

INDEXES

Index Management
• Regular Index
• db.users.ensureIndex( { user_id: 1 } )
• Multiple + DESC Index
• db.users.ensureIndex( { user_id: 1, age: -1 } )
• Sub Document Index
• db.users.ensureIndex( { address.zipcode: 1 } )
• List Indexes
• db.users.getIndexes()
• Drop Indexes
• db.users.dropIndex(“indexName”)

Known Index Issues
• Bound filter should be the last (in the index as well).
• BitMap Indexes not really working
• You should design your indexes carefully

STATS &
SCHEMA DESIGN

Sparse Matrix? I don’t Think so
• mongostat
• > db.stats();
• > db.collectionname.stats();
• Fragmentation if storageSize/size > 2
• db.collectionanme.runCommand(“compact”)
• Padding (wrong design) if paddingFactor > 2

High Availability
Going Real Time

(Do Not) Master/Slave

• In mongo.conf
• # Replication Options
• replSet=myReplSet
• > rs.initiate()
• > rs.conf()
• > rs.add(“host:port")
• rs.reconfig()
Replication Set

• rs.addArb(“host:port")
• Also:
• Low Priority
• Hidden
• (Weighted) Voting
Arbiter

Show Status: rs.status();
• {"set" : “myReplSet", "date" : ISODate("2013-02-05T10:23:28Z"),
• "myState" : 1,
• "members" : [
• {
• "_id" : 0, "name"
: "primary.example.com:27017",
• "health" : 1, "state" :
1,
• "stateStr" : "PRIMARY",
"uptime" : 164545,
• "optime" : Timestamp(1359901753000, 1),
• "optimeDate" : ISODate("2013-02-
03T14:29:13Z"), "self" : true
• },
• {
• "_id" : 1, "name"

Replica Set Recovery
• Create a new mongod
• Either install a plain vanilla
• Or duplicate existing mongod (better)
• Connect to the system
• Use the previous machine IP
• Or change configuration to remove old and add new

Sharding and Scale out:
Make a big Change
Map Reduce and Aggregation

Secondary Read Enabling

The Strategy : Sharding

MongoDB Implementation

Summary
• NoSQL
• Schemaless
• HA
• Sharding

Thank You !
Moshe Kaplan
moshe.kaplan@brightaqua.com
054-2291978

MongoDB Best Practices for Developers

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à MongoDB Best Practices for Developers

Similaire à MongoDB Best Practices for Developers (20)

Plus de Moshe Kaplan

Plus de Moshe Kaplan (20)

Dernier

Dernier (20)

MongoDB Best Practices for Developers