The slides from my talk at MongoSF 2011.

1. Mapping Flatland
Storing and Querying Location
Data with MongoDB
MongoSF 2011 (#mongosf) Grant Goodale (@ggoodale)
May 24, 2011

2. Flatland

3. Flatland?

5. A sense of scale

6. The world

7. A bit of history
Originally written for Node Knockout 2010

8. Why MongoDB?
Easy to set up and run
Fast
Reasonably robust node.js driver (even better now)
Geospatial indexing and querying

9. Geo2D in 1.8
Data and Indices
Querying
Running / Scaling Production

10. Structuring your data
tile = {
_id : BSON::ObjectId(...)
position : [0,0],
letter : "A",
wildcard : "false"
}

11. Structuring your data
tile = {
_id : BSON::ObjectId(...)
position : {x: 0, y: 0},
letter : "A",
wildcard : "false"
}

12. Watch your language
> db[‘tiles’].insert({
position : {y: 50, x: 20},
letter : "A",
wildcard : "false"
})
=> BSON::ObjectId('4dd06d037a70183256000004')
> db.[‘tiles’].find_one()
=>
{"_id"=>BSON::ObjectId('4dd06d037a70183256000
004'), "letter"=>"A", "position"=>{"x"=>20,
"y"=>50}, "wildcard"=>false}

13. Be safe!
Use array notation; guaranteed ordering = WIN
Ruby: Use 1.9.x or OrderedHash in 1.8.x
Python: Use OrderedDict (introduced in 2.7) and SON
(in the BSON package)
Javascript: Did I mention arrays?

14. Creating the index
> db[‘tiles’].create_index([[“position”,
Mongo::GEO2D]])
=> “position_2d”
> db[‘tiles’].index_information
=> {"_id_"=>{"name"=>"_id_",
"ns"=>"test.test_tiles", "key"=>{"_id"=>1}},
"position_2d"=>{"key"=>{"position"=>"2d"},
"ns"=>"test.test_tiles", "name"=>"position_2d"}}
Defaults:
Min: -180, Max: 180,
bits: 26

15. Creating the index
> db[‘tiles’].create_index(
[[“position”, Mongo::GEO2D]],
:min => -500, :max => 500, :bits => 32
)
=> “position_2d”

16. More index fun
Only one Geo2D index per collection (SERVER-2331)
But it can be a compound index:
> db[‘tiles’].create_index([
[“position”, Mongo::GEO2D],
[“letter”, Mongo::ASCENDING]
])
=> “position_2d_letter_1”

17. 1.9.x bonus feature
Geo2d indices across an array field!
> db[‘words’].insert({
“word” : “QI”,
“tiles” : [
{“letter” => “Q”, position => [1,1]},
{“letter” => “I”, position => [2,1]}
]
})
=> BSON::ObjectID('4dd074927a70183256000006')
> db[‘words’].create_index([[
“tiles.position”,
Mongo::GEO2D
]])
=> “position_2d”

18. Geo2D in 1.8
Data and Indices
Querying
Running / Scaling Production

19. A caveat: We’re weird.
VS
http://www.flickr.com/photos/toasty/1540997910/ http://www.flickr.com/photos/vizzzual-dot-com/2175221420/

20. A caveat: We’re weird.
VS

21. Problems we don’t have
Projection issues
Great Circle distance
calculation
Polar coordinate systems
Pirates
http://www.flickr.com/photos/jmd41280/4501410061/

22. Querying real location data
Search by proximity: $near
Uses native units (degrees for [-180, 180])
Use $maxDistance to bound query
> db[‘tile’].find(:position => {“$near” => [10,10]}).to_a
=> [{"_id"=>BSON::ObjectId('4dd084ca7a70183256000007'),
"letter"=>"A", "position"=>[12,9]}]
> db[‘tile’].find(:position => {“$near” => [10,10],
“$maxDistance” => 1}).to_a
=>[]

23. Querying real location data
Need distance to center as well? Use $geoNear
Also includes fun stats
> db.command('geoNear' => 'tiles', 'near' => [1830,
2002], :maxDistance => 10)
)
=> {"ns"=>"test.tiles",
"near"=>"110000000000001100011000110010101010001000001011
1111", "results"=>[{"dis"=>3.999471664428711,
"obj"=>{"_id"=>BSON::ObjectId('4dd0b0957a701852bc02bf67')
, "position"=>{"x"=>1830, "y"=>2006}, "letter"=>"A"}}],
"stats"=>{"time"=>0, "btreelocs"=>3, "nscanned"=>2,
"objectsLoaded"=>1, "avgDistance"=>3.999471664428711,
"maxDistance"=>3.999471664428711}, "ok"=>1.0}

24. Querying real location data
Region queries: $within
Example: $box (rectangle)
> db[‘tile’].find(:position => {“$within” => {“$box” =>
[[10,10], [30,30]]}).to_a
=> [{"_id"=>BSON::ObjectId('4dd084ca7a70183256000007'),
"letter"=>"A", "position"=>[12,9]}]
[30,30]
[10,10]

25. Querying real location data
Alternately: $center (circle)
> db[‘tile’].find(:position => {“$within” => {“$center”
=> [[10,10], 5]}).to_a
=> [{"_id"=>BSON::ObjectId('4dd084ca7a70183256000007'),
"letter"=>"A", "position"=>[12,9]}]
5
[10,10]

26. Querying real location data
1.9.x bonus feature: $polygon!
> db[‘tile’].find(:position => {“$within” => {“$polygon”
=> [[5,5], [5,15], [15,5]}).to_a
=> [{"_id"=>BSON::ObjectId('4dd084ca7a70183256000007'),
"letter"=>"A", "position"=>[12,9]}]
[5,15]
[5,5] [15,5]

27. Querying real location data
Spherical equivalents: $nearSphere and $centerSphere
Uses radians, not native units
position must be in [long, lat] order!
> earthRadius = 6378 #km
=> 6378
> db[‘restaurants’].find(:position => {“$nearSphere” =>
[-122.03,36.97], “$maxDistance” => 25.0/earthRadius}).to_a
=> [{"_id"=>BSON::ObjectId('4dd084ca7a70183256000007'),
"name"=>"Crow’s Nest", "position"=>[-122.0,36.96]}]

28. MapReduce
using Geo2D in MapReduce and group queries doesn’t
work (SERVER-1742)
Fixed in 1.8.2 (Yay Greg!)

29. Geo2D in 1.8
Data and Indices
Querying
Running / Scaling Production

30. Again: we’re weird.
Big index, but no need for it all to be in memory
Large regions of the world where activity => 0 as
density => 1
Single box scaling limit determined by # of active
sections of the world at a time

31. Our setup
Master/Slave, moving to Replica Set
Slaves used for
backup
Map image generation
Next stop: geoSharding

32. Sharding
Yes, you can shard on a geo-indexed field
Not recommended in 1.8 due to query performance
(SERVER-1982). Fix coming in 1.9.x.
Can’t use $near in queries, only $geoNear and
therefore runCommand(). (SERVER-1981)

33. Questions?
http://www.flickr.com/photos/wili/3361117222/
@ggoodale
grant@massivelyfun.com