Graph database super star

Notes de l'éditeur

1. Graph Databases can solve problems that your normal database struggle with. These can be hard problems, and not all of them were mentioned as graph problems in your CS classes. Using a Graph Database correctly can save the day, and make you look like a super star among your peers.\n\nFind out what a Graph Database can do for you. You might not have a classic "graph problem," but a Graph Database might still be a good solution for your data. Learn what problems Graph Databases solve, how to use one, and most importantly when to use a Graph Database. All of this with concrete examples using the Neo4j Graph Database.\n\nWith this talk, we want to:\n* Show you what kinds of data are at the sweet spot of Graph Databases.\n* How to structure your database to reap the benefits of graphy data\n* Teach you how to know when to use a Graph Database, and the fundaments of how to do so\n* Show you real application examples on the Neo4j Graph Database\n
2. \n
3. \n
4. We are assuming that you are pretty much like us. We want to build things, useful, cool things. Sure theoretical discussions are fun, but at the end of the day, I want things that work, and things that make my users go “wow”. \n
5. Database performance is like air. You don’t miss it until it’s not there. We don’t want to have to think about the performance. \n
6. This is another we don’t want to have to think about is if our data is safe. We just assume it is. If the database tells us it’s saved the data, we should trust it.\n
7. \n
8. \n
9. Relational databases are very useful, in some contexts. The pain comes when we try to shoe horn in our data. We loose semantics, and we have to do a lot of extra work to make it work.\n
10. Databases introduce accidental complexity, i.e. not complexity that has to do with the domain complexity. I want a database that gets out of my way. \n
11. First you whiteboard a sketch of how you want your data. Then you create a ER-diagram, which is then mapped to a physical model. And let’s not forget our normal forms - I’m sure everyone here makes sure that their database schema is at least in Boyce-Codd Normal Form, right? This is by no means an intuitive process - it takes time to learn. I remember clearly the first few times I had to do this.\n
12. When we shoehorn our data into a mold it doesn’t really fit in, we also loose performance. The key factor here is that as data size grows, query time grows. And modern systems today create a lot of data.\n
13. \n
14. So what are the alternatives? \n
15. So what are the alternatives? \n
16. So what are the alternatives? \n
17. So what are the alternatives? \n
18. So what are the alternatives? \n
19. So what are the alternatives? \n
20. So what are the alternatives? \n
21. Use the right shape of database for the right kind of object\n
22. When it comes to size, most databases scale just fine for most of the use cases. The second question is how much complexity your database can handle for you. You can do any complex procedure on any database, so the question is: how much can you unload on your database, and how fast does it do it.\n
23. When it comes to size, most databases scale just fine for most of the use cases. The second question is how much complexity your database can handle for you. You can do any complex procedure on any database, so the question is: how much can you unload on your database, and how fast does it do it.\n
24. When it comes to size, most databases scale just fine for most of the use cases. The second question is how much complexity your database can handle for you. You can do any complex procedure on any database, so the question is: how much can you unload on your database, and how fast does it do it.\n
25. \n
26. \n
27. \n
28. \n
29. \n
30. \n
31. \n
32. \n
33. \n
34. \n
35. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
36. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
37. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
38. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
39. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
40. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
41. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
42. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
43. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
44. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
45. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
46. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
47. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
48. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
49. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
50. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
51. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
52. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
53. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
54. nodes\nrelationships between nodes\nrelationships have types and directions\nrelationships are traversed equally fast in each direction, direction may be significant or not\nnodes have properties\nrelationships have properties\n
55. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
56. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
57. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
58. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
59. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
60. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
61. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
62. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
63. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
64. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
65. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
66. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
67. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
68. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
69. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
70. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
71. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
72. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
73. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
74. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
75. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
76. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
77. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
78. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
79. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
80. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
81. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
82. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
83. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
84. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
85. Querying a graph is done by traversing it.\nTraversing the graph means moving through it to find and gather results\n
86. \n
87. \n
88. \n
89. \n
90. \n
91. \n
92. \n
93. \n
94. \n
95. \n
96. \n
97. \n
98. \n
99. \n
100. \n
101. \n
102. \n
103. \n
104. \n
105. \n
106. \n
107. \n
108. \n
109. \n
110. \n
111. \n
112. \n
113. \n
114. \n
115. \n
116. \n
117. \n
118. \n
119. \n
120. \n
121. \n
122. \n
123. \n
124. \n
125. \n
126. \n
127. \n
128. \n
129. \n
130. \n
131. \n
132. \n
133. \n
134. \n
135. \n
136. \n
137. \n
138. \n
139. \n
140. \n
141. \n
142. \n
143. \n
144. \n
145. \n
146. \n
147. \n
148. \n
149. \n
150. \n
151. \n
152. \n
153. \n
154. \n
155. \n
156. \n
157. \n
158. \n
159. \n
160. \n
161. \n
162. \n
163. \n
164. \n
165. \n
166. \n
167. \n
168. \n
169. \n
170. \n
171. \n
172. \n
173. \n
174. \n
175. \n
176. \n
177. \n
178. \n
179. \n
180. \n
181. \n
182. \n
183. \n
184. \n
185. \n
186. \n
187. \n
188. \n
189. \n
190. \n
191. \n
192. \n
193. \n
194. \n
195. \n
196. \n
197. \n
198. \n
199. \n
200. \n
201. \n
202. \n
203. \n
204. \n
205. \n
206. \n
207. \n
208. \n
209. \n
210. \n
211. \n
212. \n
213. \n
214. \n
215. \n
216. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
217. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
218. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
219. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
220. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
221. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
222. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
223. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
224. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
225. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
226. A path is one or more nodes with relationships between them,\nthe number of relationships in a path is always one less than the number of nodes, i.e. zero or more\nA node may occur multiple times in a path, but the path defines a sequence by the interconnecting relationships\n
227. Contrary to many other non relational databases, Neo4j has full ACID transactions.\n
228. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
229. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
230. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
231. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
232. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
233. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
234. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
235. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
236. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
237. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
238. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
239. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
240. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
241. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
242. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
243. A transaction is either committed in its entirety, or not at all, never partial.\nNeo4j guarantees the integrity of relationships. The two nodes must exist.\nYou cannot access the state of an uncommitted transaction from outside of that transaction.\nNeo4j guarantees that when a transaction is committed it has been written to disk.\n
244. \n
245. \n
246. TO DECIDE: include or not?\n
247. \n
248. [AT]\n
249. \n
250. \n
251. \n
252. \n
253. \n
254. \n
255. \n
256. \n
257. \n
258. \n
259. \n
260. \n
261. \n
262. \n
263. \n
264. \n
265. \n
266. \n
267. \n
268. \n
269. \n
270. \n
271. \n
272. Schema evolution is only in the application layer. Making sense of the data is your applications responsibility, not the databases.\n
273. [AT]\nIf the data you store is hierarchical, chances are that a graph database is a good solution for you. \n
274. Has anyone in this room tried to store tree structures in a relational database? Hands up. I’ve done it a number of times - a content management system we built all the way back in 1997, and as late as last year, for an online mobile shopping site. Here are some ways you can do it.\n
275. The good old adjacency list. Simple, intuitive, and fast for writing. Reading from it is not quite as easy. This is probably the most used way of doing it, and offers no support for a lot of quite common queries you want to do.\n
276. Nested sets are the brain child of Joe Celko I believe, and a clever way of representing tree structures. The only problem is that adding to a tree, or moving branches of the tree requires roughly half the tree to move to the left or to the right.\n
277. I have actually never personally encountered this “in the wild”. But it’s an interesting way of solving the problem: just build up strings that show the path from the root to every single node in the tree. \n
278. \n
279. \n
280. \n
281. \n
282. \n
283. \n
284. \n
285. \n
286. \n
287. \n
288. [AT]\n
289. \n
290. \n
291. \n
292. \n
293. \n
294. \n
295. \n
296. \n
297. \n
298. \n
299. \n
300. \n
301. \n
302. \n
303. \n
304. \n
305. \n
306. \n
307. \n
308. \n
309. \n
310. \n
311. \n
312. \n
313. \n
314. \n
315. 1000 persons = 50’000 relationships\n
316. 1000 persons = 50’000 relationships\n
317. 1000 persons = 50’000 relationships\n
318. 1000 persons = 50’000 relationships\n
319. 1000 persons = 50’000 relationships\n
320. \n
321. \n
322. \n
323. \n
324. \n
325. \n
326. 20 million nodes\n64 million relationships between these nodes\nA* routing in this dataset - avg answer after 100 ms.\nOur worst response took three seconds, and the answer contained 5500 road segments.\n
327. 20 million nodes\n64 million relationships between these nodes\nA* routing in this dataset - avg answer after 100 ms.\nOur worst response took three seconds, and the answer contained 5500 road segments.\n
328. 20 million nodes\n64 million relationships between these nodes\nA* routing in this dataset - avg answer after 100 ms.\nOur worst response took three seconds, and the answer contained 5500 road segments.\n
329. 20 million nodes\n64 million relationships between these nodes\nA* routing in this dataset - avg answer after 100 ms.\nOur worst response took three seconds, and the answer contained 5500 road segments.\n
330. 20 million nodes\n64 million relationships between these nodes\nA* routing in this dataset - avg answer after 100 ms.\nOur worst response took three seconds, and the answer contained 5500 road segments.\n
331. \n
332. \n
333. \n
334. \n
335. \n
336. \n
337. \n
338. \n