NoSQL: onde, como e por quê
•
25 j'aime•8,120 vues
Visão geral sobre bancos de dados NoSQL e detalhes técnicos dos modelos de dados e arquiteturas das implementações Apache Cassandra e MongoDB.
![NoSQL: onde, como e por quê? Rodrigo Hjort [email_address]](https://image.slidesharecdn.com/nosql-ondecomoeporqu-hjort-111110074847-phpapp01/85/nosql-onde-como-e-por-qu-cassandra-e-mongodb-1-320.jpg)
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à NoSQL: onde, como e por quê
Similaire à NoSQL: onde, como e por quê (20)
NoSQL: onde, como e por quê
- 1. NoSQL: onde, como e por quê? Rodrigo Hjort [email_address]
- 2. Quem aqui usa banco de dados?
- 3. Por que precisamos de SQL?
- 4. Onde usamos SQL (i.e. ACID)?
- 6. Universo digital em expansão Fonte: IDC White Paper, "The Diverse and Exploding Digital Universe", 2008.
- 10. BASE otimista, aceitando que a consistência esteja em um “estado de fluxo” http://queue.acm.org/detail.cfm?id=1394128 Possibilita a escalabilidade horizontal...
- 11. NoSQL = Not Only SQL http://nosql-database.org/ distribuídos não relacionais horizontalmente escaláveis esquemas flexíveis replicáveis APIs simples
- 12. Zoologia dos bancos NoSQL Wide Column Store / Column Families Key-Value Store Document Store NoSQL Database
- 15. Disponibilidade : toda operação tem uma resposta
- 16. Partição : sistema continua operante mesmo enfrentando partições na rede Consistência Consistency Disponibilidade Availability Partição Partition Tolerance
- 20. “ A high performance, scalable, distributed storage and processing system for structured and unstructured data.”
- 21. Cassandra: um breve histórico Bigtable Dynamo
- 22. Um novo modelo de dados Row schema-less schema-optional
- 23. Exemplo: modelagem do Twitter Users Following Followers @paul segue @brigitte desde 22/08/2010 john name: John Doe pass: swordfish joined: 20091115 paul name: Paul Lane pass: thepass joined: 20091129 john paul: 20091204 brigitte: 20100815 paul john: 20091205 debora: 20100729 brigitte: 20100822 john tom: 20091128 paul: 20091205 brigitte john: 20100815 paul: 20100822
- 24. Exemplo: modelagem do Twitter Statuses (Tweets) Timeline Userline Tweets do @john Tweets dos usuários que o @paul segue data/hora tweet 12345 user: john body: Nuvem privada do @serpro! retweets: 123 12346 user: brigitte john 20100116083155: 12346 paul 20100116083002: 12345 20100116083155: 12346 john 20100116083002: 12345 20100118235914: 23457 brigitte 20100116083155: 12346 tweet body: Acabei de #acordar. tags acordar: 1
- 25. CQL (Query Language) CREATE COLUMNFAMILY users ( KEY varchar PRIMARY KEY, name varchar, pass varchar, joined bigint); INSERT INTO users (KEY, name, pass) VALUES ('jsmith', 'John Smith', 'changeme') USING CONSISTENCY QUORUM; SELECT * FROM users WHERE KEY = 'jsmith'; u'jsmith' | u'pass',u'changeme' SELECT name..pass FROM users WHERE KEY >= 'h' LIMIT 10; CREATE INDEX users_joined_idx ON users (joined); DELETE joined FROM users where KEY = 'jsmith';
- 26. “ It took two weeks to perform ALTER TABLE on the statuses [tweets] table.” – Twitter
- 30. One
- 31. Quorum: N / 2 + 1
- 33. R + W > N
- 34. Read repair ack cliente réplica réplica réplica coordenador
- 35. Comunicação entre os nós Gossip-Based Protocol
- 38. escrita: ~0,12 ms Leitura 23x mais rápida! Escrita 2500x mais rápida!
- 39. “ MongoDB (from "humongous") is a scalable, high-performance, open source, powerful, document-oriented database written in C++.”
- 40. O modelo de dados Relacional (Tabular) Orientado a Documentos
- 42. Modelo Orientado a Documentos
- 43. { _id : ObjectId("5ebf5e0fec5fab7db2b9b40e"), title : "Introdução ao MongoDB", slug : "introducao-ao-mongodb", body : "Este é o texto do post...", published : true, created : "Jun 28 2011 13:48:22 AMT", updated : "Jun 28 2011 17:01:15 AMT", comments : [ { author : "john", email : "john@doe.com", body : "Caramba!", created : "Jun 28 2011 15:01:30 BRST" } ] , tags : [ "databases", "MongoDB", "nosql" ] } Um documento JSON Array Object ID Embedded Document
- 44. Linguagem de Consulta SELECT * FROM usuarios > db.usuarios.find() SELECT nome FROM usuarios > db.usuarios.find({}, {“nome”: 1}) SELECT * FROM usuarios WHERE idade = 29 > db.usuarios.find({“idade”: 29}) SELECT * FROM usuarios WHERE idade = 29 AND ativo = true > db.usuarios.find({“idade”: 29, “ativo”: true}) SELECT * FROM usuarios WHERE idade >= 18 AND idade <= 30 > db.usuarios.find({“idade”: {“$gte”: 18, “$lte”: 30}}) SELECT * FROM usuarios WHERE nome LIKE “%admin%” > db.usuarios.find({“nome”: /admin/i})
- 45. Linguagem de Consulta SELECT * FROM usuarios ORDER BY nome > db.usuarios.find().sort({“nome”: 1}) SELECT * FROM usuarios ORDER BY idade DESC, nome > db.usuarios.find().sort({“idade”: -1, “nome”: 1}) SELECT * FROM usuarios LIMIT 3 > db.usuarios.find().limit(3) SELECT * FROM usuarios OFFSET 5 > db.usuarios.find().skip(5) SELECT * FROM usuarios LIMIT 3 OFFSET 5 > db.usuarios.find().limit(3).skip(5) SELECT * FROM usuarios ORDER BY nome LIMIT 3 > db.usuarios.find().sort({“nome”: 1}).limit(3)
- 46. Map Reduce índices capped collections Server-Side Scripting GridFS ad hoc queries
- 47. Sharding
- 49. A grande ruptura
- 51. “ NoSQL adoption is inevitable because, just as in every other walk of life, there are different tools for different jobs” – Stephen O'Graddy (RedMonk) Rodrigo Hjort [email_address] http://www.hjort.co
Notes de l'éditeur
- ...
- A grande maioria das aplicações necessita de algum tipo de persistência de dados.
- Por que precisamos de SQL? Modelo relacional está bem consolidado Linguagem com décadas de evolução Integridade referencial e transações (ACID) Conjunto rico de ferramentas É o que aprendemos É o que o mercado usa
- Onde usamos SQL (i.e. ACID)? Aplicações empresariais Agências de viagem Internet banking Compras online Cartões de crédito Transações em geral
- ...
- Informação digital criada, capturada e replicada pelo mundo Fonte: IDC White Paper, &quot;The Diverse and Exploding Digital Universe&quot;, 2008.
- Aplicações web modernas Grandes volumes de dados (escala da Internet) Altas taxas de leitura e escrita Necessidade de alta disponibilidade Frequentes mudanças nos esquemas Aplicações “sociais” não exigem os mesmos níveis de consistência que aplicações “bancárias”
- • Scaling existing Relational Databases is hard • Sharding is one solution, but makes your RDBMS unusable • Operational Nightmare
- Os modelos transacionais ACID Atomicity, Consistency, Isolation, Durability: a set of properties that guarantee database transactions are processed reliably BASE Basically Available, Soft state, Eventual consistency : as opposed to the database concept of ACID http://queue.acm.org/detail.cfm?id=1394128 Eventually Consistent http://queue.acm.org/detail.cfm?id=1466448
- O Movimento NoSQL NoSQL = Not Only SQL http://nosql-database.org/ bancos que diferem do modelo clássico relacional não relacionais distribuídos horizontalmente escaláveis com esquemas flexíveis replicáveis APIs simples BASE (e não ACID)
- Wide Column Store: Bigtable (Google) SimpleDB (AWS) Cassandra (Apache) HBase (Apache) Hypertable Document Store: CouchDB (Apache) MongoDB Key-Value Store: Riak Redis Table Storage (Microsoft Azure)
- ...
- Eric Brewer (UCB) in 2000 presented the CAP theorem , which states that of 3 properties of shared-data systems: C: data consistency A: system availability P: tolerance to network partition Only 2 can be achieved at any given time! A more formal confirmation can be found in a 2002 paper by Seth Gilbert and Nancy Lynch.
- CA – Corruption possible if live nodes can’t communicate (network partition)
- CP – Completely inaccessible if any nodes are dead
- AP – Always available, but may not always read most recent NoSQL chooses AP, but makes consistency configurable
- Cassandra highlights ● High availability ● Incremental scalability ● Eventually consistent ● Super fast writes ● Tunable tradeoffs between consistency and latency ● Minimal administration ● No SPF
- Developed at Facebook (problem of inbox search) Follows the BigTable ( Google ) Data Model - column oriented http://labs.google.com/papers/bigtable.html Follows the Dynamo ( Amazon ) Eventual Consistency model http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html Opensourced at Apache as an incubator project (2008) and then was top-leveled (2010) Implemented in Java
- Keyspace Agrupamento de famílias de colunas (~banco de dados) Column Family Agrupamento de colunas com ordenação fixada (~tabela) Row Key Chave que representa uma linha de colunas (~chave primária) Column Representação de um valor, com: - Nome (Name) - Valor (Value) - Timestamp
- ...
- ...
- Similar à linguagem SQL, mas com particularidades do Cassandra: consistency levels, TTL, slices
- Importância de esquemas flexíveis
- Every node is equal! Always at least one copy in each datacenter Alternate datacenters on the ring DHT (Distributed Hash Table) Ring
- Eventual consistency ● Synch to Washington, asynch to Hong Kong Client API Tunables ● Confirm R replicas match at read time ● Synchronously write to W replicas of N total replicas Allows for almost-strong consistency ● When W + R > N
- Gossip protocol (~P2P) is used for cluster membership and failure detection on nodes. Enables seamless nodes addition Rebalancing of keys Fast detection of nodes that goes down Every node knows about all others - no master State disseminated in O(log N) rounds
- * Dados das primeiras versões do Cassandra (v0.6) Versão 1.0 trouxe melhoras de: - 40% na escrita - 400% na leitura! http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-performance
- MongoDB is a powerful, flexible, and scalable data store. It combines the ability to scale out with many of the most useful features of relational databases, such as secondary indexes, range queries, and sorting. MongoDB is also incredibly featureful: it has tons of useful features such as built-in support for MapReduce-style aggregation and geospatial indexes. 1. (Slang.) humongous extraordinarily large.
- MongoDB basic concepts: • A document is the basic unit of data, roughly equivalent to a row in a RDBMS • Similarly, a collection can be thought of as the schema-free equivalent of a table • A single instance of MongoDB can host multiple independent databases , each of which can have its own collections and permissions Document : an ordered set of keys with associated values (i.e., map, hash, or dictionary)
- With Mongo, you do less &quot;normalization&quot; than you would perform designing a relational schema because there are no server-side joins . Generally, you will want one database collection for each of your top level objects.
- JSON-style documents with dynamic schemas offer simplicity and power.
- Rich, document-based queries.
- Index on any attribute, just like you're used to. Allows it to do queries orders of magnitude faster. Geospatial index is also provided. Map Reduce : a method of aggregation that can be easily parallelized across multiple servers. A capped collection is created fixed in size, behaving like circular queues . GridFS is a mechanism for storing large binary files in MongoDB. No need for a separate file storage architecture. JavaScript can be executed on the server.
- Sharding is the process of splitting data up and storing different portions of the data on different machines. Sharding is MongoDB’s approach to scaling out . Sharding allows you to add more machines to handle increasing load and data size without affecting your application. Scale horizontally without compromising functionality.
- From the application’s point of view, a sharded setup looks just like a nonsharded setup. There is no need to change application code when you need to scale. To set up sharding with no points of failure , you’ll need the following: • Multiple config servers • Multiple mongos servers • Replica sets for each shard • w set correctly
- Grande ruptura – IMS x RDBMS (invenção do modelo relacional)
- A segunda ruptura: RDBMS x NoSQL
- Conclusão NoSQL can’t do everything Right tool for the right job