NoSQL 동향
•
24 j'aime•4,104 vues
This document discusses the results of a NoSQL database benchmark test conducted by NHN on Cassandra, HBase and MongoDB. It describes the test environment and four test cases performed: data insertion, random reads and updates on existing data, reads only on existing data, and random reads and inserts with additional data added. The test measured average transactions per second for each database and test case. Cassandra and HBase performance varied depending on compaction levels while MongoDB update and insert performance lagged the others.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à NoSQL 동향
Similaire à NoSQL 동향 (20)
Plus de NAVER D2
Plus de NAVER D2 (20)
NoSQL 동향
- 1. SDECofflinecommunity 1.NoSQLbenchmark NHN 저장시스템개발팀 김효
- 2. 테스트환경 CPU:IntelXeonL5640,12Core(Nehalem6CoreX2CPU)2.27GHz RAM:16GB CentOSrelease5.3(Final) TestCase1 • 데이터Insert • Insertkey는랜덤(FNVhash32이용) Insert • DB는비어있는상태 TestCase2 • 임의로Read,Update수행 • Read:Update비율은50:50 • DB는50,000,000row의데이터가저장된상태(Size=약60GB) Read/Update • Key분포는Zipfian TestCase3 • Read만수행 • DB는50,000,000row의데이터가저장된상태(Size=약60GB) Read • Key분포는Zipfian • 임의로Read,Insert수행(50,000,000row에서추가로row추가) TestCase4 • Read:Insert비율은50:50 Read/Insert • DB는50,000,000row의데이터가저장된상태(Size=약60GB) • Key분포는Zipfian • 테스트시간:1시간or30분(테스트종류별로다름) • 각각의테스트를Thread128인경우에대해수행 공통사항 • 모든Operation은1Row전체에대해읽기나쓰기를한다. • 1Row=Key+10FieldsX100Bytes(RandomString)=약1000Bytes 2/10
- 3. Cassandra-0.7.6-2 Keycache=MAX(메모리한계내에서최대로) • Rowcache=사용하지않음(default) • ReplicationFactor=3 • CommitLogSync=Periodicmode(default) • ConsistencyLevel Read,Write:LevelOne YCSB Client cassandra cassandra cassandra (ReplicaSet) (ReplicaSet) (ReplicaSet) 3/10
- 4. MongoDB-1.8.1 OplogSize=100G(default:디스크의5%) • ReplicaSet:3 YCSB Client mongod mongod mongod (ReplicaSet) (ReplicaSet) (ReplicaSet) Primary Secondary Secondary 4/10
- 5. HBase-0.90.3 Hadoop-0.20-append(durableappend패치된branch) • HBaseClientcache:사용안함 • HBaseHeapSize=8G • HDFSReplicationFactor=3 YCSB Client HDFS HDFS HDFS DataNode DataNode DataNode HBase HDFS Hadoop MasterServe NameNode Zookeeper r Hbase Hbase Hbase Region Region Region Server Server Server 5/10
- 6. 테스트결과 25000 Average TPS 20000 15000 Cassandra Hbase 10000 MongoDB 5000 0 Insert Read (Compac5on 전) Read (Compac5on 후) Read/Update Read/Insert • Cassandra,Hbase의경우SSTable의Compaction정도에따라Read의성능차이가있음 • Read의경우,Compaction전과후의테스트결과 • Read/Update,Read/Insert의경우,Compaction후의테스트결과만첨부 • MongoDB의경우Update와Insert가다른명령이어서성능차이있음 • 반면,Cassandra,Hbase의경우Update와Insert가내부적으로같은명령으로성능차이거의없음 6/10
- 7. Insert 40000 30000 TPS 변화(10초 간격) Cassandra 20000 Hbase 10000 0 MongoDB 30000 20000 MongoDB 10000 0 25000 • 50,000,000row가Insert되는속도에따라그래프의길이가다름 20000 Cassandra 15000 • 주의:두그래프의세로축스케일이다름 10000 Hbase • 테스트시간:50,000,000row가Insert될때까지 5000 MongoDB 0 Insert 7/10
- 8. Read 3500 3000 2500 Compac8on 전 Cassandra 2000 Hbase 1500 MongoDB 1000 500 0 6000 5000 Compac8on 후 4000 Cassandra 3000 Hbase 2000 MongoDB 1000 0 5000 • Cassandra,Hbase의Compaction전후의TPS변화와 4000 MongoDBTPS변화의비교그래프 3000 Cassandra 2000 Hbase • 주의:두그래프의세로축스케일이다름 1000 • 테스트시간:1시간 MongoDB 0 Read (Compac5on 후) 8/10
- 9. Read/Update 9000 8000 7000 6000 Cassandra 5000 Hbase 4000 MongoDB 3000 2000 1000 0 6000 5000 • Cassandra,Hbase의경우Compaction후의테스트결과 4000 Cassandra • Hbase의경우새로Update되는데이터로인해Compact되지않은 3000 SSTable증가로시간이지날수록완만한TPS감소를보인다. Hbase 2000 MongoDB 1000 • 테스트시간:1시간 0 Read/Update 9/10
- 10. Read/Insert 8000 6000 Cassandra 4000 Hbase 2000 0 MongoDB 1000 500 MongoDB 0 6000 • Hbase의경우완전히Compaction후테스트하면 약간의averageTPS증가예상됨 4000 Cassandra Hbase • 주의:두그래프의세로축스케일이다름 2000 • 테스트시간:30분 MongoDB 0 Read/Insert 10/10
- 11. SDECofflinecommunity 2.NoSQL장애테스트 NHN 저장시스템개발팀 김효
- 12. Cassandra–level1 50000 45000 ReadWorkload,ConsistencyLevelONE 40000 35000 신규노드투입 30000 노드제거 25000 20000 15000 10000 5000 0 -Read,Read/InsertWorkload모두비슷한양상 - 초기화된신규노드추가시비정상적으로급격히TPS가높아진다. - 1~2분경과후TPS가0으로떨어진다. - 이후주기적으로TPS가급격히올라갔다가0으로떨어지기를반복하면서 점차TPS가0인상태가길어진다. 12/10
- 13. Cassandra–level1 30000 TPStarget,3000제한 25000 20000 15000 10000 5000 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91 94 97 100 103 - ReadWorkload의장애상황을테스트,테스트툴에서최대TPS를3000으로제한 - 두번의peak는갑작스런TPS증가를테스트툴에서제한하지못한문제로 CassandraCluster상태에는영향을주지않음 - TPS를3000으로제한할경우정상작동 13/10
- 14. Cassandra–levelquorum 노드제거 2500 신규노드투입 nodetool–repair수행 2000 1500 1000 500 0 - TPS가급격히높아지는현상발생하지않음 - 빈데이터를반환하지않음 - 한노드제거시,짧은시간이경과한후TPS감소를보인다. - nodetool의repair수행시시간이지날수록TPS가감소한다. 2시간테스트할때,테스트종료직전약500TPS 14/10
- 15. MongoDB • 한노드를제거하여M-S구성이된상태에서Master를제거할경우 • 남아있는Secondary는여전히Secondary로남아있으며, Master가되지못해operation수행하지못함 • Master노드에5,000,000row(약5기가)에해당하는데이터를Insert • Master노드의row수count후,Master제거 • 1262617row • MongoDB는Asynchronous하게M-S-S를유지하며, • Insert/Update중장애가발생하면,데이터유실우려가있음. 15/10
- 16. HBASE 4000 RegionServer제거 RegionServer제거 3500 RegionServer재시작 3000 2500 2000 1500 1000 500 0 - RegionServer를제거할경우약2분가량TPS가0으로떨어진다. - 일정시간(약2분)경과후TPS가올라가면서정상작동한다. 하지만,평균TPS는감소한상태로지속 -정상상태:약3500TPS -한노드를제거한상태:약2000TPS - RegionServer를재시작할경우,평균TPS가올라가초기와비슷한성능을보임 -전체RegionServer가일시적으로정지하는현상에대해원인파악하지못함 16/10
- 17. HBASE 4000 3500 DataNode제거 신규DataNode투입 3000 2500 2000 1500 1000 500 0 - DataNode를제거할경우순간적으로TPS가떨어진다. - 이후,TPS가조금씩높아져일정하게유지된다. - 정상상태:약3500TPS - DataNode를제거한상태:약2000TPS - 초기화된DataNode를추가하면TPS가더낮아진다. - 약700~1500TPS사이를진동 - 새로투입된DataNode를복구하기때문으로추정 17/10
- 18. SDECofflinecommunity 3.NoSQL동향 NHN 저장시스템개발팀 김효
- 19. BIGdata globaloutputofdata 1.8zettabyte 1billionterabyte 90% withinthepast2years 19/10
- 20. Oracle OracleNoSQLDatabase BerkelyDBJavaEditionbased Log-structuredstorage Paxosprotocolforelection 3-coyplication Singlemaster 20/10
- 21. IBMDB2 IBMDB2toincludeNoSQLFeatures basedonIBM’sRationalJazztripplestoresolution DB2,Informix고객의추가비용없이이용 21/10
- 22. Amazon AmazonDynamoDB Dynamopaperin2007 SSDhardwaredisk LessonsandlearnedfromsimpleDB: ScaleandPredictable Kindledocuments 22/10
- 23. Hadoop IBM Oracle HP EMC Steadyseller 1.0.0released Microsoft windowserverandAzure ODBCdriverforHive Hadoop-to-SQLServerconnector 23/10
- 24. Etc • MongoDB • duallicense:AGPL,commercial • Cassandra • CassandraSQL:CQL(-10%performancethanRPC) • Couchbase • CouchbaseServer2.0CouchbaseMobileSyncpoint • Redis,Riak • notshowupmuch 24/10
- 25. Summary • NoSQL은더이상신기한것이아니다. • Major업체들이뛰어드는시점 • Hadoop은기술자체에서사용솔루션으로진화 • 최초의NoSQL들의시련예상 25/10