Hadoop & Zing

PRESENTER: HUNGVV W: http://me.zing.vn/hung.vo E: [email_address] 2011-08 HADOOP & ZING

AGENDA Using Hadoop in Zing Rank Introduction to Hadoop, Hive A case study: Log Collecting, Analyzing & Reporting System ter Estimate Conclusion 1 3 2

Hadoop & Zing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Flow into Hadoop Web Servers Scribe MidTier Network Storage and Servers Hadoop Hive Warehouse MySQL

Hive – Data Warehouse ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hive Architecture HDFS Map Reduce Web UI + Hive CLI + JDBC/ODBC Browse, Query, DDL Hive QL Parser Planner Optimizer Execution SerDe CSV Thrift Regex UDF/UDAF substr sum average FileFormats TextFile SequenceFile RCFile User-defined Map-reduce Scripts

Hive DDL ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hive DML ,[object Object],[object Object],[object Object],[object Object]

Hive Query Language ,[object Object],[object Object],[object Object],[object Object],[object Object]

Multi-table Group-By/Insert ,[object Object],[object Object],[object Object],[object Object],[object Object]

File Formats ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

SerDe ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

UDF/UDAF ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

What we use Hadoop for? ,[object Object],[object Object],[object Object],[object Object],[object Object]

Data Usage ,[object Object],[object Object],[object Object],[object Object]

Where is the data stored? ,[object Object],[object Object],[object Object],[object Object],[object Object]

Log Collecting, Analyzing & Reporting ,[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Log Collecting, Analyzing & Reporting

Hadoop & Zing

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

En vedette

En vedette (20)

Similaire à Hadoop & Zing

Similaire à Hadoop & Zing (20)

Dernier

Dernier (20)

Hadoop & Zing

Notes de l'éditeur