SlideShare une entreprise Scribd logo

Is There Room For Another Elephant In Tucson

•Télécharger en tant que PPT, PDF•
•
Andy Lenards
Andy Lenards

Would you like to scale data-intensive tasks horizontally? Would you like an open source project that gave you that foundation? Well, there is: Apache Hadoop. It's a Java software framework for supporting data-intensive distributed applications. The framework was inspired by Google papers on their MapReduce framework and Google File System. Who uses Hadoop? Here's a short list: Yahoo!, A9.com, LinkedIn, Facebook, ImageShack, eHarmony, Hulu, Last.fm, and The New York Times. The highest profile user, Yahoo!, is also a major contributor to the project. They use it extensively in their web search and advertising divisions. In this talk, titled "Is there room for another elephant in Tucson?", Andrew Lenards will tell us about Hadoop and describe how it could be applied to several practical problems, even if you aren't as big as Google.

Technologie
1  sur  45
Is There Room for Another                    Elephant in               Tucson? Tucson Java User Group Andrew Lenards
andrew.lenards@gmail ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Why do I care? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Establishing our setting...
Differing World Views
Brendan's quote... ,[object Object],[object Object]
End of Moore's Law?
End of Moore's Law?
Scaling Up Endangered  Practice? Polar Bear Endangered Species
DATA!
DATA!
Zettabytes??? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 
MapReduce: the Abstraction
MapReduce ,[object Object],[object Object],[object Object],[object Object],[object Object]
MapReduce + RDBMS, not versus Source: "Hadoop: The Definite Guide, Tom White" Traditional RDBMS MapReduce Data Size Gigabytes Petabytes Access Interactive & batch Batch Updates Read and write many times Write once, read many times Structure Static schema Dynamic schema Integrity High Low Scaling Nonlinear Linear
Shared-state makes everything hard... ,[object Object],[object Object],[object Object],[object Object],[object Object]
Shared Nothing Architecture ,[object Object],[object Object],[object Object],[object Object],[object Object]
Functional inspiration, but not dogmatic ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Partitioning work... ,[object Object],[object Object],[object Object]
Raising the level of abstraction ,[object Object],[object Object],[object Object]
MapReduce Usage
Example usage... ,[object Object],[object Object],[object Object],[object Object],[object Object]
Apache Web Server Logfiles ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
crawl-66-249-71-34.googlebot.com - - [16/Aug/2009:04:40:36 -0700] "GET /tree/home.pages/searchTOL?taxon=Arna&Submit2=Find&startline=26  HTTP/1.1" 200 14693 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/img/quicknav/RightArrow.png HTTP /1.1" 200 321 "http://www.tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" localhost.localdomain - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SSiphono phorida&sp=S8149&sp=S HTTP/1.1" 200 5894 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" llf531045.crawl.yahoo.net - - [16/Aug/2009:04:40:36 -0700] "GET /Siphonophorida/8149 HTTP/1.0" 200 5894 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" localhost.localdomain - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SCampanu lotes&sp=S73605&sp=S HTTP/1.1" 200 8571 "-" "Mozilla/4.0" 65.55.108.238 - - [16/Aug/2009:04:40:36 -0700] "GET /Campanulotes/73605 HTTP/1.1" 200 8571 "-" "Mozilla/4.0" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:36 -0700] "GET /tree/img/magnify.gif HTTP/1.1" 200 124 "http://www. tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" localhost.localdomain - - [16/Aug/2009:04:40:37 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SHomo&sp =S16418&sp=S HTTP/1.1" 200 10283 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) " crawler5108.ask.com - - [16/Aug/2009:04:40:37 -0700] "GET /Homo/16418 HTTP/1.0" 200 10283 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teo ma; +http://about.ask.com/en/docs/about/webmasters.shtml)" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:37 -0700] "GET /tree/img/tinylink.png HTTP/1.1" 200 207 "http://www .tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" msnbot-65-55-105-180.search.msn.com - - [16/Aug/2009:04:40:37 -0700] "GET /onlinecontributors/app?page=ImageGallery&service=external&s p=l27570&state:ImageGallery=ZH4sIAAAAAAAAAFvzloG1nJeBgYGJgYEtLz8l1TOluIiBLyuxLFEvJzEvXc8nPy%2FduvvJhDP9yveZGBi9GFjLEnNKUyuKGAQQivxKc5N Si9rWTJXlnvKgG2hURQEDGGRfKhdgYODNTU3JTHTOSSwu9swrAZoviNAKFEhNTy0SerRgyffGdgugFZ4wKwoZ6hgYQaYAAKhZ4XSlAAAA HTTP/1.1" 200 7899 "-" "msnb ot/1.1 (+http://search.msn.com/msnbot.htm)"
Input to the Mappers... ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Output from Mappers, Input to the Reducers... ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Output from Reducers... ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Using the analysis... ,[object Object],[object Object],[object Object],[object Object],[object Object]
A Yellow Elephant Enters...
Apache Hadoop Project ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Hadoop's assumptions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Coke/Pepsi, Google/Hadoop ,[object Object]
Google/Hadoop Decoder Ring ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
NameNode, DataNodes ,[object Object],[object Object],[object Object]
Big Files, Narrow Access Pattern ,[object Object],[object Object],[object Object],[object Object]
HDFS ,[object Object]
JobTracker, TaskTracker ,[object Object],[object Object],[object Object],[object Object]
Re-execute slow running tasks ,[object Object],[object Object]
Hadoop Ecosystem ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Where to next? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Acknowledgments ,[object Object],[object Object],[object Object],[object Object]
Photo Acknowledgments ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 
http://creativecommons.org/licenses/by-nc-sa/3.0/ This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.

Recommandé

An Introduction To Map-Reduce
An Introduction To Map-ReduceAn Introduction To Map-Reduce
An Introduction To Map-ReduceFrancisco Pérez-Sorrosal
 
Introduction to Map-Reduce
Introduction to Map-ReduceIntroduction to Map-Reduce
Introduction to Map-ReduceBrendan Tierney
 
Large Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part ILarge Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part IMarin Dimitrov
 
MapReduce basic
MapReduce basicMapReduce basic
MapReduce basicChirag Ahuja
 
Introduction to map reduce
Introduction to map reduceIntroduction to map reduce
Introduction to map reduceBhupesh Chawda
 
Map Reduce Online
Map Reduce OnlineMap Reduce Online
Map Reduce OnlineHadoop User Group
 
Map Reduce
Map ReduceMap Reduce
Map ReduceVigen Sahakyan
 
Map Reduce
Map ReduceMap Reduce
Map ReduceMichel Bruley
 

Recommandé

An Introduction To Map-Reduce
An Introduction To Map-ReduceAn Introduction To Map-Reduce
An Introduction To Map-ReduceFrancisco Pérez-Sorrosal
 
Introduction to Map-Reduce
Introduction to Map-ReduceIntroduction to Map-Reduce
Introduction to Map-ReduceBrendan Tierney
 
Large Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part ILarge Scale Data Analysis with Map/Reduce, part I
Large Scale Data Analysis with Map/Reduce, part IMarin Dimitrov
 
MapReduce basic
MapReduce basicMapReduce basic
MapReduce basicChirag Ahuja
 
Introduction to map reduce
Introduction to map reduceIntroduction to map reduce
Introduction to map reduceBhupesh Chawda
 
Map Reduce Online
Map Reduce OnlineMap Reduce Online
Map Reduce OnlineHadoop User Group
 
Map Reduce
Map ReduceMap Reduce
Map ReduceVigen Sahakyan
 
Map Reduce
Map ReduceMap Reduce
Map ReduceMichel Bruley
 
Hadoop Map Reduce
Hadoop Map ReduceHadoop Map Reduce
Hadoop Map ReduceVNIT-ACM Student Chapter
 
Introduction to Map Reduce
Introduction to Map ReduceIntroduction to Map Reduce
Introduction to Map ReduceApache Apex
 
Map Reduce
Map ReduceMap Reduce
Map ReduceRahul Agarwal
 
Map Reduce
Map ReduceMap Reduce
Map ReduceSri Prasanna
 
Getting Started on Hadoop
Getting Started on HadoopGetting Started on Hadoop
Getting Started on HadoopPaco Nathan
 
Introduction to MapReduce
Introduction to MapReduceIntroduction to MapReduce
Introduction to MapReduceChicago Hadoop Users Group
 
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)npinto
 
RAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce WorkflowsRAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce WorkflowsHyunjung Park
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentationateeq ateeq
 
Map Reduce introduction
Map Reduce introductionMap Reduce introduction
Map Reduce introductionMuralidharan Deenathayalan
 
Hadoop training-in-hyderabad
Hadoop training-in-hyderabadHadoop training-in-hyderabad
Hadoop training-in-hyderabadsreehari orienit
 
Streams processing with Storm
Streams processing with StormStreams processing with Storm
Streams processing with StormMariusz Gil
 
Giraph
GiraphGiraph
Giraph주영 송
 
Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0Simon Elliston Ball
 
Hadoop
HadoopHadoop
HadoopRamakrishna Reddy Bijjam
 
An Introduction to MapReduce
An Introduction to MapReduceAn Introduction to MapReduce
An Introduction to MapReduceFrane Bandov
 
Learn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigDataLearn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigDataThanusha154
 
Hadoop Summit 2009 Hive
Hadoop Summit 2009 HiveHadoop Summit 2009 Hive
Hadoop Summit 2009 HiveZheng Shao
 
Harnessing Big Data with Spark
Harnessing Big Data with SparkHarnessing Big Data with Spark
Harnessing Big Data with SparkAlpine Data
 
MapReduce with Hadoop
MapReduce with HadoopMapReduce with Hadoop
MapReduce with HadoopVitalie Scurtu
 
Getting Started with Hadoop
Getting Started with HadoopGetting Started with Hadoop
Getting Started with HadoopJosh Devins
 
Introduction to MapReduce Data Transformations
Introduction to MapReduce Data TransformationsIntroduction to MapReduce Data Transformations
Introduction to MapReduce Data Transformationsswooledge
 

Contenu connexe

Tendances

Introduction to Map Reduce
Introduction to Map ReduceIntroduction to Map Reduce
Introduction to Map ReduceApache Apex
 
Getting Started on Hadoop
Getting Started on HadoopGetting Started on Hadoop
Getting Started on HadoopPaco Nathan
 
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)npinto
 
RAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce WorkflowsRAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce WorkflowsHyunjung Park
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentationateeq ateeq
 
Hadoop training-in-hyderabad
Hadoop training-in-hyderabadHadoop training-in-hyderabad
Hadoop training-in-hyderabadsreehari orienit
 
Streams processing with Storm
Streams processing with StormStreams processing with Storm
Streams processing with StormMariusz Gil
 
Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0Simon Elliston Ball
 
An Introduction to MapReduce
An Introduction to MapReduceAn Introduction to MapReduce
An Introduction to MapReduceFrane Bandov
 
Learn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigDataLearn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigDataThanusha154
 
Hadoop Summit 2009 Hive
Hadoop Summit 2009 HiveHadoop Summit 2009 Hive
Hadoop Summit 2009 HiveZheng Shao
 
Harnessing Big Data with Spark
Harnessing Big Data with SparkHarnessing Big Data with Spark
Harnessing Big Data with SparkAlpine Data
 

Tendances (20)

Hadoop Map Reduce
Hadoop Map ReduceHadoop Map Reduce
Hadoop Map Reduce
 
Introduction to Map Reduce
Introduction to Map ReduceIntroduction to Map Reduce
Introduction to Map Reduce
 
Map Reduce
Map ReduceMap Reduce
Map Reduce
 
Map Reduce
Map ReduceMap Reduce
Map Reduce
 
Getting Started on Hadoop
Getting Started on HadoopGetting Started on Hadoop
Getting Started on Hadoop
 
Introduction to MapReduce
Introduction to MapReduceIntroduction to MapReduce
Introduction to MapReduce
 
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
[Harvard CS264] 08b - MapReduce and Hadoop (Zak Stone, Harvard)
 
RAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce WorkflowsRAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
RAMP: A System for Capturing and Tracing Provenance in MapReduce Workflows
 
Map reduce presentation
Map reduce presentationMap reduce presentation
Map reduce presentation
 
Map Reduce introduction
Map Reduce introductionMap Reduce introduction
Map Reduce introduction
 
Hadoop training-in-hyderabad
Hadoop training-in-hyderabadHadoop training-in-hyderabad
Hadoop training-in-hyderabad
 
Streams processing with Storm
Streams processing with StormStreams processing with Storm
Streams processing with Storm
 
Giraph
GiraphGiraph
Giraph
 
Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0Riding the Elephant - Hadoop 2.0
Riding the Elephant - Hadoop 2.0
 
Hadoop
HadoopHadoop
Hadoop
 
An Introduction to MapReduce
An Introduction to MapReduceAn Introduction to MapReduce
An Introduction to MapReduce
 
Learn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigDataLearn what is Hadoop-and-BigData
Learn what is Hadoop-and-BigData
 
Hadoop Summit 2009 Hive
Hadoop Summit 2009 HiveHadoop Summit 2009 Hive
Hadoop Summit 2009 Hive
 
Harnessing Big Data with Spark
Harnessing Big Data with SparkHarnessing Big Data with Spark
Harnessing Big Data with Spark
 
MapReduce with Hadoop
MapReduce with HadoopMapReduce with Hadoop
MapReduce with Hadoop
 

Similaire à Is There Room For Another Elephant In Tucson

Getting Started with Hadoop
Getting Started with HadoopGetting Started with Hadoop
Getting Started with HadoopJosh Devins
 
Introduction to MapReduce Data Transformations
Introduction to MapReduce Data TransformationsIntroduction to MapReduce Data Transformations
Introduction to MapReduce Data Transformationsswooledge
 
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014Dataiku
 
Rendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikRendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikGraham Jones
 
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...Extreme Apache Spark: how in 3 months we created a pipeline that can process ...
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...Josef A. Habdank
 
Softshake 2013: Introduction to NoSQL with Couchbase
Softshake 2013: Introduction to NoSQL with CouchbaseSoftshake 2013: Introduction to NoSQL with Couchbase
Softshake 2013: Introduction to NoSQL with CouchbaseTugdual Grall
 
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueTheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueShay Sofer
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud
 
Google Cluster Innards
Google Cluster InnardsGoogle Cluster Innards
Google Cluster InnardsMartin Dvorak
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce AlgorithmsAmund Tveit
 
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015Codemotion
 
Introduction To Map Reduce
Introduction To Map ReduceIntroduction To Map Reduce
Introduction To Map Reducerantav
 
L19CloudMapReduce introduction for cloud computing .ppt
L19CloudMapReduce introduction for cloud computing .pptL19CloudMapReduce introduction for cloud computing .ppt
L19CloudMapReduce introduction for cloud computing .pptMaruthiPrasad96
 
Hadoop interview question
Hadoop interview questionHadoop interview question
Hadoop interview questionpappupassindia
 
An Introduction to Solr
An Introduction to SolrAn Introduction to Solr
An Introduction to Solrtomhill
 
How to avoid the latency trap and lessons about software design
How to avoid the latency trap and lessons about software designHow to avoid the latency trap and lessons about software design
How to avoid the latency trap and lessons about software designTom Croucher
 
Hadoop interview questions - Softwarequery.com
Hadoop interview questions - Softwarequery.comHadoop interview questions - Softwarequery.com
Hadoop interview questions - Softwarequery.comsoftwarequery
 
Distributed computing poli
Distributed computing poliDistributed computing poli
Distributed computing poliivascucristian
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRick Copeland
 

Similaire à Is There Room For Another Elephant In Tucson (20)

Getting Started with Hadoop
Getting Started with HadoopGetting Started with Hadoop
Getting Started with Hadoop
 
Introduction to MapReduce Data Transformations
Introduction to MapReduce Data TransformationsIntroduction to MapReduce Data Transformations
Introduction to MapReduce Data Transformations
 
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014
Dataiku - hadoop ecosystem - @Epitech Paris - janvier 2014
 
Rendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikRendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using Mapnik
 
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...Extreme Apache Spark: how in 3 months we created a pipeline that can process ...
Extreme Apache Spark: how in 3 months we created a pipeline that can process ...
 
Softshake 2013: Introduction to NoSQL with Couchbase
Softshake 2013: Introduction to NoSQL with CouchbaseSoftshake 2013: Introduction to NoSQL with Couchbase
Softshake 2013: Introduction to NoSQL with Couchbase
 
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the RescueTheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the Rescue
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
 
Google Cluster Innards
Google Cluster InnardsGoogle Cluster Innards
Google Cluster Innards
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce Algorithms
 
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
Big Data, a space adventure - Mario Cartia - Codemotion Rome 2015
 
Introduction To Map Reduce
Introduction To Map ReduceIntroduction To Map Reduce
Introduction To Map Reduce
 
L19CloudMapReduce introduction for cloud computing .ppt
L19CloudMapReduce introduction for cloud computing .pptL19CloudMapReduce introduction for cloud computing .ppt
L19CloudMapReduce introduction for cloud computing .ppt
 
Hadoop interview question
Hadoop interview questionHadoop interview question
Hadoop interview question
 
An Introduction to Solr
An Introduction to SolrAn Introduction to Solr
An Introduction to Solr
 
How to avoid the latency trap and lessons about software design
How to avoid the latency trap and lessons about software designHow to avoid the latency trap and lessons about software design
How to avoid the latency trap and lessons about software design
 
Hadoop interview questions - Softwarequery.com
Hadoop interview questions - Softwarequery.comHadoop interview questions - Softwarequery.com
Hadoop interview questions - Softwarequery.com
 
Tech
TechTech
Tech
 
Distributed computing poli
Distributed computing poliDistributed computing poli
Distributed computing poli
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
 

Plus de Andy Lenards

Rough has advantages
Rough has advantagesRough has advantages
Rough has advantagesAndy Lenards
 
The Skill That Can Impact All Skills
The Skill That Can Impact All SkillsThe Skill That Can Impact All Skills
The Skill That Can Impact All SkillsAndy Lenards
 
Software Surrounds You
Software Surrounds YouSoftware Surrounds You
Software Surrounds YouAndy Lenards
 
Code for science (rev 1)
Code for science (rev 1) Code for science (rev 1)
Code for science (rev 1) Andy Lenards
 
Code for science (rev 2)
Code for science (rev 2)Code for science (rev 2)
Code for science (rev 2)Andy Lenards
 
Loosely Coupled Thoughts
Loosely Coupled ThoughtsLoosely Coupled Thoughts
Loosely Coupled ThoughtsAndy Lenards
 

Plus de Andy Lenards (7)

Rough has advantages
Rough has advantagesRough has advantages
Rough has advantages
 
The Skill That Can Impact All Skills
The Skill That Can Impact All SkillsThe Skill That Can Impact All Skills
The Skill That Can Impact All Skills
 
Software Surrounds You
Software Surrounds YouSoftware Surrounds You
Software Surrounds You
 
Code for science (rev 1)
Code for science (rev 1) Code for science (rev 1)
Code for science (rev 1)
 
Code for science (rev 2)
Code for science (rev 2)Code for science (rev 2)
Code for science (rev 2)
 
Android
Android Android
Android
 
Loosely Coupled Thoughts
Loosely Coupled ThoughtsLoosely Coupled Thoughts
Loosely Coupled Thoughts
 

Dernier

DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 

Dernier (20)

DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 

Is There Room For Another Elephant In Tucson

  • 1. Is There Room for Another                    Elephant in               Tucson? Tucson Java User Group Andrew Lenards
  • 2.
  • 3.
  • 6.
  • 9. Scaling Up Endangered  Practice? Polar Bear Endangered Species
  • 10. DATA!
  • 11. DATA!
  • 12.
  • 13.  
  • 15.
  • 16. MapReduce + RDBMS, not versus Source: "Hadoop: The Definite Guide, Tom White" Traditional RDBMS MapReduce Data Size Gigabytes Petabytes Access Interactive & batch Batch Updates Read and write many times Write once, read many times Structure Static schema Dynamic schema Integrity High Low Scaling Nonlinear Linear
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 23.
  • 24.
  • 25. crawl-66-249-71-34.googlebot.com - - [16/Aug/2009:04:40:36 -0700] "GET /tree/home.pages/searchTOL?taxon=Arna&Submit2=Find&startline=26  HTTP/1.1" 200 14693 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/img/quicknav/RightArrow.png HTTP /1.1" 200 321 "http://www.tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" localhost.localdomain - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SSiphono phorida&sp=S8149&sp=S HTTP/1.1" 200 5894 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" llf531045.crawl.yahoo.net - - [16/Aug/2009:04:40:36 -0700] "GET /Siphonophorida/8149 HTTP/1.0" 200 5894 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" localhost.localdomain - - [16/Aug/2009:04:40:36 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SCampanu lotes&sp=S73605&sp=S HTTP/1.1" 200 8571 "-" "Mozilla/4.0" 65.55.108.238 - - [16/Aug/2009:04:40:36 -0700] "GET /Campanulotes/73605 HTTP/1.1" 200 8571 "-" "Mozilla/4.0" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:36 -0700] "GET /tree/img/magnify.gif HTTP/1.1" 200 124 "http://www. tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" localhost.localdomain - - [16/Aug/2009:04:40:37 -0700] "GET /onlinecontributors/app?service=external&page=ViewBranchOrLeaf&sp=SHomo&sp =S16418&sp=S HTTP/1.1" 200 10283 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) " crawler5108.ask.com - - [16/Aug/2009:04:40:37 -0700] "GET /Homo/16418 HTTP/1.0" 200 10283 "-" "Mozilla/5.0 (compatible; Ask Jeeves/Teo ma; +http://about.ask.com/en/docs/about/webmasters.shtml)" astdenis-105-1-18-94.w81-248.abo.wanadoo.fr - - [16/Aug/2009:04:40:37 -0700] "GET /tree/img/tinylink.png HTTP/1.1" 200 207 "http://www .tolweb.org/Echinodermata" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)" msnbot-65-55-105-180.search.msn.com - - [16/Aug/2009:04:40:37 -0700] "GET /onlinecontributors/app?page=ImageGallery&service=external&s p=l27570&state:ImageGallery=ZH4sIAAAAAAAAAFvzloG1nJeBgYGJgYEtLz8l1TOluIiBLyuxLFEvJzEvXc8nPy%2FduvvJhDP9yveZGBi9GFjLEnNKUyuKGAQQivxKc5N Si9rWTJXlnvKgG2hURQEDGGRfKhdgYODNTU3JTHTOSSwu9swrAZoviNAKFEhNTy0SerRgyffGdgugFZ4wKwoZ6hgYQaYAAKhZ4XSlAAAA HTTP/1.1" 200 7899 "-" "msnb ot/1.1 (+http://search.msn.com/msnbot.htm)"
  • 26.
  • 27.
  • 28.
  • 29.
  • 30. A Yellow Elephant Enters...
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.  
  • 45. http://creativecommons.org/licenses/by-nc-sa/3.0/ This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site.