SlideShare une entreprise Scribd logo

Hadoop + GPU

•
•
Vladimir Starostenkov
Vladimir Starostenkov

Brief intro into the problem and perspectives of OpenCL and distributed heterogeneous calculations with Hadoop. For Big Data Dive 2013 (Belarus Java User Group).

FormationTechnologie
1  sur  48
Télécharger pour lire hors ligne
© ALTOROS Systems | CONFIDENTIAL “The norm for data analytics is now to run them on commodity clusters with MapReduce-like abstractions. One only needs to read the popular blogs to see the evidence of this. We believe that we could now say that “nobody ever got fired for using Hadoop on a cluster”!
© ALTOROS Systems | CONFIDENTIAL Breaking News IBM Keynote at JavaOne 2013: Java Flies in Blue Skies and Open Clouds Java and GPUs open up a world of new opportunities for GPU accelerators and Java programmers alike.
© ALTOROS Systems | CONFIDENTIAL Breaking News Duimovich showed an example of GPU acceleration of sorting using standard NVIDIA CUDA libraries that are already available! The speedups are phenomenal — ranging from 2x to 48x faster!
© ALTOROS Systems | CONFIDENTIAL Breaking News?
© ALTOROS Systems | CONFIDENTIAL Breaking News?
© ALTOROS Systems | CONFIDENTIAL Breaking Hadoop
© ALTOROS Systems | CONFIDENTIAL Breaking Hadoop 10 000x faster
© ALTOROS Systems | CONFIDENTIAL Breaking Hadoop 10 000x faster
© ALTOROS Systems | CONFIDENTIAL Hadoop vs GPU Hadoop & GPU Hadoop + GPU HPC Big Data GPGPU in Java Heterogeneous systems Horizontal and vertical scalability
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
© ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
© ALTOROS Systems | CONFIDENTIAL Use GPU to scale vertically Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 20.5 1 1 0.5 1 1
© ALTOROS Systems | CONFIDENTIAL Profit estimation “Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU” by Intel NVidia GTX280 vs Intel Core i7-960
© ALTOROS Systems | CONFIDENTIAL Profit estimation “Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU” by Intel “OpenCL: the advantages of heterogeneous approach” by Intel NVidia GTX280 vs Intel Core i7-960
© ALTOROS Systems | CONFIDENTIAL How to use OpenCL?
© ALTOROS Systems | CONFIDENTIAL How to use OpenCL?
© ALTOROS Systems | CONFIDENTIAL How to use OpenCL? Hadoop streaming
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL?
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Execute using Java Thread Pool
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Bytecode can be converted to OpenCL? Execute using Java Thread Pool
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Bytecode can be converted to OpenCL? Convert it Execute OpenCL Kernel on DeviceExecute using Java Thread Pool
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. lambda
© ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. lambda HSA
© ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload
© ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload Code which iterates over large arrays of primitives - 32/64 bit data types preferred - where the order of iterations is not critical avoid data dependencies between iterations - each iteration contains sequential code (few branches)
© ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload Code which iterates over large arrays of primitives - 32/64 bit data types preferred - where the order of iterations is not critical avoid data dependencies between iterations - each iteration contains sequential code (few branches) Balance between data size (low) and compute (high) - data transfer to/from the GPU can be costly - trivial compute not worth the transfer cost - may still benefit by freeing up CPU for other work(?)
© ALTOROS Systems | CONFIDENTIAL HadoopCL Rice University, AMD
© ALTOROS Systems | CONFIDENTIAL HadoopCL
© ALTOROS Systems | CONFIDENTIAL HadoopCL
© ALTOROS Systems | CONFIDENTIAL HadoopCL
© ALTOROS Systems | CONFIDENTIAL HadoopCL 2 six-core Intel X5660 (48 GB mem) 2 NVidia Tesla M2050 (2*2.5 GB mem) AMD A10-5800K APU (16 GB mem)
© ALTOROS Systems | CONFIDENTIAL HadoopCL 2 six-core Intel X5660 (48 GB mem) 2 NVidia Tesla M2050 (2*2.5 GB mem) AMD A10-5800K APU (16 GB mem) WHY?
© ALTOROS Systems | CONFIDENTIAL HadoopCL
© ALTOROS Systems | CONFIDENTIAL Back to OpenCL, Aparapi and heterogeneous computing
© ALTOROS Systems | CONFIDENTIAL OpenCL, Aparapi and heterogeneous computing GPU cache GPU GDDR5 CPU cache SATA 3.0 (HDD) SATA 2.0 (SSD) 1 GBit networkFormula in terms of time: (CPU calc1) + disk read + disk write > (CPU calc2 + GPU calc + GPU-write + GPU-read) + disk read + disk write
© ALTOROS Systems | CONFIDENTIAL OpenCL future
© ALTOROS Systems | CONFIDENTIAL OpenCL future http://streamcomputing.eu/
© ALTOROS Systems | CONFIDENTIAL Questions? Big Data Experts FB group

Recommandé

CUDA performance study on Hadoop MapReduce Cluster
CUDA performance study on Hadoop MapReduce ClusterCUDA performance study on Hadoop MapReduce Cluster
CUDA performance study on Hadoop MapReduce Clusterairbots
 
Hadoop mapreduce performance study on arm cluster
Hadoop mapreduce performance study on arm clusterHadoop mapreduce performance study on arm cluster
Hadoop mapreduce performance study on arm clusterairbots
 
Using GPUs to Handle Big Data with Java
Using GPUs to Handle Big Data with JavaUsing GPUs to Handle Big Data with Java
Using GPUs to Handle Big Data with JavaTim Ellison
 
Exploiting GPUs in Spark
Exploiting GPUs in SparkExploiting GPUs in Spark
Exploiting GPUs in SparkKazuaki Ishizaki
 
Exploiting GPUs in Spark
Exploiting GPUs in SparkExploiting GPUs in Spark
Exploiting GPUs in SparkKazuaki Ishizaki
 
Easy and High Performance GPU Programming for Java Programmers
Easy and High Performance GPU Programming for Java ProgrammersEasy and High Performance GPU Programming for Java Programmers
Easy and High Performance GPU Programming for Java ProgrammersKazuaki Ishizaki
 
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production ScaleGPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scalesparktc
 
Using GPUs to handle Big Data with Java by Adam Roberts.
Using GPUs to handle Big Data with Java by Adam Roberts.Using GPUs to handle Big Data with Java by Adam Roberts.
Using GPUs to handle Big Data with Java by Adam Roberts.J On The Beach
 

Recommandé

CUDA performance study on Hadoop MapReduce Cluster
CUDA performance study on Hadoop MapReduce ClusterCUDA performance study on Hadoop MapReduce Cluster
CUDA performance study on Hadoop MapReduce Clusterairbots
 
Hadoop mapreduce performance study on arm cluster
Hadoop mapreduce performance study on arm clusterHadoop mapreduce performance study on arm cluster
Hadoop mapreduce performance study on arm clusterairbots
 
Using GPUs to Handle Big Data with Java
Using GPUs to Handle Big Data with JavaUsing GPUs to Handle Big Data with Java
Using GPUs to Handle Big Data with JavaTim Ellison
 
Exploiting GPUs in Spark
Exploiting GPUs in SparkExploiting GPUs in Spark
Exploiting GPUs in SparkKazuaki Ishizaki
 
Exploiting GPUs in Spark
Exploiting GPUs in SparkExploiting GPUs in Spark
Exploiting GPUs in SparkKazuaki Ishizaki
 
Easy and High Performance GPU Programming for Java Programmers
Easy and High Performance GPU Programming for Java ProgrammersEasy and High Performance GPU Programming for Java Programmers
Easy and High Performance GPU Programming for Java ProgrammersKazuaki Ishizaki
 
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production ScaleGPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scalesparktc
 
Using GPUs to handle Big Data with Java by Adam Roberts.
Using GPUs to handle Big Data with Java by Adam Roberts.Using GPUs to handle Big Data with Java by Adam Roberts.
Using GPUs to handle Big Data with Java by Adam Roberts.J On The Beach
 
LCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience ReportLCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience ReportLinaro
 
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDistributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDatabricks
 
JMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AIJMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AILablup Inc.
 
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd MostakLeveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd MostakDatabricks
 
Parallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARNParallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARNDataWorks Summit
 
Making Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to UseMaking Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to UseKazuaki Ishizaki
 
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleGPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleSpark Summit
 
MIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platformMIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platformGanesan Narayanasamy
 
GPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And PythonGPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And PythonJen Aman
 
MEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftMEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftLee Stott
 
Optimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for EnergyOptimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for EnergyDavid Lecomber
 
Distributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with HorovodDistributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with HorovodLin Yuan
 
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On SparkSpark Summit
 
Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?inside-BigData.com
 
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data AnalyticsOpen Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analyticsinside-BigData.com
 
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...Databricks
 
RAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceRAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceData Works MD
 
Ac922 cdac webinar
Ac922 cdac webinarAc922 cdac webinar
Ac922 cdac webinarGanesan Narayanasamy
 
Google Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningGoogle Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningDataWorks Summit/Hadoop Summit
 
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDSAccelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDSDatabricks
 
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...AMD Developer Central
 
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015Kohei KaiGai
 

Contenu connexe

Tendances

LCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience ReportLCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience ReportLinaro
 
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDistributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDatabricks
 
JMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AIJMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AILablup Inc.
 
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd MostakLeveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd MostakDatabricks
 
Parallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARNParallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARNDataWorks Summit
 
Making Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to UseMaking Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to UseKazuaki Ishizaki
 
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleGPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleSpark Summit
 
MIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platformMIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platformGanesan Narayanasamy
 
GPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And PythonGPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And PythonJen Aman
 
MEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftMEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftLee Stott
 
Optimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for EnergyOptimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for EnergyDavid Lecomber
 
Distributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with HorovodDistributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with HorovodLin Yuan
 
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On SparkSpark Summit
 
Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?inside-BigData.com
 
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data AnalyticsOpen Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analyticsinside-BigData.com
 
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...Databricks
 
RAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceRAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceData Works MD
 
Google Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningGoogle Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningDataWorks Summit/Hadoop Summit
 
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDSAccelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDSDatabricks
 

Tendances (20)

LCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience ReportLCU13: GPGPU on ARM Experience Report
LCU13: GPGPU on ARM Experience Report
 
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim DowlingDistributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling
 
JMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AIJMI Techtalk: 한재근 - How to use GPU for developing AI
JMI Techtalk: 한재근 - How to use GPU for developing AI
 
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd MostakLeveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
Leveraging GPU-Accelerated Analytics on top of Apache Spark with Todd Mostak
 
Parallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARNParallel Linear Regression in Interative Reduce and YARN
Parallel Linear Regression in Interative Reduce and YARN
 
Making Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to UseMaking Hardware Accelerator Easier to Use
Making Hardware Accelerator Easier to Use
 
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleGPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
 
MIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platformMIT's experience on OpenPOWER/POWER 9 platform
MIT's experience on OpenPOWER/POWER 9 platform
 
GPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And PythonGPU Computing With Apache Spark And Python
GPU Computing With Apache Spark And Python
 
MEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftMEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop Microsoft
 
Optimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for EnergyOptimizing High Performance Computing Applications for Energy
Optimizing High Performance Computing Applications for Energy
 
Distributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with HorovodDistributed Model Training using MXNet with Horovod
Distributed Model Training using MXNet with Horovod
 
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
Which Is Deeper - Comparison Of Deep Learning Frameworks On Spark
 
Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?Overview of Scientific Workflows - Why Use Them?
Overview of Scientific Workflows - Why Use Them?
 
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data AnalyticsOpen Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
Open Source RAPIDS GPU Platform to Accelerate Predictive Data Analytics
 
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
Project Hydrogen: Unifying State-of-the-Art AI and Big Data in Apache Spark w...
 
RAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceRAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data Science
 
Ac922 cdac webinar
Ac922 cdac webinarAc922 cdac webinar
Ac922 cdac webinar
 
Google Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine LearningGoogle Cloud Platform Empowers TensorFlow and Machine Learning
Google Cloud Platform Empowers TensorFlow and Machine Learning
 
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDSAccelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
Accelerated Machine Learning with RAPIDS and MLflow, Nvidia/RAPIDS
 

En vedette

IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...AMD Developer Central
 
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015Kohei KaiGai
 
GPUs in Big Data - StampedeCon 2014
GPUs in Big Data - StampedeCon 2014GPUs in Big Data - StampedeCon 2014
GPUs in Big Data - StampedeCon 2014StampedeCon
 
SIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
SIGGRAPH 2012: GPU-Accelerated 2D and Web RenderingSIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
SIGGRAPH 2012: GPU-Accelerated 2D and Web RenderingMark Kilgard
 
PG-Strom - GPU Accelerated Asyncr
PG-Strom - GPU Accelerated AsyncrPG-Strom - GPU Accelerated Asyncr
PG-Strom - GPU Accelerated AsyncrKohei KaiGai
 
Deep learning on spark
Deep learning on sparkDeep learning on spark
Deep learning on sparkSatyendra Rana
 
Computational Techniques for the Statistical Analysis of Big Data in R
Computational Techniques for the Statistical Analysis of Big Data in RComputational Techniques for the Statistical Analysis of Big Data in R
Computational Techniques for the Statistical Analysis of Big Data in Rherbps10
 
GTC 2012: GPU-Accelerated Path Rendering
GTC 2012: GPU-Accelerated Path Rendering GTC 2012: GPU-Accelerated Path Rendering
GTC 2012: GPU-Accelerated Path Rendering Mark Kilgard
 
Accelerating Machine Learning Applications on Spark Using GPUs
Accelerating Machine Learning Applications on Spark Using GPUsAccelerating Machine Learning Applications on Spark Using GPUs
Accelerating Machine Learning Applications on Spark Using GPUsIBM
 
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...odsc
 
Heterogeneous System Architecture Overview
Heterogeneous System Architecture OverviewHeterogeneous System Architecture Overview
Heterogeneous System Architecture Overviewinside-BigData.com
 
PyData Amsterdam - Name Matching at Scale
PyData Amsterdam - Name Matching at ScalePyData Amsterdam - Name Matching at Scale
PyData Amsterdam - Name Matching at ScaleGoDataDriven
 
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...Spark Summit
 
DeepLearning4J and Spark: Successes and Challenges - François Garillot
DeepLearning4J and Spark: Successes and Challenges - François GarillotDeepLearning4J and Spark: Successes and Challenges - François Garillot
DeepLearning4J and Spark: Successes and Challenges - François Garillotsparktc
 
How to Solve Real-Time Data Problems
How to Solve Real-Time Data ProblemsHow to Solve Real-Time Data Problems
How to Solve Real-Time Data ProblemsIBM Power Systems
 
Containerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudContainerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudSubbu Rama
 
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...Chris Fregly
 

En vedette (20)

IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
IS-4011, Accelerating Analytics on HADOOP using OpenCL, by Zubin Dowlaty and ...
 
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015PG-Strom - GPGPU meets PostgreSQL, PGcon2015
PG-Strom - GPGPU meets PostgreSQL, PGcon2015
 
PG-Strom
PG-StromPG-Strom
PG-Strom
 
GPUs in Big Data - StampedeCon 2014
GPUs in Big Data - StampedeCon 2014GPUs in Big Data - StampedeCon 2014
GPUs in Big Data - StampedeCon 2014
 
GPU Ecosystem
GPU EcosystemGPU Ecosystem
GPU Ecosystem
 
SIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
SIGGRAPH 2012: GPU-Accelerated 2D and Web RenderingSIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
SIGGRAPH 2012: GPU-Accelerated 2D and Web Rendering
 
PG-Strom - GPU Accelerated Asyncr
PG-Strom - GPU Accelerated AsyncrPG-Strom - GPU Accelerated Asyncr
PG-Strom - GPU Accelerated Asyncr
 
Deep learning on spark
Deep learning on sparkDeep learning on spark
Deep learning on spark
 
Computational Techniques for the Statistical Analysis of Big Data in R
Computational Techniques for the Statistical Analysis of Big Data in RComputational Techniques for the Statistical Analysis of Big Data in R
Computational Techniques for the Statistical Analysis of Big Data in R
 
GTC 2012: GPU-Accelerated Path Rendering
GTC 2012: GPU-Accelerated Path Rendering GTC 2012: GPU-Accelerated Path Rendering
GTC 2012: GPU-Accelerated Path Rendering
 
Accelerating Machine Learning Applications on Spark Using GPUs
Accelerating Machine Learning Applications on Spark Using GPUsAccelerating Machine Learning Applications on Spark Using GPUs
Accelerating Machine Learning Applications on Spark Using GPUs
 
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
Enabling Graph Analytics at Scale: The Opportunity for GPU-Acceleration of D...
 
Heterogeneous System Architecture Overview
Heterogeneous System Architecture OverviewHeterogeneous System Architecture Overview
Heterogeneous System Architecture Overview
 
PyData Amsterdam - Name Matching at Scale
PyData Amsterdam - Name Matching at ScalePyData Amsterdam - Name Matching at Scale
PyData Amsterdam - Name Matching at Scale
 
Deep Learning on Hadoop
Deep Learning on HadoopDeep Learning on Hadoop
Deep Learning on Hadoop
 
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
From Machine Learning to Learning Machines: Creating an End-to-End Cognitive ...
 
DeepLearning4J and Spark: Successes and Challenges - François Garillot
DeepLearning4J and Spark: Successes and Challenges - François GarillotDeepLearning4J and Spark: Successes and Challenges - François Garillot
DeepLearning4J and Spark: Successes and Challenges - François Garillot
 
How to Solve Real-Time Data Problems
How to Solve Real-Time Data ProblemsHow to Solve Real-Time Data Problems
How to Solve Real-Time Data Problems
 
Containerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudContainerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the Cloud
 
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...
Tallinn Estonia Advanced Java Meetup Spark + TensorFlow = TensorFrames Oct 24...
 

Similaire à Hadoop + GPU

HPC and cloud distributed computing, as a journey
HPC and cloud distributed computing, as a journeyHPC and cloud distributed computing, as a journey
HPC and cloud distributed computing, as a journeyPeter Clapham
 
Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster Fran Navarro
 
Introduction to Distributed Computing & Distributed Databases
Introduction to Distributed Computing & Distributed DatabasesIntroduction to Distributed Computing & Distributed Databases
Introduction to Distributed Computing & Distributed DatabasesShankar Iyer
 
OpenStack Preso: DevOps on Hybrid Infrastructure
OpenStack Preso: DevOps on Hybrid InfrastructureOpenStack Preso: DevOps on Hybrid Infrastructure
OpenStack Preso: DevOps on Hybrid Infrastructurerhirschfeld
 
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructuredevopsdaysaustin
 
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud Cloudera Analytics and Machine Learning Platform - Optimized for Cloud
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud Stefan Lipp
 
Cloud comparison - AWS vs Azure vs Google
Cloud comparison - AWS vs Azure vs GoogleCloud comparison - AWS vs Azure vs Google
Cloud comparison - AWS vs Azure vs GooglePatrick Pierson
 
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...AMD Developer Central
 
Migrating enterprise workloads to AWS
Migrating enterprise workloads to AWS Migrating enterprise workloads to AWS
Migrating enterprise workloads to AWS Tom Laszewski
 
Sanger, upcoming Openstack for Bio-informaticians
Sanger, upcoming Openstack for Bio-informaticiansSanger, upcoming Openstack for Bio-informaticians
Sanger, upcoming Openstack for Bio-informaticiansPeter Clapham
 
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
Deep Learning Frameworks Using Spark on YARN by Vartika SinghDeep Learning Frameworks Using Spark on YARN by Vartika Singh
Deep Learning Frameworks Using Spark on YARN by Vartika SinghData Con LA
 
Helix core on aws webinar
Helix core on aws webinar Helix core on aws webinar
Helix core on aws webinar Perforce
 
MySQL Fabric - High Availability & Automated Sharding for MySQL
MySQL Fabric - High Availability & Automated Sharding for MySQLMySQL Fabric - High Availability & Automated Sharding for MySQL
MySQL Fabric - High Availability & Automated Sharding for MySQLTed Wennmark
 
Oracle Cloud : Big Data Use Cases and Architecture
Oracle Cloud : Big Data Use Cases and ArchitectureOracle Cloud : Big Data Use Cases and Architecture
Oracle Cloud : Big Data Use Cases and ArchitectureRiccardo Romani
 
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018Amazon Web Services
 
High Performance Computing Pitch Deck
High Performance Computing Pitch DeckHigh Performance Computing Pitch Deck
High Performance Computing Pitch DeckNicholas Vossburg
 

Similaire à Hadoop + GPU (20)

HPC and cloud distributed computing, as a journey
HPC and cloud distributed computing, as a journeyHPC and cloud distributed computing, as a journey
HPC and cloud distributed computing, as a journey
 
Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster Simplify IT: Oracle SuperCluster
Simplify IT: Oracle SuperCluster
 
Introduction to Distributed Computing & Distributed Databases
Introduction to Distributed Computing & Distributed DatabasesIntroduction to Distributed Computing & Distributed Databases
Introduction to Distributed Computing & Distributed Databases
 
OpenStack Preso: DevOps on Hybrid Infrastructure
OpenStack Preso: DevOps on Hybrid InfrastructureOpenStack Preso: DevOps on Hybrid Infrastructure
OpenStack Preso: DevOps on Hybrid Infrastructure
 
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure
2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructure
 
AMD It's Time to ROC
AMD It's Time to ROCAMD It's Time to ROC
AMD It's Time to ROC
 
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud Cloudera Analytics and Machine Learning Platform - Optimized for Cloud
Cloudera Analytics and Machine Learning Platform - Optimized for Cloud
 
How to Win When Migrating to Azure
How to Win When Migrating to AzureHow to Win When Migrating to Azure
How to Win When Migrating to Azure
 
Cloud comparison - AWS vs Azure vs Google
Cloud comparison - AWS vs Azure vs GoogleCloud comparison - AWS vs Azure vs Google
Cloud comparison - AWS vs Azure vs Google
 
Oracle Cloud Infrastructure
Oracle Cloud InfrastructureOracle Cloud Infrastructure
Oracle Cloud Infrastructure
 
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...
Computer Vision Powered by Heterogeneous System Architecture (HSA) by Dr. Ha...
 
Migrating enterprise workloads to AWS
Migrating enterprise workloads to AWS Migrating enterprise workloads to AWS
Migrating enterprise workloads to AWS
 
Sanger, upcoming Openstack for Bio-informaticians
Sanger, upcoming Openstack for Bio-informaticiansSanger, upcoming Openstack for Bio-informaticians
Sanger, upcoming Openstack for Bio-informaticians
 
Flexible compute
Flexible computeFlexible compute
Flexible compute
 
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
Deep Learning Frameworks Using Spark on YARN by Vartika SinghDeep Learning Frameworks Using Spark on YARN by Vartika Singh
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
 
Helix core on aws webinar
Helix core on aws webinar Helix core on aws webinar
Helix core on aws webinar
 
MySQL Fabric - High Availability & Automated Sharding for MySQL
MySQL Fabric - High Availability & Automated Sharding for MySQLMySQL Fabric - High Availability & Automated Sharding for MySQL
MySQL Fabric - High Availability & Automated Sharding for MySQL
 
Oracle Cloud : Big Data Use Cases and Architecture
Oracle Cloud : Big Data Use Cases and ArchitectureOracle Cloud : Big Data Use Cases and Architecture
Oracle Cloud : Big Data Use Cases and Architecture
 
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018
Get SaaSy with Red Hat OpenShift on AWS (CON305-S) - AWS re:Invent 2018
 
High Performance Computing Pitch Deck
High Performance Computing Pitch DeckHigh Performance Computing Pitch Deck
High Performance Computing Pitch Deck
 

Dernier

Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1GloryAnnCastre1
 
Using Grammatical Signals Suitable to Patterns of Idea Development
Using Grammatical Signals Suitable to Patterns of Idea DevelopmentUsing Grammatical Signals Suitable to Patterns of Idea Development
Using Grammatical Signals Suitable to Patterns of Idea Developmentchesterberbo7
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Association for Project Management
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptxmary850239
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptxmary850239
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco
 
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDecoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDhatriParmar
 
Scientific Writing :Research Discourse
Scientific Writing :Research DiscourseScientific Writing :Research Discourse
Scientific Writing :Research DiscourseAnita GoswamiGiri
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...DhatriParmar
 
ARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptxARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptxAneriPatwari
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesVijayaLaxmi84
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQuiz Club NITW
 
Expanded definition: technical and operational
Expanded definition: technical and operationalExpanded definition: technical and operational
Expanded definition: technical and operationalssuser3e220a
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWQuiz Club NITW
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptxmary850239
 

Dernier (20)

Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1Reading and Writing Skills 11 quarter 4 melc 1
Reading and Writing Skills 11 quarter 4 melc 1
 
Using Grammatical Signals Suitable to Patterns of Idea Development
Using Grammatical Signals Suitable to Patterns of Idea DevelopmentUsing Grammatical Signals Suitable to Patterns of Idea Development
Using Grammatical Signals Suitable to Patterns of Idea Development
 
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
Team Lead Succeed – Helping you and your team achieve high-performance teamwo...
 
4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx4.16.24 21st Century Movements for Black Lives.pptx
4.16.24 21st Century Movements for Black Lives.pptx
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx4.16.24 Poverty and Precarity--Desmond.pptx
4.16.24 Poverty and Precarity--Desmond.pptx
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
 
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptxDecoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
Decoding the Tweet _ Practical Criticism in the Age of Hashtag.pptx
 
Scientific Writing :Research Discourse
Scientific Writing :Research DiscourseScientific Writing :Research Discourse
Scientific Writing :Research Discourse
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
 
ARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptxARTERIAL BLOOD GAS ANALYSIS........pptx
ARTERIAL BLOOD GAS ANALYSIS........pptx
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their uses
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITWQ-Factor General Quiz-7th April 2024, Quiz Club NITW
Q-Factor General Quiz-7th April 2024, Quiz Club NITW
 
Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"Mattingly "AI & Prompt Design: Large Language Models"
Mattingly "AI & Prompt Design: Large Language Models"
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
Expanded definition: technical and operational
Expanded definition: technical and operationalExpanded definition: technical and operational
Expanded definition: technical and operational
 
Mythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITWMythology Quiz-4th April 2024, Quiz Club NITW
Mythology Quiz-4th April 2024, Quiz Club NITW
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx
 
prashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Professionprashanth updated resume 2024 for Teaching Profession
prashanth updated resume 2024 for Teaching Profession
 

Hadoop + GPU

  • 1. © ALTOROS Systems | CONFIDENTIAL “The norm for data analytics is now to run them on commodity clusters with MapReduce-like abstractions. One only needs to read the popular blogs to see the evidence of this. We believe that we could now say that “nobody ever got fired for using Hadoop on a cluster”!
  • 2. © ALTOROS Systems | CONFIDENTIAL Breaking News IBM Keynote at JavaOne 2013: Java Flies in Blue Skies and Open Clouds Java and GPUs open up a world of new opportunities for GPU accelerators and Java programmers alike.
  • 3. © ALTOROS Systems | CONFIDENTIAL Breaking News Duimovich showed an example of GPU acceleration of sorting using standard NVIDIA CUDA libraries that are already available! The speedups are phenomenal — ranging from 2x to 48x faster!
  • 4. © ALTOROS Systems | CONFIDENTIAL Breaking News?
  • 5. © ALTOROS Systems | CONFIDENTIAL Breaking News?
  • 6. © ALTOROS Systems | CONFIDENTIAL Breaking Hadoop
  • 7. © ALTOROS Systems | CONFIDENTIAL Breaking Hadoop 10 000x faster
  • 8. © ALTOROS Systems | CONFIDENTIAL Breaking Hadoop 10 000x faster
  • 9. © ALTOROS Systems | CONFIDENTIAL Hadoop vs GPU Hadoop & GPU Hadoop + GPU HPC Big Data GPGPU in Java Heterogeneous systems Horizontal and vertical scalability
  • 10. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03
  • 11. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03
  • 12. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10
  • 13. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3
  • 14. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10
  • 15. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability file01 file02 file03 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 01 02 03 04 05 0607 0809 10 3 4 3 Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
  • 16. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
  • 17. © ALTOROS Systems | CONFIDENTIAL Hadoop horizontal scalability Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 2
  • 18. © ALTOROS Systems | CONFIDENTIAL Use GPU to scale vertically Node 1 Node 2 Node 3 01 02 03 04 05 06 07 08 09 10 Node 4 Node 5 Node 6 01 02 03 04 05 06 07 08 09 10 221 1 2 20.5 1 1 0.5 1 1
  • 19. © ALTOROS Systems | CONFIDENTIAL Profit estimation “Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU” by Intel NVidia GTX280 vs Intel Core i7-960
  • 20. © ALTOROS Systems | CONFIDENTIAL Profit estimation “Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU” by Intel “OpenCL: the advantages of heterogeneous approach” by Intel NVidia GTX280 vs Intel Core i7-960
  • 21. © ALTOROS Systems | CONFIDENTIAL How to use OpenCL?
  • 22. © ALTOROS Systems | CONFIDENTIAL How to use OpenCL?
  • 23. © ALTOROS Systems | CONFIDENTIAL How to use OpenCL? Hadoop streaming
  • 24. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class
  • 25. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL?
  • 26. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Execute using Java Thread Pool
  • 27. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Bytecode can be converted to OpenCL? Execute using Java Thread Pool
  • 28. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. MyKernel.class Platform Supports OpenCL? Bytecode can be converted to OpenCL? Convert it Execute OpenCL Kernel on DeviceExecute using Java Thread Pool
  • 29. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
  • 30. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
  • 31. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class.
  • 32. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. lambda
  • 33. © ALTOROS Systems | CONFIDENTIAL Aparapi Expands Java's “Write Once Run Anywhere” to include APU and GPU devices by expressing data parallel algorithm through extending Kernel base class. lambda HSA
  • 34. © ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload
  • 35. © ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload Code which iterates over large arrays of primitives - 32/64 bit data types preferred - where the order of iterations is not critical avoid data dependencies between iterations - each iteration contains sequential code (few branches)
  • 36. © ALTOROS Systems | CONFIDENTIAL Aparapi Characteristics of ideal data parallel workload Code which iterates over large arrays of primitives - 32/64 bit data types preferred - where the order of iterations is not critical avoid data dependencies between iterations - each iteration contains sequential code (few branches) Balance between data size (low) and compute (high) - data transfer to/from the GPU can be costly - trivial compute not worth the transfer cost - may still benefit by freeing up CPU for other work(?)
  • 37. © ALTOROS Systems | CONFIDENTIAL HadoopCL Rice University, AMD
  • 38. © ALTOROS Systems | CONFIDENTIAL HadoopCL
  • 39. © ALTOROS Systems | CONFIDENTIAL HadoopCL
  • 40. © ALTOROS Systems | CONFIDENTIAL HadoopCL
  • 41. © ALTOROS Systems | CONFIDENTIAL HadoopCL 2 six-core Intel X5660 (48 GB mem) 2 NVidia Tesla M2050 (2*2.5 GB mem) AMD A10-5800K APU (16 GB mem)
  • 42. © ALTOROS Systems | CONFIDENTIAL HadoopCL 2 six-core Intel X5660 (48 GB mem) 2 NVidia Tesla M2050 (2*2.5 GB mem) AMD A10-5800K APU (16 GB mem) WHY?
  • 43. © ALTOROS Systems | CONFIDENTIAL HadoopCL
  • 44. © ALTOROS Systems | CONFIDENTIAL Back to OpenCL, Aparapi and heterogeneous computing
  • 45. © ALTOROS Systems | CONFIDENTIAL OpenCL, Aparapi and heterogeneous computing GPU cache GPU GDDR5 CPU cache SATA 3.0 (HDD) SATA 2.0 (SSD) 1 GBit networkFormula in terms of time: (CPU calc1) + disk read + disk write > (CPU calc2 + GPU calc + GPU-write + GPU-read) + disk read + disk write
  • 46. © ALTOROS Systems | CONFIDENTIAL OpenCL future
  • 47. © ALTOROS Systems | CONFIDENTIAL OpenCL future http://streamcomputing.eu/
  • 48. © ALTOROS Systems | CONFIDENTIAL Questions? Big Data Experts FB group