SlideShare une entreprise Scribd logo
1  sur  82
Exploiting a Synergy between Greedy
Approach and NSGA for Scheduling in
Computing Clusters
A S M Rizvi1, Tarik Reza Toha2, Siddhartha Shankar Das3, Sriram Chellappan4, A. B. M. Alim Al Islam5
NSysS 2018
Dhaka, Bangladesh
1
2, 3, 5 4
Overview of This Presentation
• Background and motivation
• Proposed methodology for clusters
– Many-objective problem formulation
– Solution method using NSGA-III and greedy approach
• Real experiment and simulation
– Performance evaluation of proposed methodology
• Conclusion
2
Background of Our Study
3
Galaxy formation Planetary movements Climate changes
Modeling, simulation, and experimentation of complex real-world
phenomena demand rigorous computing
Traffic simulation Plate tectonics Weather forecasts
Cluster topology
Grid computingCloud computing
Architectures for Parallel Computing
4
Large Linux cluster in University
of Technology, Germany
Cluster machines are connected
by a local area network
Clouds and grids are
geographically distributed
Clusters are tightly coupled,
whereas clouds and grids are
loosely coupled
Home-made cluster
Motivation Behind Our Study
Clusters have conflicting outcomes
Increasing the number of
machines
Decreases
computation time
Increases
maintenance cost
Desirable Not desirable
5
Optimizations required for:
1. # of machines in the cluster
2. Machines to be selected
Related Research Work
6
• Particle Swarm Optimization (PSO) based approach
– C. Lijun et al., IJGDC, 2015
– Utilizes a variant of PSO
– Focuses on load balancing from the perspectives of
• Average workload of all servers in cloud clusters
• Deviation of the workload
• Migration cost between servers
Does not integrate any empirical performance characterization of clusters
Does not accumulate the impact of cooling energy consumption
Related Research Work (cont.)
7
• Multi-objective optimization technique based on
Ant Colony Optimization (ACO)
– Y. Gao et al., Journal of Computer and System
Sciences, 2013
– Simultaneously minimizes two different objectives in
cloud architectures
• Resource wastage
• Power consumption
• Load balancing
Does not integrate any empirical performance characterization of clusters
Does not accumulate the impact of cooling energy consumption
Issues That Are Yet to Be Explored
8
•Consideration of cooling energy consumption
Source: http://www.ha-inc.com/blog/entry/data-center-energy-consumption/
Data center cooling energy consumption can depend on several factors like room size
to cooling capacity ratio, environmental temperature, design issue, etc. A good design
can reduce the total energy consumption.
•Empirical performance characterization of clusters
 Should result in a new optimization model
Our Contributions
•We exploit a synergy between greedy method and
NSGA-III algorithm to solve a many-objective
optimization problem for clusters
̵ Incorporating cooling energy consumption
̵ Utilizing empirical characterization of clusters
•We perform rigorous experimentation to
demonstrate efficacy of our proposed solution
9
Formulation of Our Many-Objective
Optimization Problem for Computing Clusters
10
Following the empirical analysis,
we formulate our objective
functions as follows:
Computation
time
Energy
consumption
Cost
Inverse of
resource
utilization
Restriction on
assigned
workload
Constraint on
# of selected
machines
Limit of
cooling
temperature
Modification in The Existing NSGA-III
Algorithm: Modified Selection Process
11
Sharp change at
the beginning
providing results
below expectation
# of selected machines
should be greater than
a threshold. We select
the value as 6.
Modification in The Existing NSGA-III
Algorithm: Modified Crossover
12
Half Uniform Crossover
(HUX)
Greedy clustering
approach
Greedy approach to
include the best machine
and exclude the worst
machine
Modified Crossover: Half Uniform Crossover
13
Half-uniform crossover
Machine
selection
decision variables
(binary type)
Cluster
temperature
decision variable
(float type)
Parent 1:
Parent 2:
Yellow variables will have crossover within yellow variables
Modified Crossover: Greedy Clustering
Approach
Processor
speed
Memory
Network
B/W
Best
group
Worst
group
Best
group
Worst
group
Best
group
Worst
group
Find the
common
machines
Find the
common
machines
Biased to
include
machines
from this
group
Biased to
exclude
machines
from this
group
14
Modified Crossover: Greedy Approach to Include The
Best Machine and Exclude The Worst Machine
Processor
speed
Memory
Network
B/W
Best
machine
Worst
machine
Best
machine
Worst
machine
Randomly
select any one
property
Based on a
probability,
include the best
machine and
exclude the
worst machine
15
Best
machine
Worst
machine
Modification in The Existing NSGA-III
Algorithm: Solution Filtering
16
Each line indicates four objective values of modified NSGA-III
Worst objective
values for Objective
1, 3, and 4.
Best objective
value for Objective
2 only.
𝑭 𝒕𝒐𝒕𝒂𝒍 = 𝑾 𝒐𝒃𝒋𝟏 × 𝑽 𝒐𝒃𝒋𝟏 + 𝑾 𝒐𝒃𝒋𝟐 × 𝑽 𝒐𝒃𝒋𝟐 + … + 𝑾 𝒐𝒃𝒋𝑵 × 𝑽 𝒐𝒃𝒋𝑵
Pareto front showing 15 solutions of
our minimization problem
We avoid this kind of
solutions
Performance Evaluation of Our Proposed
Solution
• Real experiment over a testbed
– In a laboratory having 30 machines under a custom-built
cluster
• SimGrid simulation
– To enable experiments with more number of machines
17
Energy monitoring module for measuring
computation power and cooling power Real testbed setup
Experimental Setup for Comparing Modified NSGA-
III, PSO, and ACO in Real Testbed
18
Parameter Value
# of master machine 1
# of slave machines 29
Processor Intel Core 2 Duo
Processor base frequency 2.4 GHz, 2.66 GHz, and 2.8 GHz
Memory 1GB to 2GB
OS Ubuntu 14.04 LTS (x86)
Network B/W 5 to 100 MBPS
Total # of files 24, 16, and 11
Size of each file 787 MB
Total data size 18.9 GB, 12.6 GB, and 8.7 GB
Maximum allowable temperature 20℃-23℃
Environment temperature 28℃
Experimental Results: Comparing Modified
NSGA-III, PSO, and ACO in Real Testbed
19
Experimental Setup for Comparing NSGA-III,
PSO, and ACO in SimGrid
20
Parameter Value
# of master machines 1
# of slave machines 29, 49
Processor Intel Core 2 Duo
PC power Peak: 10 - 400 W, idle: 2.5-100 W,
power off: 5 W
Network B/W 10 to 100 kbps
Total # of files 86, 64, and 36
Size of each file 787 MB
Total data size 67.7 GB, 50.4 GB, and 28.3 GB
Maximum allowable temperature 20℃-23℃
Environment temperature 28℃
SimGrid version 3.12
Experimental Results: Comparing Modified NSGA-
III, PSO, and ACO in SimGrid with 30 machines
21
Experimental Results: Comparing Modified NSGA-
III, PSO, and ACO in SimGrid with 50 machines
22
Experimental Results: Validating Modified
NSGA-III Performs Better Than NSGA-III
23
Experimental Results: Improvement over PSO
and ACO in Real Testbed and SimGrid
24
Results for 18.9 GB and 31.5 GB workload
Conclusion and Future Work
• Cluster administrator needs to decide the right number and
right combination of machines, where a many-objective
approach is yet to be focused in the literature
• We provide a solution for this problem considering cooling
energy consumption and empirical performance
characterization
• We formulate a synergy between NSGA-III and greedy
approach to solve this problem
• Future work
 Analysis with machine failures
 Analysis on different factors for cooling energy including room
size or cooling capacity
25
Thank you
Questions are welcome!
26
BACKUP SLIDES
27
Cluster topology
Grid computingCloud computing
Architectures for Parallel Computing
28
Large Linux cluster in University
of Technology, Germany
Cluster machines are connected
by a local area network
Clouds and grids are
geographically distributed
Clusters are tightly coupled,
whereas clouds and grids are
loosely coupled
Home-made cluster
Background for Parallel Computing
29
Problem
Processor
The whole problem is distributed in small parts
to a single processor
Problem_1
Problem_2
Problem_3
Processor_1
Processor_2
Processor_3
Serial computing
Parallel computing
Related Research Work (cont.)
30
• Multi-objective optimization for virtual machine
based schemes in cloud
– R. Li et al., IEEE CLOUD, 2016
– Uses Biogeography Based Optimization (BBO)
technique for solving multi-objective optimization
– Considers only load balancing through resource
utilization in VMs placed in cloud architecture
Does not integrate any empirical performance characterization of clusters
Does not accumulate the impact of cooling energy consumption
Related Research Work (cont.)
31
• Stochastic optimization approach
– K.M. Tarplee et al., IEEE Transactions on Cloud
Computing (TCC), 2016
– Utilizes multi-objective stochastic programming
– Considers four objectives for cloud architectures
1. Up-gradation cost
2. Failure rate
3. Power consumption
4. Number of completed tasks
Does not consider any specialized many-objective optimization technique
Does not integrate any empirical performance characterization of clusters
Does not accumulate the impact of cooling energy consumption
Improvement in Parallel Computing
32
Experimental Results: Improvement over PSO
and ACO in Real Testbed
• % of improvement over PSO in real testbed:
• % of improvement over ACO in real testbed:
Workload Time
Cooling
energy
Computation
energy
CPU usage
Memory
usage
18.9 GB 74 75 63 127 5
12.6 GB 36 35 25 0 -1
8.7 GB 21 21 20 10 -7
Workload Time
Cooling
energy
Computation
energy
CPU usage
Memory
usage
18.9 GB 49 49 58 159 0
12.6 GB -18 -20 10 27 3
8.7 GB 43 42 56 130 1
33
Impacts of Selecting A Threshold
34
Threshold value
Computation time
(mins)
Cooling energy
(KWh)
Computation
energy (KWh)
2 11.1 0.43 0.92
4 9.5 0.31 0.66
6 9.7 0.24 0.53
8 10.2 0.32 0.68
Simulation results with 67.7 GB data over 30 machines
Threshold value
Total no. of
solutions in the
pareto-front
No. of solutions
with < 6 machines
No. of solutions
with >= 6
machines
2 120 34 86
4 120 27 93
6 120 13 107
8 120 14 106
Too much small threshold can
result in worse results and
introduce unnecessary
solutions
Improvement of Cooling Energy in Data Centers
by Good Design
35
Experimental Results: Improvement over PSO
in SimGrid
• % of improvement over PSO in SimGrid with 30 machines:
• % of improvement over PSO in SimGrid with 50 machines:
36
Workload Time Cooling energy
Computation
energy
67.7 GB 21 13 10
50.4 GB 36 11 10
28.3 GB 43 13 10
Workload Time Cooling energy
Computation
energy
67.7 GB 38 59 59
50.4 GB 33 36 38
28.3 GB 10 -6 0
Experimental Results: Improvement over ACO
in SimGrid
• % of improvement over ACO in SimGrid with 30 machines:
• % of improvement over ACO in SimGrid with 50 machines:
37
Workload Time Cooling energy
Computation
energy
67.7 GB 43 10 5
50.4 GB 17 8 8
28.3 GB 15 5 0
Workload Time Cooling energy
Computation
energy
67.7 GB 16 55 55
50.4 GB 41 79 79
28.3 GB 43 87 87
Experimental Setup for Validating Modified
NSGA-III Performs Better Than NSGA-III
38
Parameter Value
# of master machines 1
# of slave machines 29
Processor Intel Core 2 Duo
PC power Peak: 10 - 400 W, idle: 2.5-100 W,
power off: 5 W
Network B/W 10 to 100 kbps
Total # of files 86, 64, and 36
Size of each file 787 MB
Total data size 67.7 GB, 50.4 GB, and 28.3 GB
Maximum allowable temperature 20℃-23℃
Environment temperature 28℃
SimGrid version 3.12
Considerations for Data Centers
39
Difference between Cluster and Grid?
40
Grid are inherently distributed by its nature over a LAN,
metropolitan or WAN. On the other hand, the computers
in the cluster are normally contained in a single location.
Another difference lies in the way resources are handled. In
case of Cluster, the whole system (all nodes) behave like a
single system view and resources are managed by
centralized resource manager. In case of Grid, every node
is autonomous i.e. it has its own resource manager and
behaves like an independent entity.
Difference between Grid and Cloud?
41
Grid can have only some high computing power.
Cluster can have some application level service.
FB, Google-> cloud service.
Cloud computing is a kind of grid computing;
it has evolved by addressing the QoS (quality of service)
and reliability problems.
Outline of Our Methodology
42
Cooling
energy in
SimGrid
Cooling energy in real
test-bed
Verify one another
Cooling energy integration
in SimGrid and test-bed
Modification
in NSGA-III
Selection
FilteringCrossover
Formulation and
modification in NSGA-III
to develop a solution
Compare
Modified
NSGA-III
ACOPSO
Performance
evaluation and
comparison
Problem formulation
Our Proposed Many-Objective Optimization
Model
43
Following the empirical analysis,
we formulate our objective
functions as follows:Computation
time
Energy
consumption
Cost
Resource
utilization (CPU
and memory
usage)
Workload
should be within
machine hard
disk size
No. of selected
machines
should be
greater than 0.
Cooling
temperature
Machine
configuration
Workload
No. of
machines
Temperature
difference
Machine
configuration
No. of
machines
Per machine
workload
Lowest
possible temp.
Max.
possible
temp.
Our Proposed Modification in NSGA-III
Algorithm
44
• With some probability we perform HUX
• Only perform when the decision variables are not
same
Half Uniform
Crossover (HUX)
• Cluster machines based on each machine property
• Make a group in which all machines are in best group
• Make a group in which all machines are in worst group
• Try to take from best group and not to take from worst
group
Greedy Clustering
Approach
• For each machine property try to take the best machine
instead of taking the worst machine based on probabilistic
possibility
Greedy Best and
Worst Swap
Modified crossover
Objectives and Possible Outcomes
45
Formulate a many-
objective optimization
problem for clusters,
and solve it
Integrating cooling
energy in both real
testbed and simulation
Test our approach in
real testbed and
simulation to compare
performance with
other approaches
Particle swarm
optimization (PSO)
Ant colony optimization
(ACO)
Modified
NSGA-III
Some Related Algorithms with Bio-Geography Based
Solutions
46
• Biogeography-based optimization (BBO)
• Ant colony optimization (ACO)
• Particle swarm optimization (PSO)
• Artificial Bee Colony Optimization
PSO Based Technique
47
1. Task assignment based on avg workload, the deviation of workload, and migration
cost.
2. PSO basic: Particle location is updated based on the current location, distance
between my current location and my best location, and distance between my current
location and global best location.
3. Limitations: Local optimal- can converge fast but local optima. Keep only
current optimal point.
Solution: Keep a global storage for keeping all the optimal values.
Ant Colony Based Technique
48
1. VM assignment based on resource wastage and power consumption.
2. ACO basic: Ants search for food. They put pheromone based on the quality of food
and amount. The other ants follow the path based on the concentration of pheromone.
BBO
49
• Define the mutation probability and the elitism parameter.
• Initialize the population
• Calculate the immigration rate and emigration rate for each island. Good solutions have
high emigration rates and low immigration rates. Bad solutions have low emigration rates
and high immigration rates.
• Probabilistically perform mutation for each island.
• Replace the worst islands in the population with the previous generation’s elite islands.
• If the termination criterion is met, terminate; otherwise, repeat
Stochastic Programming
50
Stochastic programming is a framework for modeling optimization problems that
Involve uncertainty.
Stochastic programming models are similar in style but take advantage of the fact
that probability distribution governing the data are known or can be estimated.
Stochastic programming studies the case in which some of the constraints or
parameters depend on random variables.
Stochastic linear programming is an extension of linear programming, where some
of the coefficients in the objective and the constraints are random variables
NSGA-II
51
Modifications in NSGA-II to get NSGA-III
52
• Crowding distance idea has been replaced.
• NSGA-III introduces some reference points idea so that diversity is maintained.
Why we use NSGA-III as our baseline
algorithm?
Introduced in 2014
Specialized for solving many-objective optimization problems
Other earlier approaches mostly solve benchmark problems such as DTLZ. They
mostly ignore real-world problems
Popular NSGA-II only considers crowding distance, which is not a feasible
property in solving many-objective optimization problems
53
Popular MOEA/D claims to work for many-objectives, but only considered three
objectives only. Also, uses some default parameters
Algorithm Comparison
54
Earlier algorithms: O(MN3) -> M = no. of objectives, N = population size.
NSGA-II: O (MN2)
MOEA/D
NSGA-III
Improved NSGA-III
ACO
PSO
BBO/Complex
The Underneath of Cooling Energy Module Development for
Testbed Experiment: Circuit Diagram
55
1. Computation energy
2. Cooling energy
3. CPU usage
4. Memory usage
Energy
monitor
PT, CT
LM35
sensor
Psensor Energy monitoring module for measuring
computation power and cooling power
Temperature monitoring module
Circuit diagram
Experimental Results: SimGrid Energy Module: Simulation
Environment in Real Test-bed: Machine Configuration
Processor base
frequency
Memory # of machines
2.4 GHz 1 GB 1
2.4 GHz 2 GB 2
2.66 GHz 1 GB 2
2.66 GHz 2 GB 8
2.8 GHz 2 GB 17
56
How we get the equations?
57
How we get the equations? (cont.)
58
Using Matlab, we try to get the equations:
Computation time equations: (for 6GB data)
For order 1: 𝑦 = −0.562𝑥 + 17.2
For order 2: 𝑦 = 0.046𝑥2
− 2.098𝑥 + 26.8
For order 3: 𝑦 = −0.0038𝑥3
+ 0.2325𝑥2
− 4.73𝑥 + 36.4
Energy consumption equations: (for 6 GB data)
For order 1: 𝑦 = −0.0246𝑥 + .8950
For order 2: 𝑦 = 0.0023𝑥2
− 0.0996𝑥 + 1.3653
For order 3: 𝑦 = −0.0002𝑥3 + 0.0101𝑥2 − 0.2109𝑥 + 1.7722
Energy consumption equations: (for 6 GB data)
For order 1: 𝑦 = −1.4165𝑥 + 93.0156
For order 2: 𝑦 = 0.0094𝑥2 − 1.7271𝑥 + 94.9635
For order 3: 𝑦 = 0.0012𝑥3 − 0.0496𝑥2 − 0.8936𝑥 + 91.9160
Experimental Results: SimGrid Energy Module:
Cooling Energy and Computation Energy Ratio
• Cooling energy is around 83% of total energy in real
testbed
Workload
Cooling
energy
consumption
avg (KWh)
Computation
energy
consumption
avg (KWh)
Total energy
consumption
avg (KWh)
% of cooling
energy
consumption
% of
computation
energy
consumption
12.6 GB 0.367 0.088 0.455 81 19
9.44 GB 0.398 0.069 0.467 85 15
6.3 GB 0.242 0.052 0.294 82 18
3.14 GB 0.146 .030 0.176 83 17
59
Experimental Results: SimGrid Energy Module: Cooling
Energy and Computation Energy Ratio (Cont.)
• Cooling energy is around 65% of total energy in
SimGrid
Workload
Cooling
energy
consumption
avg (KWh)
Computation
energy
consumption
avg (KWh)
Total energy
consumption
avg (KWh)
% of cooling
energy
consumption
% of
computation
energy
consumption
12.6 GB 0.235 0.125 0.36 65 35
9.44 GB 0.176 0.094 0.27 65 35
6.3 GB 0.117 0.063 0.18 65 35
3.14 GB 0.058 .031 .089 65 35
60
Space vs Power Requirement?
61
We have 38*35 = 1330 sq ft room. Hence, we need 23,000 BTU/hr.
We have 3 ACs.
Each having capacity 1.5 ton = 18,000 BTU/hr.
Hence, 3 ACs will together give us 54,000 BTU.
Space vs Power Requirement? (cont.)
62
Power Requirement for Different States
63
Power State Power
Active state 100W-120W
Idle state 40W
Power-off state 5W
Why we do not consider machine on off energy?
Answer: It is not a significant amount. We make the whole simulation for about 10-60
minutes. Machine on-off needs small amount of time. Hence, it should consume small
amount of energy. Moreover, we only consider the machine runtime energy measurement.
Hence, this on-off energy is not present in all the experiments. That is why the overall
comparison should not be affected.
Experimental Results for Larger Data Size
64
Experimental Results: Improvement over PSO
and ACO in Real Testbed and SimGrid
• % of improvement over PSO comparison for 18.9 GB data in real
testbed and SimGrid:
• % of improvement over ACO comparison for 18.9 GB data in real
testbed and SimGrid:
Environment Time
Cooling
energy
Computation
energy
Real testbed 74 75 63
SimGrid 25 18 18
Environment Time
Cooling
energy
Computation
energy
Real Testbed 49 49 58
SimGrid 51 82 83
65
Experimental Results: Improvement over PSO
and ACO in Real Testbed and SimGrid
• % of improvement over PSO comparison for 31.5 GB data in real
testbed and SimGrid:
• % of improvement over ACO comparison for 31.5 GB data in real
testbed and SimGrid:
Environment Time
Cooling
energy
Computation
energy
Real testbed 43 43 44
SimGrid 28 56 58
Environment Time
Cooling
energy
Computation
energy
Real Testbed 28 9 28
SimGrid 36 82 82
66
Machine Turn-off Power
67
Machine off power
Elbow Points from Our Year-Long Collected
Data to Guess The Threshold Value
68
Computation time with 6 GB data: Elbow point: 8 Computation time with 9 GB data: Elbow
point: 7
Total energy with 6 GB data: Elbow point: 10 Total energy with 9 GB data: Elbow point: 9
69
Slaves
Master
Selection Threshold Impacts over Different
Cluster Objectives
70
Threshold
value
Computation
time (mins)
Cooling energy
(KWh)
Computation
energy (KWh)
2 11.1 0.43 0.92
4 9.5 0.31 0.66
6 9.7 0.24 0.53
8 10.2 0.32 0.68
For a random cluster, we run our modified NSGA-III algorithm with different threshold
values with 67.7 GB data.
Selection Threshold Impacts over Solutions in
The Pareto-Front
71
Threshold
value
Total no. of
solutions in the
pareto-front
No. of
solutions with
< 6 machines
No. of
solutions with
>= 6 machines
2 120 34 86
4 120 27 93
6 120 13 107
8 120 14 106
For a random cluster, we run our modified NSGA-III algorithm with different threshold
values with 67.7 GB data.
Too much small
threshold can introduce
unnecessary solutions
Effects of Different Weights in Machine
Selection
72
Weights
Computation
time (mins)
Cooling
energy
(KWh)
Computation
energy
(KWh)
No. of
selected
machines
CPU
utilization
(%)
Memory
utilization
(%)
25-75 34.8 2.6 0.3 7 57.5 91.2
75-25 23.5 1.76 0.24 11 55.2 91.0
75-25 indicates more weights
to select highly configured
machines
25-75 indicates less weights
to select highly configured
machines
Experimental Results: Improvement over PSO
and ACO in Real Testbed and SimGrid
73
Graphs showing results for 18.9 GB and 31.5 GB workload
Experimental Results: Comparing Modified
NSGA-III, PSO, and ACO in Real Testbed
74
The Underneath of Cooling Energy Module Integration in
SimGrid: An Application of Heat Dynamics
We adopt the conventional heat formula:
𝑸 = 𝑪 𝒑 × 𝑾 × 𝑫𝑻
𝐶 𝑝 = Specific heat
𝑊 = Mass of airflow per min
𝐷𝑇 = Change in temperature
𝑸 = 𝑪 𝒑 × 𝑪𝑭𝑴 × 𝑫 × 𝑫𝑻
𝐶𝐹𝑀 = Cubic feet per minute
𝐷 = Density of the air
Thus, we can write:
Putting W = CFM × D, where CFM =
cooling air volume per minute, and
D = mass of air per volume, we get:
75
𝑪𝑭𝑴 =
𝑸
𝑪 𝒑 × 𝑫 × 𝑫𝑻
•Specific heat of a room remains constant
•At a certain place where the pressure is
constant, density will also remain constant
•There remain other heat losses
𝑪𝑭𝑴 =
𝟏. 𝟕𝟔 × 𝑸
𝑫𝑻
Using 1 CFM consumes 47.82W, we get:
𝑷𝒐𝒘𝒆𝒓 = 𝑪𝑭𝑴 × 𝟒𝟕. 𝟖𝟐
The Underneath of Energy Model and Utilization
Model Integration for Testbed Experiments
76
1. Computation energy
2. Cooling energy
3. CPU usage
4. Memory usage
Energy
monitor -
PT, CT
Psensor
Energy monitoring module for measuring
computation power and cooling power
Real testbed setup
The Underneath of Energy Model and Utilization
Model Integration for Testbed Experiment (Cont.)
77
PT
CT-1
CT-2
Energy Monitor 1
PT
CT-1
CT-2
Energy Monitor 2
Cluster voltage
Cluster
current
AC voltage
AC-2
current
Cluster
AC-1 AC-2 AC-3
AC-1
current
AC-3
current
Experimental Setup for Integration of Energy Module
in SimGrid and Real Testbed
Parameter
Value
SimGrid Testbed
# of master machine 1 1
# of slave machines 4, 9, 14, 19, 24, and 29
4, 9, 14, 19, 24, and 29
PC power
38, 400-44, 800
MFLOPS
2.4 GHz, 2.66 GHz, and
2.8 GHz
Machine power consumption
Peak: 100 - 120 W, idle:
40 W, power off: 5 W
N/A
Memory N/A 1-2 GB
Total # of files 4, 8, 12, and 16 4, 8, 12, and 16
Size of each file 787 MB 787 MB
Total data size
3.14. 6.3, 9.44, and 12.6
GB
3.14. 6.3, 9.44, and 12.6
GB
Maximum allowable temperature 22◦ Celsius 22◦ Celsius
Environment temperature 25◦ Celsius
25◦ Celsius
78
Experimental Results on Computation Energy
79
Computation energy comparison for
3.14GB data
Computation energy comparison for
6.3GB data
Computation energy comparison for
9.44GB data
Computation energy comparison for
12.6GB data
SimGrid computation energy and test-bed computation
energy follow a closely similar trend
Experimental Results on Cooling Energy
80
Cooling energy comparison for
3.14GB data
Cooling energy comparison for
6.3GB data
Cooling energy comparison for
9.44GB data
Cooling energy comparison for
12.6GB data
SimGrid cooling energy and test-bed cooling energy
follow a closely similar trend
Experimental Results on Total Energy
81
Total energy comparison for 3.14GB data Total energy comparison for 6.3GB data
Total energy comparison for 9.44GB data Total energy comparison for 12.6GB data
SimGrid outputs are close to real test-bed outputs
Exploiting a Synergy between Greedy Approach and NSGA
for Scheduling in Computing Clusters
Presented by
A S M Rizvi
This work was supported by the ICT Division, Government of the
People's Republic of Bangladesh.
A S M Rizvi – University of Southern California
Tarik Reza Toha – Bangladesh University of Engineering and Technology
Siddharthe Shankar Das – Bangladesh University of Engineering and Technology
Sriram Chellappan – University of South Florida
A. B. M. Alim Al Islam - Bangladesh University of Engineering and Technology
NSysS’2018

Contenu connexe

Tendances

06 how to write a map reduce version of k-means clustering
06 how to write a map reduce version of k-means clustering06 how to write a map reduce version of k-means clustering
06 how to write a map reduce version of k-means clusteringSubhas Kumar Ghosh
 
A Study on Task Scheduling in Could Data Centers for Energy Efficacy
A Study on Task Scheduling in Could Data Centers for Energy Efficacy A Study on Task Scheduling in Could Data Centers for Energy Efficacy
A Study on Task Scheduling in Could Data Centers for Energy Efficacy Ehsan Sharifi
 
Task Scheduling in Grid Computing.
Task Scheduling in Grid Computing.Task Scheduling in Grid Computing.
Task Scheduling in Grid Computing.Ramandeep Kaur
 
Task scheduling Survey in Cloud Computing
Task scheduling Survey in Cloud ComputingTask scheduling Survey in Cloud Computing
Task scheduling Survey in Cloud ComputingRamandeep Kaur
 
Test different neural networks models for forecasting of wind,solar and energ...
Test different neural networks models for forecasting of wind,solar and energ...Test different neural networks models for forecasting of wind,solar and energ...
Test different neural networks models for forecasting of wind,solar and energ...Tonmoy Ibne Arif
 
task scheduling in cloud datacentre using genetic algorithm
task scheduling in cloud datacentre using genetic algorithmtask scheduling in cloud datacentre using genetic algorithm
task scheduling in cloud datacentre using genetic algorithmSwathi Rampur
 
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2Troy Schmidt
 
IRJET- A Statistical Approach Towards Energy Saving in Cloud Computing
IRJET-  	  A Statistical Approach Towards Energy Saving in Cloud ComputingIRJET-  	  A Statistical Approach Towards Energy Saving in Cloud Computing
IRJET- A Statistical Approach Towards Energy Saving in Cloud ComputingIRJET Journal
 
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...NECST Lab @ Politecnico di Milano
 
ECE_561_Final_Project
ECE_561_Final_ProjectECE_561_Final_Project
ECE_561_Final_Projectpranavd211190
 
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...Journal For Research
 
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in Cloud
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in CloudTime and Reliability Optimization Bat Algorithm for Scheduling Workflow in Cloud
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in CloudIRJET Journal
 
Energy Efficient Compression of Shock Data using Compressed Sensing
Energy Efficient Compression of Shock Data using Compressed SensingEnergy Efficient Compression of Shock Data using Compressed Sensing
Energy Efficient Compression of Shock Data using Compressed SensingJerrin Panachakel
 
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...MLconf
 
Sky Arrays - ArrayDB in action for Sky View Factor Computation
Sky Arrays - ArrayDB in action for Sky View Factor ComputationSky Arrays - ArrayDB in action for Sky View Factor Computation
Sky Arrays - ArrayDB in action for Sky View Factor ComputationEUDAT
 
Arrhenius.jl: A Differentiable Combustion Simulation Package
Arrhenius.jl: A Differentiable Combustion Simulation PackageArrhenius.jl: A Differentiable Combustion Simulation Package
Arrhenius.jl: A Differentiable Combustion Simulation PackageWeiqi Ji
 
BREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE Software
 
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...Cemal Ardil
 

Tendances (20)

06 how to write a map reduce version of k-means clustering
06 how to write a map reduce version of k-means clustering06 how to write a map reduce version of k-means clustering
06 how to write a map reduce version of k-means clustering
 
A Study on Task Scheduling in Could Data Centers for Energy Efficacy
A Study on Task Scheduling in Could Data Centers for Energy Efficacy A Study on Task Scheduling in Could Data Centers for Energy Efficacy
A Study on Task Scheduling in Could Data Centers for Energy Efficacy
 
Task Scheduling in Grid Computing.
Task Scheduling in Grid Computing.Task Scheduling in Grid Computing.
Task Scheduling in Grid Computing.
 
Task scheduling Survey in Cloud Computing
Task scheduling Survey in Cloud ComputingTask scheduling Survey in Cloud Computing
Task scheduling Survey in Cloud Computing
 
Test different neural networks models for forecasting of wind,solar and energ...
Test different neural networks models for forecasting of wind,solar and energ...Test different neural networks models for forecasting of wind,solar and energ...
Test different neural networks models for forecasting of wind,solar and energ...
 
task scheduling in cloud datacentre using genetic algorithm
task scheduling in cloud datacentre using genetic algorithmtask scheduling in cloud datacentre using genetic algorithm
task scheduling in cloud datacentre using genetic algorithm
 
[ppt]
[ppt][ppt]
[ppt]
 
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2
Earthquake Updates and Enhancements to Processing for Hazus-MH 3.2
 
J0210053057
J0210053057J0210053057
J0210053057
 
IRJET- A Statistical Approach Towards Energy Saving in Cloud Computing
IRJET-  	  A Statistical Approach Towards Energy Saving in Cloud ComputingIRJET-  	  A Statistical Approach Towards Energy Saving in Cloud Computing
IRJET- A Statistical Approach Towards Energy Saving in Cloud Computing
 
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...
A Highly Parallel Semi-Dataflow FPGA Architecture for Large-Scale N-Body Simu...
 
ECE_561_Final_Project
ECE_561_Final_ProjectECE_561_Final_Project
ECE_561_Final_Project
 
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...
TASK SCHEDULING USING AMALGAMATION OF MET HEURISTICS SWARM OPTIMIZATION ALGOR...
 
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in Cloud
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in CloudTime and Reliability Optimization Bat Algorithm for Scheduling Workflow in Cloud
Time and Reliability Optimization Bat Algorithm for Scheduling Workflow in Cloud
 
Energy Efficient Compression of Shock Data using Compressed Sensing
Energy Efficient Compression of Shock Data using Compressed SensingEnergy Efficient Compression of Shock Data using Compressed Sensing
Energy Efficient Compression of Shock Data using Compressed Sensing
 
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...
Anima Anadkumar, Principal Scientist, Amazon Web Services, Endowed Professor,...
 
Sky Arrays - ArrayDB in action for Sky View Factor Computation
Sky Arrays - ArrayDB in action for Sky View Factor ComputationSky Arrays - ArrayDB in action for Sky View Factor Computation
Sky Arrays - ArrayDB in action for Sky View Factor Computation
 
Arrhenius.jl: A Differentiable Combustion Simulation Package
Arrhenius.jl: A Differentiable Combustion Simulation PackageArrhenius.jl: A Differentiable Combustion Simulation Package
Arrhenius.jl: A Differentiable Combustion Simulation Package
 
BREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD ModelerBREEZE 3D Analyst for the Advanced AERMOD Modeler
BREEZE 3D Analyst for the Advanced AERMOD Modeler
 
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...
Automatic generation-control-of-multi-area-electric-energy-systems-using-modi...
 

Similaire à Exploiting a Synergy between Greedy Approach and NSGA for Scheduling in Computing Clusters

Many-Objective Performance Enhancement in Computing Clusters
Many-Objective Performance Enhancement in Computing ClustersMany-Objective Performance Enhancement in Computing Clusters
Many-Objective Performance Enhancement in Computing ClustersTarik Reza Toha
 
Architectural Optimizations for High Performance and Energy Efficient Smith-W...
Architectural Optimizations for High Performance and Energy Efficient Smith-W...Architectural Optimizations for High Performance and Energy Efficient Smith-W...
Architectural Optimizations for High Performance and Energy Efficient Smith-W...NECST Lab @ Politecnico di Milano
 
Statistical power consumption analysis and modeling
Statistical power consumption analysis and modelingStatistical power consumption analysis and modeling
Statistical power consumption analysis and modelingnadikari123
 
LSI Seminar on Marina Zapater's PhD Thesis
LSI Seminar on Marina Zapater's PhD ThesisLSI Seminar on Marina Zapater's PhD Thesis
LSI Seminar on Marina Zapater's PhD ThesisGreenLSI Team, LSI, UPM
 
Parallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix MultiplicationParallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix MultiplicationIJERA Editor
 
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...csandit
 
IRJET- Cloud Cost Analyzer and Optimizer
IRJET- Cloud Cost Analyzer and OptimizerIRJET- Cloud Cost Analyzer and Optimizer
IRJET- Cloud Cost Analyzer and OptimizerIRJET Journal
 
Tutorial-on-DNN-09A-Co-design-Sparsity.pdf
Tutorial-on-DNN-09A-Co-design-Sparsity.pdfTutorial-on-DNN-09A-Co-design-Sparsity.pdf
Tutorial-on-DNN-09A-Co-design-Sparsity.pdfDuy-Hieu Bui
 
54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptxYashAhire28
 
Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)alekn
 
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...Tulipp. Eu
 
Thinking Outside the Box – Server Design for #DataCenter Optimization
Thinking Outside the Box – Server Design for #DataCenter OptimizationThinking Outside the Box – Server Design for #DataCenter Optimization
Thinking Outside the Box – Server Design for #DataCenter OptimizationRobert Schmidt
 
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...IJERA Editor
 
Data center specific thermal and energy saving techniques
Data center specific thermal and energy saving techniquesData center specific thermal and energy saving techniques
Data center specific thermal and energy saving techniquesXiao Qin
 
Simulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud InfrastructuresSimulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud InfrastructuresCloudLightning
 

Similaire à Exploiting a Synergy between Greedy Approach and NSGA for Scheduling in Computing Clusters (20)

Many-Objective Performance Enhancement in Computing Clusters
Many-Objective Performance Enhancement in Computing ClustersMany-Objective Performance Enhancement in Computing Clusters
Many-Objective Performance Enhancement in Computing Clusters
 
presentation 2.pptx
presentation 2.pptxpresentation 2.pptx
presentation 2.pptx
 
Architectural Optimizations for High Performance and Energy Efficient Smith-W...
Architectural Optimizations for High Performance and Energy Efficient Smith-W...Architectural Optimizations for High Performance and Energy Efficient Smith-W...
Architectural Optimizations for High Performance and Energy Efficient Smith-W...
 
Statistical power consumption analysis and modeling
Statistical power consumption analysis and modelingStatistical power consumption analysis and modeling
Statistical power consumption analysis and modeling
 
LSI Seminar on Marina Zapater's PhD Thesis
LSI Seminar on Marina Zapater's PhD ThesisLSI Seminar on Marina Zapater's PhD Thesis
LSI Seminar on Marina Zapater's PhD Thesis
 
Multicore architectures
Multicore architecturesMulticore architectures
Multicore architectures
 
DOE Slides.pptx
DOE Slides.pptxDOE Slides.pptx
DOE Slides.pptx
 
Parallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix MultiplicationParallel Processing Technique for Time Efficient Matrix Multiplication
Parallel Processing Technique for Time Efficient Matrix Multiplication
 
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
Fault-Tolerance Aware Multi Objective Scheduling Algorithm for Task Schedulin...
 
FrackingPaper
FrackingPaperFrackingPaper
FrackingPaper
 
IRJET- Cloud Cost Analyzer and Optimizer
IRJET- Cloud Cost Analyzer and OptimizerIRJET- Cloud Cost Analyzer and Optimizer
IRJET- Cloud Cost Analyzer and Optimizer
 
Tutorial-on-DNN-09A-Co-design-Sparsity.pdf
Tutorial-on-DNN-09A-Co-design-Sparsity.pdfTutorial-on-DNN-09A-Co-design-Sparsity.pdf
Tutorial-on-DNN-09A-Co-design-Sparsity.pdf
 
54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx54665962-Nav-Cluster-Computing.pptx
54665962-Nav-Cluster-Computing.pptx
 
Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)
 
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...
Quantifying Energy Consumption for Practical Fork-Join Parallelism on an Embe...
 
Thinking Outside the Box – Server Design for #DataCenter Optimization
Thinking Outside the Box – Server Design for #DataCenter OptimizationThinking Outside the Box – Server Design for #DataCenter Optimization
Thinking Outside the Box – Server Design for #DataCenter Optimization
 
Energy Efficiency in Data Centers
Energy Efficiency in Data CentersEnergy Efficiency in Data Centers
Energy Efficiency in Data Centers
 
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...
Implementing Workload Postponing In Cloudsim to Maximize Renewable Energy Uti...
 
Data center specific thermal and energy saving techniques
Data center specific thermal and energy saving techniquesData center specific thermal and energy saving techniques
Data center specific thermal and energy saving techniques
 
Simulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud InfrastructuresSimulation of Heterogeneous Cloud Infrastructures
Simulation of Heterogeneous Cloud Infrastructures
 

Plus de Tarik Reza Toha

An approach towards greening the digital display system
An approach towards greening the digital display systemAn approach towards greening the digital display system
An approach towards greening the digital display systemTarik Reza Toha
 
Predicting Human Count through Environmental Sensing in Closed Indoor Settings
Predicting Human Count through Environmental Sensing in Closed Indoor SettingsPredicting Human Count through Environmental Sensing in Closed Indoor Settings
Predicting Human Count through Environmental Sensing in Closed Indoor SettingsTarik Reza Toha
 
Automatic Fabric Defect Detection with a Wide-And-Compact Network
Automatic Fabric Defect Detection with a Wide-And-Compact NetworkAutomatic Fabric Defect Detection with a Wide-And-Compact Network
Automatic Fabric Defect Detection with a Wide-And-Compact NetworkTarik Reza Toha
 
Binarization of degraded document images based on hierarchical deep supervise...
Binarization of degraded document images based on hierarchical deep supervise...Binarization of degraded document images based on hierarchical deep supervise...
Binarization of degraded document images based on hierarchical deep supervise...Tarik Reza Toha
 
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...Tarik Reza Toha
 
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...Tarik Reza Toha
 
PNUTS: Yahoo!’s Hosted Data Serving Platform
PNUTS: Yahoo!’s Hosted Data Serving PlatformPNUTS: Yahoo!’s Hosted Data Serving Platform
PNUTS: Yahoo!’s Hosted Data Serving PlatformTarik Reza Toha
 
Towards Greening the Digital Display System
Towards Greening the Digital Display SystemTowards Greening the Digital Display System
Towards Greening the Digital Display SystemTarik Reza Toha
 
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...Tarik Reza Toha
 
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...Tarik Reza Toha
 
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...Tarik Reza Toha
 
Smart Mat: A Low Cost People Counting Solution
Smart Mat: A Low Cost People Counting SolutionSmart Mat: A Low Cost People Counting Solution
Smart Mat: A Low Cost People Counting SolutionTarik Reza Toha
 
uReporter, an open public reporting system(SD)
uReporter, an open public reporting system(SD)uReporter, an open public reporting system(SD)
uReporter, an open public reporting system(SD)Tarik Reza Toha
 
uReporter, a social problem reporting system (ISD+DB)
uReporter, a social problem reporting system (ISD+DB)uReporter, a social problem reporting system (ISD+DB)
uReporter, a social problem reporting system (ISD+DB)Tarik Reza Toha
 
Euler trails and circuit
Euler trails and circuitEuler trails and circuit
Euler trails and circuitTarik Reza Toha
 
Islam, the ultimate solution
Islam, the ultimate solutionIslam, the ultimate solution
Islam, the ultimate solutionTarik Reza Toha
 

Plus de Tarik Reza Toha (19)

An approach towards greening the digital display system
An approach towards greening the digital display systemAn approach towards greening the digital display system
An approach towards greening the digital display system
 
Predicting Human Count through Environmental Sensing in Closed Indoor Settings
Predicting Human Count through Environmental Sensing in Closed Indoor SettingsPredicting Human Count through Environmental Sensing in Closed Indoor Settings
Predicting Human Count through Environmental Sensing in Closed Indoor Settings
 
Automatic Fabric Defect Detection with a Wide-And-Compact Network
Automatic Fabric Defect Detection with a Wide-And-Compact NetworkAutomatic Fabric Defect Detection with a Wide-And-Compact Network
Automatic Fabric Defect Detection with a Wide-And-Compact Network
 
Binarization of degraded document images based on hierarchical deep supervise...
Binarization of degraded document images based on hierarchical deep supervise...Binarization of degraded document images based on hierarchical deep supervise...
Binarization of degraded document images based on hierarchical deep supervise...
 
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...
Beyond Counting: Comparisons of Density Maps for Crowd Analysis Tasks—Countin...
 
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...
Towards Simulating Non-lane Based Heterogeneous Road Traffic of Less Develope...
 
PNUTS: Yahoo!’s Hosted Data Serving Platform
PNUTS: Yahoo!’s Hosted Data Serving PlatformPNUTS: Yahoo!’s Hosted Data Serving Platform
PNUTS: Yahoo!’s Hosted Data Serving Platform
 
Path shala
Path shalaPath shala
Path shala
 
Towards Greening the Digital Display System
Towards Greening the Digital Display SystemTowards Greening the Digital Display System
Towards Greening the Digital Display System
 
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...
Workload-Based Prediction of CPU Temperature and Usage for Small-Scale Distri...
 
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...
Towards Making an Anonymous and One-Stop Online Reporting System for Third-Wo...
 
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
Sparse Mat: A Tale of Devising A Low-Cost Directional System for Pedestrian C...
 
Smart Mat: A Low Cost People Counting Solution
Smart Mat: A Low Cost People Counting SolutionSmart Mat: A Low Cost People Counting Solution
Smart Mat: A Low Cost People Counting Solution
 
uReporter, an open public reporting system(SD)
uReporter, an open public reporting system(SD)uReporter, an open public reporting system(SD)
uReporter, an open public reporting system(SD)
 
uReporter, a social problem reporting system (ISD+DB)
uReporter, a social problem reporting system (ISD+DB)uReporter, a social problem reporting system (ISD+DB)
uReporter, a social problem reporting system (ISD+DB)
 
Euler trails and circuit
Euler trails and circuitEuler trails and circuit
Euler trails and circuit
 
Green Networking
Green NetworkingGreen Networking
Green Networking
 
Amplifier
AmplifierAmplifier
Amplifier
 
Islam, the ultimate solution
Islam, the ultimate solutionIslam, the ultimate solution
Islam, the ultimate solution
 

Dernier

BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdfSoniaTolstoy
 
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...PsychoTech Services
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfJayanti Pande
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdfQucHHunhnh
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfAdmir Softic
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdfQucHHunhnh
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationnomboosow
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfciinovamais
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfchloefrazer622
 
Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Celine George
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfchloefrazer622
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104misteraugie
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphThiyagu K
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingTechSoup
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxVishalSingh1417
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...EduSkills OECD
 

Dernier (20)

BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
 
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
 
Interactive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communicationInteractive Powerpoint_How to Master effective communication
Interactive Powerpoint_How to Master effective communication
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdf
 
Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdf
 
Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104Nutritional Needs Presentation - HLTH 104
Nutritional Needs Presentation - HLTH 104
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
 
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 

Exploiting a Synergy between Greedy Approach and NSGA for Scheduling in Computing Clusters

  • 1. Exploiting a Synergy between Greedy Approach and NSGA for Scheduling in Computing Clusters A S M Rizvi1, Tarik Reza Toha2, Siddhartha Shankar Das3, Sriram Chellappan4, A. B. M. Alim Al Islam5 NSysS 2018 Dhaka, Bangladesh 1 2, 3, 5 4
  • 2. Overview of This Presentation • Background and motivation • Proposed methodology for clusters – Many-objective problem formulation – Solution method using NSGA-III and greedy approach • Real experiment and simulation – Performance evaluation of proposed methodology • Conclusion 2
  • 3. Background of Our Study 3 Galaxy formation Planetary movements Climate changes Modeling, simulation, and experimentation of complex real-world phenomena demand rigorous computing Traffic simulation Plate tectonics Weather forecasts
  • 4. Cluster topology Grid computingCloud computing Architectures for Parallel Computing 4 Large Linux cluster in University of Technology, Germany Cluster machines are connected by a local area network Clouds and grids are geographically distributed Clusters are tightly coupled, whereas clouds and grids are loosely coupled Home-made cluster
  • 5. Motivation Behind Our Study Clusters have conflicting outcomes Increasing the number of machines Decreases computation time Increases maintenance cost Desirable Not desirable 5 Optimizations required for: 1. # of machines in the cluster 2. Machines to be selected
  • 6. Related Research Work 6 • Particle Swarm Optimization (PSO) based approach – C. Lijun et al., IJGDC, 2015 – Utilizes a variant of PSO – Focuses on load balancing from the perspectives of • Average workload of all servers in cloud clusters • Deviation of the workload • Migration cost between servers Does not integrate any empirical performance characterization of clusters Does not accumulate the impact of cooling energy consumption
  • 7. Related Research Work (cont.) 7 • Multi-objective optimization technique based on Ant Colony Optimization (ACO) – Y. Gao et al., Journal of Computer and System Sciences, 2013 – Simultaneously minimizes two different objectives in cloud architectures • Resource wastage • Power consumption • Load balancing Does not integrate any empirical performance characterization of clusters Does not accumulate the impact of cooling energy consumption
  • 8. Issues That Are Yet to Be Explored 8 •Consideration of cooling energy consumption Source: http://www.ha-inc.com/blog/entry/data-center-energy-consumption/ Data center cooling energy consumption can depend on several factors like room size to cooling capacity ratio, environmental temperature, design issue, etc. A good design can reduce the total energy consumption. •Empirical performance characterization of clusters  Should result in a new optimization model
  • 9. Our Contributions •We exploit a synergy between greedy method and NSGA-III algorithm to solve a many-objective optimization problem for clusters ̵ Incorporating cooling energy consumption ̵ Utilizing empirical characterization of clusters •We perform rigorous experimentation to demonstrate efficacy of our proposed solution 9
  • 10. Formulation of Our Many-Objective Optimization Problem for Computing Clusters 10 Following the empirical analysis, we formulate our objective functions as follows: Computation time Energy consumption Cost Inverse of resource utilization Restriction on assigned workload Constraint on # of selected machines Limit of cooling temperature
  • 11. Modification in The Existing NSGA-III Algorithm: Modified Selection Process 11 Sharp change at the beginning providing results below expectation # of selected machines should be greater than a threshold. We select the value as 6.
  • 12. Modification in The Existing NSGA-III Algorithm: Modified Crossover 12 Half Uniform Crossover (HUX) Greedy clustering approach Greedy approach to include the best machine and exclude the worst machine
  • 13. Modified Crossover: Half Uniform Crossover 13 Half-uniform crossover Machine selection decision variables (binary type) Cluster temperature decision variable (float type) Parent 1: Parent 2: Yellow variables will have crossover within yellow variables
  • 14. Modified Crossover: Greedy Clustering Approach Processor speed Memory Network B/W Best group Worst group Best group Worst group Best group Worst group Find the common machines Find the common machines Biased to include machines from this group Biased to exclude machines from this group 14
  • 15. Modified Crossover: Greedy Approach to Include The Best Machine and Exclude The Worst Machine Processor speed Memory Network B/W Best machine Worst machine Best machine Worst machine Randomly select any one property Based on a probability, include the best machine and exclude the worst machine 15 Best machine Worst machine
  • 16. Modification in The Existing NSGA-III Algorithm: Solution Filtering 16 Each line indicates four objective values of modified NSGA-III Worst objective values for Objective 1, 3, and 4. Best objective value for Objective 2 only. 𝑭 𝒕𝒐𝒕𝒂𝒍 = 𝑾 𝒐𝒃𝒋𝟏 × 𝑽 𝒐𝒃𝒋𝟏 + 𝑾 𝒐𝒃𝒋𝟐 × 𝑽 𝒐𝒃𝒋𝟐 + … + 𝑾 𝒐𝒃𝒋𝑵 × 𝑽 𝒐𝒃𝒋𝑵 Pareto front showing 15 solutions of our minimization problem We avoid this kind of solutions
  • 17. Performance Evaluation of Our Proposed Solution • Real experiment over a testbed – In a laboratory having 30 machines under a custom-built cluster • SimGrid simulation – To enable experiments with more number of machines 17 Energy monitoring module for measuring computation power and cooling power Real testbed setup
  • 18. Experimental Setup for Comparing Modified NSGA- III, PSO, and ACO in Real Testbed 18 Parameter Value # of master machine 1 # of slave machines 29 Processor Intel Core 2 Duo Processor base frequency 2.4 GHz, 2.66 GHz, and 2.8 GHz Memory 1GB to 2GB OS Ubuntu 14.04 LTS (x86) Network B/W 5 to 100 MBPS Total # of files 24, 16, and 11 Size of each file 787 MB Total data size 18.9 GB, 12.6 GB, and 8.7 GB Maximum allowable temperature 20℃-23℃ Environment temperature 28℃
  • 19. Experimental Results: Comparing Modified NSGA-III, PSO, and ACO in Real Testbed 19
  • 20. Experimental Setup for Comparing NSGA-III, PSO, and ACO in SimGrid 20 Parameter Value # of master machines 1 # of slave machines 29, 49 Processor Intel Core 2 Duo PC power Peak: 10 - 400 W, idle: 2.5-100 W, power off: 5 W Network B/W 10 to 100 kbps Total # of files 86, 64, and 36 Size of each file 787 MB Total data size 67.7 GB, 50.4 GB, and 28.3 GB Maximum allowable temperature 20℃-23℃ Environment temperature 28℃ SimGrid version 3.12
  • 21. Experimental Results: Comparing Modified NSGA- III, PSO, and ACO in SimGrid with 30 machines 21
  • 22. Experimental Results: Comparing Modified NSGA- III, PSO, and ACO in SimGrid with 50 machines 22
  • 23. Experimental Results: Validating Modified NSGA-III Performs Better Than NSGA-III 23
  • 24. Experimental Results: Improvement over PSO and ACO in Real Testbed and SimGrid 24 Results for 18.9 GB and 31.5 GB workload
  • 25. Conclusion and Future Work • Cluster administrator needs to decide the right number and right combination of machines, where a many-objective approach is yet to be focused in the literature • We provide a solution for this problem considering cooling energy consumption and empirical performance characterization • We formulate a synergy between NSGA-III and greedy approach to solve this problem • Future work  Analysis with machine failures  Analysis on different factors for cooling energy including room size or cooling capacity 25
  • 26. Thank you Questions are welcome! 26
  • 28. Cluster topology Grid computingCloud computing Architectures for Parallel Computing 28 Large Linux cluster in University of Technology, Germany Cluster machines are connected by a local area network Clouds and grids are geographically distributed Clusters are tightly coupled, whereas clouds and grids are loosely coupled Home-made cluster
  • 29. Background for Parallel Computing 29 Problem Processor The whole problem is distributed in small parts to a single processor Problem_1 Problem_2 Problem_3 Processor_1 Processor_2 Processor_3 Serial computing Parallel computing
  • 30. Related Research Work (cont.) 30 • Multi-objective optimization for virtual machine based schemes in cloud – R. Li et al., IEEE CLOUD, 2016 – Uses Biogeography Based Optimization (BBO) technique for solving multi-objective optimization – Considers only load balancing through resource utilization in VMs placed in cloud architecture Does not integrate any empirical performance characterization of clusters Does not accumulate the impact of cooling energy consumption
  • 31. Related Research Work (cont.) 31 • Stochastic optimization approach – K.M. Tarplee et al., IEEE Transactions on Cloud Computing (TCC), 2016 – Utilizes multi-objective stochastic programming – Considers four objectives for cloud architectures 1. Up-gradation cost 2. Failure rate 3. Power consumption 4. Number of completed tasks Does not consider any specialized many-objective optimization technique Does not integrate any empirical performance characterization of clusters Does not accumulate the impact of cooling energy consumption
  • 32. Improvement in Parallel Computing 32
  • 33. Experimental Results: Improvement over PSO and ACO in Real Testbed • % of improvement over PSO in real testbed: • % of improvement over ACO in real testbed: Workload Time Cooling energy Computation energy CPU usage Memory usage 18.9 GB 74 75 63 127 5 12.6 GB 36 35 25 0 -1 8.7 GB 21 21 20 10 -7 Workload Time Cooling energy Computation energy CPU usage Memory usage 18.9 GB 49 49 58 159 0 12.6 GB -18 -20 10 27 3 8.7 GB 43 42 56 130 1 33
  • 34. Impacts of Selecting A Threshold 34 Threshold value Computation time (mins) Cooling energy (KWh) Computation energy (KWh) 2 11.1 0.43 0.92 4 9.5 0.31 0.66 6 9.7 0.24 0.53 8 10.2 0.32 0.68 Simulation results with 67.7 GB data over 30 machines Threshold value Total no. of solutions in the pareto-front No. of solutions with < 6 machines No. of solutions with >= 6 machines 2 120 34 86 4 120 27 93 6 120 13 107 8 120 14 106 Too much small threshold can result in worse results and introduce unnecessary solutions
  • 35. Improvement of Cooling Energy in Data Centers by Good Design 35
  • 36. Experimental Results: Improvement over PSO in SimGrid • % of improvement over PSO in SimGrid with 30 machines: • % of improvement over PSO in SimGrid with 50 machines: 36 Workload Time Cooling energy Computation energy 67.7 GB 21 13 10 50.4 GB 36 11 10 28.3 GB 43 13 10 Workload Time Cooling energy Computation energy 67.7 GB 38 59 59 50.4 GB 33 36 38 28.3 GB 10 -6 0
  • 37. Experimental Results: Improvement over ACO in SimGrid • % of improvement over ACO in SimGrid with 30 machines: • % of improvement over ACO in SimGrid with 50 machines: 37 Workload Time Cooling energy Computation energy 67.7 GB 43 10 5 50.4 GB 17 8 8 28.3 GB 15 5 0 Workload Time Cooling energy Computation energy 67.7 GB 16 55 55 50.4 GB 41 79 79 28.3 GB 43 87 87
  • 38. Experimental Setup for Validating Modified NSGA-III Performs Better Than NSGA-III 38 Parameter Value # of master machines 1 # of slave machines 29 Processor Intel Core 2 Duo PC power Peak: 10 - 400 W, idle: 2.5-100 W, power off: 5 W Network B/W 10 to 100 kbps Total # of files 86, 64, and 36 Size of each file 787 MB Total data size 67.7 GB, 50.4 GB, and 28.3 GB Maximum allowable temperature 20℃-23℃ Environment temperature 28℃ SimGrid version 3.12
  • 40. Difference between Cluster and Grid? 40 Grid are inherently distributed by its nature over a LAN, metropolitan or WAN. On the other hand, the computers in the cluster are normally contained in a single location. Another difference lies in the way resources are handled. In case of Cluster, the whole system (all nodes) behave like a single system view and resources are managed by centralized resource manager. In case of Grid, every node is autonomous i.e. it has its own resource manager and behaves like an independent entity.
  • 41. Difference between Grid and Cloud? 41 Grid can have only some high computing power. Cluster can have some application level service. FB, Google-> cloud service. Cloud computing is a kind of grid computing; it has evolved by addressing the QoS (quality of service) and reliability problems.
  • 42. Outline of Our Methodology 42 Cooling energy in SimGrid Cooling energy in real test-bed Verify one another Cooling energy integration in SimGrid and test-bed Modification in NSGA-III Selection FilteringCrossover Formulation and modification in NSGA-III to develop a solution Compare Modified NSGA-III ACOPSO Performance evaluation and comparison Problem formulation
  • 43. Our Proposed Many-Objective Optimization Model 43 Following the empirical analysis, we formulate our objective functions as follows:Computation time Energy consumption Cost Resource utilization (CPU and memory usage) Workload should be within machine hard disk size No. of selected machines should be greater than 0. Cooling temperature Machine configuration Workload No. of machines Temperature difference Machine configuration No. of machines Per machine workload Lowest possible temp. Max. possible temp.
  • 44. Our Proposed Modification in NSGA-III Algorithm 44 • With some probability we perform HUX • Only perform when the decision variables are not same Half Uniform Crossover (HUX) • Cluster machines based on each machine property • Make a group in which all machines are in best group • Make a group in which all machines are in worst group • Try to take from best group and not to take from worst group Greedy Clustering Approach • For each machine property try to take the best machine instead of taking the worst machine based on probabilistic possibility Greedy Best and Worst Swap Modified crossover
  • 45. Objectives and Possible Outcomes 45 Formulate a many- objective optimization problem for clusters, and solve it Integrating cooling energy in both real testbed and simulation Test our approach in real testbed and simulation to compare performance with other approaches Particle swarm optimization (PSO) Ant colony optimization (ACO) Modified NSGA-III
  • 46. Some Related Algorithms with Bio-Geography Based Solutions 46 • Biogeography-based optimization (BBO) • Ant colony optimization (ACO) • Particle swarm optimization (PSO) • Artificial Bee Colony Optimization
  • 47. PSO Based Technique 47 1. Task assignment based on avg workload, the deviation of workload, and migration cost. 2. PSO basic: Particle location is updated based on the current location, distance between my current location and my best location, and distance between my current location and global best location. 3. Limitations: Local optimal- can converge fast but local optima. Keep only current optimal point. Solution: Keep a global storage for keeping all the optimal values.
  • 48. Ant Colony Based Technique 48 1. VM assignment based on resource wastage and power consumption. 2. ACO basic: Ants search for food. They put pheromone based on the quality of food and amount. The other ants follow the path based on the concentration of pheromone.
  • 49. BBO 49 • Define the mutation probability and the elitism parameter. • Initialize the population • Calculate the immigration rate and emigration rate for each island. Good solutions have high emigration rates and low immigration rates. Bad solutions have low emigration rates and high immigration rates. • Probabilistically perform mutation for each island. • Replace the worst islands in the population with the previous generation’s elite islands. • If the termination criterion is met, terminate; otherwise, repeat
  • 50. Stochastic Programming 50 Stochastic programming is a framework for modeling optimization problems that Involve uncertainty. Stochastic programming models are similar in style but take advantage of the fact that probability distribution governing the data are known or can be estimated. Stochastic programming studies the case in which some of the constraints or parameters depend on random variables. Stochastic linear programming is an extension of linear programming, where some of the coefficients in the objective and the constraints are random variables
  • 52. Modifications in NSGA-II to get NSGA-III 52 • Crowding distance idea has been replaced. • NSGA-III introduces some reference points idea so that diversity is maintained.
  • 53. Why we use NSGA-III as our baseline algorithm? Introduced in 2014 Specialized for solving many-objective optimization problems Other earlier approaches mostly solve benchmark problems such as DTLZ. They mostly ignore real-world problems Popular NSGA-II only considers crowding distance, which is not a feasible property in solving many-objective optimization problems 53 Popular MOEA/D claims to work for many-objectives, but only considered three objectives only. Also, uses some default parameters
  • 54. Algorithm Comparison 54 Earlier algorithms: O(MN3) -> M = no. of objectives, N = population size. NSGA-II: O (MN2) MOEA/D NSGA-III Improved NSGA-III ACO PSO BBO/Complex
  • 55. The Underneath of Cooling Energy Module Development for Testbed Experiment: Circuit Diagram 55 1. Computation energy 2. Cooling energy 3. CPU usage 4. Memory usage Energy monitor PT, CT LM35 sensor Psensor Energy monitoring module for measuring computation power and cooling power Temperature monitoring module Circuit diagram
  • 56. Experimental Results: SimGrid Energy Module: Simulation Environment in Real Test-bed: Machine Configuration Processor base frequency Memory # of machines 2.4 GHz 1 GB 1 2.4 GHz 2 GB 2 2.66 GHz 1 GB 2 2.66 GHz 2 GB 8 2.8 GHz 2 GB 17 56
  • 57. How we get the equations? 57
  • 58. How we get the equations? (cont.) 58 Using Matlab, we try to get the equations: Computation time equations: (for 6GB data) For order 1: 𝑦 = −0.562𝑥 + 17.2 For order 2: 𝑦 = 0.046𝑥2 − 2.098𝑥 + 26.8 For order 3: 𝑦 = −0.0038𝑥3 + 0.2325𝑥2 − 4.73𝑥 + 36.4 Energy consumption equations: (for 6 GB data) For order 1: 𝑦 = −0.0246𝑥 + .8950 For order 2: 𝑦 = 0.0023𝑥2 − 0.0996𝑥 + 1.3653 For order 3: 𝑦 = −0.0002𝑥3 + 0.0101𝑥2 − 0.2109𝑥 + 1.7722 Energy consumption equations: (for 6 GB data) For order 1: 𝑦 = −1.4165𝑥 + 93.0156 For order 2: 𝑦 = 0.0094𝑥2 − 1.7271𝑥 + 94.9635 For order 3: 𝑦 = 0.0012𝑥3 − 0.0496𝑥2 − 0.8936𝑥 + 91.9160
  • 59. Experimental Results: SimGrid Energy Module: Cooling Energy and Computation Energy Ratio • Cooling energy is around 83% of total energy in real testbed Workload Cooling energy consumption avg (KWh) Computation energy consumption avg (KWh) Total energy consumption avg (KWh) % of cooling energy consumption % of computation energy consumption 12.6 GB 0.367 0.088 0.455 81 19 9.44 GB 0.398 0.069 0.467 85 15 6.3 GB 0.242 0.052 0.294 82 18 3.14 GB 0.146 .030 0.176 83 17 59
  • 60. Experimental Results: SimGrid Energy Module: Cooling Energy and Computation Energy Ratio (Cont.) • Cooling energy is around 65% of total energy in SimGrid Workload Cooling energy consumption avg (KWh) Computation energy consumption avg (KWh) Total energy consumption avg (KWh) % of cooling energy consumption % of computation energy consumption 12.6 GB 0.235 0.125 0.36 65 35 9.44 GB 0.176 0.094 0.27 65 35 6.3 GB 0.117 0.063 0.18 65 35 3.14 GB 0.058 .031 .089 65 35 60
  • 61. Space vs Power Requirement? 61 We have 38*35 = 1330 sq ft room. Hence, we need 23,000 BTU/hr. We have 3 ACs. Each having capacity 1.5 ton = 18,000 BTU/hr. Hence, 3 ACs will together give us 54,000 BTU.
  • 62. Space vs Power Requirement? (cont.) 62
  • 63. Power Requirement for Different States 63 Power State Power Active state 100W-120W Idle state 40W Power-off state 5W Why we do not consider machine on off energy? Answer: It is not a significant amount. We make the whole simulation for about 10-60 minutes. Machine on-off needs small amount of time. Hence, it should consume small amount of energy. Moreover, we only consider the machine runtime energy measurement. Hence, this on-off energy is not present in all the experiments. That is why the overall comparison should not be affected.
  • 64. Experimental Results for Larger Data Size 64
  • 65. Experimental Results: Improvement over PSO and ACO in Real Testbed and SimGrid • % of improvement over PSO comparison for 18.9 GB data in real testbed and SimGrid: • % of improvement over ACO comparison for 18.9 GB data in real testbed and SimGrid: Environment Time Cooling energy Computation energy Real testbed 74 75 63 SimGrid 25 18 18 Environment Time Cooling energy Computation energy Real Testbed 49 49 58 SimGrid 51 82 83 65
  • 66. Experimental Results: Improvement over PSO and ACO in Real Testbed and SimGrid • % of improvement over PSO comparison for 31.5 GB data in real testbed and SimGrid: • % of improvement over ACO comparison for 31.5 GB data in real testbed and SimGrid: Environment Time Cooling energy Computation energy Real testbed 43 43 44 SimGrid 28 56 58 Environment Time Cooling energy Computation energy Real Testbed 28 9 28 SimGrid 36 82 82 66
  • 68. Elbow Points from Our Year-Long Collected Data to Guess The Threshold Value 68 Computation time with 6 GB data: Elbow point: 8 Computation time with 9 GB data: Elbow point: 7 Total energy with 6 GB data: Elbow point: 10 Total energy with 9 GB data: Elbow point: 9
  • 70. Selection Threshold Impacts over Different Cluster Objectives 70 Threshold value Computation time (mins) Cooling energy (KWh) Computation energy (KWh) 2 11.1 0.43 0.92 4 9.5 0.31 0.66 6 9.7 0.24 0.53 8 10.2 0.32 0.68 For a random cluster, we run our modified NSGA-III algorithm with different threshold values with 67.7 GB data.
  • 71. Selection Threshold Impacts over Solutions in The Pareto-Front 71 Threshold value Total no. of solutions in the pareto-front No. of solutions with < 6 machines No. of solutions with >= 6 machines 2 120 34 86 4 120 27 93 6 120 13 107 8 120 14 106 For a random cluster, we run our modified NSGA-III algorithm with different threshold values with 67.7 GB data. Too much small threshold can introduce unnecessary solutions
  • 72. Effects of Different Weights in Machine Selection 72 Weights Computation time (mins) Cooling energy (KWh) Computation energy (KWh) No. of selected machines CPU utilization (%) Memory utilization (%) 25-75 34.8 2.6 0.3 7 57.5 91.2 75-25 23.5 1.76 0.24 11 55.2 91.0 75-25 indicates more weights to select highly configured machines 25-75 indicates less weights to select highly configured machines
  • 73. Experimental Results: Improvement over PSO and ACO in Real Testbed and SimGrid 73 Graphs showing results for 18.9 GB and 31.5 GB workload
  • 74. Experimental Results: Comparing Modified NSGA-III, PSO, and ACO in Real Testbed 74
  • 75. The Underneath of Cooling Energy Module Integration in SimGrid: An Application of Heat Dynamics We adopt the conventional heat formula: 𝑸 = 𝑪 𝒑 × 𝑾 × 𝑫𝑻 𝐶 𝑝 = Specific heat 𝑊 = Mass of airflow per min 𝐷𝑇 = Change in temperature 𝑸 = 𝑪 𝒑 × 𝑪𝑭𝑴 × 𝑫 × 𝑫𝑻 𝐶𝐹𝑀 = Cubic feet per minute 𝐷 = Density of the air Thus, we can write: Putting W = CFM × D, where CFM = cooling air volume per minute, and D = mass of air per volume, we get: 75 𝑪𝑭𝑴 = 𝑸 𝑪 𝒑 × 𝑫 × 𝑫𝑻 •Specific heat of a room remains constant •At a certain place where the pressure is constant, density will also remain constant •There remain other heat losses 𝑪𝑭𝑴 = 𝟏. 𝟕𝟔 × 𝑸 𝑫𝑻 Using 1 CFM consumes 47.82W, we get: 𝑷𝒐𝒘𝒆𝒓 = 𝑪𝑭𝑴 × 𝟒𝟕. 𝟖𝟐
  • 76. The Underneath of Energy Model and Utilization Model Integration for Testbed Experiments 76 1. Computation energy 2. Cooling energy 3. CPU usage 4. Memory usage Energy monitor - PT, CT Psensor Energy monitoring module for measuring computation power and cooling power Real testbed setup
  • 77. The Underneath of Energy Model and Utilization Model Integration for Testbed Experiment (Cont.) 77 PT CT-1 CT-2 Energy Monitor 1 PT CT-1 CT-2 Energy Monitor 2 Cluster voltage Cluster current AC voltage AC-2 current Cluster AC-1 AC-2 AC-3 AC-1 current AC-3 current
  • 78. Experimental Setup for Integration of Energy Module in SimGrid and Real Testbed Parameter Value SimGrid Testbed # of master machine 1 1 # of slave machines 4, 9, 14, 19, 24, and 29 4, 9, 14, 19, 24, and 29 PC power 38, 400-44, 800 MFLOPS 2.4 GHz, 2.66 GHz, and 2.8 GHz Machine power consumption Peak: 100 - 120 W, idle: 40 W, power off: 5 W N/A Memory N/A 1-2 GB Total # of files 4, 8, 12, and 16 4, 8, 12, and 16 Size of each file 787 MB 787 MB Total data size 3.14. 6.3, 9.44, and 12.6 GB 3.14. 6.3, 9.44, and 12.6 GB Maximum allowable temperature 22◦ Celsius 22◦ Celsius Environment temperature 25◦ Celsius 25◦ Celsius 78
  • 79. Experimental Results on Computation Energy 79 Computation energy comparison for 3.14GB data Computation energy comparison for 6.3GB data Computation energy comparison for 9.44GB data Computation energy comparison for 12.6GB data SimGrid computation energy and test-bed computation energy follow a closely similar trend
  • 80. Experimental Results on Cooling Energy 80 Cooling energy comparison for 3.14GB data Cooling energy comparison for 6.3GB data Cooling energy comparison for 9.44GB data Cooling energy comparison for 12.6GB data SimGrid cooling energy and test-bed cooling energy follow a closely similar trend
  • 81. Experimental Results on Total Energy 81 Total energy comparison for 3.14GB data Total energy comparison for 6.3GB data Total energy comparison for 9.44GB data Total energy comparison for 12.6GB data SimGrid outputs are close to real test-bed outputs
  • 82. Exploiting a Synergy between Greedy Approach and NSGA for Scheduling in Computing Clusters Presented by A S M Rizvi This work was supported by the ICT Division, Government of the People's Republic of Bangladesh. A S M Rizvi – University of Southern California Tarik Reza Toha – Bangladesh University of Engineering and Technology Siddharthe Shankar Das – Bangladesh University of Engineering and Technology Sriram Chellappan – University of South Florida A. B. M. Alim Al Islam - Bangladesh University of Engineering and Technology NSysS’2018

Notes de l'éditeur

  1. Source: https://computing.llnl.gov/tutorials/parallel_comp/ Parallel computing is highly used for extensive computation. We can say about galaxy formation simulation or climate change prediction or weather forecast for future or even traffic simulation in busy roads. All these modeling, simulation and experimentation of complex real-world phenomena demands rigorous computing. Hence, we need parallel computing .
  2. Source: https://computing.llnl.gov/tutorials/parallel_comp/ In parallel computing multiple processors work together to make faster the overall computation. In this slide, we can see that in the case of parallel computing we can solve three problems when we have parallel computing. However, for serial computing case, we can solve only one problem.
  3. Clusters can have conflicting outcomes. If we increase the number of machines in a cluster, two conflicting results might appear. One is to decrease the computation time, and the other one is to increase the total cost. Decreasing computation time is desirable, however, increasing cost is not desirable. Hence, we have to make two optimization decisions. First we need to determine the number of machines that need to be operated. Secondly, we need to select which machines should be running. In this work, I try to give a solution for the cluster administrator to get answers of these two questions. http://www.bmpcoe.org/library/books/ex-se/17b31.html http://www.aitechnology.com/products/thermalinterface/ http://www.mtarr.co.uk/courses/ami4817_dti/u03/index.asp
  4. These are the related works with our study. At first we can mention about the work of Tarplee. This work tries to optimize around four objectives. As we know, if any optimization problem has more than three objectives we know it as a many-objective optimization problem. However, in this work, authors use a multi-objective optimization technique to solve a many-objective optimization problem which might not give expected results. Secondly, there are some works with multi-objective optimization in cloud infrastructure. However, cloud and cluster are different in architecture. Moreover, there are some works with optimization in cloud and cluster with PSO and ACO methods. These methods do not integrate any empirical performance characterization. Also, these techniques are not fully aware of cooling energy. Weighted sum method IJGDC means International Journal of Grid and Distributed Computing
  5. These are the related works with our study. At first we can mention about the work of Tarplee. This work tries to optimize around four objectives. As we know, if any optimization problem has more than three objectives we know it as a many-objective optimization problem. However, in this work, authors use a multi-objective optimization technique to solve a many-objective optimization problem which might not give expected results. Secondly, there are some works with multi-objective optimization in cloud infrastructure. However, cloud and cluster are different in architecture. Moreover, there are some works with optimization in cloud and cluster with PSO and ACO methods. These methods do not integrate any empirical performance characterization. Also, these techniques are not fully aware of cooling energy. Weighted sum method
  6. These are the related works with our study. At first we can mention about the work of Tarplee. This work tries to optimize around four objectives. As we know, if any optimization problem has more than three objectives we know it as a many-objective optimization problem. However, in this work, authors use a multi-objective optimization technique to solve a many-objective optimization problem which might not give expected results. Secondly, there are some works with multi-objective optimization in cloud infrastructure. However, cloud and cluster are different in architecture. Moreover, there are some works with optimization in cloud and cluster with PSO and ACO methods. These methods do not integrate any empirical performance characterization. Also, these techniques are not fully aware of cooling energy. Weighted sum method
  7. Look at these two graphs. From the first graph, we make the decision that if we increase the number of machines then the computation time will be decreased. From the second graph, we can say, if we increase the number of machines total energy is decreased. We make more understanding and relations with the cluster environment and cluster objectives and find these objective functions. The first equation denotes computation time. It depends on machine configuration and workload. Similarly, the second equation denotes the energy consumption……
  8. Source: https://computing.llnl.gov/tutorials/parallel_comp/ In parallel computing multiple processors work together to make faster the overall computation. In this slide, we can see that in the case of parallel computing we can solve three problems when we have parallel computing. However, for serial computing case, we can solve only one problem.
  9. Source: https://computing.llnl.gov/tutorials/parallel_comp/ In parallel computing multiple processors work together to make faster the overall computation. In this slide, we can see that in the case of parallel computing we can solve three problems when we have parallel computing. However, for serial computing case, we can solve only one problem.
  10. These are the related works with our study. At first we can mention about the work of Tarplee. This work tries to optimize around four objectives. As we know, if any optimization problem has more than three objectives we know it as a many-objective optimization problem. However, in this work, authors use a multi-objective optimization technique to solve a many-objective optimization problem which might not give expected results. Secondly, there are some works with multi-objective optimization in cloud infrastructure. However, cloud and cluster are different in architecture. Moreover, there are some works with optimization in cloud and cluster with PSO and ACO methods. These methods do not integrate any empirical performance characterization. Also, these techniques are not fully aware of cooling energy. Weighted sum method
  11. These are the related works with our study. At first we can mention about the work of Tarplee. This work tries to optimize around four objectives. As we know, if any optimization problem has more than three objectives we know it as a many-objective optimization problem. However, in this work, authors use a multi-objective optimization technique to solve a many-objective optimization problem which might not give expected results. Secondly, there are some works with multi-objective optimization in cloud infrastructure. However, cloud and cluster are different in architecture. Moreover, there are some works with optimization in cloud and cluster with PSO and ACO methods. These methods do not integrate any empirical performance characterization. Also, these techniques are not fully aware of cooling energy. Weighted sum method
  12. For a random cluster, we run our modified NSGA-III algorithm with different threshold values with 67.7 GB data. (sIMULATION)
  13. Look at these two graphs. From the first graph, we make the decision that if we increase the number of machines then the computation time will be decreased. From the second graph, we can say, if we increase the number of machines total energy is decreased. We make more understanding and relations with the cluster environment and cluster objectives and find these objective functions. The first equation denotes computation time. It depends on machine configuration and workload. Similarly, the second equation denotes the energy consumption……
  14. At first, we will try to formulate a many-objective optimization problem based on the empirical analysis of cluster environment. Then we will try to integrate cooling energy into real testbed and SimGrid. Then we will make some modification in the existing NSGA-III algorithm. We will test our approach in the real testbed and in SimGrid to compare with other approaches.
  15. PT = Potential Transformer CT = Current Transformer
  16. PT = Potential Transformer CT = Current Transformer
  17. PT = Potential Transformer CT = Current Transformer
  18. PT = Potential Transformer CT = Current Transformer
  19. https://dealnews.com/features/Summers-in-the-Air-Conditioning-7-Tips-for-Cooling-on-a-Budget/586898.html http://www.rapidtables.com/convert/power/ton-to-btu.htm
  20. http://www.apc.com/salestools/NRAN-8FL6LW/NRAN-8FL6LW_R0_EN.pdf
  21. http://www.apc.com/salestools/NRAN-8FL6LW/NRAN-8FL6LW_R0_EN.pdf
  22. http://www.apc.com/salestools/NRAN-8FL6LW/NRAN-8FL6LW_R0_EN.pdf
  23. We use different weights to see how they manipulate the experimental results. In the first case, we put weights in such a way that smaller number of machines will be selected. Thus, we should have high computation time and energy. At the same time, lower number of machines should result in a decrease in the maintenance cost as well as increase in the CPU utilization. 25-75 weight indicates our algorithm selects high configured machines 25% time and 75-25 weight means our algorithm selects high configured machines 75% time on average. The following result is for 31.5 GB workload.
  24. Ref.: A. Rizvi et al. Cooling energy integration in simgrid, NSysS, 2017 Ref: http://www.traditionaloven.com/tutorials/power/convert-atm-cfm-atmosphere-to-watts-w.html Ref: http://www.sunon.com/uFiles/file/03_products/07-Technology/004.pdf
  25. PT = Potential Transformer CT = Current Transformer
  26. PT = Potential Transformer CT = Current Transformer
  27. PT = Potential Transformer CT = Current Transformer