Spark Summit 2016 talk by Jonathan Farland (DNV GL)

1. Breaking Down Analytical and
Computational Barriers in
Energy Data Analytics
Jonathan Farland
DNV GL Energy

2. • IntroductionsWho is DNV GL?
• Overview
• Data Science
Energy Analytics
• Demonstration
Statistical
Computing Pilot
• Plans
Concepts in
Development
• DiscussionQ&A
Agenda

3. DNV GL
Policy Production
Transmission &
Distribution
Use

4. 400
offices
100
countries
16,000
employees
150
years

5. Policy, Advisory and Research
Demand Side
Management
Energy
Analytics
Load Research
Services
Market Research
and Program
Evaluation

6. Electricity Distribution Grid

7. Electricity Distribution Grid
Transmission Distribution ConsumerGeneration Transmission Distribution ConsumerGeneration
Wind
Farms
Photo
Voltaic
Aggregated
Utility Scale
2-50 MW
Utility
Scale
100kW-2MW
Distributed
Scale
25kW-100kW
Residential
Commercial
& Industrial
DistributionTransmissionGeneration
Bulk
Storage
> 50 MW
Distribution
System
End User
Bulk
System
PhotovoltaicWind
Farms

8. The Rise of The Smart Grid

9. Energy Data Science
Jonathan Farland
DNV GL Energy

10. Terminology

11. 11

12. 12

13. 13

14. 14

15. 15

16. 16

17. Forecasting Approaches
– Similar Day Matching
– Statistically Adjusted Engineering
(SAE)
– Univariate Time Series (ARIMA)
– Multiple Linear Regression
– Econometric
– Machine / Statistical Learning
– Semiparametric Regression
– Artificial Neural Networks
– Fuzzy Logic
– Support Vector Machines
– Gradient Boosting

18. Additive Semiparametric Model
𝑦" = ℎ 𝑡𝑖𝑚𝑒 + 𝑓 𝑤𝑒𝑎𝑡ℎ𝑒𝑟 + 𝛼 𝑏𝑒ℎ𝑎𝑣𝑖𝑜𝑟 + 𝜀"
Short Term
Electricity
Demand
Time of Year
Prevailing
Atmosphere
Conditions
Recent Demand
Behavior

19. Additive Semiparametric Model
𝑦" = ℎ 𝑡𝑖𝑚𝑒 + 𝑓 𝑤𝑒𝑎𝑡ℎ𝑒𝑟 + 𝛼 𝑏𝑒ℎ𝑎𝑣𝑖𝑜𝑟 + 𝜀"
Short Term
Electricity
Demand
Time of Year
Prevailing
Atmosphere
Conditions
Recent Demand
Behavior

20. Emerging Technologies
Photovoltaic
Cells (e.g., Solar)
Electric Vehicles Storage Wind
Energy Efficiency Demand
Response

21. 21
Load Shifting: Electric Vehicles
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Demand(kW)
Hour Ending
Standard Rate Electric Vehicle Rate

22. 22
-
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Load(kWh)
Hour Ending
Forecasted - DR Reduction Forecasted - DR Baseline
Forecasted - DR Impacted Load Actual DR - Reduction
Load Reduction: Demand Response

23. Databricks + Spark Pilot

24. Benefits of Big Data from Advanced Metering Infrastructure
ü A deeper understanding of demand and thereforehuman
behavior (think energy efficiency)
ü Cost effective operating costs
ü Real-timenotification of power outages
ü ImprovedSystem Planning and Reliability
ü Allows for integration of disruptive technologies like
Electric Vehicles
Statistical Computing Pilot

25. Pilot
Design
Data
Generating
Process
Analytics
Statistical Computing Pilot

26. Energy
Consumption
Climatic
- Temperature
- Humidity
- Wind Speed
- Solar
Demographic
Firmographic
Economic
Financial
Energy
Efficiency
Program
Tracking
Grid
Infrastructure
Statistical Computing Pilot
Data Diversity

27. Key Focus Areas
Performance Scalability
Granularity

28. Going Further

29. Use Cases

30. DEMONSTRATION

31. VISION OF THE FUTURE

32. Current Concepts in Development
Weather Normalization at Scale (e.g., California)
Real-time Energy Forecasting Using Statistical Learning and Spark Streaming API
Real-time Customer Sentiment Analysis
Grid Reliability Analysis
Cybercrime Protection of Electricity Grids

33. THANK YOU.
Jonathan Farland – DNV GL
jon.farland@dnvgl.com
Andrew Stryker – DNV GL
Andrew.stryker@dnvgl.com