In this deck from the Stanford HPC Conference, Thomas Francis from the UberCloud and Mohan Potheri from VMware present: On-Demand HPC/AI with UberCloud and VMware. "HPC Cloud solution providers like UberCloud can combine powerful virtualized HPC hardware with superior management capabilities to help IT dynamically provision and manage HPC clusters, deploy simulation software applications, monitor spend, and give them complete control. In this solution we combine the capabilities of the VMware platform, with the solution provided by UberCloud which leverages the automation capabilities of Terraform with the unique packaging and distributing capabilities of docker-based HPC containers to dynamically deploy HPC applications on the vSphere platform. By sandboxing applications in containers, applications become portable, and the same application can be deployed to different VMs without requiring creating multiple variations of VM images for every application. It speeds up development and testing, yielding significant acceleration to the change management process. As a proof of concept, we applied our container-based self- service environment to "Deep Learning for Fluid Flow Prediction in the HPC Cloud". Watch the video: https://youtu.be/HfQ3wUT6K1k Learn more: https://blogs.vmware.com/apps/2019/10/on-demand-hpc-on-vsphere-with-ubercloud.html and http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. 1Confidential │ ©2020 VMware, Inc.
Agile HPC with
VMware Cloud Foundation &
UberCloud
Thomas Francis, UberCloud
Mohan Potheri, VMware

2. Confidential │ ©2020 VMware, Inc.
Agenda
2
• Traditional HPC architecture
• Path to agile HPC
• vSphere announcements for Kubernetes
• Containers and HPC with UberCloud
• Example deployment

3. Confidential │ ©2020 VMware, Inc.
Traditional HPC Architecture
• Login / Head Node
• Scheduler
• Storage Gateway
• NFS Storage
• Compute Nodes
• Accelerator Nodes
• Parallel Filesystem (Lustre)
• High Speed Interconnects

4. Confidential │ ©2020 VMware, Inc. 4
​ Variety of Devices (CPU, RAM, Storage,
Networking/RDMA, GPU, FPGA)
​ Workload Management Options (Moab, PBS,
Grid Engine, Kubernetes)
​ Dormant infrastructure that is only refreshed
every 3 years
Slower innovation
Complex resource and application management
Higher Costs
Siloed operations
80% of budget spend on keeping the lights on
​ Complex Infrastructure ​ Business Impact
Enterprise HPC Today

5. Confidential │ ©2020 VMware, Inc.
Virtual Machine Benefits
hardware
hypervisor
VM
OSApp
Virtual Machines offer:
• Multi-cluster aggregation
(heterogeneity)
• Multi-tenant data security
• Fault isolation
• Reproducibility
• Fault resiliency
• Dynamic load balancing
• Performance
hardware

6. Confidential │ ©2020 VMware, Inc. 6
The Road to a Modern Data Center Architecture for HPC
6
HW
Legacy
Apps
Cloud
Native
Apps
From traditional Data Center to unified, scalable Data Center operations…
Combining existing tools with new data center technologies
• VMware VCF virtualizes Data Center resources
• Kubernetes is the orchestrator
• Docker Containers are the new kernel
• UberCloud Containers bridge to HPC
• Enabling HPC and AI to converge
• Preserving existing CAPEX investment
• Addressing rapidly evolving business requirements
• Accelerating the HPC application lifecycle

7. Confidential │ ©2020 VMware, Inc. 7
VMware vSphere 7 & VCF 4: Unifying Kubernetes and VMs vSAN
IT OperatorDeveloper
Kubernetes is embedded into the control plane of vSphere
vCenterServer
AI/ML Business
Critical
DB & Analytics Time-critical
Namespaces
VMware Cloud Foundation Services
Tanzu Runtime Services Hybrid Infrastructure Services
VM Service
Storage Service
Registry Service
vSphere Pod Service
Network Service
Hybrid Mobility Service
Tanzu Kubernetes Grid Service
vSphere vSAN NSXvRealize

8. Confidential │ ©2020 VMware, Inc.
VMware + UberCloud Provide:
• Hybrid and multi-clouds
• HPC, Big Data, AI,
Enterprise apps and
microservices
• Commercial, OpenSource,
in-house codes
• CAE, Life Sciences, EDA,
Finance, Oil & Gas, …
Choice AgilitySecurity
• Unified operations
• Rapid delivery and
response
• Cost efficiency
• HW flexibility
• Package once, move to
any cloud
• On-demand Scaling of
resources
• High Availability & Auto-
Recovery
• Safer applications
• Governance
• Chain of custody
• Threat mitigations
• Multitenancy
• HW+Network+SW isolation
• Additional Login/Password
for app container protection
• iLO with Trusted Platform
Module offer enhanced
security

9. Confidential │ ©2020 VMware, Inc.
Containers: The New Application Deployment Standard
Application
Operating System
Run Time
Middleware
Infrastructure Integration
• Portable across hybrid infrastructure
• Continuous updates
• Can bridge HPC & Enterprise IT
Cloud
Infrastructure

10. Confidential │ ©2020 VMware, Inc.
Automated Container Factory: CI/CD to build containers
• Templatized HPC containers
• Automated Build
• Integrated Testing
• Controlled Versioning

11. Confidential │ ©2020 VMware, Inc.
Deployment Capabilities On Premise Off Premise* Public Cloud
Interactive Access
(allowing full engineering workflow)
✔ ✔ ✔
Batch Access ✔ ✔ ✔
Data Isolation
(by user/group)
✔
(per cluster)
✔
(per cluster)
✔
(per cluster)
Access Security ✔
(inside network)
✔
(subnet & VPN)
✔
(VPN)
Integrated Security ✔ ✔ ✔
(Automated Threat Scanning, Controlled Code Deployment, Encrypted Communications, Role-based access
control, Policy Management and Enforcement, Image Signing)
RDMA Support ✔ ✔ ✔
GPU Support ✔ ✔ ✔
Cluster Resizing ✔ ✔ ✔
Multi-User Access ✔ ✔ ✔
On-Demand Access ✔ ✔ ✔
Single Image on platforms ✔ ✔ ✔
Certification ✔ ✔ ✔
(Certified plug-ins and Infrastructure, Certified ISV apps as containers, Certified professionals)
Automation ✔ ✔ ✔
(Built-in app reliability/High Availability, Policy-based automation, Auto health check, Auto Load Balance/Deploy/
Recovery, Containerized Application Lifecycle Management, Automated Deployment/DevOps)
Orchestration ✔
(UGE, K8s, others)
✔
(UGE, K8s, others)
✔
(UGE, K8s, others)
Add, remove compute nodes (containers), or
terminate service.
Execute Docker Run to start Login Container
and Compute Containers
Configure multi-node container environment
Load UberCloud Docker Container Image on
all VMs
Pull UberCloud Docker Container Image from
Registry to Cluster Storage
Install Docker EE (Runtime)
Configure virtual network, devices, and
storage
Create new cluster in vSphere and add hosts
login and compute resources
Deploy VCSA and Configure vCenter
(management server(s))
Deploy ESXi on servers
(management and compute)

12. Confidential │ ©2020 VMware, Inc. 12
Video Demo

13. Confidential │ ©2020 VMware, Inc.
HPC Case Studies
50 HPC Case Studies with UberCloud’s HPC Containers since 2015
• Selection of 21 Containerized Engineering Case Studies:
https://info.theubercloud.com/ubercloud-container-compendium
Using HPC Software Containers based on Docker

14. Confidential │ ©2020 VMware, Inc.
Thank You
14