2. About me
Jakub Pavlík
• Cloud Platform Engineer
• 3 years in Cloud
• 2 years in OpenStack
3. tcp cloud a.s.
• #1 deployer of OpenStack solutions in Czech Republic
• 2 years experiences with OpenStack
• Basic two products
• TCP Private Cloud – on premises, hosted in arbitrary
place
• TCP Virtual Private Cloud (TCP VPC)– hosted in TCP,
pay per use, tenant = customer (Contrail based)
4. TCP VPC Contrail/OpenStack Facts
• Contrail 1.05 with Havana on CentOS 6.4
• High Availability OpenStack architecture based on
Corrosync/Pacemaker, HAProxy and MySQL Galera
• 3x OpenStack Controllers, 2x Contrail Controllers – all
virtual machines except one OpenStack Controller
(because of Fibre Channel)
• Not deployed by Fabric, but by SaltStack
• Own solution for monitoring and metering/billing
5. Contrail/OpenStack Deployment
Not Fabric – Why?
• Fabric is great for showcase or standalone deployment
• Fabric is not suitable for deploying OpenStack, because each
deployment is different (cinder backend, glance backend)
• Fabric does not deploy HA OpenStack for now
• OpenStack config files needs to be extended for other items
• Each company prefers different configuration management
(puppet, salt, chef)
• Fabric HA deployment of Contrail is not HA
• Next releases are better
Recommendations
• There should be guide for integration Contrail with existing
OpenStack and maybe steps for manual configuration
without Fabric
6. Deployment issues with 1.05
Fabric HA is not HA
• Other config nodes refers to ip address of first node (e.g. discovery
service)
• There is no VIP address in HAProxy
Bug in nova.conf
• service_down_time = 100000
Bad configuration for nova live migration
• Nova.conf and libvirtd.conf must include parameters for doing live
migration
No options for cinder and glance backend
• Config optimization and options inside of Fabric are missing
8. OpenStack HA
TCP VPC
VIP
HAProxy HAProxy HAProxy
Contrail Config
with Analytics &
WebUI
Contrail
Control
Contrail
Database
Openstack
Controller
MySQL RabbitMQ
Contrail
Database
Openstack
Controller
GALERA
Zookee
per
Cassandra
Zookee
per
Cassandra
MySQL RabbitMQ
Contrail
Control
Contrail
Database
Cassandra
Openstack
Controller
Zookee
per
MySQL RabbitMQ
Contrail Config
with Analytics &
WebUI
Bond Interface
Pacemaker
Corosync
Contrail Config
with Analytics &
WebUI
Pacemaker
Corosync
12. HA issues
Restoring BGP peering after failure one of control
nodes
• Sometimes joining control node after failure
desynchronize BGP peering and whole
floating ips are down for 20-60 sec.
• Solved by deploying three config nodes and
two control nodes
Configuration files
• Problems can be solved better configuration.
• Native HA in Contrail release 1.10
13. General issues
Cassandra problem
• Flow records grow up very quickly. /etc/contrail/vizd_param
ANALYTICS_DATA_TTL=8
MTU configuration at MX and QFabric
• MTU size for North-South communication is set to
1550 QFabric and 1554 MX (at the end we had to
increase to 9000 MTU)
Bug in instance snapshot
• qemu-img convert has bad parameters on CentOS
14. General issues
Nova floating ip attaching
• nova floating-ip-associate does not work –
causes problems with existing PaaS tools
Details for almost all issues are at user mailing list
• All issues was discussed at list by me and
most of them are successfully solved.