holistic security for openstack clouds
TRANSCRIPT
Holistic Security for OpenStack Clouds
Major HaydenPrincipal Architect, Rackspace
@majorhayden
Photo credit: bastiend (Flickr)
Image credit: Wikipedia
Security feels like this
Image credit: Wikipedia
Securing complex systems createsmore challenges
Securing OpenStack can feel liketaking a trip to the Upside Down.
It doesn’t have to be that way(even with something as complex as OpenStack)
Image credit: Pixabay
The key istaking the right approach to secure a complex system.
Major HaydenPrincipal Architect
● At Rackspace since 2006
● Working on OpenStack since 2012
● Focused on information security for Rackspace Private Cloud
● Fedora Linux contributor; Fedora Security Team and Server Working Group member
● Has a terrible domain name purchase habit(please, no ideas for domain names today)
Holisticcharacterized by comprehension of the
parts of something as intimately interconnected and explicable only by
reference to the whole
-- Oxford English Dictionary
The holistic approach for humans considers a person to be made of a body, a mind, and a spirit.
Image credit: Pixabay
The holistic approach for OpenStack considersa cloud to be made of servers, software, and a business goal.
A holistic approach to security involves people, processes, and technologies working in tandem.
“The whole is greaterthan the sum of its parts,especially in the case of OpenStack.”
-- (partially) Aristotle
Image credit: Wikipedia
How does this apply to securing an OpenStack cloud?
Let’s do a quick security refresher.
Assume that attackerswill get inside eventually.
Image credit: Pixabay
Attackers are on offense.They can be wrong many times.
Defenders can only be wrong once for a breach to occur.
Securing only the outer perimeteris not sufficient.
We must secure our OpenStack cloud.We need to go deeper.
We just bought an expensive firewall for the perimeter. Isn’t that enough?
(no caption necessary)
Build small security improvementsat multiple layers.*
* This is the cornerstone of defense-in-depth.
Individually, these changes may not seem to have much value.
All of these changes create a strong, valuable security strategy when they are added together.
Let’s get to the good stuff.
Image credit: Pexels
Work from the outside in(just like you would at a fancy dinner)
Image credit: Wikipedia
Four layersOuter perimeter
Control and data planes
Control plane deep dive:OpenStack services and backend services
OpenStack services deep dive
Image credit: imageme (Flickr)
OUTER PERIMETER SECURITY GOAL:Convince your attackers that it’s easier to attack someone else’s cloud
Key concepts
Make it expensive for attackers to breach your perimeter defense
When they do make it through, ensure that you know about it immediately
Perimeters usually have openings on the outside and inside -- secure both of them
Tactical objectives
Require a VPN for access from external networks
Segregate internal networks using a firewall or an internally-facing VPN
Monitor all logins (successful and unsuccessful) for unusual activity
Track bandwidth usage trends using netflow data
Secure the perimeter
VPN
Internet Corporate network
Firewall
Log collector Alert system
Netflow collector
Auth system
Control and data plane
Control planekeystone, nova, glance,
cinder, neutron, horizon, rabbitmq, mysql,
memcached
Data planeHypervisors and
tenant-built items (VMs, containers, networks,
storage)
CONTROL/DATA PLANES SECURITY GOAL:Keep the inner workingsof your OpenStack cloud separated fromtenant infrastructure
Key concepts
Tenant infrastructure should have extremely limited access to the control plane, and vice versa
A misconfigured tenant VM could open a wide hole in your secure network
Protect your cloud from VM exit exploits that allow attackers to gain hypervisor access
Tactical objectives
Separate control plane, hypervisors and tenant infrastructure with VLANs and strict firewall rules (and monitor dropped packets)
Use SELinux or AppArmor on hypervisors to reduce the impact of VM and container exit exploits
Hypervisor
Linux Security Module refresher
Three popular implementations: SELinux, AppArmor, and TOMOYO
sVirt (in libvirt) ensures that all processes are labeled properly (SELinux) or have profiles configured (AppArmor)
VM exit exploits are confined in most situations
Tenant VM
Storage Network
Linux Security Module
Do not disableSELinux or AppArmoron your hypervisors.
(Seriously. Leave it enabled.)
Control plane deep dive:OpenStack and backend services
Image credit: Wikipedia
CONTROL PLANE SECURITY GOAL:Heavily restrict lateral movement and restrict access to the “crown jewels”
“crown jewels” are the databases and message queuesin your OpenStack cloud
Control plane deep dive
OpenStack serviceskeystone, nova, glance,
cinder, neutron, horizon
Backend servicesmysql, rabbitmq,
memcached, syslog
The “crown jewels” are hereThe map to the “crown jewels” is here
Key concepts
Allow the least amount of access possible from the OpenStack services to backend services
Further restrict access to specific ports, sources, and destinations
Deploy services into containers to apply fine-tuned network and process restrictions
Tactical objectives
Use a load balancer or firewall to create a “choke point” between OpenStack and backend services
Monitor messaging and database performance closely to look for anomalies or unauthorized access
Use unique credentials for each MySQL database and RabbitMQ virtual host
OpenStack services deep dive
Image credit: Wikipedia
OPENSTACK SERVICES SECURITY GOAL:Know what valid communication looks like and alert oneverything else
OpenStack has many (predictable) interactions
Key concepts
OpenStack services are heavily interconnected, but the connections are predictable
Limit access between OpenStack services and monitor any invalid questions
Tactical objectives
Use iptables rules to limit access between OpenStack services; alert on any invalid connections
Give each service a different keystone service account (with different credentials)
Monitor closely for high bandwidth usage and high connection counts
Let’s wrap up
Analyze.Isolate.Monitor.Repeat.
These small security changesadd up to a strong defense
Image credit: Wikipedia
Try OpenStack-Ansible
OpenStack-Ansible deploys enterprise-grade OpenStack clouds using Ansible.
Security and reliability are two of the core priorities for the project. Most of the security changes in this talk are already implemented.
Learn more: http://bit.ly/openstack-ansible
RACKSPACE PRIVATE CLOUDPOWERED BY OPENSTACK®
Learn more about ourproven operational expertise,
industry-leading reliability,and OpenStack Everywhere.
Join us at the Rackspace booth (A22)in the OpenStack Marketplace.
RACKSPACE INVENTED OPENSTACK® – NOW WE'RE PERFECTING IT
Thank you!Major Hayden
Photo credit: bastiend (Flickr)