integrate 3rd party security solution into cloudstack
DESCRIPTION
TRANSCRIPT
About me
• About me– Project manager from TCloud Computing Ltd, Beijing, China– Apache CloudStack Committer– About TCloud
• Provides cloud platform product/service to Chinese customers• Based on CloudStack
• About this topic– Based on the practice and on-going work– Goal: bring some insights into integration between CS and security
product
Outline
• Background• Three categories of solutions• Integration goal and problems• Architecture• Workflow• Future work
Background
• CS does an excellent job for building clouds• Growing needs for solutions to secure the Cloud
• Seek security solutions to work with CS• Works for both private and public cloud• Multi-tenant ,self-service, • Easy to provision• Resource effective• Easy to integrate
Agent-based
Guest Network
Guest VM 1
Guest VM
Guest VM
Guest VM
CSVirtual Router
Agent
Agent
Agent
Agent
• Widely implemented in physical environment
• Relies on the agent installed on guest OS
• Network connection to pattern/mgmt server
• Not much IAAS level integration • Cons• Instant-on gap• Resource contention
Pattern/MgmtServer
Gateway-based
• Agentless, widely adopted in physical environment
• Detect and prevent by capturing network traffics
• Works fine with simple requirements
• Cons• Network traffic based• Uneasy to provide Inter-user and
inter-VM protection
Router
L3 Core Switch
…… …
Pod 1 Pod N
Offline mode
Available Zone
Inline mode
Hypervisor
Hypervisor-based
• Dedicated for virtual environment• Agentless for user VM, need an
‘agent’ on hypervisor• Capture network traffics• Scan VM memory
• Can work with any complex network environment
User VM
User VM
Module
Brief intro to ElasterShield
• Hypervisor based security product• Features
• Firewall (L2, L3, L4)• IDS/IPS• Application Control• Web server protection• Anti-malware (roadmap)
• For IT admin, no multi-tenant• Rules: defines what triggers event• Profiles: a set of enabled rules
ElasterShield
<VM, Profile/Rules>
Events/Alerts
Brief intro to ElasterShield (cont.)
• ElasterShield Manager (ESM)• Storing rules/profiles• Provide UI/ web service API• Send command to ESVA
• Security Virtual Appliance (ESVA)• One for each hypervisor, stateless• Hypervisor redirects VM’s traffic to
ESVA• Includes an engine to do the
detection/prevention
• Bi-directional communicate between ESM and ESVA
• No impact to user VM if ESVA stops
ESM
Hypervisor
User VM
Hypervisor
User VMESVA
ESVA
Integration goal
• CS Integrates ES• Security as a service, ES as a service provider
• Multi-tenant, defined by admins, subscribed by users, provisioned to VMs
• Security offering == Security Profiles• Export new CS APIs
• Generic to hypervisor-based solution (hopefully)• No change to ES code, keep it as it is• Minimum change to CS 4.0 existing code
Problems : Networking
• Bi-directional communication is required between ESM and ESVA
• Deploy ESVA as a user VM– Connects to guest network– Basic Zone
• Ingress rules– Advanced zone
• Virtual router NAT problem• Another dedicated shared network?
• Solution: Via management network– Works for both basic and adv zone– ESVA as a special VM managed by CS– Connect it to management network
programmatically
ESM
ESVA
User VM
User VM
VRouter
Shared Guest Network
Isolated Guest Network
Public Network
?Management Network
Problems: ESVA management
• Make sure ESVA is running on each hypervisor– When a new host is added, ESVA should be deployed
automatically on the host– When hypervisor is rebooted, ESVA should be started
automatically– Can be destroyed manually and re-deployed
• Similar to SystemVM/VRouter– Managed by CS, transparent to end user– Per hypervisor basis
Problems: VM life cycle conflicts
• User VM State mismatch– ES can detect VM state changes by polling events directly from
Hypervisor• Enable VM’s profile when it gets started• Disable VM’s profile when it gets stopped• Un-assign profile when it gets destroyed
– It does not work. CS removes VM from hypervisor when it gets stopped
– Solution: CS controls profile (un)assignment to VM , based on VM state in the CS
Stopped Destroyed
Destroyed (Missing)
CS VM state
Hypervisor VM state
Removed
Architecture Overview
CloudStack Manager
ES Plug-in ESM
Hypervisor
ESVA
User VM
Hypervisor
ESVA
User VM
ESM APINew CS API
Mgmt network
Mgmt network
Managed by CS
Integrate as a CS Plug-in
• Provide security related APIs• Treat ESM as a resource and send commands to it• ESVA monitor and management• Tailor User VM lifecycle with existing framework• Tailor Hypervisor management
Plug-in
ESM
Reso
urce
ESM
Reso
urce
Rest API
Plug
in A
PI
Data Access Layer
ESVAManagerESVAManager
ESMManagerESMManager
ESMUserVMManagerImplESMUserVMManagerImpl
…
• New manager/service for ESVA and ESM
• Extends several CS manager to tailor process
• A new resource for calling ESM APIs
• DAOs for security related database operation
Workflow: Add ESM
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
2) Connect to ES manager1) Call plug-in API
Database 3) Persist into DB
1) Admin user or external UI calls Plug-in API (addElasterShield)• Parameters include ESM url/account/password• One ESM per zone or per installation?
2) Plug-in calls ESM’s APIs to:• test connection with specified authentication• Get ESM info such like version, license, etc.
3) Plug-in persists the ESM info into CS database
Workflow: Enable Security protection
CloudStack Manager
ES Plug-in ESM
3) Activate ESVA
1 ) Call plug-in API: enableSecurityProtection1) Admin user or external UI calls plug-in
API (enableSecurityProtection)• Cluster wide enable
2) ES Plug-in deploys ESVA• Deploy ESVA on each host of the
cluster• With specified service offering
and template• Start ESVA
3) Activate ESVA• Plug-in calls ESM API to activate
each ESVA
Hypervisor
2) Start ESVA on each hypervisor of the cluster
Hypervisor
ESVA ESVA
Workflow: List Security Offering
CloudStack Manager
ElasterShield Plug-in
ElasterShield Manager
2) Call ESM API, listSecurityProfiles
1) Call plug-in APIlistSecurityOffering
1) User or external UI calls Plug-in API (listSecurityOffering)• Fetch all security offerings with query parameters• Id, name, description
2) Plug-in calls ESM’s APIs to:• List security profiles from ESM
Workflow: Define Security Profiles
CloudStack Manager
ElasterShield Plug-in
ElasterShield Manager
2) Call ESM API listSecurityProfiles
1) Call plug-in APIlistSecurityOffering
• For this Stage, this will be not implemented in plug-in, because it’s product-specific
• Admins use ESM’s UI to define profiles• Users use Plug-in API to list and apply profiles
Admin defines Rules/profiles
Workflow: Apply security offering
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
3) Call ESM API, assignSecurityProfile
1) Call plug-in APIapplySecurityOffering
1) User or external UI calls plug-in API (applySecurityOffering)• Specify VM id and security offering id
2) Plug-in persists <VM, offering> mapping into DB• Address multi-tenant problem
3) Plug-in calls ESM API to assign corresponding profile if VM is running
Database 2) Persist into DB
Workflow: Start Virtual Machine
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
4) Call ESM API, assignSecurityProfile
1) Call CS APIstartVirtualMachine
1) User or UI calls CS API (startVirtualMachine)2) CS checks ESVA status and starts user VM as usual3) After user VM is started, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStart)4) Plug-in call ESM API to assign profile to the VM5) ESM notifies ESVA in the same hypervisor with VM to take effect profile
Database
3) Get profile
HypervisorESVA
User VM
2) Start User VM
5) ESM notifies ESVA to take effect
Workflow: Stop Virtual Machine
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
4) Call ESM API, unassignSecurityProfile
1) Call CS APIstopVirtualMachine
1) User or UI calls CS API (stopVirtualMachine)2) CS powers off user VM as usual3) After user VM is shut down, plug-in gets this VM’s profile
(VirtualMachineGuru.finalizeStop)4) Plug-in calls ESM API to unassign profile to the VM5) ESM notifies ESVA in the same hypervisor with VM to take profile effect
Database
3) Get profile
HypervisorESVA
User VM
2) Stop user VM
5) ESM notifies ESVA to take effect
Workflow: Destroy Virtual Machine
CloudStack Manager
ElasterShield Plug-in
1) Call CS APIdestroyVirtualMachine
1) User or UI calls CS API (destroyVirtualMachine)• VM is destroyed as usual by CS
2) When VM gets expunged, plug-in removes <VM, profile> mapping from DB• VirtualMachineGuru.finalizeExpunge
Database2) Remove from DB
Workflow: Add a new host
CloudStack Manager
ElasterShield Plug-inElasterShield
Manager
5) Call ESM API to activate
1) Call CS APIaddHost
1) User or UI calls CS API (addHost)2) CS add a new hypervisor host as usual3) Deploy ESVA 4) Plug-in persist ESVA info
Hypervisor
ESVA
2) Add host 3) Deploy ESVA
Workflow: Maintain a host
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
3) Call ESM API to de-activate ESVA
1) Call CS APIprepareHostForMaintenance
1) User or UI calls CS API (prepareHostForMaintenance)2) Stop ESVA on it
• update ESVA status into DB3) Plug-in call ESM API to deactivate ESVA
• unassign security profiles to VMs that cannot be migrated
Hypervisor
ESVA
2) Power off ESVA
Workflow: Query Events/Alerts
CloudStack Manager
ElasterShield Plug-in
1) Call plug-in APIquerySecurityEvents
1) User or external UI calls plug-in API (querySecurityEvents)• With query parameters such as time, VM id, user id, etc
2) Plug-in calls ESM’s API to fetch events/alerts
2) Call ESM API
ElasterShield Manager
Workflow: ESVA upgrading
CloudStack Manager
ElasterShield Plug-in ElasterShield Manager
2) Call ESM API to upgrade
1) Call plug-in APIupgradeESVA
1) Call plug-in API to upgrade ESVA , with a URL parameter2) Plug-in forward the API to ESM3) ESM notifies all ESVA to upgrade4) ESVA downloads upgrading package from the URL specified5) ESVA upgrade itself, it still functions during the upgrading.
Hypervisor
ESVA
3) Notify ESVA
Web server
4) Download upgrade package
Summary & Future work
• Summary– Develop plug-in with existing CS framework– Leverage CS to manage security virtual appliance– Tailor process
• Future work– Make the security offering more generic– Make it easier for CS to introduce a new system
VM