collaborative network security in multi-tenant data center for cloud computing

13
TSINGHUA SCIENCE AND TECHNOLOGY ISSN ll 1007-0214 ll 09/10 ll pp82-94 Volume 19, Number 1, February 2014 Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing Zhen Chen , Wenyu Dong, Hang Li, Peng Zhang, Xinming Chen, and Junwei Cao Abstract: A data center is an infrastructure that supports Internet service. Cloud computing is rapidly changing the face of the Internet service infrastructure, enabling even small organizations to quickly build Web and mobile applications for millions of users by taking advantage of the scale and flexibility of shared physical infrastructures provided by cloud computing. In this scenario, multiple tenants save their data and applications in shared data centers, blurring the network boundaries between each tenant in the cloud. In addition, different tenants have different security requirements, while different security policies are necessary for different tenants. Network virtualization is used to meet a diverse set of tenant-specific requirements with the underlying physical network, enabling multi-tenant datacenters to automatically address a large and diverse set of tenants requirements. In this paper, we propose the system implementation of vCNSMS, a collaborative network security prototype system used in a multi-tenant data center. We demonstrate vCNSMS with a centralized collaborative scheme and deep packet inspection with an open source UTM system. A security level based protection policy is proposed for simplifying the security rule management for vCNSMS. Different security levels have different packet inspection schemes and are enforced with different security plugins. A smart packet verdict scheme is also integrated into vCNSMS for intelligence flow processing to protect from possible network attacks inside a data center network. Key words: data center network; network security; software defined network; collaborative network security; multi- tenant; network virtualization; intelligent flow processing; cloud computing 1 Introduction A cloud data center is an infrastructure that supports Zhen Chen and Junwei Cao are with Research Institute of Information Technology and Tsinghua National Lab for Information Science and Technology, Tsinghua University, Beijing 100084, China. E-mail: fzhenchen, jcaog@tsinghua.edu.cn. Wenyu Dong is with the Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China. Hang Li is with Department of Computer Science and Technology, PLA Univ. of Info. & Eng., Zhengzhou 450001, China. Peng Zhang is with Department of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China. Xinming Chen is with Department of Electrical and Computer Engineering, University of Massachusetts, MA 01003, USA. To whom correspondence should be addressed. Manuscript received: 2013-12-18; accepted: 2013-12-24 Internet services. A cloud data center may be defined from a variety of perspectives, and the most popular ones are categorized by IaaS, PaaS, and SaaS proposed by the NIST [1] and include public cloud, private cloud, hybrid cloud, and other different categories. Other categories include computing, networking, storage from a system’s perspective or using (in use), archiving (at rest), and transmission (in motion) from a data perspective. Specific to the cloud network, there are different characteristics of a cloud (remote access), within a cloud, and between cloud networks. The debut of VMware NSX provides the virtualization of networks with Software-Defined Network (SDN) inside a data center. The Google B4 network [2] also uses an OpenFlow-based SDN [3, 4] to implement all the interconnections among cloud data centers in different locations.

Upload: junwei

Post on 25-Dec-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

TSINGHUA SCIENCE AND TECHNOLOGYISSNl l1007-0214 l l09 /10 l lpp82-94Volume 19, Number 1, February 2014

Collaborative Network Security in Multi-Tenant Data Center forCloud Computing

Zhen Chen�, Wenyu Dong, Hang Li, Peng Zhang, Xinming Chen, and Junwei Cao�

Abstract: A data center is an infrastructure that supports Internet service. Cloud computing is rapidly changing

the face of the Internet service infrastructure, enabling even small organizations to quickly build Web and mobile

applications for millions of users by taking advantage of the scale and flexibility of shared physical infrastructures

provided by cloud computing. In this scenario, multiple tenants save their data and applications in shared data

centers, blurring the network boundaries between each tenant in the cloud. In addition, different tenants have

different security requirements, while different security policies are necessary for different tenants. Network

virtualization is used to meet a diverse set of tenant-specific requirements with the underlying physical network,

enabling multi-tenant datacenters to automatically address a large and diverse set of tenants requirements. In this

paper, we propose the system implementation of vCNSMS, a collaborative network security prototype system used

in a multi-tenant data center. We demonstrate vCNSMS with a centralized collaborative scheme and deep packet

inspection with an open source UTM system. A security level based protection policy is proposed for simplifying

the security rule management for vCNSMS. Different security levels have different packet inspection schemes and

are enforced with different security plugins. A smart packet verdict scheme is also integrated into vCNSMS for

intelligence flow processing to protect from possible network attacks inside a data center network.

Key words: data center network; network security; software defined network; collaborative network security; multi-

tenant; network virtualization; intelligent flow processing; cloud computing

1 Introduction

A cloud data center is an infrastructure that supports

� Zhen Chen and Junwei Cao are with Research Instituteof Information Technology and Tsinghua National Labfor Information Science and Technology, TsinghuaUniversity, Beijing 100084, China. E-mail: fzhenchen,[email protected].�Wenyu Dong is with the Department of Computer Science and

Technology, Tsinghua University, Beijing 100084, China.�Hang Li is with Department of Computer Science and

Technology, PLA Univ. of Info. & Eng., Zhengzhou 450001,China.� Peng Zhang is with Department of Electronic and Information

Engineering, Xi’an Jiaotong University, Xi’an 710049, China.�Xinming Chen is with Department of Electrical and Computer

Engineering, University of Massachusetts, MA 01003, USA.�To whom correspondence should be addressed.

Manuscript received: 2013-12-18; accepted: 2013-12-24

Internet services. A cloud data center may be definedfrom a variety of perspectives, and the most popularones are categorized by IaaS, PaaS, and SaaS proposedby the NIST[1] and include public cloud, private cloud,hybrid cloud, and other different categories. Othercategories include computing, networking, storagefrom a system’s perspective or using (in use), archiving(at rest), and transmission (in motion) from a dataperspective. Specific to the cloud network, there aredifferent characteristics of a cloud (remote access),within a cloud, and between cloud networks. Thedebut of VMware NSX provides the virtualizationof networks with Software-Defined Network (SDN)inside a data center. The Google B4 network[2] alsouses an OpenFlow-based SDN[3, 4] to implement all theinterconnections among cloud data centers in differentlocations.

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 83

The challenge posed by SDN is the dynamiccharacteristic of network boundaries that is the “twin”of the network topology with expanded flexibilityprovided by virtualization. In other words, the originalstatic, natural, and physical boundaries within thetraditional network are replaced by the dynamic andvirtual logical boundaries of SDN.

1.1 State-of-the-art network security in data centernetworks

Traditional security devices such as Firewalls, IDS,WAF, and other devices are deployed with theMiddleboxes model inside and outside networks. Withthe development of cloud computing technology, thedeployment of Middleboxes is facing new challengesin the large-scale data center network environment[5].

(1) In multi-tenant cases, the network boundariesare blurring.

With the increase of Internet users, the data centernetwork topology becomes more complicated. Multipletenants save their data in the same server, and thesame tenant may store data on different serverswith multiple hot backups, causing the networkboundaries between each user to become blurred, asopposed to the set boundaries found in traditionalphysical isolation. The original static, natural, andphysical boundaries within the network are replacedby dynamic and virtual logical boundaries. Hence,network security within the cloud will be moredependent on dynamic deployment, configuration,and management of security policies and securitycomponents, and more dependent on the networksecurity system for flow and traffic awareness, decision-making, and response. Undoubtedly, this causesnetwork management to become more complex. Howto ensure network security is a challenge in such acomplex network environment.

(2) The deployment location of Middleboxes alsohas new changes.

In traditional networks, the data of several hosts arefrom the same gateway, and the entire network mayhave several gateways. So long as security devicesare deployed in network vantage points and ensuredata through the gateway is safe and reliable. In thedata center network, the physical vantage points havebeen replaced by virtual logical gateways. To protectthe security of virtual logical boundaries betweentenants, Firewall, IDS/IPS, and other devices arerequired to collaborate with the traffic controller to

adapt to performance and safety requirements of eachtenant or security domain, provide security of dynamicboundaries caused by virtual machine migration, andmeet the dynamic security requirements of virtualmachines[6-10]. Therefore, to meet these requirements,how to properly deploy these security devices is also animportant issue.

(3) Security requirements for different tenants aredifferent.

Since different tenants have different securityrequirements, it is necessary to create different securitypolicies for different tenants, and this is undoubtedlya major challenge for traditional security devices. Thetraditional approach is to set rules for the devicewhere data passed through. Obviously, in a data centernetwork, this strategy no longer applies, and now wehave to address the question how to meet diversesecurity requirements when multiple tenants’ data passthrough the same security device and how to provideeffective enforcement.

(4) The migration of the virtual machine results inthe switchover of security domain.

To meet the security needs of a single tenant, it needsto configure multiple security devices to control trafficand thus completely implement the tenant’s securitypolicy. When service host migrates to other locationsin the network, the topology of the network changesand appropriate security policies also migrate with thetenant host. The original security configuration of thesecurity device will no longer work, the migration tothe new security domain also requires tenants on thatserver to apply some configuration updates. Therefore,to protect the mapping of the security policies from thelogical network to the physical network and maintain itscorrectness and consistency will be a major challenge tothe control function of a data center network.

1.2 Software-defined network in data centernetwork

To enable network virtualization, SDN is one of thesupporting technologies to build cloud data centervirtual networks. Cloud data center virtual networksneed to ensure tenants’ security domains have completeand isolated network boundaries. This is not asimple network security technology, because the virtualnetwork itself provides services for multiple tenantsor basic virtual private cloud services that should beguaranteed in the implementation of the network. Dueto the different virtual machines of different tenants

84 Tsinghua Science and Technology, February 2014, 19(1): 82-94

sharing the same physical resources, system securityguarantees such as preventing the virtual machine fromescaping its boundaries are also the issue of networkvirtualization security.

Comparing software-defined network security (e.g.,OpenFlow based on SDN) with a traditional physicalnetwork, the changes are in the control mechanisms ofswitches, routers, and other forwarding devices. Theforwarding devices worked according to a flow tableissued by the controller, thereby more efficient andless costly. The controller also collects network statusinformation, discovers network topologies, checks thenetwork forwarding policies, and generates and releasesnew flow table according to status update.

Thus, handling of the first network packets headerof a flow should not happen in the gateway, but in thecontroller. The traditional Access Control List (ACL)scheme or network packet filtering Firewall shouldbe deployed in the controller. However, the controllerusually only receives the first packet header and cannotperform an deep inspection of the network flows. Thestateful Firewalls and deep packet inspection that filterpacket payloads should be deployed in the data planeor in the data plane and control plane. Therefore,the network access control deployed on the physicalnetwork gateway should still be reasonable for a logicalgateway.

Through Openflow protocols and controllers, datacenter network security based on SDN addresses theproblems of Middleboxes positioning and separation ofsecurity rules by setting a flow table and guiding trafficthat matches the corresponding policies to the propersecurity devices.

It is possible to optimize the controller byconfiguration and management, and update rules fordynamic migration and reconfiguration to solve securitypolicy inconsistencies caused by Middlebox movementand virtual machine migration.

2 Related Work

2.1 Network security in data center networks

Among the security research in data center networks,SDN security is a hot topic both for academia andindustry.

FRESCO[11-13] introduces a new security applicationdevelopment framework. It is used to solve several keyissues when implementing security service componentson demand. OpenFlow is an open standard that is able

to provide simplified and convenient design of complexnetwork security applications and their integrationinto large networks. However, so far there remains alack of convincing applications regarding the securityaspects of OpenFlow. FRESCO provides a scriptinglanguage Application Programming Interface (API) thatprogrammer can use to write security monitoring logicwith a modular library of basic processing unit inFRESCO. A FRESCO module allows customization ofstream processing rules to provide an effective responseto detected networks threats.

Slick[14] is a proposed programming framework thatseparates the controller and Middleboxes and providesan interface for communication. There are more andmore programs running on controllers which cross dataplanes and control planes to call actions, and there isno complete solution to meet this requirement. WhileOpenFlow provides a programmable control plane,it has a relatively simple data plane. In contrast,Middleboxes are able to effectively extend the dataplane and provide a complex data plane, but cannotprovide effective integration with the control plane.

Slick extends data plane programmability in twoaspects. First, Slick arranges complex data planefunctions dynamically in the network, and also guidesa subset of packet through the appropriate functionprocessing queue that adapts to the position of thelayout and response as these change over time to meetthe changeable network conditions and transmissionmode. Second, to achieve modularity, reusability,and integration strategy across multiple applicationsand network resources, Slick allows programmers tocoordinate multiple actions among different entities inthe data plane.

SIMPLE, based on SDN, is proposed in Refs. [15-18], and is an implementation of strategy layerthat directs traffic to specific Middleboxes. Today’snetworks must rely on Middleboxes to provide highthroughput, high security, and efficient decision-processing capabilities. To achieve these goals, weneed to ensure that traffic goes directly to the requiredMiddleboxes queue, an operation that once requiresextensive manual effort and operator’s expertise. In thisrespect, an SDN provides a promising option, but alsointroduces some functions that do not belong to thetraditional L2/L3 functions, such as policy components,resource management, and packet manipulation.

SIMPLE allows network operators to specify routingpolicies in a logical Middlebox, and automatically

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 85

update forwarding rules according to the physicaltopologies, switching capacity, and resource constraintsof Middleboxes to guide traffic to the properMiddleboxes queues. Under the premise of existingSDN functions and without modifying the existingimplementation of Middleboxes, SIMPLE increasesthe flexibility of Middleboxes deployment effectivelyby flow oriented control, and also generates andloads new rules to maintain network stability whenany Middleboxes fail and/or network transmission isoverloaded.

2.2 Network virtualization with VMware NSX

VMware NSX[19] is a network virtualization platformfor data center networks. VMware NSX provides anetwork management and network security system forits software-defined data center that combines SDNand Network Functions Virtualization (NFV), and isdesigned to achieve the rapid deployment of multiplelayers of a logical network without complicatedconfiguration of physical devices that greatly improvesthe flexibility of network deployment. It provides 2-7 layers’ NFV and provides a software-based securitymodel to achieve the decoupling of the underlyingnetwork hardware, thus making full use of the existingnetwork infrastructure.

The core components of VMware NSX are logicswitches, logical routers, NSX APIs, logical Firewalls,logical load balancing, logical VPN, etc. NSX providesa network virtualization approach that allows datacenter operators to treat the physical network as an on-demand system and capacity tool. The overall structureof NSX is shown in Fig. 1.

Fig. 1 Network virtualization platform of VMware NSX[19].

The scalability in NSX provides services of othernetwork security vendors. The NSX platform usesa distributed service framework that allows multiplehosts to integrate the network service, and can easilyinsert new service modules using the NSX API. NSXoffers a variety of logical device interfaces, but it alsoneeds cooperation from other vendors to achieve correctmapping of logical-to-physical connections when facedwith a wide variety of hardware devices.

2.3 NSX and network security

NSX aims to integrate security functions offered byother vendors. To simplify the third-party securityproducts and integrate services, authorized vendors usea single specific API that links the security platformand the network. NSX’s Service Composer tool isused to deploy third-party Firewalls, anti-malwares,vulnerability management, data loss protection,intrusion detection, and intrusion prevention (IDS/IPS) platforms centrally.

In addition to the security measures discussed above,the network service gateways bridge the physical andvirtual environments, and application delivery servicesinclude load balancing, application delivery, and WANoptimization.

There are internal stateful Firewalls in the NSXthat can provide distributed Firewall detection for eachvirtual router port. Firewall management uses a varietyof rules, audit, and inspection technologies. It alsoprovides the logic state, and integrates routing into athird-party security platform.

Brocade VCS Gateway allows the connection ofvirtual and physical facilities while providing the typesof applications required by a data center operatorto maintain a unified network. By using BrocadeVCS Fabric technology, one can use their currentinfrastructure to implement network managementdeployment. The implementation structure of BrocadeVCS Gateway is shown in Fig. 2.

Cumulus Linux system provides programmablefeatures and automated tools that use the samecomputing environment on network devices to quicklyconfigure the physical network and increase cost-efficiency when adding new network capacity byreducing the difficulty of adding new devices. Itprovides support for network virtualization boundaryfunctions by realizing coverage of the second layergateway and avoiding use of the virtual networkof the VXLAN tunnel endpoint, and registers NSX

86 Tsinghua Science and Technology, February 2014, 19(1): 82-94

Fig. 2 Brocade supported NSX switch gateway[19].

gateway services to further simplify management bythe controller. Combined with NSX, Cumulus Linuxprovides a network virtualization boundary to achievephysical workload at high levels of connectivity. Theintegrated solution of Cumuluss NSX is shown in Fig. 3.

Dell S6000 data center switching gateway for NSXprovides programmability, automation features, andscalability. S6000 is a high-performance gateway forNSX that extends a virtual network to the physicalserver, or connects the physical workload that isaccessed by virtual LANs to the logical network viathe second layer network services, also provides themigration from an existing virtual environment to thepublic cloud, creates a hybrid cloud, etc. The specificstructure of the S6000 data center switching gatewayfor NSX is shown in Fig. 4.

In addition, for the security of the data center,Palo Alto presents its next generation securityplatform[20, 21]. As shown in Fig. 5, its core componentis a powerful Firewall that provides identification andcontrol functions, and is always able to detect threatswhile ensuring the safe operation of the program. PaloAlto PA-5000 offers a variety of defense options tonetwork threats, such as IPS and anti-malware softwareto defeat known threats, and automated sandboxanalysis of suspicious files to detect unknown maliciousprograms or APT attacks.

Fig. 3 The integrated structure of Cumuluss Linux NSX[19].

Fig. 4 The integration of Dell S6000 switching gateway andNSX[19].

Fig. 5 The integration of the Palo Alto network and NSX[19].

3 vCNSMS

In this section, we propose vCNSMS, a collaborativenetwork management prototype system derivedfrom CNSMS[22-26] for multi-tenant data centernetworks. In our experiments, we demonstrate thatvCNSMS can be integrated into virtualized cloudenvironments. vCNSMS is a prototype system witha home-brewed version of an untangle open sourcemultifunction gateway[27, 28].

3.1 The principle of collaborative network securityin DCN

3.1.1 Basic network topologyThe Security Center and peer-UTM are deployed in thedata center network. In the bootstrap stages, the peer-UTM is running a registration process for the SecurityCenter.

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 87

The Security Center receives the registrationinformation and displays the registered UTMs. Figure 6shows the vCNSMS configuration and the registrationprocess in the bootstrap stage.

3.1.2 Collaborative security in DCN3.1.2.1 Security center interacts with the peer-UTMsWe assume each peer-UTM manages a virtual domainfor a tenant in the Data Center, and the peer-UTM is managed by commands from the SecurityCenter. Figures 7 and 8 show the collaborative networksecurity process in vCNSMS prototype.

(1) Security Center issues rules: There is a helpoption for rule creation and an option for the rule issued,in web User Interface (UI).

(2) Peer-UTM reports events: The Security Center

Fig. 6 vCNSMS configuration settings and registrationprocess in the bootstrap stage.

needs a web interface to display the security eventsreported by peer-UTMs.

(3) Events inform between peer-UTMs: There is aweb-based UI in the peer-UTM collaborative securitymodule. The web UI shows the different peer-UTMs tothe Security Center and displays any occured securityevents.3.1.2.2 Antivirus module in collaborative securityWorkflow of Antivirus module is shown in Fig. 9.

Fig. 7 Security rules issued.

Fig. 8 Security rules interwork.

Fig. 9 Workflow of signature database updating inAntivirus module.

88 Tsinghua Science and Technology, February 2014, 19(1): 82-94

(1) The Security Center imports the virus signaturedatabase: There is an option to import the virussignature database in the Security Center.

(2) The Security Center issues the virus signaturedatabase: There is an option to issue the virus signaturedatabase in the Security Center.

(3) An interface displays that the virus database hasbeen updated: The time changes.

(4) Virus signature database synchronization betweenpeer-UTMs is performed using the p2p mode.3.1.2.3 Firewall module in collaborative securityWorkflow of Firewall module is shown in Fig. 10.

(1) Importing Firewall rules to Security Center: Theoption of importing Firewall rules to the SecurityCenter.

(2) Security Center issues the Firewall rules: Theoption of issuing the Firewall rules.

(3) Interface displays that the Firewall rules havebeen updated: There is a web UI in the peer-UTMcollaborative security Firewall module. The web UIshows the new Firewall rules that will be updated.

(4) The peer-UTMs choose to implement the rulesthat are updated by the Security Center: In the displayUI of the update rules, the peer-UTMs are allowed to leta certain rule lapse if that rule is not applicable.3.1.2.4 Protocol control and content matching

module in collaborative securityFunctions and principles are the same as Section3.1.2.3.

3.1.3 Security rule centerThe security rule center in security center runs onDebian Linux, and includes a rule distribution moduleand an event alarm module.

The rule distribution module is divided into server

Fig. 10 Workflow of rules issuing to Firewall module.

and client, and the server program is a socketcommunication module written in Java that is manuallyactivated in the Security Center. The client program isrunning in the peer-UTM Firewall and Rules controlmodule. When the server and client programms arerunning normally, the Security Center can quicklytransfer rules in a specific folder to the peer-UTM’s rulecontrol module.

The event alarm module is a web application based onApache Tomcat that opens port 443 for this service. Itlistens for any security event reported from the peer-UTM, and dynamically displays these on the web UI.

The rule format for the Security Center and peer-UTM communication mechanism is shown in AppendixA, in details.

3.1.4 Virtualization network based on SDNAn experimental platform based on Openflow SDN isshown in Fig. 11, and the detailed configuration isshown in Table 1. Network topology and the host IPsare allocated in the following 2 rules:

(1) Each virtual machine is bridged to eth0(192.168.0.0) segment for easy ssh control.

(2) Another controller is also present in this sectionand controls Openflow switch communication.

In the Openflow switch, there are three Ethernetports that can be connected according to experimentalneeds. Each client has an Ethernet port that canbe connected to any Openflow switch accordingto experimental needs. We use Openflow in theexperimental environment. Here are some details of the

Fig. 11 Virtualization network based on SDN.

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 89

Table 1 Detailed configuration settings in virtualization network based on SDN.

VMs IP Hostname Host alias System

Controller 192.168.0.60 Saturn-controller Control Ubuntu 13.10 (flood light)

OpenFlow1 192.68.0.61 Saturn-openFlow1 Of1 Ubuntu 13.04

OpenFlow2 192.68.0.62 Saturn-openFlow2 Of2 Ubuntu 13.04

OpenFlow3 192.68.0.63 Saturn-openFlow3 Of3 Ubuntu 13.04

Client1 192.68.0.64 Client1 Client1 Ubuntu 13.04

Client2 192.68.0.65 Client2 Client2 Ubuntu 13.04

set up deployment.(1) In Openflow 1.0, there is only a single userspace

datapath. The dpct that played an important rolein previous versions is replaced by ofdatapath andofprotocol.

(2) Ofdatapath is the implementation of a datapath. Itis responsible for forwarding packets based on theflow table and communicates through ofprotocol andfloodlight.

(3) Ofprotocol is a middle program that controlsofdatapath and is controlled by floodlight.

(4) dpctl can still be used to view the flow table inthe Openflow switch, but cannot be used to manage thedatapath.

(5) To change the behavior of the Openflow switch,the component (C or python) must pass throughfloodlight, and the component will be loaded withfloodlight. (Specific usage can be referred to from themain page for ofdatapath, ofprotocol, and dpctl).

3.2 Deep security check in vCNSMS

3.2.1 Function settingsThe Security Center, centrally manages security rules,collects the feedback information from the ruledeployment, and stores the data into the security log.

(1) Security rules are incrementally downloaded. Inthe Security Center, duplicate rules are removed fromthe new rule set and make the new rules available. Thenew rules will be added in a package and issued to thepeer-UTMs.

(2) Firewall module. Firewall rules will bedownloaded from the Security Center and loaded in theFirewall module, and the corresponding functions willbe activated. Security events are collected and returnedback to the Security Center.

(3) UDP content filtering module. It blocks or dropsthe specified types of data packets under the currentrules. The format for UDP rules and content matchingis presented in Appendix A.

3.2.2 Enhanced security functionsProtocol Control module in the prototype systemmainly enforces UDP protocol rules.

(1) UTMs routinely update rules. UTMs regularlyobtain Firewall rules. The configuration informationincludes the rules of content inspection for UDP andthe blacklist based on the content filtering (source anddestination addresses, ports, protocol, URL).

(2) Filtering based on the content of a UDPpacket. According to the rules, any packets that containspecified signatures will be matched. For example, theDNS requests’ packets that contain the specified textstring “abc” will be matched and logged.

(3) Blocking function with a blacklist. Blockrule based on “quintuple” filtering, e.g., block thenetwork connection with a command such as “TCP1.2.3.4:443”. The Firewall module can quickly applythe update rule.

3.2.3 Implementation in peer-UTM(1) Update Firewall rules. The administrator puts anew Firewall rule file in the specified folder namedFirewallRule in the Security Center. The daemonprogram periodically checks this folder, and if thereis a new set of rules in the folder, the Security Centerwill send announcement messages to all online peer-UTMs to inform them of a new rule update. A peer-UTM receives messages from the Security Center andcompares these with its own rules’ serial number inthe rule database. If there is an update, the peer-UTMwill send a download request to the Security Centerfor the updated rules. The corresponding rules are thendownloaded to the local rule database and are activatedin real time.

(2) Firewall module blocks a specified networkflow. The Firewall module provides a blocking functionof rules based on “quintuple” filtering, and sends thecorresponding log info to the Security Center throughan announcement message too. After receiving themessage, the web UI of Security Center will present this

90 Tsinghua Science and Technology, February 2014, 19(1): 82-94

information in real-time.(3) UDP content filtering. UDP content filtering

module uses the same mechanism as the Firewallmodule, and has the specified tag indicated in theannouncement message. When a peer-UTM receivesthe announcement from the Security Center, it will senda download request message to the Security Center forthe rules. When the rules are downloaded and activated,the peer-UTM will call the getPatterns function in theLoadPatterns class to activate the rules loaded in UDPcontent filtering module. (We modify the prototypesystem to load multiple UDP rules.) When the UDPpacket is parsed, the findMatch() function in theEventHandler class will be called to match the contentand rules. The createRegExPattern function in thePatternFactory class is called to process the content ofthe UDP packet. The JAVA regular expression matchingand matcher functions in the java.util.regex class arethen used to match the rules and filter out any specifiedcontent. Finally, the UDP packets that contain specifiedcontent will be dropped.

3.3 Intelligent flow processing in vCNSMS

Intelligent flow processing is an advanced methodin Refs. [29-31] for intrusion detection. Intelligentflow processing in vCNSMS is based on the smartpacket verdict scheme. In this section, we propose thesecurity level based protection policy for intelligentflow processing in vCNSMS.

3.3.1 Security level based protection policyThe security level based protection policy is shown asfollows:

(1) Security level: Red, Yellow, Orange, and Green.(2) Intelligence flow processing: According to the

security level, different security rule sets and packetverdict schemes are used with different performancesand load costs.

(3) Multi-function security gateway: UTM canconfigure different security plugins on the demand ofdifferent security levels, and incurs different processingcosts. The working mode of peer-UTMs can alsobe categorized into Red, Yellow, Orange, and Greenconfigurations.

3.3.2 Implementation of security level withsecurity function plugins

We modify the untangle Shield module[27, 32] andenhance it with the smart packet verdict algorithm. Thedetailed functions are as follows.

(1) Security plugins’ rule set: S1 (urgent), S2(important), S3 (less important), and S4 (trivial).

(2) Security plugin module: plugin 1, plugin 2, plugin3, � � � , plugin N.

(3) Security plugin module’s processing: Taggingeach packet with a Block mark, Pass mark, or Suspectmark.

(4) Packet verdict on network packets is based on ascoring system.

(5) Packet processing obeys the following strategy:� Let the benign flows pass as soon as possible;�Recheck the suspected flow with additional rule sets

or security plugins;� Provide a verdict on the packet with the context,

i.e., current level of security, such as Red or Green.(6) Packet verdict processing also takes traffic load

and performance penalties into consideration.3.3.2.1 Smart packet verdict scheme with untangle

ShieldOriginally, Shield is an untangle module whose functionis to prevent DoS attacks. Shield reads a packetfrom nfqueue and starts four threads including amain thread, an event dispatch thread, an admissionpacket processing thread, and a frontend microhttpddaemon. The main task of Shield is to collect packetprocessing threads, and there is no parallel structurein Shield. Figure 12 shows the smart packet verdictscheme used in vCNSMS.3.3.2.2 Process level parallelization of ShieldWe separate the untangle Shield module and enhance itwith our smart packet verdict scheme. For performanceimprovement, we also modify Shield to integrate

Fig. 12 Smart packet verdict scheme used in vCNSMS.

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 91

xtables[33] for parallelization.The Shield test environment is shown in Fig. 13. The

parallelization scheme of Shield may be accomplishedin two ways:

(1) Use multiple NFQUEUEs and open multipleShield processes that perform the same as theSnort inline program[34].

(2) Rewrite the Shield code to implement a dispatcherfor the corresponding queue in the program to achievemulti-threaded processing.

The detailed parallelization scheme is given asfollows:

(1) Install xtables-addons-1.12-nslab that is anNFQUEUEX module with the shunt function.

(2) Specify iptables -A FORWARD -j NFQUEUEX–queue-num 4. The last parameter establishes fourNFQUEUEs in this example.

(3) Start four Shield processes, with different queuenumbers and port numbers assigned to each.

To achieve the forward operation, it needs not only toset routing table of client1 and client2, but also to runthe command (echo1>/proc/sys/net/ipv4/ip forward) toenable the kernel forwarding function.3.3.2.3 Performance of Shield parallelizationBecause of the amount of traffic is throttled froma single IP, UDP packets sent by traffic generatorSmartBit will lead the original Shield to block thesepackets. To avoid this, we modify Shield to make allpackets pass through. The performance measured in thismanner should reflect the capability of Shield module.

The test environment is described as follows:(1) Hardware: Intel(R) Core(TM) 2 Quad CPU

Q9400 @ 2.66 GHz, 4 cores, 2 GB memory, 8 Gbitports.

(2) Software: Ubuntu 13.04, Shield untangle module.

Fig. 13 Performance evaluation in Shield parallelizationimplementation.

(3) SmartBit settings: 4-way UDP streams withdifferent IP and Ports. Packet loss ratio is collected ineach experiment. Each experiment is conducted in a 1-Gbps link with varied input traffic ratio from 10% to100%.

As shown in Fig. 14, five parallel Shield processescan handle 400-500 Mbps. The CPU usage is about40%. Memory consumption is quite low. As Shieldonly deals with the headers of packets in this test, theperformance result is expectable, but it is slower thanthe Snort[35].3.3.2.4 Bottleneck analysis of ShieldWe use Intels VTune tools in the compile stageof the Shield module. Running performanceevaluation experiments of the Shield moduleproduces useful information of the function callsinside the Shield module. The most time consumingoperations in packet handling are concentrated inbarfight net nfqueue read() and handle packet()wherein the packet verdict scheme is running. Inaddition, the debug function is time consuming.

As we run the Shield module in a virtual machine,performance issue in virtualized environments is acritical and open problem, which has already beendiscussed in Ref. [36]. We can further improveperformance with vPF ring[37] in virtualizedenvironments.

4 Conclusions

In this paper, we propose vCNSMS to addressnetwork security in multiple-tenants data center anddemonstrate vCNSMS with a centralized collaborativescheme. vCNSMS can further integrate a smartpacket verdict scheme for packet inspection to defendfrom possible network attacks inside the data centernetwork. An SDN-based virtualization network in a

Fig. 14 Throughput for smart packet verdict scheme invCNSMS.

92 Tsinghua Science and Technology, February 2014, 19(1): 82-94

data center can deploy vCNSMS for flexibility andscalability to protect multiple tenants with differentnetwork policies and security requirements. The smartpacket verdict scheme can be further investigated andimproved with parallelization.

5 Future Work

Based on the practical deployment and operationalexperience gained from evaluating vCNSMS in a datacenter network, vCNSMSs security center is ableto deploy more and more security rules and collectdata of network events. It should be possible todetect network policy violations and intrusion with anartificial intelligence based on unsupervised learningmethods. This is a promising area for exploration in thefuture.

Acknowledgements

This work was supported in part by the National KeyBasic Research and Development (973) Program of China(Nos. 2013CB228206 and 2012CB315801), the NationalNatural Science Foundation of China (Nos. 61233016 and61140320).

This work was also supported by the Intel ResearchCouncil with the title of “Security Vulnerability Analysisbased on Cloud Platform with Intel IA Architecture” andHuawei Corp.

Appendix A

References

[1] NIST definition of cloud computing, http://csrc.nist.gov/publications/PubsNISTIRs.html, 2007.

[2] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A.Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla,U. Hozle, S. Stuart, and A. Vahdat, B4: Experience witha globally-deployed software defined WAN, in Proc. ACMSIGCOMM 2013 Conference on SIGCOMM, Hong Kong,China, 2013, pp. 3-14.

[3] J.D. Liu, A. Panda, A. Singla, B. Godfrey, M. Schapira,and S. Shenker, Ensuring connectivity via data planemechanisms, presented at 10th USENIX Symposiumon Networked Systems Design and Implementation,Lombard, IL, USA, 2013.

[4] J. D. Liu, B. H. Yan, S. Shenker, and M. Schapira, Data-driven network connectivity, in Proc.10th ACM Workshopon Hot Topics in Networks, New York, USA, 2011, p. 8.

[5] Qihoo 360 Internet Security Center, Developmenttrend of enterprise security in the internet ages, http://www.gartner.com/technology/mediaproducts/pdfindex.jsp?g=Qihoo issue1, 2013.

[6] X. M. Chen, B. P. Mu, and C. Zhen, NetSecu: Acollaborative network security platform for in-networksecurity, in Proc. 3rd International Conference onCommunications and Mobile Computing, Qingdao, China,2011, pp. 59-64.

[7] D. H. Ruan, C. Lin, Z. Chen, and J. Ni, Handlinghigh speed traffic measurement using network processors,presented at International Conference on CommunicationTechnology, Guilin, China, 2006.

[8] J. Ni, C. Lin, and Z. Chen, A fast multi-pattern matchingalgorithm for deep packet inspection on a networkprocessor, presented at the IEEE International Conferenceon Parallel Processing, Xi’an, China, 2007.

[9] Z. Chen, C. Lin, J. Ni, D.H. Ruan, B. Zheng, Y. X. Jiang,X. H. Peng, Y. Wang, A. A. Luo, B. Zhu, Y. Yue, and F.

Zhen Chen et al.: Collaborative Network Security in Multi-Tenant Data Center for Cloud Computing 93

Y. Ren, AntiWorm NPU-based parallel bloom filters forTCP/IP content processing in giga-Ethernet LAN, in Proc.the IEEE International Conference on Communications,2006, pp. 2118-2123.

[10] Z. Chen, C. Lin, J. Ni, D. H. Ruan, B. Zheng, Y. X.Jiang, and F. Y. Ren, AntiWorm NPU-based parallel bloomfilters for TCP/IP content processing in Giga-EthernetLAN, in Proc. the IEEE International Conference on LocalComputer Networks, Sydney, Australia, 2005, pp. 748-755.

[11] S. Shin, P. Porras, V. Yegneswaran, M. Fong, G. F.Gu, and M. Tyson, FRESCO: Modular composablesecurity services for software-defined networks, presentedat Network and Distributed Security Symposium, 2013.

[12] L7 filter project, http://l7-filter.sourceforge.net/Pattern-HOWTO, 2008.

[13] V. Sekar, M. K. Reiter, W. Willinger, H. Zhang,R. R. Kompella, and D. G. Andersen, cSamp: Asystem for network-wide flow monitoring, in Proc. 5thUSENIX Symposium on Networked Systems Design andImplementation, San Francisco, USA, 2008, pp. 233-246.

[14] B. Anwer, T. Benson, N. Feamster, D. Levin, and J.Rexford, A slick control plane for network middleboxes, inProc. Association for Computing Machinery, Hong Kong,China, 2013, pp. 147-148.

[15] Z. A. Qazi, C. C. Tu, L. Chiang, R. Miao, V. Sekar, and M.Yu, SIMPLE-fying middlebox policy enforcement usingSDN, in Proc. Association for Computing Machinery,Hong Kong, China, 2013, pp. 27-38.

[16] K. Wang, Y. Qi, B. Yang, Y. Xue, and J. Li, LiveSec:Towards effective security management in large-scaleproduction networks, in Proc. IEEE 32nd InternationalConference on Distributed Computing Systems Workshops,Macau, China, 2012, pp. 451-460.

[17] X. Wang, Z. Liu, Y. Qi, and J. Li, LiveCloud: A lucidorchestrator for cloud datacenters, in Proc. IEEE 4thInternational Conference on Cloud Computing Technologyand Science, Taipei, China, 2012, pp. 341-348.

[18] VMWare Network security, http://www.vmware.com/products/nsx/resources.html, 2013.

[19] VMware NSX network virtualization platform,http://www.vmware.com/products/nsx, 2013.

[20] Y. Zhang, F. Deng, Z. Chen, Y. B. Xue, and C. Lin,UTM-CM: A practical control mechanism solution forUTM system, in Proc. IEEE International Conferenceon Communications and Mobile Computing, Shenzhen,China, 2010, pp. 86-90.

[21] Bro, http://www.bro-ids.org, 2013.

[22] F. Han, Z. Chen, H. Xu, H. Wang, and Y. Liang, Acollaborative botnets suppression system based on overlaynetwork, International Journal of Security and Networks,vol. 7, no. 4, pp. 211-219, 2012.

[23] Z. Chen, F. Han, J. Cao, X. Jiang, and S. Chen,Cloud computing-based forensic analysis for collaborativenetwork security management system, Tsinghua Scienceand Technology, vol. 18, no. 1, pp. 40-50, 2013.

[24] X. Chen, K. Ge, Z. Chen, and J. Li, AC-Suffix-Tree: Bufferfree string matching on out-of-sequence packets, in Proc.2011 ACM/IEEE Seventh Symposium on Architecturesfor Networking and Communications Systems, IEEEComputer Society, Brooklyn, NY, USA, 2011, pp. 36-44.

[25] T. Li, F. Han, S. Ding, and Z. Chen, LARX: Large-scale antiphishing by retrospective data-exploring basedon a cloud computing platform, in Proc. IEEE 20thInternational Conference on Computer Communicationsand Networks, Maui, HI, USA, 2011, pp. 1-5.

[26] B. Mu, X. Chen, and Z. Chen, A collaborativenetwork security management system in metropolitan areanetwork, in Proc. IEEE 3rd International Conference onCommunications and Mobile Computing, Qingdao, China,2011, pp. 45-50.

[27] Untangle open source appliance, https://gitorious.org/untangle, 2013.

[28] Y. D. Lin, R. H. Hwang, and F. Baker, Computer Networks:An Open Source Approach. McGraw-Hill, February 2011.

[29] Y. D. Lin, H. Y. Wei, and S. T. Yu, Building anintegrated security gateway: Mechanisms, performanceevaluations, implementations, and research issues, IEEECommunications Surveys & Tutorials, vol. 4, no. 1, pp. 2-15, 2002.

[30] Y. D. Lin, C. W. Jan, P. C. Lin, and Y. C. Lai, Designingan integrated architecture for network content securitygateways, Computer, vol. 39, no. 11, pp. 66-72, 2006.

[31] C. N. Lu, C. Y. Huang, Y. D. Lin, and Y. C. Lai, Sessionlevel flow classification by packet size distribution andsession grouping, Computer Networks, vol. 56, no. 1, pp.260-272, 2012.

[32] D. Morris, J. Irwin, and R. Scott, Methods and systems forreputation based resource allocation for networking, USPatents US20070043738A1, February 22, 2007.

[33] Xtables-addons, http://xtables-addons.sourceforge.net,2013.

[34] Snort-inline, http://snort-inline.sourceforge.net, 2009.[35] Snort, http://www.snort.org, 2010.[36] M. J. Schultz and P. Crowley, Performance analysis

of packet capture methods in a 10 Gbps virtualizedenvironment, in Proc. IEEE 21st International Conferenceon Computer Communications and Networks, Munich,Germany, 2012, pp. 1-8.

[37] A. Cardigliano, L. Deri, J. Gasparakis, and F. Fusco,vPFRING: Towards wire-speed network monitoring usingvirtual machines, in Proc. ACM SIGCOMM Conferenceon Internet Mearsurement Conference, Berlin, Germany,2011, pp. 533-548.

94 Tsinghua Science and Technology, February 2014, 19(1): 82-94

Zhen Chen is an associate professorin Research Institute of InformationTechnology at Tsinghua University. Hereceived his BEng and PhD degreesfrom Xidian University in 1998 and2004, respectively. He once workedas postdoctoral researcher in NetworkInstitute of Department of Computer

Science and Technology in Tsinghua University during 2004 to2006. He was also a visiting scholar in UC Berkeley ICSI in2006. He joined Research Institute of Information Technology atTsinghua University since 2006. His research interests includenetwork architecture, computer security, and data analysis. Hehas published around 80 academic papers.

Junwei Cao is currently a professorand deputy director of ResearchInstitute of Information Technology,Tsinghua University, China. He is alsothe director of Open Platform andTechnology Division, Tsinghua NationalLaboratory for Information Science andTechnology. His research is focused on

advanced computing technology and applications. Beforejoining Tsinghua in 2006, Junwei Cao was a research scientistof Massachusetts Institute of Technology, USA. Before thathe worked as a research staff member of NEC Europe Ltd.,Germany. He got his PhD in computer science from Universityof Warwick, UK, in 2001. He got his MEng and BEng degreesfrom Tsinghua University in 1998 and 1996, respectively. Hehas published over 130 academic papers and books, cited byinternational researchers for over 3000 times. He is a seniormember of the IEEE Computer Society and a member of theACM and CCF.

Wenyu Dong is currently a master studentof Department of Computer Science andTechnology at Tsinghua University. Hegot his BEng degree from PLA Univ. ofInfo. & Eng. in 2009. Currently hismain research interests focus on computernetwork security and mobile safety.

Hang Li is currently a master studentof Department of Computer Science andTechnology at PLA Univ. of Info. &Eng. She got her BEng degree from PLAUniv. of Info. & Eng. in 2013. Hermain research interest focuses on computernetwork security.

Peng Zhang received his BEng degreefrom Beijing University of Posts andTelecommunications in 2008, andPhD degree from Tsinghua Universityin 2013. He was a visiting assistantin Research with the Department ofComputer Science, Yale University from2012 to 2013. He is now an assistant

professor in Department of Computer Science and Technology,Xi’an Jiaotong University, China. His research interests includenetwork security and privacy, software-defined network, andinformation-centric networks.

Xinming Chen is currently a PhDcandidate in Dept. of Electrical andComputer Engineering, University ofMassachusetts, Amherst. He received hisBEng and MEng degrees from TsinghuaUniversity, China, in 2009 and 2011,respectively. He is also a member of theNetwork Systems Laboratory. His research

interests include Internet architecture, multi-core processing,and deep packet inspection.