view article

Copyright 2013 by Modern Scientific Press Company, Florida, USA

International Journal of Modern Engineering Sciences, 2013, 2(1): 17-27

International Journal of Modern Engineering Sciences

Journal homepage:www.ModernScientificPress.com/Journals/IJMES.aspx

ISSN: 2167-1133

Florida, USA

Article

Scalability Aspects in BGP/MPLS VPN

N. H. Almofary1,*

, H. S. Moustafa2, F. W. Zaki

3

1Faculty of Engineering, Sana'a University, Sana'a, Yemen

2Faculty of Engineering, Mansoura University, Mansoura, Egypt

3Faculty of Engineering, Mansoura University, Mansoura, Egypt

* Author to whom correspondence should be addressed; E-Mail: [email protected]

Article history: Received 19 January 2013, Accepted 20 February 2013, Published 28 February 2013.

Abstract: All Virtual Private Network (VPN) should provide users with the isolation and

security associated with private networks, but at lower costs. This may be made possible by

implementing these networks over some type of shared infrastructure. Provider provisioned

VPN allow enterprises to Participate their private backbone networks to service providers.

One of the most common provider provisioned VPN technologies is BGP/MPLS VPN

which uses multiprotocol label switching (MPLS) as tunneling technology for customer

flow isolation and border getaway protocol (BGP) to propagate routing information

between VPN sites. The scalability is one of the main reasons for the wide deployment of

this technology. However, it still suffers from some scaling problems in large scale

networks resulting from the BGP protocol properties. In this paper the basics of

BGP/MPLS VPNs are explained in conjunction with scalability. OPNET modeler was used

to study the effects of route reflectors solution to cope with the scalability problem. This is

due to the fact that route reflectors enhance the scalability of the network by reducing the

number of routing states that must be maintained in each provider edge router (PE).

Keywords: BGP, MPLS, VPN, provider edge router.

1. Introduction

The Internet Engineering Task Force (IETF) provides the standardized definition of a Virtual

Private Network (VPN) as A network in which connectivity among multiple private Wide Area

Networks (WANs) is deployed using shared IP infrastructure with the same policies as a private

Int. J. Modern Eng. Sci. 2013, 2(1): 17-27


18

network. In other words (VPN) allows enterprises to connect their remote sites and users within

single secured network. Three clearly distinct categories of VPN networking technology exist, these

are:

Traditional VPNs based on layer 2 frame relay (FR) and asynchronous transfer mode (ATM)

technology.

Customer Premises Equipment (CPE) based VPNs based on protocols such as layer 2 tunneling

protocol (L2TP) and IPSec [1] .

Provider-provisioned VPNs (PP-VPNs) based on layer 2 switching such as Ethernet, or layer 3

IP-based paradigms such as BGP/MPLS VPNs [2,3].

2. BGP/MPLS VPN

It is interested in Provider-provisioned VPN where Enterprises have sites spread across distant

locations that need to be interconnected. Instead of having fully dedicated links between their sites,

many enterprises prefer to contract a Virtual Private Network (VPN) service from a VPN service

provider, thereby reducing the connection costs. This service model is known as the provider

provisioned VPN service. In this model, the VPN provider shares its physical network

infrastructure among multiple enterprises, guaranteeing isolation of virtual networks [4].

There has been a tremendous standardization effort within the IETF to specify routing

mechanisms for provider provisioned VPNs [5, 6, 7, 8, 9]. Among the proposed standards, BGP-

MPLS-IP-VPNs [7] stand as the most popular. This technology uses BGP as a control plane to provide

VPN routing and MPLS as a transport technique to achieve isolation between customer traffic. Its

popularity as a result of the high number of customers supported (thousands of customers and

hundreds of thousands of VPN sites)[4].

2.1. Provider Goals and Desired Properties

Isolation is a key word in the VPN service provisioning. The different VPNs have to be isolated

as if they had dedicated links between their different sites. Two levels of isolation is defined, control

plane isolation and data plane isolation.

VPN control plane isolation is concerned with signaling and addressing, both should be isolated.

Each VPN can have its own addressing and different VPNs can have overlapping addresses. Also,

signaling of one VPN must not affect signaling of another VPN.

Data plane isolation concerns performances and privacy. The performances of one VPN should

not be affected by the fact that other VPNs share the same infrastructure [4].



19

2.2. BGP/ MPLS VPN Structure

The key core network elements of a provider-provisioned BGP/MPLS VPN network are

provider edge (PE) and provider core (P) routers as shown in Fig 1, whereas the customer edge (CE)

router is not considered part of the providers core network. It acts as a peer of the PE router, but not a

peer to other CE routers.

Fig. 1: MPLS VPN Structure

The routers that link the customer sites to the provider network are called customer edge (CE)

routers, whereas the service provider routers to which the CE routers are connected are called provider

edge (PE) routers. In most cases, the provider network is made up of more than just the PE routers;

those other routers are called P devices [10].

PE routers take the charge of both accessing VPN service and forwarding packets from private

intranet to public internet, whereas P routers only have basic forwarding and typically are not have

directly connecting customer access circuits [11].

All PE and P routers run label switching so that they can build MPLS label switched paths

(LSPs) from each PE to each other PE. This is achieved through the use of the label distribution

protocol (LDP) in conjunction with the interior gateway protocol, such as open shortest path first

(OSPF).

When a PE forwards a VPN-addressed packet across the core, it adds two MPLS labels, one

external which identies the PE in the provider backbone and the other internal which identies the

interface inside the PE. Any intermediate P or PE routers switch the packet to the egress PE using the

outer label only. The inner label is used by the egress PE to determine the VPN/port to which the

packet should be forwarded [2, 12].

Each PE router supports multiple routing/forwarding tables, called virtual route forwarding

(VRF) tables. Every site to which the PE is attached must be mapped to one of those forwarding tables.

When a packet is received from a particular site, the forwarding table associated with that site is

consulted in order to determine how to route the packet [3].



20

3. VPN Route Distribution

PE routers use BGP to distribute VPN routes to each other, but a BGP speaker can only install

and distribute one route to a given address prefix. Yet each VPN allowed having its own address space,

which means that the same address can be used in any number of VPNs where in each VPN the

address denotes a different system.

BGP allows installing and distributing multiple routes to a single IP address prefix. These goals

were meet by the use of a new address family.

3.1. The VPN-IP Address Families

The BGP Multiprotocol Extensions [3] allow BGP to carry routes from multiple address

families. The notion of the VPN- IPv4 address family and VPN- IPv6 address family are introduced.

3.2. The VPN-IPv4 Address Family

A VPN-IP v4 address is a 12-byte, beginning with an 8-byte "Route Distinguisher (RD)" and

ending with a 4-byte IPv4 address [11]. If two VPNs use the same IPv4 address prefix, the PEs

translate these into unique VPN-IPv4 address prefixes. This ensures that if the same address is used in

two different VPNs, it is possible to install two completely different routes to that address, one for each

VPN [7].

The purpose of the RD is just to allow one to create distinct routes to a common IP address

prefix. The RD can also be used to create multiple different routes to the same system. This can be

achieved by creating two different VPN-IPv4 routes that have the same IP part, but different RDs. This

allows BGP to install multiple different routes to the same system, and allows policy to be used to

decide which packets use which route.

Note that VPN-IPv4 addresses and IP addresses are always considered by BGP to be

incomparable.

3.3. The VPN-IPv6 Address Family

A VPN-IPv6 address is a 24-byte quantity, beginning with an 8-byte "Route Distinguisher"

(RD) and ending with a 16-byte IPv6 address[13].

When a site is IPv4- and IPv6-capable, the same RD can be used for the advertisement of IPv6

addresses and IPv4 addresses.

3.4. Controlling Route Distribution



21

When a PE router is attached to a CE of particular VPN, it learns some of that VPNs IP routes.

Routes learned from the CE routing peer over a particular attachment circuit may be installed in the

VRF associated with that attachment circuit.

These routes are then converted to VPN-IP routes, and "exported" to BGP. If there is more than

one route to a particular VPN-IP address prefix, BGP chooses the "best" one, using the BGP decision

process. This route is then distributed by BGP to the set of other PEs that needs to know about it. At

the other PEs, BGP will again choose the best route for a particular VPN-IP address prefix. The

chosen VPN-IP routes are converted back into IP routes, and "imported" into one or more VRFs.

Every VRF is associated with one or more Route Target (RT) attributes. When a VPN-IPv4

route is created (from an IP route that the PE has learned from a CE) by a PE router, it is associated

with one or more Route Target attributes. Any route associated with Route Target T must be

distributed to every PE router that has a VRF associated with Route Target T. When such a route is

received by a PE router, it is eligible to be installed in those of the PEs VRFs that are associated with

Route Target T [7].

3.5. Route Distribution among PEs by BGP

If two sites of a VPN attached to PEs that are in the same Autonomous System, the PEs can

distribute VPN-IPv4 routes to each other by means of an IBGP connection between them. The term

"IBGP" refers to the set of protocols and procedures used when there is a BGP connection between

two BGP speakers in the same Autonomous System. This is distinguished from "EBGP", the set of

procedures used between two BGP speakers in different Autonomous Systems. Alternatively, each

can have an IBGP connection to a route reflector (BGP-RR).

PE routers distribute Labeled VPN-IPv4 routes, when other PEs processes a received packet

that has this label at the top of the stack, the PE will pop the stack, and processes the packet

appropriately.

In a BGP/MPLS VPN network, logical state is required to maintain connectivity and

reachability between PEs in the form of LSPs and BGP peerings. LSPs are created in the control plane

by label distribution protocol (LDP) and once established, facilitate MPLS packet forwarding in the

data plane. BGP peerings are created in the control plane via TCP sessions which in turn facilitate the

exchange of customer-specific VPN routing information stored in PE routers [7,3].

4. Scalability

Scalability of BGP-MPLS VPN networks can be defined as the ability to grow the service to

any number of VPNs, with any number of sites, by adding more routers and without the requirement to

redesign the network or reconfigure a large number of devices.



22

From a practical point of view, this may be translated into the following requirements:

No single device in the network should carry information about all the VPN sites or about all

the VPN routes in the network.

The addition of a new device, VPN, or site should not require reconfiguring all other devices,

VPNs, or sites in the network.

Control plane information should be distributed to the subset of devices that require it, rather than

to all devices in the network.

Now the scaling properties are discussed by examining the amount of control plane states that

is maintained and the cost of maintaining these states, these aspects are translated directly into memory

and CPU requirements on the routers [14].

4.1. Scalability Aspects for P and CE Routers

P routers do not participate in routing exchanges for VPN traffic and as such, are not required

to maintain VPN routing information. The only state maintained by the core routers is for the PE-to-PE

transport tunnels required for forwarding the traffic tagged with VPN labels between PEs. This state

depends on the number of PEs of the core network rather than the number of VPNs, because the load

from different VPNs can be forwarded along the same PE-PE tunnel.

Thus, adding a new site to the VPNs does not increase the peering the CE must maintain and

does not affect the configuration of existing CEs. So adding more VPNs to the deployment does not

impact the scaling of the CE device.

The effect of adding new VPNs or VPN sites on the P and CE routers are discussed above and

now we examine the effect of provision a new VPN or site [14].

4.2. Scalability Aspects for PE Routers

A PE advertises all routes from all sites connected to it using BGP. The number of BGP

sessions is not dependent on the number of VPN sites attached to the PE. Instead, it is proportional to

the number of peers to which the routing information is distributed, which in the worst case, is the

number of other PE routers in the network (full mesh)[14 , 15], see Fig 2.

However, a PE receives incoming route advertisements for all VPN routes in the network,

receiving all the incoming advertisements would be a problem if the PE was required to maintain all

these routes, as the growth of the VPN service in the network would be limited by the total number of

routes that any given PE can support [14].

As the size of VPN growth, more PE routers are needed resulting in more PE-PE peers and also

more MPLS - LSPs tunnels.



23

Solving the scaling problem resulted from increased number of LSPs is done at the level of

MPLS technology in [10]. In the next section the effect of using the RRs to solve the scalability issue

arises from increase of peering between PEs will be studied. This effectiveness will be proven by

comparing two models; one with RR and another without.

Fig. 2: Full mesh BGP peers between PEs

5. Network Simulation

The main task of this part is to analyze the behavior of MPLS VPN and study the

scalability problem according to particular network design. To accomplish this task VoIP traffic is

used across MPLS VPN backbone that consists of interior gateway protocol (IGP) OSPF, and exterior

gateway protocol (EGP) BGP. OPNET V14.5 package was used to analyze the behavior of MPLS

VPN backbone and the effect of scalability problem solution.

Depending on how the MPLS VPN is implemented by using full mesh between PE

routers or by using rout reflector , we have the following scenarios are considered:

MPLS VPN backbone with IGP without rout reflector (full mesh).

MPLS VPN backbone with rout reflector (RR).

5.1. Network Topology

Three VPNs called VPNA, VPNB, and VPNC are constructed in the OPNET modeler by

using:

5 PE routers (PE1 PE5).

1 P router (P).

6 CE routers (CEA1, CEA2, CEB1, CEB2, CEC1, CEC2).



24

6 subnets each consist of 100 clients.

In the first scenario P router only forward the packets, but in the second scenario it is

configured as RR and all PEs are configured as clients to it. VoIP traffic is send from site A1 to site A2

in the VPN A. The VoIP calls are established in the two scenarios using traffic flow which generates

12 Mbits/second and 100packet/second of G711 voice. Both scenarios were simulated in order to

obtain VPN delay, CPU usage. In each PE BGP is enabled, and peers is determined from BGP

parameters neighbors.

In the first scenario, for each PE all other PEs and the CE attached to it are selected as peers. In

the second scenario Route Reflectors are used to reduce the required number of IBGP connections. RR

set up in the network model by configuring a single compound attribute on the P router that acts as a

route reflector [16].

6. Results and Analysis

Fig 3 shows the End-to-End delay for traffic through an MPLS-BGP VPN. This delay is

measured as time elapsed between traffic entering the "Provider's Network" through Ingress PE

and traffic leaving the "Provider's Network" through Egress PE. Bear mind that VPN delay is not a

physical link delay. The maximum end-to-end delay for MPLS VPN backbone is 400ms.The

recommended value is 150 ms by TTU-T [17].

Fig. 3: VPN delay

Fig 3 shows that VPN delay decreases when RR is used in the network. This is due to the fact

that RR improves the overall performance of the network by reducing the routing state in the

backbone.

When RR is used in the network each PE is peering with the RR only instead of peer with all

other PEs see Fig 4. Thus the number of peers that PE must maintain remains constant regardless of

VPN with RR.VPN Delay (seconds)

Simulation Time (sec)

VP

N d

elay

(s

ec)



25

the number of PEs in the network. This reduces the routing state that PE must maintain which in turn

reduce the CPU usage in the PE router as shown in Fig 5. Another benefit of using RRs is obtained

when adding new PEs to the network, only route reflector need to be reconfigured.

Fig. 4: BGP peers with RR

Small CPU usage appear in Fig 5 because we take it for the PE router which only have control

traffic to ensure the CPU is used only to maintain the routing states and also the network is small so

that the number of routing stat that PE router must maintain is very small, but when using large

number of VPNs the effect of using RR appeared clearly.

RR overcome the problem of provisioning new devices to the network but a new scale problem

appear because the RR maintain all the VPN routes in the network (memory limitation) and RR must

propagate all VPN changes (CPU limitation) .

One way to overcome maintaining all VPN routes in single device is partitioning them in

several RRs, So that each RR maintains subset of the routes. But as the VPN deployment grows,

increasing number of PEs end up peering with all the RRs, as a result RR must send updates to

increasing number of PEs.

As the RR groups grow in the number of clients the efficiency of update processing decreases

and the CPU load increases. So this technique can't contain routing table size explosion. In fact, the

number of BGP routes in such networks may reach around 2 million routes In order to cope with these

scalability issues, incremental solution proposes which is RT constraints [18].

Some PEs receive the updates that not interested in it. Instead of that PE advertise to the RR set

of route target (RT) that is interested in it and the filtering doesnt in the RR which result in reducing

the number of VPN rout advertisement.



26

Fig. 5: CPU Utilization

6. Conclusion

OPNET modeler is used to study the effects of route reflectors solution to cope with the

scalability problem in BGP-MPLS VPNs. Route Reflectors enhance the scalability of the network by

reducing the number of routing states that must be maintained in each provider edge router (PE). Also,

from the above discussion it can be seen that BGP is not appropriate for large scale VPN routing

because it causes some routers to keep state for all routes in the network. RRs make the PE

routers only keep state for constant number of routers regardless of number of PEs in the network. This

reduces the state that PEs need to be maintained. And solve the problem of provisioning new devices.

In large scale network RRs itself keep sates of all routes in the network which prohibits RR to

contain the resulting large routing tables. Rout target filtering (RT) can be used to reduce the number

of VPN route advertisements sent to a BGP peer. This scalability improvement is important for

deployments using route reflectors, because it reduces the processing load placed on the route

reflector.

References

[1] T. Li, CPE based VPNs using MPLS, Juniper network, Internet Draft, available on

http://tools.ietf.org/html/draft-li-mpls-vpn-00, accessed at September, 2012.

[2] P. Veitch, Scalability and Functionality Challenges for MPLS VPN Networks, The Journal

of The Communications Network, 6(2)(2007): 38-44.

[3] E. Rosen and Y. Rekhter, BGP/MPLS VPNs, IETF RFC 2547, March, 1999.

VPN with RR_PE4.CPU Utilization (%)

VPN witout RR -PE4.CPU Utilization (%)

Simulation Time (sec)

VP

N d

elay

(s

ec)



27

[4] Z. Houidi and M. Meulle, A new VPN routing approach for large scale networks, 18th IEEE

International Conference on Network Protocols (ICNP), Kyoto, Japan, p124-133, 2010.

[5] D. Fedyk and Y. Rekhter et.al, Layer 1 VPN Basic Mode, IETF RFC 5251, July 2008.

[6] L. Andersson and E. Rosen, Framework for Layer 2 Virtual Private Networks (L2VPNs), IETF

RFC 4664, Sept. 2006.

[7] E. Rosen and Y. Rekhter, BGP/MPLS IP Virtual Private Networks (VPNs), IETF RFC 4364,

February 2006.

[8] K. Kompella and Y. Rekhter, Virtual Private LAN Service (VPLS) Using BGP for Auto-

Discovery and Signaling, IETF RFC 4761, January 2007.

[9] P. Knight, H. Ould-brahim, and B. Gleeson, Network based IP VPN Architecture Using Virtual

Routers, Internet Draft, available on http://tools.ietf.org/html/draft-ietf-l3vpn-vpn-vr-03, accessed

on Sept., 2012.

[10] I. Peplnjak, J. Guichard, MPLS and VPN Architectures, 3rd

ed., Cisco Press, 2001.

[11] L. Jun and L. Ying, Research for Service Deployment Based on MPLS L3 VPN Technology,

International Conference on Mechatronic Science, Electric Engineering and Computer, Jilin,

China, p1484-1488, August 2011.

[12] L. Ming-hui and X Jing-bo, Research and Simulation on VPN Networking Based on MPLS, 4th

International Conference on Wireless Communications, Networking and Mobile Computing

(WiCOM ), Dalian, China, pp26 -31, Oct. 2008.

[13] T. Nguyen, G. Gastaud, and D. Ooms, BGP-MPLS VPN extension for IPv6 VPN over an IPv4

infrastructure, IETF Internet Draft, Nov. 2001.

[14] I. Minei and P. R. Marques, Scalability Considerations in BGP / MPLS IP VPNs, IEEE

Communications Magazine, 45(4)(2007): 26-31.

[15] F. Palmieri, VPN scalability over high performance backbones Evaluating MPLS VPN against

traditional approaches, Proceedings of the Eighth IEEE International Symposium on Computers

and Communication (ISCC ), 2(2003): 975 - 981.

[16] OPNET Technologies, Inc., OPNET Documentation, 1986-2008.

http://www.opnet.com/support

[17] A. Alvarez, QoS for IP/MPLS Network, 1st ed., Cisco Press, 2006.

[18] P. Marques, Constrained Route Distribution for Border Gateway Protocol/ MultiProtocol Label

Switching (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks (VPNs), IETF RFC

4684, Nov. 2006.

view article

Documents