Member MapAnnual Report 2011

n London Internet Exchange

Messages from the Board

As I welcome you to LINX’s 17th Annual report, I find my

thoughts split between considering the necessary and appropriate

review of 2011, and eager anticipation of the future.

Trading conditions have not notably altered since our last report,

but the organisation has successfully achieved continued solid

growth and performance whilst undertaking the single

biggest network architecture change in its history. The

bold selection of Juniper networking equipment

to replace the primary LAN was made and

successfully executed, establishing LINX as

Juniper’s first IXP to deploy its hardware.

Key staff hires and restructuring have bedded in well and the Public

Affairs team have had, as ever, a plethora of challenging subjects

to either actively be involved with, or to report upon. Interaction

and collaboration with members with respect to all aspect of the

business has developed well with improved attendance and input

at member meetings alongside the work of the newly formed

Membership Relations team. Evidence of this was portrayed with the

LINX Council elections which successfully encouraged the highest

voting numbers on record. Also, at year end, the LINX London office

moved to more substantial premises with superior working areas and

facilities in line with requirements for now, and in the future.

In the position LINX is in, growth is, and should be, expected. All

worthy metrics for LINX to deliver increasing value on existing, and

some new, fronts are in place, so I look forward to reporting on a

very successful 2012 in a year’s time.

Letter from the Chairman, Grahame Davies

Chief Executive’s Report, John Souter

2011 was a very challenging year and definitely not one to be

complacent! As I write this, in early 2012, I can look back in

satisfaction that we not only successfully rose to those issues

but we came out stronger, more experienced, and perhaps most

crucially, with increased confidence from our members.

What we faced last year is covered in the pages of this annual

report but for the basis of this summary I will look forward to

2012.

In the Spring our Connexions partnership programme was

launched and our regional IXP strategy began to be rolled out,

beginning with IXManchester in April. This year has also presented

a number of engineering challenges including the early adoption

of the Juniper PTX equipment, renumbering of both LANs, the

Extreme LAN expansion program and 100GE member facing ports.

London’s hosting of the Olympic Games has focussed the teams

on making timely critical architectural decisions and carefully

chosen project timelines to cope with uncertainty of port

orders, traffic levels and engineering freezes accompanying the

Games.

In all my time at LINX I’ve never seen such an intense period

of development and it shows no sign of stopping yet.

What happens during 2012 is going to shape how

the exchange operates for many years to come

and I strongly believe we are ready to face the

demands of whatever this industry throws at us.

The Year Ahead

n A Momentous Year

The whole world’s eyes will be on London in 2012.

The London Olympic games are being billed the

digital games, and that means huge volumes of data

originating out of London and across IP networks

to the whole world.

It’s therefore not surprising that the Olympics is a

pivotal event in the LINX calendar and much of

the year is being planned around that four week

global festival.

n LINX Network

LINX will be part of the early adopter

programme for the new range of Juniper PTX

switches which is partly driven by ever increasing

traffic levels, but also by the need to have all

network upgrades completed before we enter a

self-imposed engineering change freeze during the

Games.

2012 also sees a necessary upgrade of our second

peering LAN. This upgrade comes under the same

timescale pressures as the primary LAN and if we

don’t have an acceptable long term solution we can

action before the Games there will be an interim

capacity management programme actioned.

n Port Growth

When it comes to member ports we expect to

pass two significant milestones during the year :

going over five hundred member 10GE ports and

provisioning our first 100GE member facing ports.

n Regional Peering

Strategically the timing, economics and market

demand are right for us to support a regional

exchange initiative throughout the UK. By the

end of the year we hope to be running, or be in

the processes of starting, at least three regional

exchanges.

n Reseller Programme

In the wider global market our ConneXions

reseller programme will be fully rolled out with

promotion and sponsorship at events in EMEA,

North America and Asia. ConneXions is an

extension of our previous LINX from Anywhere

remote access programme and enables third

parties to offer an end-to-end solution using VLAN

technology.

n Community & Regulation

Our work in the Internet community and for The

Good of the Internet will continue through our

Public Affairs activity and support of the many

national and international organisations that help.

On the regulation side there are sure to be

further developments to the Communications

Bill, which may become known as the Electronic

Communications Act, and LINX will make

ourselves available for consultation on behalf of the

membership and their interests.

“In 2012 all eyes will be on London”Ben Hedges, LINX Head of Marketing & Business Development

LINX Engineering

2011 was a year of challenge and significant change

for the LINX peering network and the LINX

Engineering team responsible for its design, build and

operation.

The year began with pressure on both the primary

and secondary LANs to meet rising capacity

demands against a background of operational

problems that were affecting service to LINX

members. These problems were overcome through

close collaboration between LINX, Brocade

and Extreme, allowing LINX to meet service

and capacity needs and then to move on to the

replacement of the primary, Brocade LAN.

The contract to replace the primary LAN was

awarded to Juniper and a successful migration to a

new VPLS based LAN took place during September

and October, after months of joint planning with

Juniper.

We ended 2011 with both the new primary and

secondary LANs stable and meeting capacity

demands, which had risen from 800Gb/s peak traffic

in January to 1.2Tb/s by December. Added to this,

600Gb/s plus of private interconnect traffic facilitated

and provided via LINX, sees the LINX peering

platform facilitating the exchange of around 1.8Tb/s

of peering traffic for our members.

n Early Year Challenges

As traffic peaked towards 800Gb/s at the end of

2010, LINX suffered stability problems on both LANs.

The primary LAN suffered a number of incidents

at the end of 2010 into early 2011. Traffic loss was

being regularly experienced between the MLX

switches used at our densest sites, caused by loss of

individual LAG ports. This also contributed to ring

instability as protection mechanisms kicked in, often

leading to flapping between main and standby paths.

We also suffered a high number of errors associated

with Traffic Manager cards.

By working closely with Brocade, we identified the

required solutions, comprising a software upgrade

to address the LAG port incidents and hardware

changes to the switch fabrics and line cards. These

measures resolved the ring instability and traffic error

problems we were seeing on the MLX switches.

This joint plan with Brocade gradually returned the

Brocade LAN to stability, allowed further capacity

upgrades within the densest sites and enabled the

deployment of 32x10Gb/s LAGs on ISLs. These

changes were completed in April 2011 and supported

traffic growth on the Brocade LAN, which rose from

600Gb/s in January to 791Gb/s by September.

Inter switch capacity demand on the secondary

LAN was also rising beyond the capacity of the

Extreme 8810 switches at our Telehouse and

Sovereign House sites. This demand was met by

the deployment of new BD20k switches. These

provided the core capacity to upgrade the inter

switch links between these sites to 12x10Gb/s, and

provide additional member port capacity.

In addition, a software upgrade on the 8810

switches elsewhere in the secondary LAN fixed a

problem with load balancing on LAGs. We were

then able to upgrade the LAN rings to our remote

sites, delivering a minimum of 8x10Gb/s on all

LAN ring segments. This successfully delivered

the capacity plans committed to LINX members

for 2011, as traffic rose from 200Gb/s to 240Gb/s

through the year.

n Juniper LAN Migration

In 2010, LINX had decided to move away from

ring based LANs to VPLS. The issues described in

late 2010/early 2011 had delayed these plans, but

progress was still made through extensive proof

of concept testing with Brocade and Juniper, that

proved that both vendors had potentially viable

solutions.

A request for quotation was issued to both vendors

in March 2011, followed by intensive work on high

level designs, migration strategy, capacity planning,

support strategies and roadmap reviews.

n 2011 - A Challenging Year

LINX Engineering

At the end of this process, LINX decided to award

the contract for the primary LAN replacement

to Juniper, receiving LINX Board approval on

14 June. This was Juniper’s first major Internet

Exchange contract win and it was achieved on the

back of a strong technical proposal and executive

commitment from Juniper at the highest level.

The design was based on Juniper’s MX router

technology, but also required a major re-design of

the LINX transport and fibre network to underpin

the new LAN, with MRV, Kylia, Laser 2000 and

Geo Networks providing the transport solution.

A programme structure was established for

the primary LAN replacement. LINX hired an

experienced programme manager, Tony West from

Nxtera, who had previous experience with major

network migrations and with Juniper.

The programme team comprised LINX Engineering,

IT, Member Relations, Juniper and Telindus. Close

collaboration between LINX, Juniper and Telindus

ensured a detailed low-level design, testing and

migration strategy that was jointly underwritten by

all parties.

“The Juniper migration was a great success”Derek Cobb, LINX Chief Technical Officer

Systems developments were planned and delivered

through a close partnership between LINX

Engineering and IT.

Member communication was delivered through a

special member facing web-site that detailed the

overall plan and allowed members to see when

their ports were scheduled to move from the

old Brocade LAN to the new Juniper LAN. This

was delivered by co-ordinated activity between

Engineering, IT and Member Relations.

Following intensive operational testing and

acceptance into service, member migration began

on 26 September and completed on 29 October.

The migration was a complete success. All work

was completed within the designated maintenance

windows and without any unplanned interruption

to service.

n By Year End

At the end of December, combined traffic across

the primary and secondary LANs had reached

1.2Tb/s, a 50% increase across the year involving

a problematic start, a series of interim capacity

upgrades and a major LAN migration.

The LAN migration had been a major success, but

not without some mistakes and near misses along the

way. The lessons learned from the migration were

built in to a re-shaped Engineering team, which will

give us an improved focus on architecture, capacity

planning, engineering, operations and service delivery.

n And for 2012?

Planned growth on the primary LAN will see the

introduction of four new Juniper PTX switches,

providing greater traffic handling capacity and enabling

ISL expansion to 32x10Gb/s. The switch fabrics of

all our MX routers will also be upgraded, enabling

new edge port and ISL capacity, a new PoP will open

at Telehouse West and we will launch a 100Gb/s

member access.

The secondary LAN will be upgraded from its

current ring based architecture to a VPLS design. A

contract has now been agreed with Extreme that will

provide capacity for the Olympic Games and enable

the migration to the new architecture later this year.

In addition, LINX will be carrying out a major LAN

re-numbering exercise in 2012 to support further

member growth, which is particularly important to

support the new ConneXions programme.

And in the middle of all this is the service and

capacity challenge that will be caused by the 2012

London Olympics. It promises to be another

interesting year!

The Juniper programme delivered the following:

• 10 LINX POP sites migrated

• 110 x 100 Mb/s ports • 196 x 1Gb/s ports • 277 x 10 Gb/s ports• 791Gb/s peak traffic

“The Network Architecture Refresh Programme has taken the LINX infrastructure to the next level. It is robust, resilient and has the capacity for further expansion in the future.”

Derek Cobb,LINX Chief Technical Officer

Public Policy

n 2011 in Review

2011 was a year bookended by battles over human

rights, and marked for history by the Arab Spring.

For the world of Internet policy, the Arab Spring

was a seminal moment, as governments rushed

to react to the implications of a popular revolt

empowered by online communications.

The Arab Spring itself began on 17th December

2010, when a young street vendor called Mohamed

Bouazizi set himself on fire in protest at his

harassment by a corrupt police force. The wave

of Internet-coordinated protest marches that

followed toppled first the 23-year old regime of

President Zine El Abidine Ben Ali and then, as

people learned they could organise online to defeat

seemingly impregnable dictatorships, swept across

the entire region. Within a year governments had

been overthrown in Egypt, Yemen and even - after

a NATO-backed civil war - Libya, while in five more

countries government leaders were dismissed or

agreed to step down peacefully.

It wasn’t just the Internet that emboldened the

peoples of a whole region to cast down long

standing despots: Al-Jazeera, the ubiquity of mobile

phones, and the muezzin’s call to Friday prayers

all played an important role. But the importance

in this of the new-found ability of people to

come together and share their grievances, and to

discover that they are not alone, was recognised

by protestors and governments alike. Government

blocking of sites like Facebook and Twitter did not

abate, for all its futility; the operator of a Tor Internet

proxy node in Tunisia was arrested and held in

“Increasingly, our members tell us that public policy developments are some of the most important challenges facing their businesses” - Malcolm Hutty, LINX Head of Public Affairs

From the Digital Economy Act to the Data Retention Directive, LINX

and its members have been at the heart of Internet policy in 2011.

BT and TalkTalk sought and won a judicial review of the Digital

Economy Act, which, though ultimately unsuccessful, won crucial

concessions for ISPs in the area of costs.

At European level, LINX and EuroISPA played an important role in

representing ISPs interests in the review of the EU Data Retention

Directive.

Meanwhile, as well as working on the day to day negotiations

of policy detail in the UK and EU, we continued to engage with

institutions that are the pre-cursors of domestic policy, including UN,

OECD and the Council of Europe.

Facing PageAnalysis of the major global events

of 2011 and their impact on the global Internet public policy debate

“World events such as the Arab Spring played an unprecedented role in shaping Internet policy in 2011.”

Malcolm Hutty, LINX Head of Public Affairs

secret (he later became Minister for Youth). In

Egypt, for five days, the government disconnected

the country from the global Internet entirely.

The lessons Internet policy-makers worldwide

have drawn from this period have been mixed.

Global leaders like U.S Secretary of State Hillary

Clinton positioned themselves as being on the

side of the people, committing their country to

the cause of freedom on the Internet. International

treaty organisations like the OECD and the

Council of Europe proclaimed new charters of

Internet rights which, though easy to despise

as woolly and toothless, will have lasting impact

in debates on national legislation, at least in the

developed world.

At the same time, government security services

reacted with fear, rather than inspiration, and

while the White House insisted on its need for

an “Internet kill-switch” UK Prime Minister David

Cameron reacted to riots in London by mulling the

possibility of blocking access to social networking

sites, just as had been done in Tunisia. This quickly

looked ridiculous in the light of local communities’

use of Facebook to spontaneously organise post-

riot clean-up and anti-rioter vigilance.

When Cameron and Foreign Secretary William

Hague convened the “London Cyberconference” to

lecture world governments on the need to ensure

cyber-security’s compatibility with human rights,

the Prime Minister’s initial instinct for Internet

suppression seemed a political embarrassment.

In 2011 British security officials continued to

lobby Ministers to whittle away at their promise

to abandon the unlamented Interception

Modernisation Programme, and American federal

and British law enforcement each pressed DNS

registries for the unrestrained power to cancel

domains of suspected malefactors.

Otherwise, it was mostly the commercial interests

of the copyright lobby that justified ever stricter

government suppression of online communications.

Progress towards ratification of the secretly

negotiated ACTA treaty continued, prompting

millions of Eastern Europeans onto the streets in

protest at moves to undermine their recently won

rule of law, and threaten the Internet access surveys

said they valued more than television or chocolate.

BT and TalkTalk failed to convince a court to

overturn the Digital Economy Act, largely on

the grounds that their complaint was premature

while it remained conceivable the Act could

be implemented in a manner compatible with

European law. The European Court of Justice

proved more robust, and in two precedent-setting

cases threw out demands for Internet filtering from

Belgian music industry body SABAM. The Court

based its ruling not in procedural niceties or the

technicalities of statutory interpretation, but in a

ringing endorsement of defence of fundamental

rights as constitutionally protected across Europe by

the Charter of Fundamental Rights and Freedoms.

It was a moment that would have made an Internet

activist of the Jasmine revolution proud.

“The number of routes peering at LINX now stands at 334,000 - more than any other Internet Exchange”

Ben Hedges, LINX Head of Marketing & Business Development

Exchange Growth

n Membership

For many membership organisations the only

way to measure growth is by the number of

members it has. At LINX this is just one of the

ways the exchange grows, and in many respects

is the one with the smallest impact. In 2011 the

LINX membership continued to grow at a good

rate and although tough economic conditions

meant there was still some churn due to mergers,

acquisitions and cancellations, we had the highest net

membership increase ever, adding 37 new members

over the year.

The number of new membership applications

during 2011 totalled 49 which is an incredible

performance considering the first half of the year

was overshadowed by network stability issues.

Applications continue to come in at a 2:1 ratio for

International : domestic new members, re-enforcing

LINX’s position as one of the must-

join Internet Exchange Points.

n Routes

Whilst size of the membership is an important

factor for an IXP because it can given an indication

of the peering opportunity, this is only one of

many factors as the geographic spread of that

membership and the routes they represent on the

global routing table begin to give those numbers

some context.

The Global Routing Table is the set of all Internet

address prefixes, or autonomous system numbers

(ASNs) announced into the Default-Free

Zone (DFZ). Together they make the entire

public Internet. The Global Routing Table does

not physically exist anywhere, and the use of

route-filtering anyway means that no router has

a complete view of all routes. However through

our route collectors, which are mandatory for

members to connect to, LINX is able to quantify

its coverage of the Internet better than any other

Internet exchange in the world.

By the end of 2011 routes peered at LINX

accounted for more than 80% of the global routing

table. During the course of the year the number

of routes peered at LINX increased by nearly

100,000 to 334,000.

LINX is also continually growing it’s global footprint

and by the end of the year had members passing

traffic from 50 countries across the globe.

n Ports

The economy and size of LINX is best reflected

through the number of 10GE member ports

that are connected to the exchange. This growth

primarily comes from existing members who have

increased traffic going over their networks that

they wish to peer at LINX.

This kind of repeat and recurring business gives

us a very real barometer with regards to member

satisfaction and there was an acute demonstration

of this in 2011. For the first six months of the year,

when the network was experiencing stability issues,

there were only 39 new 10GE ports added to the

network. The second half of the year, particularly

after the migration to the new LAN was complete

saw orders jump by more than 307% with 63

more new ports connected and a further 32

still at the provisioning stage. That saw us finish

2011 with 410 member 10GE ports connected

to the exchange, a year on year increase, after

cancellations, of 102.

n New Products

For much of the previous 17 years LINX has

concentrated all its efforts on technical innovation

and service delivery. This continued during 2011

with LINX becoming the first Internet Exchange in

the world to move their primary LAN to Juniper.

In addition to this, 2011 saw the first new product

launch in many years as we unveiled our new

channel partner programme, with the product name

of ConneXions being derived from the fact our new

partners would be making multiple new member

connections to the exchange.

The product, which was only made possible by

updating the network to be MPLS/ VPLS enabled,

allows third parties to sell ports at the London

Internet Exchange for the first time ever.

They will do this by selling smaller fractions of a

10GE port in denominations as small as 100M. In

theory, one single 10GE port could accommodate

100 new members connecting at that level but in

practice the channel partners will introduce a range

of new members to LINX on varying port sizes,

including many being connected at over 1GE.

The ConneXIons product has been launched to

stimulate further growth in membership numbers

and increase the number of routes available to

them, but will not, in the short term at least, have

any substantial impact on the broader 10GE port

numbers or traffic figures.

Exchange Growth

“LINX is the ‘must have’ peering point for any large content provider”Patrick Gilmore, Akamai

n Traffic

The other key growth measurement for an

Internet Exchange is the amount of traffic being

exchanged there. At LINX we provide settlement

free peering of packets between our members

in two ways. All members connect to the main

exchange and pass some of their traffic this way.

We also provide a service where members who

are passing significant amounts of traffic with

specific other members can do this through our

private interconnect service. The total traffic

facilitated through the exchange is the sum of

these two figures.

By the end of 2011 traffic on the main

exchange had peaked at over 1.25Tb/s. This

was supplemented by approximately 800Gb/s

of traffic over private interconnections making

LINX the first exchange in the world to carry

more than 2Tb/s of member traffic. This is a

huge increase on the 1.4Tb/s at the end of 2010

(826Gb/s on main exchange and 640Gb/s on PI).

Growth in Connected 10GE Member Ports450

400

350

300

250

200

150

100

50

0

Con

nect

ed 1

0GE

Mem

ber

Port

s

2007 2008 2009 2010 2011

100M Ports 1GE Ports 10GE Ports

Profit & Loss Account 2011 2010

£ £

Turnover 8,186,551 6,729,462

Distribution costs 26,276 14,305

Administrative expenses 7,753,999 6,138,420

Other operating income 740,111 727,365

Operatingprofit 1,146,387 1,304,102

Interest receivable 13,549 10,055

Profit on ordinary activities before taxation 1,159,936 1,314,157

Tax on profit on ordinary activities 18,986 10,584

Profitonordinaryactivitiesaftertaxation,beingprofitforthefinancialyear 1,140,950 1,303,573

Balance Sheet 2011 2010

£ £

Fixedassets 5,908,716 2,527,648

Currentassets 2,136,127 3,972,042

Currentliabilities 1,284,490 879,288

Netassets 6,761,353 5,620,402

Members’funds 6,761,353 5,620,402

Financial Report

n Financial Commentary

The annual LINX budget is presented to members

for approval at the last quarterly meeting of the

year. It was a good year for LINX financially, and we

are proud of what we have achieved in 2011.

Turnover increased by £1,457,089 (21.7%) to

£8,186,551, reflecting the growth of LINX - both

in terms of membership and the number of ports

being used. Included in this was also an increase in

revenue from reselling services (mainly colocation,

but also some fibre) to members.

There was quite a substantial increase in

expenditure, corresponding to the increased

activity due to growth. This expenditure was

also significantly affected by the considerable

capital expenditure - connected to the complete

replacement of our primary network - in fact the

largest single year capital expenditure in the history

of LINX. Expenditure grew by £1,615,599 to

£7,753,999.

Despite being a not-for-profit organisation,

the surplus for the year was £1,140,950. LINX

surpluses are always either re-invested in the

network and/or used to cut prices.

The investment in the network has obviously been

reflected by the value of fixed assets in the balance

sheet for the year.

A full copy of the financial statements and auditors report is available to members

LINX in the Wider WorldMessages from the Board

LINX is an

“Organisational Member”

of ISOC, the Internet

Society. ISOC’s policy and

international engagement activities are rooted in the

organisation’s fundamental belief that the Internet

should be available to people everywhere.

The organisation – and its members and Chapters

– work with governments, national and international

organisations, civil society organisations, the private

sector, and other parties and stakeholders to reach

decisions about the Internet that conform to its core

values. LINX strongly supports the ISOC’s mission

to preserve and protect the open, collaborative,

distributed, multi-stakeholder model that has defined

the successful development of the Internet to date.

n LINX in the Internet Community

LINX continues to enjoy a supportive and

collaborative relationship with its community

affiliates. The result of LINX relationships with

the wider community affords members the

opportunity of representation at a global level,

the chance to benchmark performance, a role in

helping make the Internet a safer, more trusted

space and the chance to influence European

policies and procedures.

The organisations LINX work with include

EuroISPA, ISOC, Euro-IX and the IWF.

EuroISPA, the pan-

European association

for organisations

representing the ISP

industry, is a key partner for LINX, enabling us

to influence policy and legislation at the EU level.

Malcolm Hutty, LINX’s Head of Public Affairs, was

elected as President of EuroISPA in 2009.

LINX is a founder member

of Euro-IX, the European

Internet Exchange

Association. LINX and the

IX community meet to exchange best practice,

promoting an open interchange of ideas between

exchange point operators, developing common

procedures, and sharing and publishing statistics.

LINX was instrumental in the

creation and start-up of the

Internet Watch Foundation

(IWF) as a way to help hosting

providers remove child abuse

images from their server. LINX works with the IWF

to ensure that the practical imperatives of LINX

members are fully incorporated in IWF policy.

R

The blue areas on the map shows the 50 countries around the world where LINX has members.

Member MapContact LINX

LondonInternetExchangeLtd21-27 St Thomas StreetLondon SE1 9RY United Kingdom

LondonInternetExchangeLtdTrinity Court Trinity Street Peterborough PE1 IDA United Kingdom

Phone: +44 20 7645 3501Fax: +44 20 7536 0720Email: info@linx.netWeb: www.linx.net

Front cover image: © Mike Hellers 2012