CANARIE

“The Customer Empowered Networking Revolution”

http://www.canarie.ca

http://www.canet3.net

Background Papers on Gigabit toThe Home and Optical InternetArchitecture Design AvailableOptical Internet News list:Send e-mail to Bill@Canarie.ca

Bill.St.Arnaud@canarie.ca

Outline The Message CANARIE CA*net 3 CA*net 4 Gigabit Internet to the School and Home

The Message In mid 1990s the prevailing wisdom was that commercial sector would drive

design of Internet infrastructure R&E networks would focus on applications or specialized services

As a result in North America R&E networks were commercialized or discontinued e.g NSFnet & CA*net

However new network technologies and most importantly dark fiber is allowing R&E networks to once again redefine telecommunications not only for themselves but also for businesses and most importantly the last mile to the home

R&E networks may become the cornerstone of municipal fiber to the home networks

Over time the current hierarchical “connection oriented” telecom environment will look more like the Internet which is made up of autonomous peering networks.

These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community.

Mission: To facilitate the development of Canada’s communications infrastructure and stimulate next generation products, applications and services

Canadian equivalent to Internet 2 and NGI private-sector led, not-for-profit consortium consortium formed 1993 federal funding of $300m (1993-99) total project costs estimated over $600 M currently over 140 members; 21 Board members

CANARIE Inc

GigaPOP

CA*net 3 National Optical Internet

Vancouver

Calgary ReginaWinnipeg

Ottawa

Montreal

Toronto

Halifax

St. John’s

FrederictonCharlottetown

ORAN

BCnet

Netera SRnet MRnet

ONet RISQ

ACORN

ChicagoSTAR TAP

CA*net 3 Primary Route

Seattle

New YorkLos Angeles

CA*net 3 Diverse Route

Deploying a 4 channel CWDM Gigabit Ethernet

network – 400 km

Deploying a 4 channel Gigabit

Ethernet transparent optical DWDM–

1500 km

Multiple Customer Owned Dark Fiber

Networks connecting

universities and schools

16 channel DWDM-8 wavelengths @OC-192 reserved for CANARIE-8 wavelengths for carrier and other customers

Consortium Partners:Bell Nexxia

NortelCisco

JDS UniphaseNewbridge

Condo Dark Fiber Networks

connecting universities and

schools

Condo Fiber Network linking all

universities and hospital

Customer Empowered Networks School boards and municipalities throughout North America are deploying

their own dark fiber networks in partnership with next generation carrier Individual institutions – the customers – own and control their own strands

of fiber Fiber are configured in point to point private networks; or Connect to local ISP or carrier hotel

Low cost LAN architectures and optics are used to light the fiber Control and management of the optics and wavelengths is under the domain of the

LAN customer at the edge, as opposed to the traditional carrier in the center These new concepts in customer empowered networking are starting in the same

place as the Internet started – the university and research community.

Customers will start with dark fiber but will eventually extend further outwards with customer control and ownership of wavelengths

Extending the Internet model of autonomous peering networks to the telecom world

Examples Customer Empowered Networks

Universities in Quebec are building their own 3500km “condominium” fiber network in partnership with 6 next gen carriers- $US 2million

Will deploy and manage their own optics and long haul transmission gear Universities in Alberta are deploying their own 400 km 4xGbe dark fiber

network - $US 200K Deploy and manage their own optics and long haul transmission gear

City of Montreal is second most fibered city in the world because of municipal owned open access conduit

In Ottawa is deploying a 85km- 144 strand “condominium” network connecting 26 institutions – cost $1m US

Peel County – Missassuaga & Brampton has built a 200km public sector fiber network - $US 5m

Many other cities including Ashland OR, Halifax, Toronto are looking at similar initiatives

Market Drivers First - low cost

Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in

network Network Restoral & Protection can be done by customer using a variety of

techniques such as wireless backup, or relocating servers to a multi-homed site, etc

Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video

Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3rd party at a carrier

neutral collocation facility

À venir

Bande passante louée

Projet démarré

Construit

Observatoire Mont-Mégantic

Val d’Or/Rouyn

MAN de Montréal

MAN de Québec

MAN de Sherbrooke

MAN d’Ottawa/Hull

Quebec University Condo Network

St-Laurent/Vanier

Lanaudière

Maisonneuve

Marie-Victorin

Champlain

Rosemont

Sorel-Tracy

Montmorency

Édouard-Montpetit

Vieux-Montréal

Bois-de-Boulogne

Ahuntsic

Lionel-Groulx

Vers Québec

Gérald-Godin

John-Abbott

André-Laurendeau

Dawson

À venir

Bande passante louée

Projet démarré

Construit

Montreal Public Sector Condominium Networks

Schoolboard Condominium Builds

Typical Capital Costs Fixed One Time Capital Costs Include

Management, engineering and construction costs Negotiating support structure agreements Fiber optic cables Fusing of fibers OTDR sweeps, Premise termination, etc.

Average total cost between $7 and $15 per meter as follows: Engineering and Design:

$1 - $3 per meter for engineering, design, supervision, splicing Plus Installation:

$7 to $10 per meter for install in existing conduit; or $3 to $6 per meter for install on existing poles

Plus Premise termination: Average $5k each

Plus cost of fiber: 15¢ per strand per meter for 36 strands or less 12¢ per strand per meter for 96 strands or less 10¢ per strand per meter 192 strands or less 5¢ per strand per meter over 192 strands

Examples of Dark Fiber costs University network Urban Fiber Builds

Varennes: 50 km - $406K (maintenance $26K/year) Montreal East: 14 km - $120K (maintenance $9K/year) Laval: 33km - $213K (maintenance $15K/year)

University network Rural Fiber Builds Sorel: 54km - $266K (maintenance $19K/year) Megantic: 40km -$273K (maintenance $14K/year)

Schoolboards Victoriaville school board -Average price for fiber(s) $2 - $7 per meter Spokane School District - $US 800/mo for first 5 years then $US 400/mo Over 50 schools Stockholm - $1200/mo – over 100 schools

Las Vegas School district – 240 schools – Telcordia (Bellcore) prime contractor Many, many others in the works Companies like Telcordia (Bellcore), IBM, etc are now leading development of

dark fiber networks for schools

Condo Fiber Build Examples Des affluents: Total cost $1,500,00 ($750,00 for schools)

70 schools 12 municipal buildings 204 km fiber $1,500,000 total cost average cost per building - $18,000 per building

Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools) 80 schools 18 municipal buildings 223km $21,428 per building

Laval: Total cost $1,800,000 ($1,000,000 for schools) 111 schools 45 municipal buildings 165 km $11,500 per building

Peel county: Total cost $5m – 100 buildings Cost per building $50,000

Ottawa Fiber Condominium

Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes

26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second

build under way

Typical Payback for school(Real example – des affluents – north of Montreal)

DSL to 100 schools - $400 per month per school Over 3 years total expenditure of $1,440,000 for DSL service Total cost of dark fiber network for 100 schools $1,350,000 Additional condominium participants were brought in to lower cost to

school board to $750,000 School board can now centralize routers and network servers at each

school Estimated savings in travel and software upgrades $800,000

Payback typically 8 –16 months Independent Study by Group Secor available upon request

Before

After fiber

fiberAntennas 780Novell Servers 82 1SQL Servers 13 3Lotus Notes Servers 21Tape Backup Servers 12 4Ethernet switches/hubs 10 98Routers 1083Cache/proxy (Linux) 120Fire walls (Linux) 11

Reduction in the number of servers

CA*net 4

VancouverCalgary

Regina Winnipeg

Ottawa

Montreal

Toronto

Halifax

St. John’s

Fredericton

Charlottetown

Chicago

Seattle

New York

Europe

Dedicated Wavelength or

SONET channel

OBGP switches

Optional Layer 3 aggregation service

Large channel WDM system

Overall Objective

To deploy a novel new optical network that allows GigaPOPs at the edge of the network (and ultimately their participating institutions) to setup and manage their own wavelengths across the network and thus allow direct peering between GigaPOPs on dedicated wavelengths and optical cross connects that they control and manage

To allow the establishment of wavelengths by the GigaPOPs and their participating institutions in support of QoS and grid applications

To allow connected regional and community networks to setup transit wavelength peering relationships with similar like minded networks to reduce the cost of Internet transit

To offer an “optional” layer 3 aggregation service for those networks that require or want such a facility

O-BGP (Optical BGP) Control of optical routing and switches across an optical cloud is by the customer – not

the carrier Use BGP peering at network configuration stage for process to establish light path cross

connects Customers control of portions of OXC which becomes part of their AS Optical cross connects look like BGP speaking peers

All customer requires from carrier is dark fiber, dim wavelengths, dark spaces and dumb switches

Building “carrier free” networks Traditional BGP gives no indication of route congestion or QoS, but with DWDM wave

lengths edge router will have a simple QoS path of guaranteed bandwidth May allow smaller ISPs and R&E networks to route around large ISPs that dominate the

Internet by massive direct peerings with like minded networks Wavelengths will become new instrument for settlement and exchange eventually

leading to futures market in wavelengths

The biggest challenge of all…To foster and accelerate

broadband Internet to the home

The basic assumptions The good, the bad and the ugly..

Monopolies are bad Duopolies are ugly Facilities based competition is good

The private sector, in an open competitive market, is far more effective at responding to consumer’s needs and introducing new services at lower prices than any kind of government regulation

But government has a responsibility to foster competition and ensure a level playing field

Where a natural monopoly exists government has a responsibility to regulate that monopoly, but only as a last resort

First it should make every attempt to develop mechanisms for introducing private sector competition rather than depending on legislative fiat

Regulation should be seen as a last resort

Facilities based competition in the residential neighborhood?

Facilities based competition is alive and well in downtown core The biggest challenge for governments is manage and coordinate the

digging up of streets Outside of downtown in big cities

Usually only a monopoly telecom provider At best a duopoly

How do we introduce facilities based competition into this market (or at least come as close as possible to true facilities based competition)?

As well how can we assure scalable high speed Internet services to the home that eventually will support Gigabit speeds or higher?

Critical role for governments and universities

Municipal dark fiber networks increases facilities based competition, levels the playing field and provides greater choice to the consumer

Governments can play a critical role in paying for dark fiber to all public sector buildings

Private sector can extend the fiber to businesses and homes ( via wireless, fiber, DSL, etc)

Universities can play critical role in organizing municipal condominium fiber builds in their community and serve as the “anchor tenant”

Governments and universities can also encourage building carrier neutral collocation facilities

In downtown cores will likely be done by private sector In suburbs will probably have to be public facility like school board

office, university, etc

Networked NationCA*net 4

Provincial research and education network

Usually one GigaPOP per province

Usually one access facility in every major town and city

School board office City Hall University

School HospitalLibrarySchool School

Colo Colo

Option B: Home owners are aggregated at node by service provider of their

choice

Option A: Home owners and businesses have fused

connections all the way to service provider at supernode

SuperNodes

Nodes

Colo

Colo

Splice Box

Homes

Splice Box

CommercialInternet

CommercialInternet

Benefits to Industry For cablecos and telcos it help them accelerate the deployment of high speed

internet services into the community Currently deployment of DSL and cable modem deployment is hampered

by high cost of deploying fiber into the neighbourhoods Cable companies need fiber to every 250 homes for cable modem service,

but currently only have fiber on average to every 5000 homes Telephone companies need to get fiber to every 250 homes to support

VDSL or FSAN technologies Wireless companies need to get fiber to every 250 homes for new high

bandwidth wireless services and mobile Internet It will provide opportunities for small innovative service providers to offer

service to public institutions as well as homes For e-commerce and web hosting companies it will generate new business in out

sourcing and web hosting For Canadian optical manufacturing companies it will provide new opportunities

for sales of optical technology and components

CANARIE's 6th Advanced Networks Workshop"The Networked Nation"

November 28 and 29, 2000Palais des Congrès

Montreal, Quebec - Canada

"The Networked Nation", will focus on application architectures ("grids") made up of customer owned dark fiber and next generation Internet networks like CA*net 3 that will ultimately lead to the development of the networked nation where eventually every school, home and business will have high bandwidth connection to the Internet.

Three tracks: Customer owned dark fiber for schools, hospitals, businesses and homes. Next generation optical Internet architectures that will be a natural and seamless

extension of the customer owned dark fiber networks being built for schools, homes and businesses.

"application grids", which are a seamless integration of dark fiber and optical networks to support specific collaborative research and education applications.