Fiber – from 2 Strand to Parallel OptionsFiber from 2 Strand to Parallel OptionsCarrie Higbie, RCDD/NTS

Global Director Data Center Solutions and ServicesC i Hi bi @ iCarrie_Higbie@siemon.com

TechTarget – Ask the Experts and columnistSearchNetworking, SearchEnterprise Voice, SearchDataCentersMember IEEE, USGBC, WGBC, Uptime Institute, 7x24exchange,

C l i t P f N t ki S hCIOColumnist Performance Networking, SearchCIO,SearchMobile, ZeroDowntime, ComputerWorld,

IT World, Network World, Financial Review, Strategic Facilities Inside NetworksStrategic Facilities, Inside Networks

PowerPower• According to the Uptime Institute the three year cost of

powering and cooling servers is currently one and a half times the cost of purchasing the server hardware.p g

• Server energy demand has doubled from 2000-2005• 2% of the US electrical usage, 3% by 2011• Cooling and electrical costs represent up to 44 percent of a

data centers TCOdata centers TCO • EU directive to drive a 20% reduction in energy by 2020• U.S. Federal Executive Order 13423

– Improve energy efficiency and reduce greenhouse gas emissions through reduction of energy intensity by:

– 3 percent annually through the end of fiscal year 2015 for a total reduction of 30%

• Data Center Pulse• Data Center Pulse

BudgetsBudgets

Facilities Security

Networking Servers

Why End to End? What to Considery• Every network connection is a combination of

– An end device port– Patch cordsPatch cords– Outlet or jack– Cabling channel– Patch panel(s), consolidation point

S it h– Switch gear– Cabinets, pathways, spaces– Heating and Cooling– Router– Internet connections– Security and application software– End user(s)– Redundant componentsRedundant components

• Any one component’s failure or replacement affects the entire end to end usability – this includes maintenance

End to End – Facilities and Electronics

(3)

Day 2 Maintenance (8)

Channel Type

(1) Channel

Cost

(2) Equipment

Day 1 Costs

MaintenancePatches and Upgrades *

years in service

(4) Total Cost

(5) Cost to

Upgrade Channels

(6) Cost to

Upgrade Equipment

(8)Projected

Yearly Downtime

Costs

(9) Weighting

Factor (Optional)

Category 6Category 6

Augmented 6

Category 7/Class F

Single Mode Fiber M lti d Fib 50Multimode Fiber 50 micron Laser optimized 50 micron

Twinax/Coax/misc.

The need for speedThe need for speed

• VirtualizationC lid ti• Consolidation

• Increasing storage requirements• More analytics• More analytics• Blade technologies• Instant demand for data• Instant demand for data

Courtesy of InfoneticsCourtesy of Infonetics

TR 942 Design Considerations

Top of Rack is Allowed as an exception here only!

Data Center Design – The StandardISO IEC 24764 (D ft)ISO-IEC 24764 (Draft)

Top of Rack is Allowed as an exception here only and must be in rack or close proximity!

TIA-942• Horizontal and Vertical channels shall be run accommodating growth so g g

these areas do not have to be revisited• No more bus or direct connections unless specifically REQUIRED by the

manufacturer• No shared sheath media• Category 6A recommended (echoed by Cisco® 6A or 7) Note: this is an

update from the original TIA-942 document now TIA 942-2OM3 i d f fib• OM3 min grade of fiber

ISO/IEC 24764• Horizontal and Vertical channels shall be run accommodating growth so

th d t h t b i it dthese areas do not have to be revisited• All systems shall be connected via a structured cabling system, point to

point connections are allowed for short run cables in server cabinets only• Category 6 minimum (UTP or F/UTP) Class E• Category 6A minimum (UTP or F/UTP) Class EA

• Category 7 and 7A/Class F and FA

• OM3 min grade of fiber

10G Options10G Options

•10GBASE-EX, 10GBASE-LX, 10GBASE-SX (fiber)•10GBASE LX4 10GBASE LRM (fiber)•10GBASE-LX4, 10GBASE-LRM (fiber)•10GBASE-T (Copper)•10GBASE-CX4 (Twinax point to point Copper - limited 5-15m)•Infiniband (CX4 Cable, Fiber, 4 Pair Copper)( )•ATM/SONET Equivalents (Varies with installation)•SFP+ (limited to 1-5m)

Other Points to KnowOther Points to Know•Maintenance costs are normally based on original equipment purchase prices•Fiber Channel and FC-BASET (2,4,6,8,16,32,64, 128 Gig over twisted pair)•100G and 40G under development in IEEE as next speed increase•Twisted pair higher speed to follow•EEE – new technology to address power – may also include PoE

Short 10GBASE-T historyShort 10GBASE T history

• First chips 10-17W per port (proof of concept)• Gigabit was first introduced at 6.5W per port now <1• New chips are 2.5W in short reach (<30m) mode 3.5W at full 100m• Now that power has decreased 10GBASE-T is becoming viableNow that power has decreased, 10GBASE T is becoming viable• Expected introduction dates Q1 2010 by most manufacturers• Removes distance limitations of SFP+

R d it h t d b i ti f t• Reduces switch counts and oversubscription of ports• Cost released in 2010 $399 / port• Power is now roughly equal to fiber

Fiber Distances

Application Wavelength62.5

160/50062.5

200/50050

500/50050

2000/500 SMF

100BASE-SX 850nm 300m 300m 300m 300m

1000BASE-SX 850nm 220m* 275m* 550m 550m

1000BASE-LX 1300nm 550m 550m 550m 550m 5km

10GBASE-SX 850nm 28m 28m 86m 300m

10GBASE-LX 1310nm 10km

10GBASE EX 1550nm 40km10GBASE-EX 1550nm 40km

10GBASE-LRM 1310nm 220m* 220m* 220m* 220m

10GBASE-LX4 1310nm 300m* 300m* 300m* 300m 10km

*Mode conditioning patch cords will add to channel costs. Only 50 micron 2000/500 and SMF are in 100G/40G IEEE HSSG

End to End Cost over Time

Application Range Channel Cost*

MSRP for Module

Maintenance@15% of

cost/3 years

End to End Cost

Fiber1000BASE-SR 220m-550m $ 381.64 $ 995.00 $ 447.75 $1,824.39 1000BASE-LR 550m $ 381.64 $ 500.00 $ 225.00 $1,106.64 10GBASE-SR 28m-300m $ 381.64 $3,700.00 $ 1,665.00 $5,746.64 10GBASE-LRM 220m-550m $ 381.64 $1,495.00 $ 672.75 $2,549.39 10GBASE-LX4 300m $ 381.64 $4,000.00 $ 1,800.00 $6,181.64

Copper 1000BASE-T 100m $ 376.09 $ 395.00 $ 177.75 $ 948.84 $ $ $ $10GBASE-CX4 3m-15m $ 495.00 $ 600.00 $ 270.00 $1,365.00 Infiniband 3m-15m $ 495.00 $1,295.00 $ 582.75 $2,372.75 10GBASE-T 100m^ $ 376.09 $1,185.00 $ 533.25 $2,094.34

$399.00 /2 $ 179.55 $ 954.64

MSRP for modules based on Cisco®10GBASE-LRM /LX4requires Mode Conditioning patch cords for <OM3 which adds $300 each end and no future support for 40/100G

Emerging Applications – Beyond 10Gbps

Proposed 40Gbps Ethernet• Target: 100m, 850nm laser-optimized 50/125m multimode optical fiber (OM3)

OM4 t 150 ( t h f 125 )• OM4 up to 150m (recent change from 125m)• Target: 10m, copper (twinax) cabling assembly Reduced to 7m Proposed 100Gbps Ethernet• Target: 100m, 850nm laser-optimized 50/125m multimode optical fiber (OM3)• OM4 up to 150m (recent change from 125m)• Target: 10m, copper (twinax) cabling assembly Reduced to 7m tentatively

Ethernet AllianceEthernet Alliance

• Next higher speed study groups – Twisted pair 40/100G– Terabit Ethernet

Fiber Structured SystemsFiber Structured Systems

• Backbone Applications• Single mode for longer runs and higher speedsSingle mode for longer runs and higher speeds

• Storage applications (Ethernet, Infiniband, Fiber Channel, etc.)

• High Speed/Supercomputing point to point connections• Services to adjacent floors and telecom rooms

Si l d f l d hi h d– Single mode for longer runs and higher speeds

• Connectivity options include:– Standard Trunks (LC connectors)( )– MTP/MPO trunks (12 or 24 strand)– Standard Termination methods

MPO/MTP Polarity OptionsMPO/MTP Polarity Options

• Specified within TIA-568-B.1-7 “Guidelines For Maintaining Optical Fiber Polarity With Systems Utilizing Array Connectors”Optical Fiber Polarity With Systems Utilizing Array Connectors and pending TIA-568-C.0-2009 (Annex B)– These are SAMPLE methods only

• ISO/IEC 14763-2 (new version- draft)ISO/IEC 14763 2 (new version draft)• 3 Methods Specified (not interoperable but are changeable)

– Method A: Patch cord mixMethod B: Key matching– Method B: Key matching

– Method C: Pair-wise flip in trunk

Array Polarity OptionsArray Polarity Options

• 3 sample connectivity methods (A, B, C) outlined

• Duplex and parallel configurations covered

• Array components specified within TIA 568 C 3 2008• Array components specified within TIA-568-C.3-2008– Adapters

• Type-A (Key Up/Key Down)• Type-B (Key Up/Key Up)yp ( y p y p)

– Patch Cords• Type-A• Type-B

T C• Type-C

KeyKey

Method A – Patch Cord Mix

Near Far

Key UP Key UP

1 1

2 2

3 3

4 4

5 5

6 6

7 7

8 8

9 9

10 10

11 11

12 12

Method A – ParallelMethod A Parallel

Method B – Key-matching

Near Far

Key UP Key UP

12 1

11 2

10 3

9 4

8 5

7 6

6 7

5 8

9 9

3 10

2 11

1 12

Method B – ParallelMethod B Parallel

Method C – Pair-wise Flip in CableNear Far

Key UP Key UP

1 2

2 1

3 4

4 3

5 6

6 5

7 8

8 7

9 10

10 9

11 12

12 11

Method C – Parallel (Proposed)Key up to Key up

t d ti t

Rx 1Rx 2...Tx 2

mated connection totransceiver

Position 1

A

Key up to Key downmated connection

Position 1

PU

LL

PU

SH

Position 1

PU

LL

PU

SH

PU

LL

PU

SH

Tx 2Tx 1

Type-B arrayconnectorpatch cord

Position 12 Position 12 Position 12

Type-C arrayconnector

patch cordType-C arrayconnector cable

patch cord

Rx 1

Key up to Key upmated connection totransceiver Key up to Key down

mated connection

P iti 1Position 1 Position 1Rx 2...Tx 2Tx 1

A PU

LL

PU

SH

Position 1

Position 12

Position 1

Position 12 Position 12

Position 1

PU

LL

PU

SHP

ULL

PU

SH

TIA-568-C.0 Component SummaryTIA 568 C.0 Component Summary

• Method C not currently listed for parallel

Method C Parallel Signal ProposalMethod C Parallel Signal Proposal

• Type-C array patch cord specified within ‘568-C.3• Type-B and Type-C array patch cords can be used to create valid

Method C polarity option for parallel signals

C C A One Type-B and one Type-C

Infiniband

• There are 3 interface widths; 1x, 4x, and 12x.1X defines a interface with 2 differential pairs Same... 1X defines a interface with 2 differential pairs . Same

for Fiber, 1 Transmit, 1 Receive [Maximum bit rate is 2.5Gbps]... 4X defines a interface with 8 differential pairs (4 per direction). Same for Fiber, 4 Transmit, 4 Receive [Maximum bit rate is 10Gbps][ p ]... 12X defines a interface with 24 differential pairs (12 per direction). Same for Fiber, 12 Transmit, 12 Receive [Maximum bit rate is 30Gbps][Maximum bit rate is 30Gbps]

Two Strand Fiber ApplicationsTwo Strand Fiber Applications

Application Wavelength62.5

160/50062.5

200/50050

500/50050

2000/500 SMF

100BASE-SX 850nm 300m 300m 300m 300m

1000BASE-SX 850nm 220m* 275m* 550m 550m

1000BASE-LX 1300nm 550m 550m 550m 550m 5km

10GBASE-SX 850nm 28m 28m 86m 300m

10GBASE-LX 1310nm 10km

10GBASE-EX 1550nm 40km

10GBASE-LRM 1310nm 220m* 220m* 220m* 220m

10GBASE-LX4 1310nm 300m* 300m* 300m* 300m 10km

*Mode conditioning patch cords will add to channel costs. Only 50 micron 2000/500 and SMF are in 100G/40G IEEE HSSG

Fiber is Changing!Fiber is Changing!

Data Fiber Implementation No. of Length Rate Fiber Implementation Fibers Length

40GBASE-SR4 40Gb/s OM3 4 * 10Gb/s (parallel) 8 100m

40GBASE-LR4 40Gb/s OS1 4 * 10Gb/s (WDM) 2 10km40GBASE LR4 40Gb/s OS1 4 10Gb/s (WDM) 2 10km

100GBASE-SR10 100Gb/s OM3 10 * 10Gb/s (parallel) 20 100m

100GBASE-LR4 100Gb/s OS1 4 * 25Gb/s (WDM) 2 10km( )

100GBASE-ER4 100Gb/s OS1 4 * 25Gb/s (WDM) 2 40km

Infiniband Single (SDR) Double (DDR)

Quad (QDR)

1X 2 Gbit/s 4 Gbit/s 8 Gbit/s4X 8 Gbit/s 16 Gbit/s 32 Gbit/s12X 24 Gbit/s 48 Gbit/s 96 Gbit/s

InfinibandSingle Double Quad g(SDR) (DDR) (QDR)

1X 2 Gbit/s 4 Gbit/s 8 Gbit/s

4X 8 Gbit/s 16 Gbit/s

32 Gbit/s

12X 24 48 96 12X Gbit/s Gbit/s Gbit/s

40GBASE-SR40GBASE SR

• New Term: Bit Skew

A Few Words about Bit Scew

• 40GbE – Packets are numbered for disassembly and reassembly– Zero Skew cable is not needed– Skew from field terminated to MTP likely not be acceptable– 26ns is the Goal (Infiniband)26ns is the Goal (Infiniband)

• 100GbE numbers are not set but are not expected to be an issue with MTP cables

• In short- Parallel optics are is the end of field terminated MMF

• 2 Strand applications can use field terminated SMF• 2 Strand applications can use field terminated SMF but will be more expensive

Plug & Play Assemblies

Much LOWER labor

TrunksTrunks

Plug and PlayPlug and Play

Colo considerations for the ownerColo considerations for the owner

• Space planning!• Preconfigured versus

configure on demandM d i• Managed services

• Cabling as a technical advantageg

• Cabling as a revenue stream

• Subletting space

Managed Backbone Services Using MapIT G2Managed Backbone Services Using MapIT G2

Benefits of MTP/MPO TrunksBenefits of MTP/MPO Trunks

• Saves on consumables• No worry with sharps• No field terminationNo field termination• Lower onsite waste (greener)

S t f hi h d• Support for higher speeds• Faster installation• Don’t need the highest quality installers

Coordination needed with trunksCoordination needed with trunks

• Site must be well definedP th h ld b d fi d– Pathways should be defined

• Slack space must be allowed• Polarity has to be tracked• Should always be testedShould always be tested• Installation plan should always be

providedprovided

Questions?Questions?

• Carrie_higbie@siemon.com• Skype ID: chigbie