1

Broadband Network Design -

CERNET Experience

Xing LiCERNET

Feb. 22, 2005

2

Outline

! Introduction! The user demand analysis! The management and security response! The QoS experience and the tools! The 40G IP backbone trial! IPv6 challenges! Conclusion

3

Introduction

! The university campus networks have been

practicing broadband services for a very

long time. This presentation will introduce

the CERNET’s campus broadband services

and discuss the lessens learnt.

4

CERNET Transport Network大�

北北北北京京京京

�州州州州

石家石家石家石家庄庄庄庄

武武武武�

合合合合肥肥肥肥

南南南南京京京京

上上上上海海海海

�沙沙沙沙

南南南南昌昌昌昌 杭杭杭杭

州州州州

广广广广州州州州

福福福福州州州州

成成成成都都都都

昆昆昆昆明明明明

重重重重�

厦�

海海海海口口口口

桂林桂林桂林桂林

沈沈沈沈阳阳阳阳

哈哈哈哈��

天天天天津津津津

�南南南南

南南南南宁宁宁宁

深深深深圳圳圳圳

太太太太原原原原

青青青青�

5

CERNET Backbone (1)

6

CERNET Backbone (2)

! 18 national Pops are connected via multiple 2.5Gbps DWDM links

! 8 provincial Pops are connected via multiple 155Mbps SDH links

! 38 GigaPops distributed in 36 cities, covering all the provinces in

Mainland China.

! 300 campus networks connect to their nearest Pops via 100Mbps-

1Gbps links.

! 1,500 education and research institutions connected

! 15 million users

! 320 million students/school kids in mainland China

7

CERNET2 Backbone (1)

8

CERNET2 Backbone (2)

! 5 10 GPops are connected via 10G DWDM links

! 15 GigaPops are connected via 2.5G links

! 25 GigaPops distributed in 20 cities.

! 100+ campus networks connect to their nearest Pops via 1Gbps links.

! Pure IPv6 service

9

X.25NODE

X.25 NODE

X.25NODE

PAD

PAD

PADIBM ELXSI

¼ÆËã ÖÐ ÐÄ

IBM DPS8¼ÆËã ÖÐ ÐÄ

ADM5Aͼ Êé ¹Ý

ADM5Aͼ Êé ¹Ý

IBMPC/ XTË® Àû ϵIBM VAX

ͼ Ïñ ÖÐ ÐÄ

IBMPC/ XT»ú е ϵ

IBM5550

IBM5550

IBMPC/ XTУ »ú ¹Ø

PAD PAD

1988

TUNET 1988

10

FDDI»·

MB

MB

MB

CC

C.C.

Mini Super

B

R

MR

SUN

NS

PABX

TSSN

ÖÐ ¹ú ¹« ¹² ·Ö ×é½» »» ÍøCN- PAC

CIMS NCFC

Çå »ª ó́ ѧ У Ô° Íø TUNET1993Äê

FHUB

FC

FC

FC

F

F

NHUB

NHUB

C

C

S

NHUB

C

C

FHUB

FC

FC

FC

FC

FC

F

C

NHUB

1993

TUNET 1993

11

R9

R10

R11

R12

R8

R2

¼Æ Ëã ÖÐ ÐÄ ¡¢ ¹¤ Îï ϵ µÈ ?15 LANs

Rout e r Swi t c hHUBRn

1 0 0BASE- TXÇå »ª ó́ ѧ У Ô° Íø( TUne t )

R1

CERNET

¼Æ Ëã »ú ϵ ¡¢ µç ×Ó Ïµ µÈ 1 2 LANs

ÖÐÑëÖ÷Â¥

A D WS PC

DNS NMEmai l

Ô¶ ³Ì ·Ã ÎÊ·þ Îñ Æ÷

Mo de m³Ø

ͼ Êé ¹Ý

µç »° ÊÒ ¹¤×ÖÌü

¹âÅÌÂ¥

¾«ÒÇϵ

Î÷ Ö÷ Â¥

¶« Ö÷ Â¥

Ȩ́ Ïß ¹â À ·Ö Ö§ ¹â À»¥ Á¬ Éè ±̧

ѧ Éú Çø

¶« ±± Çø

TUne tBac kbo ne

FDDI Swi t c h

R3

R4

R5R6R7

ѧÉúÇø

µç »ú ϵ µÈ 8 LANs

¾« ÒÇ Ïµ µÈ 8 LANs×Ï ¹â ¼̄ ÍÅ µÈ

1 0 LANsУ »ú ¹Ø µÈ

1 2 LANs

ͼ Êé ¹Ý µÈ12 LANs

×Ô ¶̄ »̄ ϵ µÈ 9LANs

ATMTe s t Be d2

ATMTe s t Be d1

ATM½» »» »ú1998

TUNET 1998

12

• • • •• • • •

FDDI

FDDI

FDDI

FDDI

FD D IFD D I

FDD

IFD

DI

CERNET

F D D I

FDD I

FDD IFD D I

FD D I

FDD

IFD

DI

W W W DNS 1 D NS 2 M ail M a ils

DH CP 1 D H CP 2 FT P CA D B

ͼ Êé ¹Ý

¹« ¹² ½Ì ÊÒ

¼Æ

À¶ Æì Óª

16ºÅ Â¥ µØ Çø

GSR GSR GSR GSR

Íø ¹Ø

»ª Òµ ó́ ÏÃ

Öй¤ ×Ö Ìü

ͼ Êé ¹ÝУ Ô° ¶« Çø

FDD I

FDD I

FDD I

FDD I

FDD

IFD

DI

9 0 00Co re B ui lde r 9 0 00

¶« Ö÷ Â¥Î÷ Ö÷ Â¥9003 µç »° ÊÒ

Co re B ui lde r 9 0 00Co re B ui lde r

1#Â¥ 2#Â¥ ÐÂ Õ«

• • • •• • • •5#Â¥ 12#Â¥ 13#Â¥ 14#Â¥ 34#Â¥

• • • •• • • •

Çå »ª ó́ ѧ У Ô° Íø TUNET

2002

TUNET 2002

13

2004

TUNET 2004

14

THUNET Facts

THUtotal student num 30000border bandwidth (M) 2000local bandwidth (M) 70000IP address 65536

border bandwidth per head (M) 0. 07local bandwidth per head (M) 2. 33address per head 2. 18

15

Design Considerations

! High performance

! Low operation cost

! Effective management

! Quick emergence response

16

Emergence Response

17

Technical Design! Packet forwarding

– Routing – switching

! Network management– throughput/performance

! User management– Gateway– PPPoE– 802.1x– VALAN

18

Top 20 Usage Distribution

! 0.3% of the users

! 30% of the bandwidth

19

Top 20 Case Study

! Normal case

! Abnormal case

20

Chinese University Connectivity Demand

! 16 Million students enrolled in 2003 universities and institutes located

in more than 260 cities in Mainland China in year 2002.

! The average number of enrollment per university/institute is about

8,000 students.

! If 1 Mbps connectivity is required per student, an average sized

campus network will need about 8 Gbps of aggregated bandwidth.

! When 80 vs. 20 rule applied, the CERNET backbone should provide 4

Tbps aggregated bandwidth reaching more than 260 cities.

! For one IP address per student assignment, 1 /8 space is needed.

21

40G IP Backbone Trial

22

IPv6 Transition! Hosts

– FreeBSD/Linux/UNIX– WindowsXP

! Layer 2 Switch–

! Layer 3 Switch/Router– !!!

! Access Server– !!!

! DNS– !!!

! Applications– !!!

23

Conclusion

! High performance

! Source address authentication

! User behavior control

! 10G-40G backbone ports

! IPv6 transition