introduction kisti sciencedmz · •kisti sciencedmz – two types dtns (sdn/l3 based) • 100g:...
TRANSCRIPT
![Page 1: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/1.jpg)
National Institute of Supercomputing and Networking
National Institute of
Supercomputing & Networking
Introduction KISTI ScienceDMZ
(Including PRP, NRP, Asia PRP)
Jeonghoon Moon8th Aug 2018
APAN 46th Meeting@Auckland, APRP Session
![Page 2: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/2.jpg)
2 /592
Highlights of KREONET
1. 100G from Daejonto Chicago/StarLight
2. SDN-WAN Connection
to Chicago/StarLight,
Add Seattle and Hongkong (Aug)
3. 100G Ring linking major cities
4. 17 GigaPoPs with 1G, 10G or 40G
100G
KREONET2 & GLORIAD-Kr & SDN Deployment (KREONET-S)
Additional 100G from Daejeon to Seattle and Hongkong early 2019
100G
100G
![Page 3: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/3.jpg)
• KISTI ScienceDMZ– Two types DTNs (SDN/L3 based)
• 100G : International & Giant Group Research & Study
• 10~40G : Domestic Scientific Research Group / ~1G : Normal User (Fionette)
– Participated in 1st PRP Workshop since 2015 & 1st NRP Workshop since 2017
• Deploy ScienceDMZ and Activites
– Domestic
• Focus on Giant Group Research & Study(Data-Intensive Science) areas
• Examples : K*GENOME, GENOME RDC, LSST, HEP(CMS, ALICE), SDO, HPC,
GSDC, KSTAR, KBSI TEM, eKVN etc.
• Especially, eKVN(Korea eVLBI) and LSST deploy 100G connection (KASI)
• Deploy KISTI New Supercomputer with a DTN Cluster on 100G
• Dark Matter research using astronomical/particle big data on ScienceDMZ
– International
• Participate in PRP, NRP, APRP
• Asia PRP : Collaboration with APAN members for Deploy and use ScienceDMZ
• Participate in ‘Peta Scale DTN Project’
KISTI ScienceDMZs
eKVN
KBSI TEM KSTAR
K*GENOME
SDOHEP (CMS, ALICE)
LSST
GSDC
Genome RDC
Supercomputer(5th)
![Page 4: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/4.jpg)
KISTI ScienceDMZs
• SDN based DTN– Seoul & Daejeon on ONOS(SDN/VDN,
Currently, 10G)
– For SDN based applications of KREONET service
– Enabled SDN environment between Daejeon,
Chicago(now), Seattle, and Hongkong(in Aug
– Enabled SDN environments for 5 major places
in Domestic
• L3 based DTN– For International with 40G & 100G connection
for PRP and Peta Scale DTN
– From Aug, 2018, 100G test for Domestic
& International(SC18)
– Participated in PRP PS MaDDash (Prof. Tomas DeFanti & John Graham)
– Deployment for several areas
• Such as KISTI Supercomputer, eKVN, LSST, HEP, Climate Service and so on
• Expand 17 Giga Pops in KREONET(Domestic)
KISTI Supercomputer
KISTI SDN baseed DTN
![Page 5: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/5.jpg)
SDN status of KREONET(Intl’ &Domestic)
Intl’ Exchange
(via StarLight)
Seoul-Core
Busan-Core
ChangWon-Core
GwangJu-Core
Chicago-Core
ScienceDMZ/DTN
ScienceDMZ/DTN
GwangJu-Edge
Deajeon-Core
Deajeon-Edge
ChangWon-Edge
Busan-Edge
Chicago-Edge
Seoul-Edge
KIMS
Tongmyong
Univ.
Ulsan Observatory
Tamna Observatory
Yonsei Observatory
PLSI
(via KISTI)
OverCloud-Box
(via GIST)
Cloud Gateway
(COREEN)
PLSI
(via GIST)
OverCloud-Box
(via GIST)
Seattle-Core
Intl’ Exchange
(via PacificWave)
Chicago-Edge
Hongkong-Core
Hongkong-EdgeIntl’ Exchange
(via HKLight)
![Page 6: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/6.jpg)
Performance of KISTI ScienceDMZfor Domestic users 1/3
• Transfer performance of KISTI ScienceDMZ over SDN – Transfer performance between Seoul and Daejeon (over 160Km)
Iperf Verification(9.89Gb/s on 10GE)
GridFTP Verification(9.43Gb/s on 10GE)
SDN/ONOS/VDN Topology
Seoul-Daejeon transfer performance vi DTN
Transfer performance bwteen Seoul and Daejeon
- Iperf : 9.89Gb/s on 10GE- GridFTP : 9.43Gb/s on 10GE
VDN service on SDN/ONOSVDN service of TTA given connection and bandwidth
- supported
![Page 7: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/7.jpg)
Performance of KISTI ScienceDMZ 2/3SC16@Salt Lake City(Nov.2016) Data Transfer Demonstration
• Long Distance Data Transfer Demo with Harvey Newman Team
• Between Daejeon(Korea) – Chicago – Salt Lake City : 10GE
• Approximately 10,000mi
• Result : FDT Transfer with normal TCP • AVG : 9.16Gb/s on 10GE
• Max : 9.48Gb/s on 10GE
• DTN – FIONA (Flash I/O Node Appliance) • Dedicated Data Transfer Node
FDT transfer with normal TCP
on 10GE between Daejeon-Salt Lake City
![Page 8: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/8.jpg)
Performance of KISTI ScienceDMZ 3/3Expanding to the Global Research Platform
Via CENIC/Pacific Wave, Internet2, and International Links from Larry Smarr
PRP
PRP’s Current
International
Partners
Korea Shows Distance is Not the Barrier to Above 5Gb/s Disk-to-Disk Performance
Netherlands
Guam
Australia
Korea
Japan
Source: Larry Smarr(Calit2), 1st NRP Workshop@Bozeman 2017
![Page 9: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/9.jpg)
LSSTLarge Synoptic Survey Telescope:
potential usage case of the DTN
⚫ LSST (Large Synoptic Survey Telescope; http://lsst.org)
− 10-year (2022 ~ 2032) survey of Southern Sky support
ed by DOE, NSF, international contributors, and private
donors.
− Production of raw data about at a rate of 15TB~30TB p
er night.
− 0.5Exabyte is expected around 2032.
− NCSA is in charge of processing, archiving, and serving
the data.
− Engineering “First Light: anticipated in 2019
− Science “First Light “ in 2021
⚫ KASI (Korea Astronomy and Space Science Institute)
is one of the international contributors on behalf of
the LSST Korea (http://lsst.kasi.re.kr).
⚫ KASI – Deployment of ScienceDMZ and PRP
![Page 10: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/10.jpg)
ScienceDMZ for LSST in KASI
Data Volume of ~ 30TB per night• Data size : 0.5Exabyte for imaging (over 50PB for the catalog DB)
The LSST Network requirements• High demand end-to-end applications like LSST require that all networks in the path support
QoS and Programmability
• Each 12.7GB data set (6.4GB picture + 6.3GB metadata) must be transmitted to the U.S. in 5 seconds and Database synchronization, etc.
LSST data (part of data set) transfer from Chile and NCSA(US) to KASI• LSST data transfer demonstration at SC18 (Nov)
100G
100G
100G
![Page 11: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/11.jpg)
ScienceDMZ Architecture for HPC(Collaboration with NERSC & Esnet)
• ScienceDMZ for KISTI 5th Supercomputer – Dedicated DTN connection with NERSC DTN
• Peta Scale DTN transfer Project– DTN Cluster on 100GE
• Globus Online– GridFTP
DTN
DTN
DTN
DTN
100G Switch
SDN(VPN/VDN)
5th Super-computer
File System
Normal User
…
DTN Cluster
F/W
슈퍼컴퓨터 5호기 100G/SDN
Peta Scale transfer environment over KISTI 5th Supercomputer
Collaboration with NERSC ScienceDMZ
DTN Cluster
ScienceDMZ architecture for HPC
![Page 12: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/12.jpg)
4Gbps FabricInfiniband10GbE Network
Sun X4270 36EA / J4400 72EALustre OSS Servers
MPP Computer Cluster(3200 Nodes)
SUN X4600 4EASAM-QFS MDS Servers(Dual PCI-E(4) : Disks, PCI-E(1) : 10GbE, Dual PCI-X(2) : Tapes)
Brocade 48K
STK 6140 (24 Tray)(500GB SATA, ??TB Usable)
STK 8500 (LTO4 16 Drives)(1050 LTO3 Media)
IBMSMP
10GbE Network
6
6
(8 Port for Disk per Svr)(4 Port for Tape per Svr)
(1 Port for 10GbE per Svr)
2
22
22
22
22
66
66
6
66
6
Brocade 48K
SUN V215(ACSLS Server)
BACKUP2
[117TB]
2
4
Sun X4270 6EA / J4200 5EALustre MDS Servers
DTN 4 node(Dual PCI-E(1) : IB, PCI-E(1):10GbE)
66
6
6
6
Sun X4270 2EA [admin01.02]Provisioning Servers
HO
ME01/A
PPLIC
/SCRATCH
METAD
ATA
M9-648*8EA
M2-36 *6EA
KISTI Supercomputer architecture with DTN
DTN
![Page 13: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/13.jpg)
SC18 Demonstration & LSST Project in Korea
• 100G connection with KISTI Booth@SC18
– 100G DTN demonstration between SC18 KISTI booth and Daejeon
– 100G switch and DTN will be set up at booth
– PRP/APRP collaboration
• Participate in Global 100G DTN demonstration with Caltech (Harvey Newman)
– Participate in several demonstrations with Global partners
• 100G LSST Demo (Working on a scenario)
– Focus on real LSST data from Chile to real user of Korea via US
– KASI(Korea Astronomy and Space Science Institute)• One of 100G users in KREONET
• One of the international contributors on behalf of the LSST Korea(http://lsst.kasi.re.kr)
– Two Endpoints will be available
• KASI and KISTI(Daejeon)
• 100G DTN will be set up before SC18
![Page 14: Introduction KISTI ScienceDMZ · •KISTI ScienceDMZ – Two types DTNs (SDN/L3 based) • 100G: International & Giant Group Research & Study • 10~40G: Domestic Scientific Research](https://reader035.vdocuments.net/reader035/viewer/2022062507/5fca04689eefce246377e13a/html5/thumbnails/14.jpg)
✓ Asia Pacific Research Platform(APRP) WG in APAN- Chair : Jeonghoon Moon, KISTI, Korea
- Co-Chair : Andrew Howard, NCI, Australia
- Secretary : Asif Khan, Perdana Univ. Malaysia
- Committee Member : Yves Poppe(NSCC, Singapore), Kenneth Ban (NUS, NSCC, Singapore),
Lin Gan(Tsinghua Univ. NSCC Wuxi, PRC), Garry Swan(CSIRO, Australia),
Suhaimi Napis(Putra Univ. Malaysia)
✓ APRP Project- 100G HK-Marseille link in June 2018 at TNC18 as first 100G Ring-Around-The-World
- Cooperation of Asia – USA – Europe to build a Global Research Platform with PRP, NRP,
ERP and APRP
- Asia-Pacific Bioinformatics Network (APBioNet) to propose genomics link with ELIXIR
- Ability for us to submit projects to TEIN*CC for Asia@Connect funding
- Related Research & Work for HPC in Asia
✓ APRP Roadmap- 2nd NRP Workshop at Bozeman, Aug, 2018
- APAN 46th Meeting at Auckland, Aug, 2018
- SC18 at Dallas, Nov, 2018
- APAN 47th at Daejeon Feb. 2018
- SCAsia19 at Singapore Mar, 2019 (APRP Conference will be held)
APRP - Asia Pacific Research Platform