egee-iii infso-ri-222667 enabling grids for e-science egee and glite are registered trademarks...
TRANSCRIPT
![Page 1: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/1.jpg)
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
Etienne Dublé - CNRS/UREC EGEE SA2
Mario Reale – GARR EGEE SA2
Friday, March 6, 2009 – Catania – OGF 25/ EGEE UF4
gLite IPv6 complianceoverall status and on going activities
![Page 2: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/2.jpg)
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
content
• Overview on gLite IPv6 compliance activities and their outcome
• Current status of the IPv6 compliance of gLite
• Outlook over forthcoming work
![Page 3: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/3.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
gLite IPv6 compliance activities and their
outcomeEGEE SA2 IPv6 task and collaborations
3
![Page 4: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/4.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 4
• EGEE SA2 started working on the problem of the IPv6 compliance of gLite in september 2006 – during EGEE II
• Initial emphasis has been on defining a methodology– Gathering and implementing required tools– Defining an operational workflow
• Given its close relationship to the management of the gLite release, ETICS has been involved in a fruitful collaboration on this issue (since February 07) (initially also EuChinaGRID).
• It was immediately realized that IPv6 evangelization within the EGEE community played a key role to foster the transition to IPv6– IPv6 network programming tutorials for gLite developers have
been organized
• SA2 IPv6 workplan for EGEE III defined in May 08
a (very little) bit of history
![Page 5: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/5.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (1/4)• We performed prototypal functionality (manual) tests of
the Workload Management System and the BDII in IPv6. IPv4 turned off. IPv6 on.– Results showed immediately IPv6 was breaking provided
functionality
• Set up an ETICS pilot project called glite_IPv6 to exploit the automation of IPv6 functionality testing– We followed the ETICS developments on multi-nodes
testing and gLite modules deployment tests– Included IPv6 resources in the ETICS metronome pool– Thanks to the ETICS team, we got IPv6 match-making
available to drive jobs to IPv6 nodes in the Web Application
• We have developed an IPv6 compliant version of the BDII
5
![Page 6: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/6.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (2/4)
• Distributed gLite IPv6 (IPv6/IPv4/Dual Stack) testbed in place– 2 sites (UREC CNRS Paris, GARR Rome)– many relevant deployment modules installed and configured:
VOMS, UI, WMS, lcg-CE, CREAM, WNs, DPM-SE, LFC, PX, BDII, LB
– gL 3.1 on SLC 4.6/4.7 nodes, Dual Stack– Protocol conversion NAT-PT available for specific C-S studies
• Carried out first comprehensive analysis of the gLite source code in september 08– Using the IPv6 metric (IPv6 code checker) in ETICS– Around 110 bugs on non-compliant function calls and data
structures in the code posted
![Page 7: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/7.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (3/4)
7
• We performed an overall analysis of all gLite external dependencies to assess their level of IPv6 compliance– Showing that there is a very limited number of non
compliant ones
• We have developed a new software package called IPv6 CARE acting as an IPv6 checker– It’s a very general purpose tool to assess IPv6 compilance– It’s publicly available on sourceforge
https://sourceforge.net/projects/ipv6-care
– Requires running services on the nodesIPv6 CARE
![Page 8: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/8.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Achievements so far (4/4)• Performed specific in deep studies on the IPv6
compliance of relevant packages to the gLite community (see list later on)
• Written a general document on how to check the IPv6 compliance of a socket server
• Written guide for IPv6 compliant network programming (in C/C++,Java, Perl, Python)
• Organized 2 general tutorial on IPv6 for the gLite developers community– Rome Jan 16 2008– Prague Nov 6 2008
![Page 9: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/9.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 11
SA2 gLite IPv6 testbeds
11
VOMS
.236 :d
LB
WMS
CE WN1
WN2
BDIISE
LFC
PX
.233
.226 .227
.228
.232
.231
.234
LB serverVOMS Server
UREC site BD-II
Workload management server
LFC File Catalog
LCG Computing Element
Worker Node (Torque/PBS)
DPM Storage Element
MyProxy server
.229
.235
2001:660:3302:7006::1
Gateway IPv6
:a
:8
:3:4
:5
UI
User Interface
.230 :7
:9
:b
:6
:c
UI2
VOMS2 .59
LB
WMS
DPM1
LFC
.50
.27
.22
.51
LB server
SA2 top level BD-II
RGMA-BDII
.24
GARR site BD-II
User Interface
Workload management server
LFC File Catalog
Worker Node (Torque/PBS)
CE WN1
WN2CREAM.23
.56
LCG Computing Element
CREAM Computing Element
Storage Element
BDII
.30
.29
.21
DEV.34
Grid Job monitoring DB
.29.11
Gateway2001:760::159:242/64
IPv4/IPv6 Internet:Renater/GEANT/GARR
GARR/ROME UREC/PARIS
![Page 10: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/10.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Other SA2 provided tools and documents
• Reference documents on IPv6 for gLite developers: (all on SA2 EDMS or Wiki page)– IPv6 Programming methods:
Guide to IPv6 compliant programming in C/C++, Java, Python and Perl:• Provides a sample TCP client and server for each programming language• Explains advantages/drawbacks/limitations of each language regarding IPv6
– IPv6 Testing methods: How to make sure the IPv6 behavior of your application is as expected
– IPv6 Tests reports: Assessment of the current status of the gLite external packages overall Selected IPv6 compliance studies for specific packages: gSOAP, Axis / Axis2,
Boost:asio, gridFTP, PythonZSI, PerlSOAPLite Assessment of the IPv6 compliance of gLite components: DPM, LFC
• Provisioning of specific IPv6 introductory tutorials for gLite developers
• In collaboration with ETICS: – IPv6 resources inside the metronome pool– A whole dedicated ETICS project for IPv6 studies
1212
![Page 11: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/11.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Example of SA2-developed tool:
Assessing IPv6 compliance with
IPv6 CARE(*)
(*) IPv6 Compliance Automatic Runtime Experimenthttp://sourceforge.net/projects/ipv6-care
13
![Page 12: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/12.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Usage example
• Usage example: we want to test the program called “nc”– We run the program as usual except that we prefix the command
by “ipv6_care check”:
– The standard behavior of ‘nc’ is not affected (the program works as it should).
– The diagnosis (network function calls log file, non-IPv6 compliant calls detected, etc.) can be reviewed in the mentioned directory.
14
$ ipv6_care check nc www.google.fr 80
^C
----------------------------------------------------------------------
IPv6 diagnosis for 'nc www.google.fr 80' was generated in:
/tmp/ipv6_diagnosis/nc/by_pid/pid_7823
----------------------------------------------------------------------$
![Page 13: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/13.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
IPv6 CARE mechanism
Program <name>-------------------------Main(){ …gethostbyname(…)…}
Program <name>-------------------------Main(){ …gethostbyname(…)…} C Standard
Shared Library----------------------gethostbyname(){…}…
C Standard Shared Library
----------------------gethostbyname(){…}…
15
C Standard Shared Library
----------------------gethostbyname(){…}…
C Standard Shared Library
----------------------gethostbyname(){…}…
Preloaded libipv6_care.so library-----------------------------------------------gethostbyname(…){Diagnose problem in /tmp/ipv6_diagnosis/<name>/…Call RTLD_NEXT gethostbyname()}...<other_non_ipv6_compliantfunctions>
Preloaded libipv6_care.so library-----------------------------------------------gethostbyname(…){Diagnose problem in /tmp/ipv6_diagnosis/<name>/…Call RTLD_NEXT gethostbyname()}...<other_non_ipv6_compliantfunctions>
LD_PRELOAD=/path/to/libipv6_care.so
![Page 14: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/14.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Current stand in supporting IPv6 with
gLiteHow far are we from having gLite fully IPv6
compliant ?
17
![Page 15: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/15.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Current stand ongLite and IPv6
Full IPv6 compliance – for the production version
Full IPv6 compliance – for a prototype version
IPv6 compliance to be tested/verified – gLite part of the deployment module claimed to be IPv6 compliant
IPv6 porting currently on-going
IPv6 porting plan exist
No porting plan yet (ew are not aware of)
IPv6 compliance
LFC DPM globus-url-copy/gridFTP
BDII(perl)
CREAM
WMproxy/Job submission
VObox
lcgutilsVOMS
PX MON dCache Torque C/S MPIutils
Condorutils AMGA
gfalblahFTS
BDII(python)
![Page 16: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/16.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
JRA1 workplan for IPv6 porting
BLAH
WMS / WMproxy
Job Management
GFAL
LCG util
FTS
VOMS Client and APIs
Apr 09
VOMS Server
RGMA
Jun 09 Apr 10Aug 09 Oct 09 Dic 09 Feb 10Feb 09
![Page 17: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/17.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
OutlookWhat will keep us busy in the next months ?
20
![Page 18: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/18.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Future work / Outlook
• IPv6 Bugs Verification and follow up– New systematic analysis
• gLite functionality tests with IPv6– After code is claimed to have been ported to IPv6
WMproxy ( next weeks) Test new BDII implementation (Python) (next weeks) …..
• Keep up the collaboration with ETICS on– Integration of IPv6 CARE into their metrics
• Possible addition of missing gLite deployment modules to our testbed ( FTS, ..)
• Further IPv6 Programming tutorial to organize– if required by other EGEE activities
• General purpose support on IPv6 related issues for the gLite community
![Page 19: EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Etienne Dublé - CNRS/UREC EGEE SA2 Mario](https://reader034.vdocuments.net/reader034/viewer/2022051412/5514d8af550346b0338b5474/html5/thumbnails/19.jpg)
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 2222
Thank You.
https://twiki.cern.ch/twiki/bin/view/EGEE/IPv6FollowUp