campus bridging with globus services
DESCRIPTION
Talk given at XSEDE 2012 conference in Chicago. The highlight were Dan Milroy and Brock Palen's presentations on experiences at Colorado and Michigan. Paper is at https://www.globusonline.org/files/2012/07/XSEDE12-Globus-Campus-Bridging.pdf As science becomes more computation and data intensive, computing needs often exceed campus capacity. Thus we see a desire to scale from the local environment to other campuses, to national cyberinfrastructure providers such as XSEDE, and/or to cloud providers—in other words, to “bridge” to the wider world. But given the realities of limited resources, time, and expertise, campus bridging methods must be exceedingly easy to use: as easy, for example, as are Netflix and Amazon movie streaming services. We report here on experiences with a service called Globus Online, which seeks to do for campus bridging what Netflix and Amazon do for movies: that is, use powerful cloud-hosted services and simple, intuitive web interfaces to make it “so easy that your grandparent can do it.” Specifically, we describe Globus Transfer, which addresses the important campus bridging use case of moving or synchronizing data across institutional boundaries. We describe how Globus Transfer achieves both ease of use for researchers and ease of administration for campus IT staff. We provide technical details on the Globus solution; quantitative data on usage by more than 25 early adopter campuses; and experience reports from two early adopters, the University of Michigan and the University of Colorado Boulder.TRANSCRIPT
![Page 1: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/1.jpg)
www.globusonline.org
globus online
Campus Bridging Made Easy via Globus Services Ian Foster, Rajkumar Kettimuthu, Stuart Martin, Steve Tuecke: Chicago and Argonne Thomas Hauser, Daniel Milroy, Jazcek Braden: Colorado Brock Palen: Michigan
![Page 2: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/2.jpg)
www.globusonline.org
“the seamlessly integrated use of cyberinfrastructure operated by a scientist or engineer with other cyberinfrastructure on the scientist’s campus, at other campuses, and at the regional, national, and international levels as if they were proximate to the scientist” -- NSF Advisory Committee for Cyberinfrastructure Task Force on Campus Bridging Final Report, March 2011.
Campus bridging
![Page 3: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/3.jpg)
www.globusonline.org
“Use of data resources from campus on XSEDE, or from XSEDE at a campus”*
• Researchers often use a range of resources and must move data among them
• Desktop, campus clusters, remote instruments, national computing facilities, commercial clouds, …
• Researcher desktops and campus clusters often lack sophisticated data movement tools • Transient network and system failures have to be dealt with • Each resource has its own security domain • Firewalls and other problems often get in the way too
*Campus Bridging Use Cases, XSEDE Project, 2012.
![Page 4: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/4.jpg)
www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface – Intuitive GUI for file transfer – No interruptions for transient failures – Transfer efficiency 2) System administrators – Easy integration of a campus resource into campus and national cyberinfrastructure – Easy management in terms of adding users, tracking usage, etc.
Two distinct groups of stakeholders
![Page 5: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/5.jpg)
www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface – Intuitive GUI for file transfer – No interruptions for transient failures – Transfer efficiency 2) System administrators – Easy integration of a campus resource into campus and national cyberinfrastructure – Easy management in terms of adding users, tracking usage, etc.
Two distinct groups of stakeholders
![Page 6: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/6.jpg)
www.globusonline.org
Reliable file transfer. - Fire-and-forget - Automatic fault recovery - High performance - Across security domains
No IT required. - Intuitive Web 2.0 interface - No client software install - New features available automatically - Consolidated support and troubleshooting
Globus Transfer: Data movement
Works with existing GridFTP servers; also Globus Connect
as a Service
![Page 7: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/7.jpg)
![Page 8: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/8.jpg)
www.globusonline.org
XSEDE-aware
![Page 9: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/9.jpg)
www.globusonline.org
![Page 10: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/10.jpg)
www.globusonline.org
Globus Connect
GlobusConnect
"MyDesktop"
GridFTP server"SiteA"
Globus OnlineUser (1) Globus Connect
client registers with Globus Online
(2) User makes requestto Globus Online: e.g.,"transfer data from MyDesktop to SiteA"
(3) Globus Onlineforwards requeststo Globus Connect
(4) Globus Connect establishes data channelconnection to SiteA and transfers data
![Page 11: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/11.jpg)
www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface ✔ – Intuitive GUI for file transfer ✔ – No interruptions for transient failures ✔ – Transfer efficiency ✔ 2) System administrators – Easy integration of a campus resource into campus and national cyberinfrastructure – Easy management in terms of adding users, tracking usage, etc.
Two distinct groups of stakeholders
![Page 12: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/12.jpg)
www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface – Intuitive GUI for file transfer – No interruptions for transient failures – Transfer efficiency 2) System administrators – Easy integration of a campus resource into campus and national cyberinfrastructure – Easy management in terms of adding users, tracking usage, etc.
Two distinct groups of stakeholders
![Page 13: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/13.jpg)
www.globustoolkit.org www.globusonline.org
Installation • Download, untar, configure, make
Security configuration (server admins) • Obtain and install X.509 host certificate from well-known CA • Configure trust roots
Security configuration (users) • Obtain and install user certificate from well-known CA • Configure trust roots
Setup authorization (both users and admins) • DN to local username mapping in gridmap file • '/DC=org/DC=doegrids/OU=People/CN=Rajkumar Kettimuthu
227852' rajk
Too complex for many users and small labs
GridFTP security configuration, old way
13
![Page 14: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/14.jpg)
www.globustoolkit.org www.globusonline.org
• What is GCMU? • Multi-user version of Globus Connect • Packages a GridFTP server and MyProxy CA, pre-configured for
use with Globus Online • Why GCMU?
• Create transfer endpoints in minutes • Avoid complex GridFTP install • Avoid frequent sources of user and administrator error
• To download: https://www.globusonline.org/gcmu/
Globus Connect Multi-User
“We used GCMU to form a campus-wide GSI authentication service spanning multiple servers. Now my users have a fast, easy way to get their data wherever it needs to go, and the setup process was trivial." --University of Michigan
“As a resource admin, I've found GCMU an exceedingly useful tool.... With GCMU, setting up a GridFTP server and handling authentication for multiple users is easy." --Oak Ridge National Lab
![Page 15: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/15.jpg)
www.globustoolkit.org www.globusonline.org
Make GridFTP deployment trivial • GridFTP transfers can be achieved “instantly” even by
non-experts
Automate the process of configuring security • Avoid the need for any end-user or system administrator
involvement in security configuration
Reduce burden on both users and administrators • Eliminate frequent sources of errors in GridFTP
configuration and use.
GCMU makes deploy and config trivial
15
![Page 16: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/16.jpg)
www.globustoolkit.org www.globusonline.org
Globus Transfer / GCMU Interaction
![Page 17: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/17.jpg)
www.globustoolkit.org www.globusonline.org
Globus Transfer / GCMU Interaction
![Page 18: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/18.jpg)
www.globustoolkit.org www.globusonline.org
• Site passwords flow through Globus Online • Globus Online does not store passwords • Just pass along to MyProxy servers at site • Still a security concern for some sites
• OAuth • Sites run an OAuth
server • Users enter username
and password only on a site’s webpage
• Globus Online gets an X.509 credential via Oauth protocol
OAuth protocol to protect passwords
![Page 19: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/19.jpg)
www.globustoolkit.org www.globusonline.org
Globus Connect Multi User with OAuth (coming soon)
MyProxy Online CA
PAM
Local Authentication System (LDAP, RADIUS, Kerberos etc)
Username password
certifficate
Transfer request
certificate
Step 5
Step 7 Step 8
Step 9
Local Storage
GridFTP Server certificate
Access files
Step 10
Step 11
Authentication & Data Transfer
Authorization
Step 1 Access Endpoint
GridFTP Server
Cam
pus
Clu
ster
GCMU
Globus Online (Hosted Service)
Remote Cluster / User’s PC
Oauth Server
Username password
certificate
certificate
Redirect Step 3
Step 4
Step 6
Username password
Step 2
![Page 20: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/20.jpg)
www.globustoolkit.org www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface – Intuitive GUI for file transfer – No interruptions for transient failures – Transfer efficiency
2) System administrators – Easy integration of a campus resource into ✔
campus and national cyberinfrastructure – Easy management in terms of adding users, ✔
tracking usage, etc.
Two distinct groups of stakeholders
![Page 21: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/21.jpg)
www.globustoolkit.org www.globusonline.org
GCMU deployments (as of April 2012)
21
![Page 22: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/22.jpg)
www.globustoolkit.org www.globusonline.org
GCMU endpoints and users
![Page 23: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/23.jpg)
www.globustoolkit.org www.globusonline.org
GCMU – Bytes transferred
![Page 24: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/24.jpg)
www.globustoolkit.org www.globusonline.org
GC users
![Page 25: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/25.jpg)
www.globustoolkit.org www.globusonline.org
GC – Bytes transferred
![Page 26: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/26.jpg)
www.globustoolkit.org www.globusonline.org
• Janus Supercomputer - 16,416 Westmere cores, 2GB memory per core - Four Dell PowerEdge R710s as GridFTP servers - Dedicated 10Gb ethernet per node - RC network: “private VLANs”
• Globus Online endpoints - colorado#gridftp 122 TB transferred from 22 TB transferred to
- colorado#jila, colorado#nsidc --data-interface <vlan>
Campus bridging at CU-Boulder
![Page 27: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/27.jpg)
www.globustoolkit.org www.globusonline.org
• Globus Transfer and “manual tuning” • CLI transfer with -cc 4 -p 4 -pp 4
• In “external” transfers, we noticed 44% increase in transfer rate for default packets and 26% for MTU 9000
• Problem with jumbo frames • Path MTU discovery and ICMP filtering • Probably the issue- reverting to default packets
solved the problem • Determined to be the issue with JILA transfers
Campus bridging at CU-Boulder (contd)
![Page 28: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/28.jpg)
www.globustoolkit.org www.globusonline.org
CU-Boulder
Data transferred from colorado#gridftp 122.5 TB
Data transferred to colorado#gridftp 21.6 TB
Peak transfer rate between distinct endpoints 2.9 Gb/s
Peak transfer rate to/from Janus (disk) 5.9 Gb/s
Peak transfer rate to/from Janus (memory) 9.5 Gb/s
![Page 29: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/29.jpg)
www.globustoolkit.org www.globusonline.org
Single MyProxy Server for Campus • Users: PAM+Kerberos+LDAP • Built from GCMU
Multiple GridFTP Servers • Not all under umich# • Offer documentation and help to setup endpoints • Built from GCMU
http://cac.engin.umich.edu/resources/loginnodes/globus.html
GridFTP at Michigan
![Page 30: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/30.jpg)
www.globustoolkit.org www.globusonline.org
Many small users
![Page 31: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/31.jpg)
www.globustoolkit.org www.globusonline.org
Data transferred from umich#nyx 9.8 TB
Data transferred to umich#nyx 10.4 TB
Data transferred from umich#flux 20.4 TB
Data transferred to umich#flux 6.5 TB
Campus bridging at UMichigan
• UMichigan has five Globus Transfer endpoints • Two endpoints at College of Engineering HPC systems • The other three endpoints at other departments
![Page 32: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/32.jpg)
www.globustoolkit.org www.globusonline.org
1) Individuals (researchers, educators, students) – Easy installation of access layer interface – Intuitive GUI for file transfer – No interruptions for transient failures – Transfer efficiency
2) System administrators – Easy integration of a campus resource into
campus and national cyberinfrastructure – Easy management in terms of adding users,
tracking usage, etc.
Two distinct groups of stakeholders
![Page 33: Campus Bridging with Globus Services](https://reader033.vdocuments.net/reader033/viewer/2022052822/554ea0d2b4c905977e8b4600/html5/thumbnails/33.jpg)
www.globustoolkit.org www.globusonline.org
• Globus Transfer – simple file transfer service • SaaS methods for easy fire-and-forget transfers, high
performance, automatic fault recovery • Web 2.0; integrated knowledge of XSEDE resources • (Leverages Globus Nexus – identity management; sign
in from federated identity systems such as InCommon and from OpenID providers such as Google)
• Globus Connect – one click GridFTP for desktops
• Globus Connect Multi User (GCMU) – easy-to-install GridFTP and security package
• Globus Storage – user-managed storage [soon]
Globus and Campus Bridging