SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Cyberinfrastructure Requirements and Best

PracticesLessons from a study

of TeraGrid

Ann ZimmermanResearch Assistant ProfessorUM School of InformationOGF Workshop, May 27, 2009

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Outline

Background Challenges People Methods used Analysis Conclusions

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Background: Learning from TeraGrid

NSF-funded study to examine:– The TeraGrid collaboration– user needs and requirements– impact on research practice & outcomes– education, outreach & training activities

Research Team– Tom Finholt, PI; Ann Zimmerman, co-PI– Magia Krause, PhD student

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Key Questions: User Needs What factors affect users’ computing

needs and requirements? What factors affect users' behavior as it

relates to their use (or non-use) of TeraGrid/HPC?

How are the needs of users expected to change over the next five years?

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Key Questions Continued

Where do users currently spend time that does not count as doing science?

What research questions do they want to answer but currently cannot? What are the barriers?

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Data Collection

June 2006-May 2007 7 site visits, including 4 TeraGrid sites Interviews (n=~90) Participant observations User workshop Document analysis and review Surveys

– Survey of current TeraGrid users– Surveys of tutorials at TG ’06 & TG ‘07

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

TeraGrid Planning Process The goal of the planning process was to

– develop options for delivering TeraGrid resources and services

– based on the diverse needs of science and engineering communities

http://www.teragridfuture.org/

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Challenges

Heterogeneous users Potentially thousands of users Distributed environment

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

People included in the study Current TeraGrid users “Target” TeraGrid users

– Non-users– Science gateway developers

Cyberinfrastructure “experts” TeraGrid personnel

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

METHODS

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Workshops

Strengths– Learn by seeing users interact– Gain both broad & detailed information– Relatively efficient

Challenges– Require careful & creative planning (pre-

and post-workshop)– Invite 3-4 times as many people as you

want to participate in the workshop

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Interviews

Strengths– Gain detailed information– Information informs survey development

Challenges– Time-consuming (to conduct & to analyze

data)– Resources limit the number of people who

can be interviewed

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Category DefinitionNumber of

intervieweesTeraGrid Users

Individual Researchers

Individuals associated with a project that had a TeraGrid allocation at the time of the interview

26

TeraGrid Users

Science Gateway Developers

Individuals who on a day-to-day basis spend some portion of their time working on a project designated as a TeraGrid Science Gateway

27

TeraGrid Personnel Individuals employed by one of the TeraGrid RP sites who have a formal or informal role in the TeraGrid project

26

Non-TeraGrid Users of HPC Resources

Individuals who use HPC computing resources other than TeraGrid

3

Cyberinfrastructure Experts Individuals with extensive knowledge of high-performance computing

4

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Survey

Advantages– Findings are generalizable across a larger

population– Inexpensive (in $ not in person time)

Challenges– Developing a good survey is hard– Getting a good response rate takes a lot of

effort

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Participant Observation

Advantages– Learn about all the factors that affect the

ability to serve users– Learn about user needs from a variety of

sources Challenges

– Time-consuming– Capturing and analyzing data

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

FINDINGS

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Components of User Behavior The nature of the research problem Alignment between infrastructure and

scientific practice Computational readiness Ease of use

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Achieving Transformative Science “Easy” things can be show stoppers Many complexities to manage

– virtual organization – diverse user needs– changes in science

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Conclusions

Different methods provide different kinds of information

Involve more than users in your study Current methods are effective, but time-

consuming and resource-intensive– New methods required. For example,

“mine” and analyze sources of information (wikis, user support logs and databases, user sites)

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

More information

TeraGrid evaluation study reports & Planning Process workshop reports– http://deepblue.lib.umich.edu (browse for

documents by Ann Zimmerman) Other TeraGrid Planning Process

materials– http://teragridfuture.org

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Acknowledgments

TeraGrid Research participants NSF grants OCI-0602525 and OCI-

0724300

SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu

Contact me!

Sorry I couldn’t be here! Contact me at:

– asz@umich.edu– http://www-personal.si.umich.edu/~asz/index.html