Embedding e-Science Applications: Designing and Managing for

Usability

Marina Jirotka and Sharon LloydOxford University

NeSC The Changing Landscape16th April 2009

Embedding e-Science Applications: Designing and Managing for Usability

• A 3-year project funded by the EPSRC

• Project staff: Marina Jirotka (PI) Anne Trefethen (CI) Sharon Lloyd (Advisor) Dimitrina Spencer (Researcher) Ralph Schroeder (Researcher) and Grace de la Flor (DPhil student) (Andrew Warr -researcher Year 1)

• Working closely with the Oxford e-Social Science (OeSS) project, the e-Horizons Institute and the UK e-Science Usability Task Force (UTF)

Objectives• To develop an online toolkit defining processes and practices for managing

collaboration to facilitate usability in e-Science projects

• Through engagement with project collaborators, to investigate approaches, tools and techniques that may enable the development of shared understanding of e-Science project expectations, management and implementation

• To develop recommendations, guidelines and procedures that facilitate the effective integration of e-Science technologies with existing work practices whilst also allowing potentially new ways of working

• To consider usability of nationally provided services through a broker, and to draw upon case studies to provide insights into the use of the NGS and large-scale resources like it

• To consider specific tools and technologies that allow user engagement with projects such as personal Access Grids.

• To develop a set of case studies leading to recommendations for managing usability on e-Science projects

Embedding

Use of large scale infrastructure

Use of services/applications

Use of collaborative tools

and approaches

Task Forces STC, ETF, UTF

OGF

NGS

HPCx

CSAR

Research Projects: dealing with specific problem

Dame, e-DiaMoND, Integrative Biology, NeuroGrid, Carmen…

Access Grids, PIGS

Commercial Solutions

OMII - Product development

What is Usability?

• As defined by the ISO standard ISO 9241 Part 11, usability can be measured only by taking into account the context of use of the system– who is using the system, what they are using it for, and the environment in which they

are using it

• Furthermore, measurements of usability have several different aspects– effectiveness (can users successfully achieve their objectives) – efficiency (how much effort and resource is expended in achieving those objectives) – satisfaction (was the experience satisfactory)

• Who is the system built for? Management? End users?

Who are the users?

Typically…

Middleware/Core infrastructure development

Users of

Standards, Products and Trends

Applications

MiddlewareDevelopers

Portals

Infrastructure/Services - challenges

• to 'productise' the outputs from e-science projects/initiatives and to ensure outputs were developed in a scalable and robust fashion.

• how to ensure that what is developed is usable for everyone?

• What do users expect of an infrastructure? - robust and sustainable egs HPCx vs CSAR - user forums very technical

• Less uptake than expected– Inadequate understanding of kinds of services

– Insufficient resources to make it happen

• National vs local solution– Whose responsibility is to ensure usability?

Infrastructure - lessons learned

• Mode 1 Provision of national services and infrastructure. Seamless and sustainable provision - different mode of engagement with users: training - handholding - documentation..

• Mode 2 Use of standards and services to develop own infrastructure for a specific scientific problem - need visible/transparent infrastructure where users can see what it is doing and modify it

• Applications and infrastructure co-evolve• Gap exercises with users• Localisation - local staff, system administrators, groups of users

– But many projects involve cross-institutional work• No large scale data sets to work across institutions• No one solution as an institution to investigate• Researchers have to work out own mechanisms for long-term collaboration

Users and Applications - challenges identified

• In depth qualitative studies of several key e-Science projects reveal lack of impact - applications not being used beyond lifetime of the project

• User requirements not clearly understood - little expertise in elicitation or how they fit into the development cycle

• Different types of ‘users’ - middleware developers, end users..• Stakeholder requirements often poorly conceptualised - who

is a stakeholder?• Embedding of applications seen as an additional

requirement once system developed

Users - lessons learned

• Early strong user/stakeholder buy in and feedback - engaging people who are v busy - communication across different groups translation exercise

• Engage in project vision - recalibrate throughout project lifetime

• Showcase technical possibilities - milestones in project plan

• Understanding current activities tools and techniques

• Develop stakeholder analysis to ensure right partners and people in the community

• Ensure enough time and resources for engaging users efficiently

• Focus on exploitation and impact - is this research? and who funds?

Collaboration and Communication - Challenges

• Instantaneous methods for communicating are important in dynamic teams e.g. use of video conferencing requiring a weeks notice is problematic - or limited skill set

• Partners may have favourite audio/video services - either initiate change, or ensure interoperability of services - applies to visualisation tools also

• Methods for recording and recollecting information are vitally important for wider dissemination of project knowledge

• Translation of information to communities requires consideration of target audience constantly and often may require external development - good for public engagement

• Support for training people and institutional buy in to provide tools such as AG

Collaboration and Communication - lessons learned

• Methods and tools need to be inclusive - so select an approach

that interoperates with all platforms - compliant technologies -

Neurogrid - Cancergrid commercial solution (coffee room - open

activities could have been intrusive)

• Using shared repositories and Wikis to collate decision making

processes - who does this and maintains these repositories even

after the project is over

• Certain tools such as Crewe and Memetic potentially interesting for

recording - but seemingly no active embedding

Project Management - challenges

• Scientific vs operational management

– Imperative that there is clear role definition and identification of who takes the scientific lead and who takes the operational lead – or one and the same person? Neutral PM beneficial.

• Project initiation – key to ensuring project progresses with clear understanding by all project members of what is expected of them.

• Communication methods

– Developing and maintaining shared visions and objectives

• Closedown and Sustainability

– How do you assess the delivery of a project and what legacy do they leave behind?

Project Management - lessons learned (1)• Taught methods not wholly applicable

– Process skills needs to be coupled with social /management skills (e.g. NLP)– ‘Executing plans’ , but if cannot motivate the team … ?

• Strategic vs operational management – Scientific vs operational - is PI both?

– Different skill sets - may not be for entire duration of project • Project initiation

– Defining roles and activities, key to maintaining the project teams – Stakeholder questionnaires, ‘cheap’ means of monitoring project team health

• Communication methods - for different purposes– Teleconferences, wikis, maillist, websites, showcases, newsletters

• Development and exploitation plan– Can be used to communicate how individuals fit into the bigger picture– Consider sustainable routes - users involved to sustain activity– For every deliverable what can be done to push beyond project

Project Management - lessons learned (2)

• Management Styles - Use of language important ‘We’ ‘Our’ - Empowerment drives problem ownership - Embarrassment drives delivery!

• Project closedown– Needs time…– Must indicate what will happen to output – Delivering exploitation plan– Consider what to do with website, wikis, reports, documents etc and whether others can benefit from them– Lessons Learned exercises - useful for all contributors– Blueprint documents useful to document what could not be achieved - content often results in new proposals for follow on

projects and enables projects to publish knowledge that is not research.

ConsiderScientific and operational concernsInvolving users from project conception to closedown and beyond - strategiesInitiation activities Showcasing technical potentialClosedown activitiesExploitation plansLessons learned activitiesBlueprintOpen, modifiable and transparent infrastructure (not only by SA)Ongoing agreements between users, users and developers, and other stakeholders

Build it and they will come

Technical decision making in isolation from users

Users determining requirements

Disciplinary silos

PM as requirements engineer

Rigid inflexible technical vision

No stakeholder analysis at project inception and/or throughout project lifetime

Fixed waterfall development

No management of user expectations

Suggestions

• Provide training and support in operational management of large scale multi-disciplinary projects?

• Can we learn from the EU Network of Excellence approach bringing together communities of interest?

• Perhaps OMII + open tools + knowledge base + training = ‘toolkit’ for communities of interest?