BSC-CNS, RES and PRACE: Services for science

RES Scientific Seminar !Supercomputing and Next Generation Sequencing !

March 17th, 2011 - Málaga!

Montserrat González!Technical Coordinator of RES

2!

National Supercomputing Center (BSC-CNS)!

Mission of BSC-CNS!

•  Research, develop and manage technology to facilitate the advancement of science!

Objectives of BSC-CNS!

•  R&D in Computer Sciences, Life Sciences and Earth Sciences.!

•  Supercomputing support for research external to BSC-CNS!

BSC-CNS is a consortium that includes:!

•  Government of Spain (MICINN) – 51%!

•  Government of Catalonia (DIUE) – 37%!

•  Polytechnic University of Catalonia (UPC) – 12%!

BSC-CNS: Computer Sciences!

Benchmarking, analysis and "prediction tools:!•  Tracing scalability!•  Pattern and structure identification!•  Visualization and analysis!•  Processor, memory, network, system!

Computer architecture: !•  Superscalar and VLIW!•  Hardware multithreading!•  Design space exploration for multicore " chips and Hw accelerators!•  Transactional memory (Hw, Hw-assisted)!•  SIMD and vector extensions/units!

Grid!

Grid and cluster computing: !•  Programming models!•  Resource management!•  I/O for Grid!

Operating environments:!•  Autonomic application servers!•  Resource management for heterogenous workloads!•  Coordinated scheduling and resource management!•  Parallel file system scalability!

Programming models: !•  Scalability of MPI and UPC!•  OpenMP for multicore, SMP " and ccNUMA!•  DSM for clusters!•  CellSs, streaming!• Transactional Memory!•  Embedded architectures!

3!

BSC-CNS: Applications in Science & Engineering!

Computational fluid dynamics!

Geophysics! ITER: Plasma physics!

Ab-initio molecular dynamics!Bio-mechanics!

4!

BSC-CNS: Life Sciences!

Atomic (and electronic) modeling of protein biochemistry and biophysics.!

Identification of the structural bases of protein-protein interaction. !

Micro and mesoscopic modeling of macromolecules. Drug design.!

Protein-protein interaction networks.!Systems biology.!

Web services, applications, databases.!

Analysis of genomes and networks to model diseases, systems and evolution

of organisms.!

5!

BSC-CNS: Earth Sciences!

Air Quality Forecast!

Mineral dust transport!

Volcanic ash transport!

Developement of a global model of mineral dust !

Technology transfer: Environmental impact

studies!

Climatic change!

6!

7!

MareNostrum: a living sea (2004)!

•  MareNostrum2004!

-  42,35 Teraflops!

-  4812 PowerPC 970 cores!

-  2406 JS20 2.2 GHz nodes!

-  10 TB of memory!

-  4 GB per node!

-  270 TB of storage capacity!

•  3 networks!

-  Myrinet!

-  Gigabit!

-  10/100 Ethernet!

•  Operating system!

-  Linux 2.6 (SuSE)!

8!

MareNostrum: a living sea (2006)!

•  MareNostrum2006!

-  94,21 Teraflops!

-  10240 PowerPC 970 cores!

-  2560 JS21 2.3 GHz nodes!

-  20 TB of memory!

-  8 GB per node!

-  480 TB of storage capacity!

•  3 networks!

-  Myrinet!

-  Gigabit!

-  10/100 Ethernet!

•  Operating system!

-  Linux 2.6 (SuSE)!

9!

MariCel: Spanish prototype in PRACE!

MariCel is in 2 racks of MareNostrum’s computer room!

IBM QS22 Blade Center!

MariCel!

•  72 QS22 IBM Blade servers!

•  PowerXCell 8i 3.2Ghz CPUs !

•  864 GB of total RAM!

•  1296 cores!

•  12 JS22 IBM Blade servers!

•  Power6 64 bits 4.0 GHz dual core!

•  96 GB RAM total!

•  48 cores!

•  4xDDR InfiniBand (16GB)!

•  MPI, GPFS!

•  4 x 24 switch ports!

•  Peak performance 14.4 Tflops from 2 rack units!

•  20 kW of consumption!

10!

CNAG, National Center for Genome Analysis

•  BSC-CNS provides services to CNAG HPC, data and IT !

•  Next generation sequencing !

Aligned results !250-500 GB/run!

Sequence analysis,!

Alignment and !clustering!

Raw Data:!1-2TB/run!

2 runs/week!10 machines!

Image processing !to generate !

sequence data!

Rapid sequencing "of whole individuals"

Detailed studies cellular processes!

11!

Spanish Supercomputing Network (RES)!

BSC-CNS (MareNostrum)!Processor: !10240 PowerPC 970 2.3 GHz!Memory: !20 TB!Disk: !280 + 90 TB!Networks: !Myrinet, Gigabit, 10/100!System: !Linux!

UPM (Magerit)!Processor: !2408 PowerPC 970 2.2 GHz!Memory: !4.7 TB!Disk: !63 + 47 TB!Networks: !Myrinet, Gigabit, 10/100!System: !Linux!

IAC, UMA, UC, UZ, UV (LaPalma, Picasso, Altamira, Caesaraugusta,

Tirant)!Processor: !512 PowerPC 970 2.2 GHz!Memory: !1 TB!Disk: !14 + 10 TB!Networks: !Myrinet, Gigabit, 10/100!System: !Linux!

Canary Islands Government – ITC (Atlante)!

Processor: !336 PowerPC 970 2.3 GHz!Memory: !672 GB!Disk: !3 TB!Networks: !Myrinet, Gigabit, 10/100!System: !Linux!

12!

RES: supercomputing backbone service in Spain!

Magerit"Polytechnic University of Madrid! Atlante!

ITC!La Palma!

IAC!

CaesarAugusta"University of "

Zaragoza!

Picasso"University of "

Malaga!

Altamira"University of "

Cantabria!

Tirant"University of Valencia!

MareNostrum!BSC-CNS!

13!

Scientific projects in RES!

Astronomy, Space and Earth

Sciences!

Biomedicine and Life Sciences!

Chemistry and Science and

technology of Materials!

BSC-CNS!

12! 17! 12! 16! 21! 14! 14! 11! 14! 12! 14! 11! 14! 18! 16! 216!

20! 26! 18! 18! 26! 24! 20! 28! 24! 22! 23! 26! 29! 24! 19! 347!

19! 24! 17! 20! 32! 29! 22! 21! 32! 26! 24! 21! 23! 21! 25! 356!

21! 27! 23! 18! 33! 32! 31! 26! 29! 26! 22! 22! 16! 23! 26! 375!

14! 19! 14! 14! 22! 29! 17! 17! 20! 17! 17! 16! 16! 17! 17! 258!

2005! 2006! 2007! 2008! 2009! 2010! 1552!

Physics and Engineering !

How to request access to the RES

RES Area (www.bsc.es/RES)!

•  Publication of calls for access to RES!

•  See terms and conditions of use!

•  User registration !

•  Submission of access requests !

! !- Activity title !

! !- Project and activity description !

! !- Numerical libraries and software !

! !- Description of the research team!

! !- Required resources !

! !- Abstract of the activity for its publication!

14!

Access Committee!

Distribution of resources usage!•  20% own use for research in each site!•  80% assigned by Access Committee!

Single Access Committee!•  44 scientists (not limited to Spanish scientists)!

•  4 panels!•  Astronomy, Space and Earth Sciences!•  Biomedicine and Life Sciences!•  Physics and Engineering!•  Chemistry and Science and Technology of Materials!

•  Core Team!•  Triennial renewal!•  Appointed by the MICINN, independent of the BSC-CNS!•  Technical assistance from BSC-CNS staff!•  Quarterly access assignment !

15!

Researcher!

BSC-IT!

Web form! Shape defect!ANEP!

General project peer evaluation ! General project !

not previously evaluated!

Expert Panel!

General project !previously evaluated!

1. Quality of the activity (>30%)!2. Impact of the developed calculation !3. Experience in HPC!4. Need for supercomputing (>30%)!5. Adaptation to architecture (>30%)!

Access Committee!

4 lists of activities, !categories and priorities!

BSC-CNS!Prioritization!

Decision !

16!

Evaluation of activities!

Class A!Class B!

Without access!

Average resources of a RES user

17!

•  Processors: 245!

•  Calculation time (thousands of CPU hours):13.700 !

•  Disk space: 98 TB!

•  Tape space: 288 TB!

•  Services: support, mobility and training. !

(Average according to the RES assignments in the second period of 2010)!

Support offered to RES users!

18!

•  Code optimization for its use in RES supercomputers!

•  Parallelization and scalability improvements !

•  Optimization of Input/Output processes!

•  Code porting (architecture, GridSuperscalar…)!

•  Code debugging (Totalview, Paraver, DDT…)!

•  User training!

•  Resolution of technical and administrative issues!

- Account creation!

- Help with the resources request !

- Weekly reports!

…!

Mobility services to RES users!

•  Test activities!

•  ICTS financial assistance!

Visits to the site allocated to get support in-situ!

http://www.bsc.es/ayuda-mobilidad-icts !

•  Mobility program HPC- Europa2!

http://www.hpc-europa.eu!

19!

Objectives!

• To convey the opinion and advise the RES coordinator about the services and resources provided.!

• Promote effective use of RES resources sharing information about user experience, suggesting further research and being the users voice. !

RES Users’Committee (CURES)!

20!

Members!

• 8 scientists (two for each RES area) who had been PI of an activity developed in the RES. !

• 1 representative chosen by the RES. !

History!• Governance of BSC-CNS RES began its formation in October 2009. !• CURES was established in February 2010. !

Trainings for RES users!

User training: Parallel computing !20th October 2010, Santander!

User training: Programming models (STARSs)!From 20th to 21st September 2010 in Barcelona.!Soon in Tenerife.!

User training: Introduction to supercomputing!November 2011, in Barcelona.!

Scientific seminar: Parallel simulations in the network!10th November 2010 in Zaragoza.!

Supercomputing and Next Generation Sequencing !Right now in Málaga. !

Scientific seminar: Image Processing !Soon in Santander. !

More information at: http://www.bsc.es/res/next-events!

21!

Technical seminars!

Implementation of ITIL approach to the technical management of RES services !

From 18th to 19th May 2010 in Santander. !

Energy management and cooling of a DPC!From 19th to 20th May 2010 in Santander. !

Storage management in the RES !14 September 2010 in Barcelona. !

Function and application of Perfminer !15 September 2010 in Barcelona. !

Visualization and virtual reality !Soon in Barcelona. !

22!

More information at: http://www.bsc.es/res/next-events!

RES Users’meeting Presentation of RES, dissemination of research in different areas of science, operation of

the Access Committee and the CURES. !

I RES Users’meeting (23 April 2008, Barcelona)!•  Speakers: Francisco Javier Luque, Arcadi Navarro, Javier Jiménez Sendín, Fernando Martín, Gustavo Yepes,

Fernando Moreno Insertis, Francesc Illas, Rubén Pérez, Fernado Javier de la Cruz.!

II RES Users’meeting (23 September 2009, Santander)!•  Speakers: Sjors Scheres, Carme Rovira, Assumpta Parreño García, Francisco Castejón, Marcel Swart,

Nuria Lopez, Pablo Fosalba, Enrique Martinez Gonzalez, Ramón Beivide, Joaquin Serrano, Sergi Girona.!

III RES Users’meeting (1 July 2010, Zaragoza)!•  Speakers: Alfonso Tarancón, Mateo Valero, Víctor Castelo, Sergi Girona, Carles Serrat, Carlos Pena,

Javier Fernández, Cristina Díaz, Rosa Domínguez, Yves Soufflet, Ludovico Sutto, Ronen Zangi, Victoria Ley, Jordi Torra. !

IV RES Users’meeting (24 November 2010, Madrid)!•  Speakers: Vicente Martín Ayuso, Víctor Castelo, Sergi Girona, Manuel Alcamí, Federico Gago, Oswaldo

Dieguez, Xavier Luri, David Vicente, Jordi Torra.!

23!

Support to seminars and conferences!

•  RES offers its collaboration to jointly prepare seminars and conferences:!

!- Providing expertise and experts!

!- Conducting trainings!

- Co-funding the celebration of events of scientific popularization!

24!

Scientific dissemination!

•  Scientific publications

•  BSC-CNS and RES annual report

25!

26!

PRACE!

ESFRI Vision for a European HPC service!

•  Tier-0: 3-5 European centers!

•  Tier-1: national centers!

•  Tier-2: regional centers/universities !

•  European HPC systems involve all participants: !

!- HPC service providers in every tier!

!- Grid infrastructures!

!- Communities of scientific and industrial users!

!- The European HPC software and hardware industries!

tier-0

tier-1

tier-2

PRACE

DEISA

27!

capa

bilit

y!

# of systems!

28!

PRACE

New general members - since May 2008

Tier 0 – Hosting members

Tier 1 – General members

Available machines!

JUGENE IBM BlueGene/P – FZJ, Jülich, Germany:!

http://www.fz-juelich.de/jsc/service/sco_ibmBGP!

!Storage capacity: !

http://www.fz-juelich.de/jsc/jump/usage/FAQ/data-limits!

Next machine available – GENCI, France!

!Summer 2011!

PRACE

29!

PRACE Calls – JUGENE (Jülich)!

Million core hours!month!

time!07/10 09/10 11/10

01/11 03/11 05/11

07/11 09/11 11/11

01/12 03/12 05/12

30

60

…

…

2nd Regular PRACE Call!

1st Regular PRACE Call!

PRACE Early Access Call!

1st allocation!period!

2nd allocation !period!

30!

More information at: http://www.prace-project.eu/hpc-access!

PRACE Early access call

•  Initial call for a reduced number of large projects!

•  Highly relevant scientific projects, mature enough to be able to achieve publishable results in a short allocation period!

•  1 call with two allocation periods of 4 months each!

•  First allocation period from 1st August to 30th November 2010!

•  Second allocation period from 1st December 2010 to 31st March!

•  Projects in second allocation period may request technical support (preparatory access) in the first allocation period!

•  Only proposals from academia and where the organisation of the project leader is homed in a European Union or a PRACE initiative country are eligible!

31!

Fully operational, secure on-line application system!

•  Fully electronic submission!

•  Automatic rejection of post-deadline and incomplete applications!

•  On-line peer review (including reviewers’ reports and applicant’s reply)!

•  Track application status (PRACE staff and applicant)!

•  Electronic submission of reviewers’ reports!

32!

Access through: http://www.prace-project.eu/hpc-access!

PRACE regular calls!

•  Preparatory access – code testing and optimisation, technical support if requested, continuous call, fast track assessment, maximum allocation of 6 months!

•  Project access – 2 calls per year, 1 year allocation, allocation in November and May!

•  Programme access – large projects of a research group, 2 years allocation, calls coincide with the calls for project access, possible review at the end of the first year allocation!

Remark: Final report mandatory for all types of proposals!

33!

BSC-CNS offers:!

Assistance in preparing the application calls !• Application preparation !

- Testing if the code is suitable to run in the PRACE facility!

- Assessment about the amount of resources needed!…!

• Scalability tests in MareNostrum or in the PRACE facilities!

Assistance during the execution!• Code optimization !• Benchmarking!

PRACE!

34!

Thanks for your attention " http://www.bsc.es!