last update: 08/12/2015 14:00 datagrid overview & status cnap meeting 13th october 2000...
TRANSCRIPT
last update: 21/04/23 12:09
DataGridDataGrid
Overview & StatusOverview & Status
CNAP Meeting 13th October 2000CNAP Meeting 13th October 2000
R.P.Middleton (DataGrid UK Manager)R.P.Middleton (DataGrid UK Manager)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 2
Rapid OverviewRapid Overview
Proposal favourably reviewed -> now negotiating contract with Proposal favourably reviewed -> now negotiating contract with EUEU
10 Meuro, 21 partners (6 principal) -> BIG consortium10 Meuro, 21 partners (6 principal) -> BIG consortium Collaboration with Earth Observation & Bio-sciencesCollaboration with Earth Observation & Bio-sciences 4 + 1 “funded” posts for UK4 + 1 “funded” posts for UK ( WP3: 3+1; WP5: 1 )( WP3: 3+1; WP5: 1 ) UK Industrial Partner : IBMUK Industrial Partner : IBM PMB - 6 month rotating chair; PTB includes experimentsPMB - 6 month rotating chair; PTB includes experiments CERN Project Office (Project Manager + Architect + Secretariat)CERN Project Office (Project Manager + Architect + Secretariat) Accompanying Measures : “Research & Industry Forum” Accompanying Measures : “Research & Industry Forum” Frequent WP managers phone conferencesFrequent WP managers phone conferences Prototype work already starting (e.g. Globus installation & tests)Prototype work already starting (e.g. Globus installation & tests) Commitment to Open Source for common software componentsCommitment to Open Source for common software components Presented at DANTE/Geant, Terena conference, ECFA, HEP-CCCPresented at DANTE/Geant, Terena conference, ECFA, HEP-CCC High visibility in EU & their IST programmeHigh visibility in EU & their IST programme
(http://europa.eu.int/comm/information_society/eeurope/news/index_en.htm)(http://europa.eu.int/comm/information_society/eeurope/news/index_en.htm)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 3
So What is it ?So What is it ?
Principal goals:Principal goals: Middleware for fabric & Grid management Linking together of “national” PP Grid infrastructures Large scale testbed - major fraction of one LHC experiment Production quality HEP demonstrations
“mock data”, simulation analysis, current experiments “Other science” demonstrations on testbeds
Three year phased developments & demosThree year phased developments & demos Anticipated start (of EU money) ~Jan 2001
Complementary to other EU GRID projectsComplementary to other EU GRID projects EuroGrid: Uniform access to parallel supercomputing resources
Synergies being developedSynergies being developed Instumental in merge of GridForum & EGRID into Global Grid
Forum Associated Industry and Research Forum (Accompanying
Measure ?) Close contact with Globus team (with informal input to
DataGrid)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 4
……and What it is Notand What it is Not
No funding for kitNo funding for kit get from national resources (CERN ?)
No substitute for a national Grid programmeNo substitute for a national Grid programme but an external link to LHC data
No money for writing PP specific codeNo money for writing PP specific code EU funds “economic development”, not physics research
Not THE final solution for LHCNot THE final solution for LHC but a very important stepping stone
Not a full-scale implementationNot a full-scale implementation but a realistic prototype
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 5
ParticipantsParticipants
Main partners: CERN, INFN(I), CNRS(F), PPARC(UK), Main partners: CERN, INFN(I), CNRS(F), PPARC(UK), NIKHEF(NL), ESA-Earth Observation NIKHEF(NL), ESA-Earth Observation
Other sciences: KNMI(NL), Biology, Medicine Other sciences: KNMI(NL), Biology, Medicine Industrial participation: CS SI/F, DataMat/I, IBM/UKIndustrial participation: CS SI/F, DataMat/I, IBM/UK Associated partners: Czech Republic, Finland, Associated partners: Czech Republic, Finland,
Germany, Hungary, Spain, Sweden, (- mostly from Germany, Hungary, Spain, Sweden, (- mostly from computer science)computer science)
Formal collaboration with USA being establishedFormal collaboration with USA being established Industry and Research Project Forum with Industry and Research Project Forum with
representatives from:representatives from: Member Countries + Denmark, Greece, Israel, Japan,
Norway, Poland, Portugal, Russia, Switzerland, +...
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 6
WorkpackagesWorkpackages
WP 1 Grid Workload Management (C. Vistoli/Italy)WP 1 Grid Workload Management (C. Vistoli/Italy) WP 2 Grid Data ManagementWP 2 Grid Data Management (B. Segal/CERN) (B. Segal/CERN) WP 3 Grid Monitoring servicesWP 3 Grid Monitoring services (R. Middleton/ (R. Middleton/UKUK)) WP 4 Fabric ManagementWP 4 Fabric Management (T. Smith/CERN) (T. Smith/CERN) WP 5 Mass Storage Management (J. Gordon/WP 5 Mass Storage Management (J. Gordon/UKUK)) WP 6 Integration TestbedWP 6 Integration Testbed (F. Etienne/France) (F. Etienne/France) WP 7 Network ServicesWP 7 Network Services (C. Michau/France) (C. Michau/France) WP 8 HEP ApplicationsWP 8 HEP Applications (F. Carminati/CERN) (F. Carminati/CERN) WP 9 EO Science ApplicationsWP 9 EO Science Applications (L. Fusco/ESA) (L. Fusco/ESA) WP 10 Biology ApplicationsWP 10 Biology Applications (C. Michau/France) (C. Michau/France) WP 11 Dissemination WP 11 Dissemination (G. Mascari/Italy) (G. Mascari/Italy) WP 12 Project ManagementWP 12 Project Management (F. Gagliardi/CERN) (F. Gagliardi/CERN)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 7
Workpackages - UKWorkpackages - UK
WP 1 Grid Workload Management WP 1 Grid Workload Management A.Martin-QMWA.Martin-QMW WP 2 Grid Data ManagementWP 2 Grid Data Management A.Doyle-GlasgowA.Doyle-Glasgow WP 3 Grid Monitoring servicesWP 3 Grid Monitoring services R.Middleton-RALR.Middleton-RAL WP 4 Fabric ManagementWP 4 Fabric Management A.Sansum-RALA.Sansum-RAL WP 5 Mass Storage Management WP 5 Mass Storage Management J.Gordon-RALJ.Gordon-RAL WP 6 Integration TestbedWP 6 Integration Testbed D.Newbold-BristolD.Newbold-Bristol WP 7 Network ServicesWP 7 Network Services P.Clarke-PPNCG/UCLP.Clarke-PPNCG/UCL WP 8 HEP ApplicationsWP 8 HEP Applications N/A (?)N/A (?) WP 9 EO Science ApplicationsWP 9 EO Science Applications ( c/o R.Middleton-( c/o R.Middleton-
RAL )RAL ) WP 10 Biology ApplicationsWP 10 Biology Applications ( c/o P.Jeffreys-RAL )( c/o P.Jeffreys-RAL ) WP 11 Dissemination WP 11 Dissemination P.Jeffreys-RALP.Jeffreys-RAL WP 12 Project ManagementWP 12 Project Management R.Middleton-RALR.Middleton-RAL
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 8
Simplified Workpackage Simplified Workpackage RelationshipsRelationships
HEP Apps (WP8) EO Apps (WP9) Bio Apps (WP10)
Workload Management (WP1)
Data Management (WP2) Monitoring Services (WP3)
Globus Middleware
Fabric Mangage-ment (WP4)
Networking (WP7)Mass Storage
Management (WP5)
Applications
Data Grid Services
Core Middleware
PhysicalFabric
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 9
UpdateUpdate
Negotiating EU ContractNegotiating EU Contract deadline 25th October 3rd iteration, but slowly converging !! background doc. on Testbed (accepted)
Turn project/management from re-active to pro-activeTurn project/management from re-active to pro-active Forging links : Globus, GriPhyN, PPDG, GF/GGF/EgridForging links : Globus, GriPhyN, PPDG, GF/GGF/Egrid Network negotiation - UKERNA, Dante, etcNetwork negotiation - UKERNA, Dante, etc Search for Project Architect, PTB ChairmanSearch for Project Architect, PTB Chairman Formation of ATFFormation of ATF
Attend GF5, meet with Globus, GriPhyN, PPDG DataGrid “kick-off” in AmsterdamDataGrid “kick-off” in Amsterdam
Joint with GGF1 ? (confirm next week in Boston)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 10
Organogram ?Organogram ?
PMB
Executive (5 pers)
Arch Task Force
Proj Tech Board
WP1WP1WP1WP1Middleware
WP1-5Testbed & NetworkWP1-5WP1-5HEP/EO/BIO
WP1-5WP11 & 12
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 11
PoliticsPolitics
DataGrid has VERY HIGH profile in EU & Government(s)DataGrid has VERY HIGH profile in EU & Government(s) On lips at OST… Seen (in OST) as THE link to international activity,
standards, etc. Has declared programme bringing in bio- & earth sciences
DataGrid has much support in high placesDataGrid has much support in high places also has critics (enemies) in high places
DataGrid MUST DELIVERDataGrid MUST DELIVER failure is not possible late delivery / poor results ==> acutely embarassing …and LHC computing could fragment if no Grid-scale
solution UK domestic funding taken VERY seriously by DataGrid UK domestic funding taken VERY seriously by DataGrid
partnerspartners even our domestic programme is very exposed !!
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 12
DataGrid : DomesticDataGrid : Domestic
ProblemsProblems JIF held over - was basis for DataGrid planning in UK Ability to attend meetings Testbed (0) and Demos MUST start to deliver ==> ramp up effort
Working with PPARC…no solution yetWorking with PPARC…no solution yet Travel £15k Enabling “unfunded” effort ramp-up to 15 FTE 10% Yr 1 of JIF (CPU & disk) for Jan 2001 £105k
EU posts open for 5 FTE (actually 4.3)EU posts open for 5 FTE (actually 4.3) 1 Mass storage 3 Monitoring Service (consortium requires cluster) 1 HEP Applications (significant time at CERN)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 13
Jan 2001 @ Tier-1Jan 2001 @ Tier-1
Need Resource NOW which can be dedicated to GridNeed Resource NOW which can be dedicated to Grid no impact on existing services shared internationally
JIF Yr 1 (neglect tape & network)JIF Yr 1 (neglect tape & network) 830 CPU processors (in PC99 units = 450 MHZ PC = 1
SpecInt95) 25 TBytes of disk space
10% Proposal for Jan 200110% Proposal for Jan 2001 20 Dual CPU Pentium processors @ ~£2.5k each £ 50k
(each CPU roughly 2 x PC99 performance) 2 TByte of disk space @ ~£20k per Tbyte £ 40k networking and infrastructure £ 15k
TOTALTOTAL£105k£105k
4-6 weeks of FTE Do not want to continue in 10% size chunks !!Do not want to continue in 10% size chunks !!
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 14
EffortEffort
PPARC PPARCWP Title Unfunded Funded (staff months)1 Workload Management 18 02 Data Management 54 03 Monitoring Services 66 1084 Fabric Management 18 05 Mass Storage Management 54 366 Testbeds & Demonstrations 108 07 Networking 72 08 HEP Applications 144 129 Earth Sciences 0 0 (invisible to EU)10 Bio Sciences 0 0 (3.6 budgeted)11 Dissemination 0 0 (3.6 budgeted)12 Project Management 0 0 (18 budgeted)
TOTAL 534 156
Grand Total 690
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 15
Effort profilesEffort profiles
Middleware (WP1-5)Middleware (WP1-5) 210 staff months210 staff months Testbed/Apps (WP6-8)Testbed/Apps (WP6-8) 324 staff months324 staff months To date…To date…
guess < 4 FTE in use around UK (incl RAL) virtually no Middleware development
…and little sign of effort here from ppUK !! ppUK focussed on deployment
Globus testbed Networking Applications
Ramp-up by Jan 2001 is CRITICALRamp-up by Jan 2001 is CRITICAL
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 16
Marseille Workshop 4-6 Sept 2000Marseille Workshop 4-6 Sept 2000
Rush to setup Sept workshopRush to setup Sept workshop Difficult to organise an agenda over summer Impossible to get people to commit to come during holiday
season However, forward progress ...However, forward progress ...
100 attendees (8 (9) from UK = RAL) Cross-workpackage issues aired Worked on fine detail that timescales had not permitted
before ! “Team Building” for many workpackages
Who are we ? What is there to do and who’s going to do it ?
Mentor feedback - Carl Kesselmann - EXTREMELY usefulMentor feedback - Carl Kesselmann - EXTREMELY useful
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 17
Marseille DecisionsMarseille Decisions
Project ManagementProject Management Internal Reviewer contract (with EPCC)
Project Technical BoardProject Technical Board 2 (per WP) + 4 (HEP) + 4 (EO) + 2 (Bio) + PM 35 members first meeting early October
Architecture Task Force (Project Architect ?)Architecture Task Force (Project Architect ?) Project-wide milestonesProject-wide milestones
focussed, major goals (i.e. take a problem & solve it) internal testbed cycle of 6 months external testbed cycle annually
MeetingsMeetings 4 * DataGrid weeks per year Annual review (incl. deliverables) by external referees Annual project conference (aligned with IST event ?)
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 18
Contract Meeting - 11th Sept 2000Contract Meeting - 11th Sept 2000
No Show-stoppersNo Show-stoppers Very high profile / visibility within EUVery high profile / visibility within EU
seen as a key project within IST programme ! Updates to management structure satisfied EUUpdates to management structure satisfied EU A shopping list of details to fixA shopping list of details to fix
missing deliverables uniformity of deliverables risk analysis dissemination activities with external groups (e.g. standards) Research & Industry forum (Accompanying Measure)
Rumour : EU planning new, specific Grid action in 2001Rumour : EU planning new, specific Grid action in 2001
DataGrid Status - CNAP Meeting 13-Oct-00 - R.P.Middleton13th September 2000 19
TimelineTimeline
JulyJuly Proposal accepted --> Contract NegotiationsProposal accepted --> Contract Negotiations 25th Aug25th Aug CPF & draft Technical Annex to EUCPF & draft Technical Annex to EU 4-6 Sep4-6 Sep Technical Workshop in MarseilleTechnical Workshop in Marseille 11th Sep11th Sep Contract negotiation in BrusselsContract negotiation in Brussels 22nd Sep22nd Sep Revisions submitted to EURevisions submitted to EU this weekthis week More revisions submitted to EUMore revisions submitted to EU . . .. . . 25th Oct25th Oct Last date for contract input !!Last date for contract input !!
(otherwise miss 2000 budget)(otherwise miss 2000 budget) Dec 2000Dec 2000 DataGrid/GriPhyN/PPDG Management DataGrid/GriPhyN/PPDG Management
BoardsBoards(technical collaboration)(technical collaboration)
1st Jan 20011st Jan 2001 Current aim for project startCurrent aim for project start Feb 2001Feb 2001 GGF1 & DataGrid “Kick-off” meetingGGF1 & DataGrid “Kick-off” meeting