12th November 2003 LHCb Software Week 1

UK Computing

Glenn PatrickRutherford Appleton Laboratory

12th November 2003 LHCb Software Week 2

RAL Tier-1

12th November 2003 LHCb Software Week 3

RAL Tier-12002 - 312 cpus4 racks holding 156 dual 1.4GHz Pentium III cpus.

March 2003 - extra 160 cpus 80 x dual processor P4, 2.66GHz, Xenon

Dec 2003 - extra 400/500 cpu200 - 250 dual systems

Operating system now RedHat 7.3. Batch = PBS. CSF Legacy Equipment 250 cpus (450MHz - 1 GHz) Total ~1000 cpus

12th November 2003 LHCb Software Week 4

RAL Tier-1 Mass Storage

March 2003 - Extra 40TB11 new disk servers, each with 2 x 1.8TB filesystems

Dec 2003 - Extra ~70TB

2002 - 40TB26 disk servers, each with 2 x 0.8TB filesystems.

Disk Cluster

Total ~150TB

12th November 2003 LHCb Software Week 5

RAL DataStore

STK 9930 (Powderhorn) tape robot

June 2003Updated with 8xSTK 9940B drives.Transfer speed 30MB/sec each driveTape capacity 200GB

5,500 slots = 1PB potential capacity

Current capacity limited by number of tapes.

12th November 2003 LHCb Software Week 6

Switch_1 Switch_2

RS6000 RS6000RS6000 RS6000

fsc0 fsc1 fsc1fsc0

9940B 9940B 9940B 9940B 9940B 9940B 9940B 9940B

1 2 3 4 5 6 7 8

11 14 11 1415

fsc1fsc0fsc1fsc0

12 13 12 13 15

rmt1 rmt4rmt3rmt2rmt5-8 rmt5-8rmt5-8rmt5-8

A A A A A A A A

STK 9310 “Powder Horn”

Gbit network

1.2TB 1.2TB 1.2TB 1.2TB

12th November 2003 LHCb Software Week 7

GRID at RALLCGCurrently 5 worker nodes in testbed.15.8.2003 LCG On-Line10.9.2003 Upgraded to LCG1-1_0_022.9.2003 Upgraded to LCG1-1_0_1

Amount of future hardware deployed in LCG depends on experiments and GRIDPP.

EDGEDG 2.1 Deployed on development testbedEDG 2.0 On main production testbed.EDG 1.4 Gatekeeper into main production farm.

12th November 2003 LHCb Software Week 8

LHCb

CMS

BaBarATLAS

12th November 2003 LHCb Software Week 9

2003 Data Challenge 2004 (same sites)?Site SPECint2k*hour

s% Share

SPECint2k*hours

Storage(GB)

Bristol 5,520,705 0.9% 37,019,600 183

Cambridge

12,881,646 2.1% 86,379,067 428

Imperial 78,516,698 12.8% 526,500,978 2,606

Oxford 7,360,940 1.2% 49,359,467 244

RAL 54,593,642 8.9% 366,082,711 1,812

ScotGrid 47,232,701 7.7% 316,723,244 1,568

TOTAL 206,106,332 33.6% 1,382,065,067

6,841

Factor of 6.7

Still hope for ~10% share from 3 largest centres?

12th November 2003 LHCb Software Week 10

Tier-1 Resources for LHCb

Requested for DC04 (April - June):From Marco’s numbers (assuming same share as DC03)

CPU requirement - 366M SI2k*hours 6TB of disk for "permanent copy" of all DSTs (may reduce to 1TB if pre-selection is used) to be used for analysis. Existing disk servers (3.2TB) used to store MC production from RAL and other UK sites before transfer to tape/CERN.. Mass storage of 7TB to store SIM+DIGI+ data from all UK sites.

But actual resources will depend on competition from other experiments.

12th November 2003 LHCb Software Week 11

0

100

200

300

400

500

600

700

800

900

1000

UKQCD

Other

D0

Alice

LHCb

Atlas

CMS

BaBar

GPP-only

90%

Capacity

CPU Requirements (KSI2K)

LHCb

x3

LHCb

LHCb need~20% of farm for 3 months

12th November 2003 LHCb Software Week 12

UK Tier-2 Centres

NorthGridDaresbury, Lancaster, Liverpool,Manchester, Sheffield

SouthGridBirmingham, Bristol, Cambridge,Oxford, RAL PPD

ScotGridDurham, Edinburgh, Glasgow

LondonGridBrunel, Imperial, QMUL, RHUL, UCL

12th November 2003 LHCb Software Week 13

Tier-2

Number of CPUs

Total CPU

(KSI2000)

Total Disk (TB)

Total Tape (TB)

London 677 335 26 8

NorthGrid 815 263 43 28

SouthGrid 436 229 32 8

ScotGrid 259 126 18 20

Total 2187 953 118 64

Tier-2Number of CPUs

Total CPU

(KSI2000)

Total Disk (TB)

Total Tape (TB)

London 2454 1996 99 20

NorthGrid 2718 2801 209 332

SouthGrid 918 930 67 8

ScotGrid 368 318 79 0

Total 6458 6045 455 360

Existing Hardware(April 2003)

Estimated Hardware (Sept 2004)

12th November 2003 LHCb Software Week 14

Liverpool New MAP2 facility now installed

940 3GHz/1GB/128GB P4 Dell Nodes

SCALIManage installed, RH9 20 CPU CDF facility now

installed, Fermi RH Linux and 5.9TB disk

MAP memory upgrade (270 nodes)

EDG 2.0 being installed

10% of DC04 would take17 days on all processors.

Initial LHCb schedulingproposal:50% of farm for ~1 month.

~1.1M SPECint2k

12th November 2003 LHCb Software Week 15

ScotGrid

ScotGRID Processing nodes at Glasgow (128 cpu)ScotGRID Processing nodes at Glasgow (128 cpu)• 59 IBM X Series 330 dual 1 GHz Pentium III with 2GB memory • 2 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory • 3 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory • 1TB disk • LTO/Ultrium Tape Library • Cisco ethernet switches

ScotGRID Storage at Edinburgh (5TB)ScotGRID Storage at Edinburgh (5TB)• IBM X Series 370 PIII Xeon • 70 x 73.4 GB IBM FC Hot-Swap HDD

Phase 1complete

Phase 2 - now commissioningUpgrade database server in Edinburgh16-20TB disk storage in Edinburgh5TB disk storage in Glasgow (relocation from Edinburgh)Edge servers for EdinburghNew kit for Glasgow - CDF and eDIKT

12th November 2003 LHCb Software Week 16

ImperialViking at London e-Science Centre:Upgrade to ~500 cpu cluster(33% HEP + bioinformatics +…)Ready to join LCG1

Ulrik - “Factor of 5 seems realistic” running across~3 months.

Note: Other potential resources coming online inLondon Tier 2...Royal Holloway ~100cpuUCL ~100 cpuBrunel(BITLab)64 dual Xenon nodes + 128 more nodesTimescale? LHCb use?

12th November 2003 LHCb Software Week 17

Manpower

Currently, little dedicated manpower.Rely on effort shared with other tasks. Gennady has been main technical link for Tier1.

Easy installation/maintenance of production& analysis software and tools