high energy physics & computing grid
DESCRIPTION
High Energy Physics & Computing Grid. Univ. of Texas @ Arlington Dr. Yu. Outline. High Energy Physics The problem A solution An example of implemented solution Accomplishments Future plans Summary. Matter. Molecule. Atom. Nucleus. Baryon. Quark. (Hadron). u. Electron. (Lepton). - PowerPoint PPT PresentationTRANSCRIPT
HEP and Grid ComputingDr. Jaehoon Yu
High Energy Physics & Computing Grid
Univ. of Texas @ Arlington
Dr. Yu
Feb. 3, 2005 2HEP and Grid ComputingDr. Jaehoon Yu
Outline• High Energy Physics• The problem• A solution• An example of implemented solution• Accomplishments• Future plans• Summary
Feb. 3, 2005 3HEP and Grid ComputingDr. Jaehoon Yu
High Energy Physics
Structure of Matter
10-10m 10-14m 10-15m
<10-18m
10-9m
Matter Molecule Atom Nucleus
u
Quark
<10-19mprotons, neutrons,
mesons, etc.
top, bottom,charm, strange,
up, down
Condensed matter/Nano-Science/ChemistryAtomic Physics
NuclearPhysics
Baryon(Hadron)
Electron(Lepton)
10-2m
Feb. 3, 2005 4HEP and Grid ComputingDr. Jaehoon Yu
Mysteries in High Energy Physics? The “STANDARD MODELSTANDARD MODEL” has been extremely successful (Precision 10-6)
BUT… many mysteries
Why so many quarks/leptons??
Why four forces?? Unification?
Why is there large particle- antipaticle asymmetry?
Does Higgs particle exist?
Where does mass come from??
Are there other theories??
Feb. 3, 2005 5HEP and Grid ComputingDr. Jaehoon Yu
High Energy Physics• Definition: A field of Physics pursues for fundamental
constituents of matter and basic principles of interactions between them How is universe created, and how does it work?
• Use large particle accelerators • Use large particle detectors
Feb. 3, 2005 6HEP and Grid ComputingDr. Jaehoon Yu
The Standard Model• Assumes the following fundamental structure:
Discovered in 2000
Discovered in 1995
Feb. 3, 2005 7HEP and Grid ComputingDr. Jaehoon Yu
Fermilab Tevatron and LHC at CERN• Present world’s Highest Energy
proton-anti-proton collider – Ecm=1.96 TeV (=6.3x10-7J/p
13M Joules on 10-4m2) Equivalent to the kinetic energy of
a 20t truck at a speed 80 mi/hr
Chicago
Tevatron p
p CDF
DØ
• World’s Highest Energy proton-proton collider in 2 years – Ecm=14 TeV (=44x10-7J/p
1000M Joules on 10-4m2) Equivalent to the kinetic energy of
a 20t truck at a speed 6150 mi/hr
Feb. 3, 2005 8HEP and Grid ComputingDr. Jaehoon Yu
High Energy Physics• Definition: A field of Physics pursues for fundamental
constituents of matter and basic principles of interactions between them How is universe created, and how does it work?
• Use large particle accelerators • Use large particle detectors• Large, distributed collaborations
– ~600/experiment for currently operating experiments– ~2000/experiment for future experiments – WWW grew out of HEP to expedite communication
between collaborators
Feb. 3, 2005 9HEP and Grid ComputingDr. Jaehoon Yu
Particle Detection
InteractionPoint
electron
photon
jet
muonneutrino -- or any non-interacting particle missing transverse momentum
B
Scintillating FiberSilicon Tracking
Charged Particle Tracks
Calorimeter (dense)
EM hadronic
Energy
Wire Chambers
Mag
net
Muon Tracks
We know x,y starting momenta is zero, butalong the z axis it is not, so many of our measurements are in the xy plane, or transverse
Feb. 3, 2005 10HEP and Grid ComputingDr. Jaehoon Yu
DØ Detector: Run II
30’
30’
50’
• Weighs 5000 tons• Can inspect
3,000,000 collisions/second
• Will record 50 collisions/second
• Records ~12.5M Bytes/second
• Will record 2 Peta bytes in the current run.
Feb. 3, 2005 11HEP and Grid ComputingDr. Jaehoon Yu
DØ Central Calorimeter 1990
Feb. 3, 2005 12HEP and Grid ComputingDr. Jaehoon Yu
How are computers used in HEP?
Digital Data
Data Reconstruction
pp
Feb. 3, 2005 13HEP and Grid ComputingDr. Jaehoon Yu
qT
ime
p p
q g
K
“par
ton
jet”
“par
ticle
jet”
“cal
orim
eter
jet”
hadrons
CH
FH
EM
Highest ET dijet event at DØHighest ET dijet event at DØ
0.69 GeV, 472E
0.69 GeV, 475E21
T
11T
How does an Event Look in the DØ Detector?
Feb. 3, 2005 14HEP and Grid ComputingDr. Jaehoon Yu
• Current Experiments at Tevatron – Has been taking data for the past 3 years and will continue
throughout much of the decade The immediacy!!!– Current data size close to 1PB and will be over 4 PB by
the end (~100km stack of 100GB disk drives)• 10 – 20 times (~100PB) increase at the future experiments
The Problem
Feb. 3, 2005 15HEP and Grid ComputingDr. Jaehoon Yu
~50M events/mo
Feb. 3, 2005 16HEP and Grid ComputingDr. Jaehoon Yu
• Current Experiments at Tevatron – Has been taking data for the past 3 years and will continue
throughout much of the decade The immediacy!!!– Current data size close to 1PB and will be over 4 PB by
the end (~100km stack of 100GB disk drives)• 10 – 20 times (~100PB) increase at the future experiments
– Detectors are complicated Need many people to construct and make them work
– Collaboration is large and scattered all over the world
The Problem
Feb. 3, 2005 17HEP and Grid ComputingDr. Jaehoon Yu
~700 Collaborators~80 Institutions18 Countries
Typical HEP Collaboration at Present
Feb. 3, 2005 18HEP and Grid ComputingDr. Jaehoon Yu
First Beams: Summer 2007Physics Runs: from Fall 2007
TOTEM
LHCb: B-physics
ALICE : HI
pp s =14 TeV L=1034 cm-2 s-1
27 km Tunnel in Switzerland & France
Large Hadron Collider (LHC) CERN, Geneva: 2007 Start
Large Hadron Collider (LHC) CERN, Geneva: 2007 Start
CMS
Atlas
5000+ Physicists 250+ Institutes 60+ Countries
H. Newman
Feb. 3, 2005 19HEP and Grid ComputingDr. Jaehoon Yu
• Current Experiments at Tevatron – Has been taking data for the past 3 years and will continue throughout much of the
decade The immediacy!!!– Current data size close to 1PB and will be over 4 PB by the end (~100km stack of
100GB disk drives)• 10 – 20 times (~100PB) increase at the future experiments
– Detectors are complicated Need many people to construct and make them work
– Collaboration is large and scattered all over the world– Development and improvements at remote institutions– Optimized resource management, job scheduling, and monitoring tools– Efficient and transparent data delivery and sharing
• Use the opportunity of having large data set in furthering grid computing technology– Improve computational capability for education – Improve quality of life
The Problem
Feb. 3, 2005 20HEP and Grid ComputingDr. Jaehoon Yu
What is a Computing Grid?• Grid: Geographically distributed computing resources
configured for coordinated use• Physical resources & networks provide raw capability• “Middleware” software ties it together
Feb. 3, 2005 21HEP and Grid ComputingDr. Jaehoon Yu
How do HEP physicists communicate?
Feb. 3, 2005 22HEP and Grid ComputingDr. Jaehoon Yu
Old Deployment ModelsStarted with Fermilab-centric SAM infrastructure in place, …
…transition to hierarchically distributed Model
Feb. 3, 2005 23HEP and Grid ComputingDr. Jaehoon Yu
Desktop Analysis Stations
Institutional Analysis Centers
Regional Analysis Centers
Normal InteractionCommunication PathOccasional Interaction Communication Path
Central Analysis Center (CAC)
DAS DAS…. DAS DAS….
IAC ... IAC IAC…IAC
RAC …. RAC
DØ Remote Analysis Model (DØRAM)Fermilab
Feb. 3, 2005 24HEP and Grid ComputingDr. Jaehoon Yu
UTA has the first and the only US RAC
Mexico/Brazil
OU/LU
UAZ
RiceLTU
UTA
KUKSU
Ole Miss
DØRAM Implementation
MainzWuppertal
Munich
AachenBonn
GridKa
(Karlsruhe)DØ Southern Analysis Region (DØSAR) formed around UTA
Feb. 3, 2005 25HEP and Grid ComputingDr. Jaehoon Yu
What can accomplished in an analysis region?• Construct end-to-end service environment in a smaller, manageable
scale• Train and accumulate local expertise and share them• Form a smaller group to work coherently and closely• Draw additional resources from variety of funding sources
– Promote interdisciplinary collaboration• Increase intellectual resources for the experiment
– Enable remote participants to be more actively contribute to the collaboration• Form a grid and use it for DØ
– Simulated data (Monte Carlo) production– Actual data reconstruction– Actual and simulated data analyses
• Promote and improve IAC’s group stature
Feb. 3, 2005 26HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Consortium First Generation IAC’s
University of Texas at Arlington Louisiana Tech University Langston University University of Oklahoma Tata Institute (India)
• Second Generation IAC’s– Cinvestav, Mexico– Universidade Estadual Paulista, Brazil – University of Kansas– Kansas State University
• Third Generation IAC’s– Ole Miss, MS– Rice University, TX– University of Arizona, Tucson, AZ– USTC China– Korea University, Korea
Each 1st generation institution is paired with a 2nd generation institution to help expedite implementation of D0SAR capabilities
Both 1st and 2nd generation institutions can then help the 3rd generation institutions implement D0SAR capabilities
Feb. 3, 2005 27HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Accomplishments• The only established US analysis region within DØ• Constructed and activated a Regional Analysis Center• Formed and activated five new MC production farms• Data access capability implemented in 70% of the sites
• Employed and developed and implemented many useful monitoring tools– Ganglia and MonaLISA– McFarmGraph, McPerM, McQue, and McFarmDB
Feb. 3, 2005 28HEP and Grid ComputingDr. Jaehoon Yu
UTA, The New Way•100 P4 Xeon 2.6GHz CPU = 260 GHz•64TB of Disk space
•84 P4 Xeon 2.4GHz CPU = 202 GHz•7.5TB of Disk space
•Total CPU: 462 GHz•Total disk: 73TB•Total Memory: 168Gbyte•Network bandwidth: 68Gb/sec
Feb. 3, 2005 29HEP and Grid ComputingDr. Jaehoon Yu
Various Monitoring ApplicationsGanglia: Operating since Apr. 2003 McGraph: Operating since Sept. 2003
McPerM: Operating since Sept. 2003McQue: Operating since June 2004
HEP and Grid ComputingDr. Jaehoon Yu
ot
MonaLISA Grid Resource Monitoring
Feb. 3, 2005 31HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Accomplishments• The only established US analysis region within DØ• Formed and activated five new MC production farms• SAM stations are installed in eight sites (three more to go..)• Constructed and activated a Regional Analysis Center• Employed and developed and implemented many useful monitoring tools• Started contributing beyond MC production
• Accumulated large expertise in many areas• Successfully brought in additional computing and
human resources
Feb. 3, 2005 32HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Computing & Human ResourcesInstitutions CPU(GHz) [future] Storage (TB) People
Cinvestav 13 1.1 1F + …
Langston 22 1.3 1F+1GA
LTU 25+[12] 3.0 1F+1PD+2GA
KU 12 2.5 1F+1PD
KSU 40 3.5 1F+2GA
OU 19+[270] 1.8 + 120(tape) 4F+3PD+2GA
Sao Paulo 115+[300] 4.5 2F + Many
Tata Institute 78 1.6 1F + 1Sys
Ole Miss 300 1.5 1F + 1Sys
UTA 520 74 2.5F+ 1sys + 1.1PD + 3GA
Total 1144 + [582] 95 + 120 (tape) 15.5F+3sys+6.1PD+10GA
Feb. 3, 2005 33HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Accomplishments• The only established US analysis region within DØ• Formed and activated five new MC production farms• SAM stations are installed in eight sites (three more to go..)• Constructed and activated a Regional Analysis Center• Employed and developed and implemented many useful monitoring tools• Started contributing beyond MC production• Accumulated large expertise• Successfully brought in additional computing and human resources
• Formed the DØSARGrid and producing simulated events on it
Feb. 3, 2005 34HEP and Grid ComputingDr. Jaehoon Yu
GUI to Access Grid
Feb. 3, 2005 35HEP and Grid ComputingDr. Jaehoon Yu
How does the DØSARGrid work?
Client Site
DØ Grid
Sub. Sites
Reg. Grids
Exe. Sites
Desktop. Clst.
Desktop. Clst.
Ded. Clst.
Ded. Clst.
SAM
JDL
Feb. 3, 2005 36HEP and Grid ComputingDr. Jaehoon Yu
DØSAR Accomplishments• The only established US analysis region within DØ• Formed and activated five new MC production farms• SAM stations are installed in eight sites (three more to go..)• Constructed and activated a Regional Analysis Center• Employed and developed and implemented many useful monitoring
tools• Started contributing beyond MC production• Accumulated large expertise• Successfully brought in additional computing and human resources• Developed McFarm interface to SAMGrid• Formed the DØSARGrid and start producing MC events on the grid
• Promote inter-disciplinary collaborations
Feb. 3, 2005 37HEP and Grid ComputingDr. Jaehoon Yu
What next?• We will participate in large scale DØ data processing in a few
months– 5 TB of data has been pre-staged in preparation
• Must perform data analysis in the region using the regional resources– Large number of data sets has been transferred to UTA– HEP Students are working on their theses analyses using these data
sets– Physics Undergraduate students are working on their class projects
using this data• Preparing the transition into future experiment and exploit it in
DØ• Improve local infrastructure, such as network bandwidths
Feb. 3, 2005 38HEP and Grid ComputingDr. Jaehoon Yu
Network Bandwidth Usage at UTA
DPCC online
DØ and ATLAS Production
Feb. 3, 2005 39HEP and Grid ComputingDr. Jaehoon Yu
What next, cnt’d?• Transform into a legitimate, active Virtual Organization within
the global grid picture Should be gradually done within the next year– Participate in existing US (Open Science Grid) and European
(Enabling Grid for E-science in Europe) – Fully utilize the involvement with future experiments– Turn DØSAR into DOSAR (Data Oriented Super Analysis Region)
• Continue promoting interdisciplinary collaboration• Actively participate and lead grid computing efforts in the
respective states• Employ the grid computing technology not just for research but
also for education
Feb. 3, 2005 40HEP and Grid ComputingDr. Jaehoon Yu
NLR – National LambdaRail
Denver
Seattle
Sunnyvale
LA
San Diego
Chicago Pitts
Wash DC
Raleigh
Jacksonville
Atlanta
KC
Baton Rouge
El Paso - Las Cruces
Phoenix
Pensacola
Dallas
San Ant.Houston
Albuq. Tulsa
New YorkClev
10GB/sec connections
Feb. 3, 2005 41HEP and Grid ComputingDr. Jaehoon Yu
Grid in other disciplines?• Nuclear physics• Bioinformatics • Genetics• Meteorology• Medical science and medicine• Homeland security
Feb. 3, 2005 42HEP and Grid ComputingDr. Jaehoon Yu
Conclusions• To understand the fundamentals of nature, High Energy Physics
– Uses accelerators to look into extremely small distances– Uses large detectors to explore nature– Uses large number of computers to process data– Large amount of data gets accumulated need computing grid to
perform expeditious data analysis• Computing grid needed for other disciplines with large data sets• HEP is an exciting endeavor in understanding nature• Physics analyses at one’s own desktop using computing grid is
close to be a reality• UTA plays a leading role in HEP research and shaping the
future of computing Grid• Computing grid will soon revolutionize everyday lives soon…