ieee nss dresden 22 oct. 2008 paolo valente for the computing & software group for the computing...
DESCRIPTION
IEEE NSS Dresden 22 Oct Paolo Valente for the computing & software group for the computing & software group NA48 (K L & K S ) + NA48/1 (K S only) + NA48/2 (K ± ): 1997,1998, , , ,1998, , , 2004 Up to 135 days/year of runningUp to 135 days/year of running Up to 5×10 10 K decays/yearUp to 5×10 10 K decays/year 10 kHz trigger rate10 kHz trigger rate 65k events/burst, 600 MB/burst65k events/burst, 600 MB/burst From few tens up to 200 TB/year of raw dataFrom few tens up to 200 TB/year of raw data NA48/n Quasi-online reconstructionQuasi-online reconstruction Fortran-based reconstruction/simulationFortran-based reconstruction/simulation Script-based framework forScript-based framework for Data-recording (CDR, pioneered by Bernd Painzer-Steindl)Data-recording (CDR, pioneered by Bernd Painzer-Steindl) Re-processing, data preparationRe-processing, data preparation User analysisUser analysis Main result: Direct CP violation established and precisely measuredTRANSCRIPT
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 Software & NA62 Software & ComputingComputing
Outlook:Outlook:
•From From NA48NA48 data model to data model to NA62NA62•Computing model for the 2007/2008 runComputing model for the 2007/2008 run•Computing model for the KComputing model for the K experimentexperiment•Software architectureSoftware architecture
www.buconero.euwww.buconero.eu Supp
ort I
talia
n re
sear
chSu
ppor
t Ita
lian
rese
arch
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA48 & NA62NA48 & NA62The kaon beam at the SPS North The kaon beam at the SPS North Area:Area:in the heart of the LHC!in the heart of the LHC!
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA48NA48 (K (KLL & K & KSS)) + + NA48/1NA48/1 (K (KSS only) only) + + NA48/2NA48/2 (K (K±±))::•1997,1998,19991997,1998,1999 + + 2000, 20012000, 2001 + + 2003, 20042003, 2004•Up to 135 days/year of runningUp to 135 days/year of running•Up to 5Up to 5××10101010 K decays/year K decays/year•10 kHz trigger rate10 kHz trigger rate•65k events/burst, 600 MB/burst65k events/burst, 600 MB/burst•From From few tensfew tens up to up to 200 TB/year200 TB/year of raw data of raw data
NA48/nNA48/n
•Quasi-online reconstructionQuasi-online reconstruction•Fortran-based reconstruction/simulationFortran-based reconstruction/simulation•Script-based framework for Script-based framework for
•Data-recording (CDR, pioneered by Bernd Painzer-Steindl)Data-recording (CDR, pioneered by Bernd Painzer-Steindl)•Re-processing, data preparationRe-processing, data preparation•User analysisUser analysis
Main result:Direct CP violation established and precisely measured
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 NA62 for Ke2/Kfor Ke2/K2 precise measurement2 precise measurement•Collected 120000 signal events Collected 120000 signal events (were 4000)(were 4000)•Data taking in 2007Data taking in 2007 (+ few weeks in 2008) (+ few weeks in 2008)•same DAQ/data recording of NA48/2… same DAQ/data recording of NA48/2… •……added added level-3 data filteringlevel-3 data filtering•300 TB300 TB of raw data of raw data 100 TB 100 TB to tapeto tape
From NA48 to NA62From NA48 to NA62
•Quasi-online reconstruction and Quasi-online reconstruction and event filteringevent filtering•Essentially the Essentially the oldold Fortran-based reconstruction/simulation Fortran-based reconstruction/simulation framework…framework…•… … but with “but with “maintenancemaintenance” of the CDR and reprocessing ” of the CDR and reprocessing environmentenvironment
•Castor2, new streams, event filtering and re-filtering, Castor2, new streams, event filtering and re-filtering, obsolescence of some piece of databases, software…)obsolescence of some piece of databases, software…)
A bridge towards A bridge towards the future the future experimentexperiment……
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
The future experimentThe future experiment**
•Theoretically cleanTheoretically clean test of Standard Model in the flavor sector test of Standard Model in the flavor sector•Look at KLook at K events (penguin diagram with heavy quarks largely dominating, events (penguin diagram with heavy quarks largely dominating, small hadronic corrections)…small hadronic corrections)…
•ComplementaryComplementary to the high-energy (LHC) approach to the high-energy (LHC) approach•Unfortunately, KUnfortunately, K processes are processes are very rarevery rare! (~10! (~10-11-11) )
•Need Need 12 order of magnitudes12 order of magnitudes rejection power rejection power•Need a very Need a very intenseintense (hadron) (hadron) beambeam… (it’s … (it’s therethere!)!)•Track kaons and track pionsTrack kaons and track pions to use the to use the kinematicskinematics (missing E (missing E from from ))
•with exceptional timing performances to match the tracks in the upstream with exceptional timing performances to match the tracks in the upstream (kaon) downstream (pion) spectrometer(kaon) downstream (pion) spectrometer
•Identify kaonsIdentify kaons andand identify identify pions pions (with dedicated Cerenkov (with dedicated Cerenkov detectors)detectors)•VetoVeto all particles other than all particles other than : :
•new, dedicated veto detectorsnew, dedicated veto detectors for for and and •AimAim: collect : collect ~100 events in two years~100 events in two years… …
•……corresponding to corresponding to ~~10101313 kaon decays kaon decays• New DAQ, new trigger, new computing schemeNew DAQ, new trigger, new computing scheme
**in just 2 slidesin just 2 slides
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
The future experimentThe future experiment**
Produce hadron beamProduce hadron beam
Define beam momentumDefine beam momentum
Identify kaonsIdentify kaons
Measure kaon momentumMeasure kaon momentum
Look for kaon decays in vacuum…Look for kaon decays in vacuum…
Veto photonsVeto photons
Measure pion momentumMeasure pion momentum
Identify pionsIdentify pions
Veto photonsVeto photons
Veto Veto muonsmuons
**in just 2 slidesin just 2 slides
Veto photonsVeto photons
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 detectorsNA62 detectors•Re-use the liquid Krypton calorimeter:Re-use the liquid Krypton calorimeter:
•13,000 cells, new readout electronics13,000 cells, new readout electronics, , no zero-suppression in hardwareno zero-suppression in hardware
•NewNew Silicon pixel kaon tracker Silicon pixel kaon tracker•150 ps time resolution, 150 ps time resolution, 54,000 54,000 channelschannels
•Photon vetoes: Photon vetoes: •re-use OPAL calorimeter lead-glasses, re-use OPAL calorimeter lead-glasses, 2,500 photo-multipliers2,500 photo-multipliers, , new readoutnew readout
•NewNew straw chambers spectrometer: straw chambers spectrometer: •4 stations,4 stations, 8,000 total wires 8,000 total wires , , new new readoutreadout•RICH: RICH: 2,100 photo-multipliers2,100 photo-multipliers, , new readoutnew readout
•CEDAR kaon Cerenkov detector, muon veto, hodoscope:CEDAR kaon Cerenkov detector, muon veto, hodoscope:•newnew readout systems, readout systems, ~500 channels~500 channels
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 TDAQ schemeNA62 TDAQ schemeMuon Muon vetoveto
Photon Photon vetoveto
L0 L0 processorprocessor
RICHRICH
L0L0triggertrigger
L0L0primitivesprimitives
PCPCPCPC PCPCPCPCPCPC PCPC
Liquid Liquid KryptonKrypton
Straw Straw chamberschambers
Kaon beamKaon beampixelpixel
L2 triggerL2 triggerEvent building Event building
Detector
Detector
elements
elements
AnalogAnalog
FEEFEE
ADC
ADC
TDC
TDC
FPGA
FPGA
Buffer
Buffer
L0L0triggertrigger
16.8 s/34.8 s CNGS+fixed target16.8 s/34.8 s CNGS+fixed target 0 s/22.8 s LHC+CNGS0 s/22.8 s LHC+CNGS 0 s/21.6 s LHC filling0 s/21.6 s LHC filling~ 30% of spill-time/total time, not synchronous events~ 30% of spill-time/total time, not synchronous events(no bunch crossing!) (no bunch crossing!)
Kaon Kaon CerenkovCerenkov
PCPCPCPCPCPCPCPCPCPCPCPCPCPC ReadoutReadoutL1 triggerL1 trigger
Sub-detectorSub-detector
primitivesprimitives
DATADATA
DATADATASwitchSwitch
Now under development, Now under development, based on the based on the TELL1TELL1 board (LHCb)board (LHCb)
To data To data processinprocessing & g & storagestorage
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 data volumeNA62 data volume
L1 + L2L1 + L2 in software, reduction by a factor in software, reduction by a factor ~50~50~ 3 GB/s~ 3 GB/s from Krypton from Krypton withoutwithout zero zero suppression (average event size 250 kB)suppression (average event size 250 kB)~ 100 TB/day ~ 100 TB/day O(10 PB/year) O(10 PB/year)
Only a fraction to be logged to tape:Only a fraction to be logged to tape:•NeedNeed L3 trigger L3 trigger and/or Krypton and/or Krypton zero-zero-suppressionsuppression (with halo expansion) (with halo expansion)
•Reduce by a Reduce by a factor 10-20factor 10-20 •Use pre-scaling & control samplesUse pre-scaling & control samples•Need on-line monitoringNeed on-line monitoring•Keep the system Keep the system highlyhighly flexible/adaptableflexible/adaptable
10 MHz kaon decays 10 MHz kaon decays
Maximum Maximum 1 MHz1 MHz at the at the output of L0output of L0 triggertrigger
Log to tape O(1 PB)/yearLog to tape O(1 PB)/year
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
NA62 status and plansNA62 status and plansNA62NA62
•Many new groups joined the NA48 coreMany new groups joined the NA48 core•Very advanced R&D status for all detectors:Very advanced R&D status for all detectors:
• Photon and muon vetoes ready, straw chambers ready to Photon and muon vetoes ready, straw chambers ready to start construction; start construction;
• Silicon pixel beam tracker Silicon pixel beam tracker demodemo (detector & readout) in (detector & readout) in 20092009
•2009-20112009-2011: construction of all new detectors, finalize : construction of all new detectors, finalize TDAQTDAQ, finalize , finalize computingcomputing model, run model, run Monte CarloMonte Carlo productions, develop productions, develop softwaresoftware infrastructure infrastructure•2012-20132012-2013 and in case 2014: and in case 2014: data takingdata taking, , processingprocessing•AnalysisAnalysis lasting ~2 years more lasting ~2 years more
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Data processing [now]Data processing [now]
RAWRAW
COmPACTCOmPACT(RECO)(RECO)
SuperCOmPACTSuperCOmPACT(DST)(DST)
HyperCOmPACTHyperCOmPACT
ReconstructionReconstruction
Drop variables neededDrop variables neededonly for reconstruction only for reconstruction and calibrationand calibration
Only beam geometry Only beam geometry & spectrum& spectrum
CalibrationCalibration
AnalysisAnalysis
Monte Carlo tuningMonte Carlo tuning
Dat
a si
zeD
ata
size
CPU
pow
erCP
U p
ower
1/61/6
1/21/2
CERN
CERN
Off
-site
Off
-site
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
CERN CERN computing computing
centercenter
Experiment Experiment computing computing resources resources
Off-site Off-site computing computing resources resources
Data recordingData recordingProcessing (pass-1) / L3 Processing (pass-1) / L3
filteringfilteringnetworknetwork
Data processing [now]Data processing [now]
CalibrationCalibration
networknetwork
Event buildingEvent buildingL1-L2 triggerL1-L2 trigger
RAWRAW
““burstlets”burstlets”
COmPACTCOmPACTMet
a-da
taM
eta-
data
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
CERN CERN computing computing
centercenter
Experiment Experiment computing computing resources resources
Off-site Off-site computing computing resources resources
User analysisUser analysisSimulationSimulation
networknetwork
Data processing [now]Data processing [now]
Re-processing (pass-2+)Re-processing (pass-2+)networknetwork
RAWRAW
COmPACTCOmPACTMet
a-da
taM
eta-
data
Super-COmPACTSuper-COmPACT
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
CERN CERN computing computing
centercenter
Experiment Experiment computing computing resources resources
Off-site Off-site computing computing resources resources
User analysisUser analysisSimulationSimulation
networknetwork
Data processing [now]Data processing [now]
Re-processing (pass-2+)Re-processing (pass-2+)networknetwork
Super-COmPACTSuper-COmPACT
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
networknetwork
networknetwork
CERN CERN computing computing
centercenter
Experiment Experiment computing computing resources resources
Off-site Off-site computing computing resources resources
Event buildingEvent buildingL1-L2 triggerL1-L2 trigger
Processing on-line Processing on-line (pass 1)(pass 1)
L3 filteringL3 filtering
Data processingData processing
Move processing to the Move processing to the experiment farm?experiment farm?- Need enough computing power Need enough computing power in order to avoid back-logs to in order to avoid back-logs to DAQDAQ
Data recordingData recording
Re-processing (pass-2+)Re-processing (pass-2+)CalibrationCalibration
User analysisUser analysisSimulationSimulation
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
networknetwork
networknetwork
CERN CERN computing computing
centercenter
Experiment Experiment computing computing resources resources
Off-site Off-site computing computing resources resources CalibrationCalibration
Re-processing (pass 2+)Re-processing (pass 2+)
Event buildingEvent buildingL1-L2 triggerL1-L2 trigger
Processing on-line Processing on-line (pass 1)(pass 1)
L3 filteringL3 filtering
Data processingData processing
Move re-processing off-Move re-processing off-site?site?- Need access to (filtered) RAW - Need access to (filtered) RAW datadata
Data recordingData recording
User analysisUser analysisSimulationSimulation
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Software infrastructureSoftware infrastructureTr
igge
r &
Filt
erTr
igge
r &
Filt
er
Reco
nstr
uctio
nRe
cons
truc
tion
Anal
ysis
Anal
ysis
Sim
ulat
ion
Sim
ulat
ion
LibrariesLibraries
FrameworksFrameworks
No big deal in demonstrating the need of a No big deal in demonstrating the need of a new infrastructure:new infrastructure:•Cannot rely on old technologies Cannot rely on old technologies
•At least for support problems: FORTRAN At least for support problems: FORTRAN code, geant3, cernlib, zebra, hepdb, …code, geant3, cernlib, zebra, hepdb, …
•Existing “framework” based on [different] Existing “framework” based on [different] scripting languages: scripting languages:
•Hard to maintainHard to maintain•Certainly, not GRID-compliantCertainly, not GRID-compliant•AlreadyAlready need to use GRID for Monte Carlo… need to use GRID for Monte Carlo…
•Need to handle 1 PB/yearNeed to handle 1 PB/year•~1 order of magnitude wrt past experiment ~1 order of magnitude wrt past experiment (and cannot scale man-power accordingly…)(and cannot scale man-power accordingly…)
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Software frameworkSoftware framework
….WHY DON’T YOU USE ALIROOT / GAUDI / … ?
DO I REALLY NEED THAT FOR NA62 ???
Use cases…Use cases…
Data modelData model
Software Software architecturearchitecture
Software frameworkSoftware framework
User softwareUser software
TheoryTheory ExperimentExperiment
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
What the framework should do for us…
• Use casesUse cases• Define the algorithms to run in the codeDefine the algorithms to run in the code• Handle input/output file catalogs Handle input/output file catalogs • Handle meta-data (geometry, conditions, calibrations)Handle meta-data (geometry, conditions, calibrations)
• handle different databaseshandle different databases• Handle different running configurations (online, reprocessing, Monte Handle different running configurations (online, reprocessing, Monte Carlo, test, …) of software:Carlo, test, …) of software:
• different configuration for input, output, algorithms to run, load/save meta-different configuration for input, output, algorithms to run, load/save meta-data, connection to databases…data, connection to databases…
• manage versioning of software, multi-user development of codemanage versioning of software, multi-user development of code• manage different versions of datamanage different versions of data• manage changes in OS/system callsmanage changes in OS/system calls• jobs submissionjobs submission• error recoveryerror recovery
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
AlgorithmAlgorithm
TransientTransientdatadata PersistentPersistent
datadata
ServicesServices
Exploring GAUDI:Exploring GAUDI:•Satisfies NA62 requirementsSatisfies NA62 requirements•Exists (and works!)Exists (and works!)•Good support (and lasting at least as long as LHC)Good support (and lasting at least as long as LHC)•Experience in some NA62 InstitutesExperience in some NA62 Institutes•Natural integration with GRID technology through GANGANatural integration with GRID technology through GANGA
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Monte CarloMonte Carlo•Several kinds of Monte Carlo already existing:Several kinds of Monte Carlo already existing:
•It is not It is not conceivableconceivable to generate 10 to generate 101313 kaon decays with full kaon decays with full tracking of particles inside a detailed GEANT description of the tracking of particles inside a detailed GEANT description of the experiment…experiment…•GEANT4 detailed simulations of detectors responseGEANT4 detailed simulations of detectors response
•““Fast” simulation using:Fast” simulation using:•Physics generatorsPhysics generators•Accurate beam and halo parametrizationsAccurate beam and halo parametrizations•Simulation of setup as passive materialsSimulation of setup as passive materials•Parametrization of detectors response from GEANT4Parametrization of detectors response from GEANT4
•Full reconstruction in C++Full reconstruction in C++•Root persistencyRoot persistency
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
•NA62 is awaiting its NA62 is awaiting its finalfinal approval approval•Even though:Even though:
•Some detectors are still finalizing FEE and readout schemeSome detectors are still finalizing FEE and readout scheme•Data volume is largely dependent from Liquid Krypton readoutData volume is largely dependent from Liquid Krypton readout•TDAQ quickly evolving towards its final formTDAQ quickly evolving towards its final form
some some key pointskey points are firm: are firm:•Need a “smart” way of reducing data (Liquid Krypton zero-Need a “smart” way of reducing data (Liquid Krypton zero-suppression and/or L3 filtering)suppression and/or L3 filtering)•~ 1 PB/year of raw data to:~ 1 PB/year of raw data to:
•log to tape, reconstruct, calibrate, re-process, convert to log to tape, reconstruct, calibrate, re-process, convert to data-summary formatdata-summary format
•Need large Monte Carlo productionsNeed large Monte Carlo productions•Need to do all that from now to ~2015: new [or not-so-old] Need to do all that from now to ~2015: new [or not-so-old] technologies!technologies!
Final remarks
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Beyond NA62Beyond NA62
NA??NA??•2015-20202015-2020•Increase beam intensity for ~1000 KIncrease beam intensity for ~1000 K events events•Improve detector for KImprove detector for KLL
IEEE NSSDresden22 Oct. 2008
Paolo ValentePaolo Valente for the computing & software groupfor the computing & software group
Step 2Step 2: ~1000 K: ~1000 K++ eventsevents
Step 3Step 3: ~100 K: ~100 KLL eventsevents
Step 1Step 1: ~100 K: ~100 K++ events, events, NA62NA62