the architecture of the zeus second level global track trigger gtt
DESCRIPTION
The Architecture of the ZEUS Second Level Global Track Trigger GTT. Satish Dhawan Yale. Outline. Interaction. MVD. CTD. The ZEUS Experiment and Trigger Why a GTT ? Interfacing to ZEUS detector components The Global Track Trigger Performance and first experience with real data - PowerPoint PPT PresentationTRANSCRIPT
IEEE RT 2003, Montreal, 18-23 May 2003 1 S. Dhawan
ZEUS MVD and GTT Group: ANL, Bonn Univ., DESY-Hamburg -Zeuthen, Hamburg Univ., KEK-Japan, NIKHEF, Oxford Univ., Bologna, Firenze, Padova, Torino Univ. and INFN, UCL, Yale, York.
The Architecture of the ZEUS The Architecture of the ZEUS Second Level Global Track Trigger Second Level Global Track Trigger
GTTGTT
Satish Dhawan Yale
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan2
OutlineOutline
The ZEUS Experiment and TriggerThe ZEUS Experiment and Trigger Why a GTT ?Why a GTT ? Interfacing to ZEUS detector componentsInterfacing to ZEUS detector components The Global Track TriggerThe Global Track Trigger Performance and first experience with real dataPerformance and first experience with real data Summary and OutlookSummary and Outlook
MVDMVD
CTDCTD
STTSTT
InteractionInteraction
TriggerTrigger
epep
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan3
The ZEUS DetectorThe ZEUS Detector HERA
– e± proton collider
ZEUS
– Multi-purpose ep experiment with tracking and calorimetry
CTD
– Central Tracking Detector
MVD
– Si. Micro Vertex Detector
STT
– STT Straw Tube Tracker
e±
27.5 GeV
p
920 GeV
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan4
The ZEUS TriggerThe ZEUS Trigger
bunch crossing interval: 96 ns
ZEUS: 3-Level Trigger System
Level GFLT GSLT TLTRate 500Hz 50Hz 5 HzLatency 0.7μs 10ms none
Event BuilderEvent Builder
Third Level TriggerThird Level Trigger
cpucpucpucpu cpucpu cpucpu cpucpu cpucpu
CALCAL CTDCTD
Offline TapeOffline Tape
Global Second Global Second Level TriggerLevel Trigger
GSLT Accept/RejectGSLT Accept/Reject
Global First Global First Level TriggerLevel Trigger
GSLT Accept/RejectGSLT Accept/Reject
CTDCTDFront EndFront End
CALCALFront EndFront End
Other Other ComponentsComponents
Other Other ComponentsComponents
CTDCTDSLTSLT
CALCALSLTSLT
CALCALFLTFLT
CTDCTDFLTFLT
~10 ms
5Hz5Hz
40Hz40Hz
500Hz500Hz
101077 Hz Hz
Eve
nt
Bu
ffer
s
Eve
nt
Bu
ffer
s
55 s
pip
elin
es
pip
elin
e
55 s
pip
elin
es
pip
elin
e
~0.7 s
ZEUS trigger design implemented by 1992
– First high rate (96 ns) pipelined system
– With a flexible 3 level trigger
– Main building blocks were transputers (20Mbit/s)
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan5
Why a GTT ?Why a GTT ? Conceptual Development path
– MVD participation in GFLT not feasible, readout latency too large.
– Participation at GSLT possible:
Pushing ADC data over FastEthernet gave acceptable rates/latencies performance.
But track and vertex information poor due to low number of planes.
– Expand scope to interface data from other tracking detectors:
Initially Central Tracking Detector (CTD) - overlap with barrel detectors
Later Straw Tube Tracker (STT) - overlap with wheels detectors.
– Implement GTT as a PC farm with TCP data and control path
Trigger Aims
– Higher quality track reconstruction and rate reduction at GSLT
– Primary Z vertex resolution 9 cm (CTD only) 400 m (+MVD)
– Decision required within existing SLT (<15 ms)
– Eventually sensitive to heavy quark secondary verticesDijet MC event
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan6
Interfaces to ZEUS detector components Interfaces to ZEUS detector components
The ZEUS Experiment was based on Transputers
CTD duplicate data sent to existing CTD-SLT
STT new component reused forward detector electronics
use CTD soln.
MVD new component interface use MVD to provide at GTT
access MVD data
GSLT handling
CTD or STT
LOCALFLT
GSLT
DIGI-TIZEDDATA
BUFFER
EVB
GTT
MVD
DATAPIPELINE
DATAPIPELINE
CLUSTERFIFO
STRIPFIFO
GFLT
TPSPLITER
LOCALSLT
INTER-FACE
INTER-FACE
RESULT ANDDATA BUFFERS
OTHER COMPONENTS
INTER-FACE
GFLT ACCEPTGFLT ACCEPT
GSLT DECISION GSLT DECISION
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan7
Interfaces to MVD component Interfaces to MVD component
MVD readout system is VME based
– Data gathering and readout control using LynxOS 3.01 Real Time OS on network booted Motorola MVME2400/MV2700 PPC VME Computers
– Send CLUSTER event data with Fast Ethernet TCP to GTT
– Use custom VME “all purpose Latency clock + interrupt board” Full DAQ wide latency measurement system
* Nikhef NIM A332, 263 (1993)
Design mean event size:
MVD cluster 5kB
Timing resolution 16μs
GSLT 2TP modules
Lynx OS
CPU
Lynx OS
CPU
NIM + Latency
NIM + Latenc
y
Slow control + Latency Clock modules
CPU Boot Server and Control
ADCM modules
Lynx OS
CPU
AnalogLinksNIM + Latency
Clock +Control
GSLT VME interface MVD VME Readout Crates
Latency Clock
F/E Network
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan8
Interfaces to CTD+STT components Interfaces to CTD+STT components
Interface is VME based
– Component trigger event data received on TP links by NIKHEF 2TP board* and copied to TPM
– Event copied via VME to LynxOS
– Send data with Fast Ethernet TCP to GTT
* Nikhef NIM A332, 263 (1993)
Design mean event sizes:
CTD 5 kBSTT 5 kB
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan9
GTT hardwareGTT hardwareImplementation
– MVD readout
3 Motorola MVME2400 450MHz
– CTD/STT interfaces
NIKHEF-2TP VME-Transputer
Motorola MVME2400 450MHz
– PC farm
12 DELL PowerEdge 4400 Dual 1GHz
– GTT/GSLT result interface
Motorola MVME2700 367MHz
– GSLT/EVB trigger result interface
DELL PowerEdge 4400 Dual 1GHz
DELL Poweredge 6450 Quad 700 MHz
– Network switches
3 Intel Express 480T Fast/Giga 16 ports.
– Thanks to Intel Corp. who provided switch and PowerEdge hardware via Yale grant .
CTD/STT interface MVD readout
PC farm and switches
GTT/GSLT interface EVB/GSLT result interface
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan10
Sizing the GTTSizing the GTT Naïve estimate of GTT node multiplicity
– Ignore network transit times
– Assume higher rate than expected
– GTT latency at GSLT must not be worse than existing CTD component
– Control credit based identification of next free GTT node (not Round-Robin)
Simulate mean and max waiting time for node
~ 10 nodes needed
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan11
GTT node configuration and GTT node configuration and controlcontrol
Each node contains:
Multi-threaded algorithm process: 1 thread per input source
1 thread per algorithm (Barrel = CTD+MVD, Forward = STT+MVD)
1 timeout thread sending PASS result after 40ms
Plot server pushing shared memory histograms
Statistics server pushing shared memory stats
Simulation + Monte Carlo or Dumped data → development
Run and process control is provided by the MVD:
State transition diagram
Active nodes configured automatically on SETUP
Interprocess messages contain: Short (64 byte) fixed length XDR header (GFLT#, etc)
and, optionally, an opaque data block
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan12
GTT
3
41 2
1
23
4
5
67
8
9 10
11
8
12
TOGSLT
1
TOEVB
12
MVD
1 2 3 4
FROMGSLT
2 3 1
STT
12
CTD
1 2
1. credit allocation
2. credit-socket resolution
3. credit list
4. free credit
5. data to algorithm
6. algorithm result
7. algorithm finished free credit
8. GSLT trigger result
9. algorithm result banks and MVD cluster data
10. MVD strip data
11. latency measurements
12. histogram and pedestal
SETUP transition:
ACTIVE state:
GTT network connectionsand message transfer
definitions
GFLT accept trigger decision
GSLT decision GTT decision Event Builder
bind+accept
connect
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan13
GTT Algorithm DescriptionGTT Algorithm Description Modular Algorithm Design
– Two concurrent algorithms
(Barrel/Forward) foreseen
– Process one event per host
– multithreaded event processing: data unpacking
concurrent algorithms
time-out
– Test and Simulation results: 10 computing hosts required
“Control Credit” distribution preferred on Round-Robin
At present barrel algorithm implemented Forward algorithm in development phase
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan14
Find tracks in the CTD, extrapolate into the MVD to resolve pattern recognition ambiguity
– Find segments in Axial and Stereo layers of CTD
– Match Axial Segments to get r- tracks
– Match MVD r- hits
– Refit r- track including MVD r- hits
After finding 2-D tracks in r-, look for 3-D tracks in z-axial track length,s:
– Match stereo segments to track in r- to get position for z-s fit
– Extrapolation to inner CTD layers
– If available use coarse MVD wafer position to guide extrapolation
– Match MVD z hits
– Refit z-s track including z hits
Constrained or unconstrained fit– Pattern recognition better with constrained
tracks
– Secondary vertices require unconstrained tracks
Unconstrained track refit after MVD hits have been matched
Barrel algorithm descriptionBarrel algorithm description
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan15
First GTT latency resultsFirst GTT latency results
2002 running HERA after lumi upgrade compromized by high background rates
– Mean datasizes from CTD and MVD were larger than the design
Sept 2002 runs used to tune datasize cuts
– Allowed GTT to run with acceptable mean latency and tails at the GSLT
– Design rate of 500 Hz appears possible
CTD VME readout latency with respect to MVD
MVD VME SLT readout latencyms msms ms
ms
GTT latency after complete trigger processing
Mean GTT latency vs GFLT rate per run
MVD-GTT Latency as measured by GSLT
Low data occupancy rate tests
HERA
Montecarlo
Hz
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan16
Resolution including MVD from MC ~400 μm
First tracking resultsFirst tracking results
Dijet Montecarlo Vertex Distribution
mm
Run 42314 Event 938
Run 44569 Vertex Distribution
Collimator C5backscattering
Nominal Vertex
GTT eventdisplay
Physics datavertex distribution
ep candidate
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan17
Resolution including MVD from MC ~400 μm
First tracking resultsFirst tracking results
Dijet Montecarlo Vertex Distribution
mm
Yet another backgroundevent
Run 44569 Vertex Distribution
Collimator C5backscattering
Nominal Vertex
GTT eventdisplay
Physics datavertex distribution
IEEE RT 2003, Montreal, 18-23 May 2003 S. Dhawan18
Summary and OutlookSummary and Outlook
The MVD and GTT system have been successfully integrated into the ZEUS experiment 267 runs with 3.1Mio events recorded between 31/10/02 and 18/02/03 with MVD on
and DQM (~ 700 nb-1) The MVD DAQ and GTT performance (latency, stability and efficiency) are satisfactory
Next steps: Utilize results of the barrel algorithm at the GSLT Finalize development and integration of the forward algorithm
So far very encouraging results. Looking forward to routine high
luminosity data taking. The shutdown ends in June 2003...
Why does the GTT work…
Use 2002 CPU+Network technology → performance increase
No proportional increase in data size