l2cal pulsar upgrade proposal in 24 minutes an executive summary ted liu for the l2cal upgrade team...
TRANSCRIPT
L2CAL Pulsar Upgrade Proposal in 24
minutes An Executive Summary
Ted Liu For the L2CAL upgrade team
Pisa/Purdue/Rockefeller/UC/Upenn
/Madrid/Padova/Academica Sinica/
FNAL
Trigger Upgrade Review, July 27 2006
•This is the talk shown at the review on July 27th, 2006.• With some new slides on schedule and people power at the end (backup slides). Updated Aug.3rd, 06.
A brief history of time …
It has been a long way towards this proposal … just like the Pulsar project history Early 2001, 1st time Ted met Pacman while working with Monica on L2CAL
commissioning… impressed with the simplicity of pacman’s algorithm, worried about jet merging at higher luminosity …
Year 2002, Pulsar was designed for general purposes (some CDF people hated it, yet many liked it):CDF/BABAR/CMS/ATLAS/TOTEM/ALICE/…compatible with L2CAL too
Sept. 23rd, 2004: Henry’s email to Ted, asking if we can improve L2 clustering for taus and jets… I was really busy with L2 Pulsar upgrade back then…. Only answered Henry’s email on March 23, 2005. The answer was: YES, but needs physics motivation….
Nov. 22nd, 2004 and Feb. 22, 2005: Viktor Veszpremi showed his study on L1MET and L2MET at exotics meeting, with higher resolution….
April 2005, Ted asked a few others to look for jet merging… lots of hall-way talking… Early 2005: Ming&Mario mentioned the L2CAL upgrade idea at TDWG, a few people were
very much pissed at them…:( June 18th, 2005: Mary showed a few event display with jet merging in ROF. Vadim and Ted confirmed Mary’s finding, and showed the “phase transition”… However, it was really Paola Giannetti’s full support that convinced Ted to go ahead and
push this proposal forward … Paola is willing to put her full team behind this upgrade! Anwar/Mary/Gene developed L2Cone algorithm, Gene did extensive studies on it, and
Laura significantly optimized the L2Cone algorithm speed in the spare L2 decision CPU…
CDF Trigger Performancein 24 seconds…
L1A bandwidth: ~ 30KHz, no longer a bottleneck for RunIIb L2 Decision crate and SVT upgrade XFT upgrade will also improve the L1 purity
L2A bandwidth: ~1 KHz, not enough for RunIIb Tons of work went into EVB/L3 upgrade, DAQ improvements … Significant improvements from ~ 300 Hz to 1 KHz Still not enough for RunIIb, L2A already at ~800Hz at ~180E30 The problem: L2A cross section grows rapidly with luminosity Why? Mostly JUNK L2 does not have enough information to control the background
Most recent run (July 23, 2006)
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Slope: > ~100Hz/10E30
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Soft Readoutdeadtime wall
HardBusydeadtimewall
The two great Wallscompeting for attention…
L2 triggers with high growth term:two types of L2 triggers gone wild…
. Track/mu related:
e.g. CMX etc
L2CAL related:
e.g. MET+2JETs
Why? Probably Bad names…
JET: JUNK Enhanced TriggerMET: MESS Enhanced Trigger
CMX: Could Match beamX
Examples of L2 triggers gone wild
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
CJET10_JET10_L1_MET25(MET+2JET HIGGs trigger)
Other examples of JET/MET related triggers will go wild in rate at higher luminosity: MET35, Higgs high-Pt b-jet, Higgs multi-jets …JET40/60/90…
also go wild in inefficiency:Triggers require multi-jets at L2, e.g.
Top_multi_jets (L2_FOUR_JET15_SUMET175)could go TOPLESS at higher luminosity due to jets merging…
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
L2_CMX6_PT15:
XFT upgrade improves this
Bottom line:
Higgs/Top/QCD physics is at stake. Once bandwidth saturated, ALL physics triggers at stake…
600 nb @ 170E30: ~100 Hz
CDF Trigger Performancein 24 seconds…
L1A bandwidth: ~ 30KHz, no longer a bottleneck for RunIIb L2 Decision crate and SVT upgrade XFT upgrade will also improve the L1 purity
L2A bandwidth: ~1 KHz, not enough for RunIIb Tons of work went into EVB/L3 upgrade, DAQ improvements … Significant improvements from ~ 300 Hz to 1 KHz Still not enough for RunIIb, L2A already at ~800Hz at ~180E30 The problem: L2A cross section grows rapidly with luminosity Why? Mostly JUNK L2 does not have enough information to control the background
The most effective way to solve the L2A bandwidth problem is to:
improve the trigger purity at L2 …
How to improve L2 purity?send more information into L2 CPU
We are already doing it ! Send XFT stereo information from L1
Segment Finders to L2 CPU via Pulsars, to improve CMX etc trigger purity …
The information is made available by designing the new stereo Finder boards …
Today, we are proposing the same thing for JET/MET related triggers (L2CAL path) Send all 10-bit trigger tower information from
L1 DIRACs to L2 CPU via Pulsars, to improve JET/MET trigger purity… The information has already been there in the
system, and we have not made use of them! This is a natural thing to do… Will take some real work for sure …
Wait a second! Do we really have to do this? (I)
Are you sure the machine luminosity will ever go beyond 200E30?
We have every reason to believe that it will (past year experience!) True, every single machine I know is a moving target We are not 100% for sure it will ever reach 300E30 We cannot even say for sure it will not go beyond 300E30 (CESR&PEP-II&KEKB all went way beyond design luminosity at the
end)
It does not matter:
One may hope for the worst (to minimize the work), we will have toprepare for the best. Otherwise, why waste our life in the CDF trailer?
Wait a second! Do we really have to do this?
(II)
OK, let’s assume the luminosity will reach 300E30, but there must be other easy&simple&cheap&lazy way to deal with this! Sure. The simplest way is to DPS all high rate L2 triggers, and call it the final
RunIIb trigger table It will only take a few hours work for TDWG. We have been practicing this
technique a lot in the past few years, mostly to charm and beauty triggers We have been pretty good at this already. Quite a few Higgs high-Pt or multi-
jets triggers are already DPSed in the default trigger table This would minimize the work load for ALL of us …:)
Is this really what you want?
The simplest idea of all: beg the machine folks to keep the luminosity low... It may reduce their work load as well…Want more discussions on luminosity leveling?
Wait a second! Do we really have to do this?
(III)
OK, But what if one could improve the L2A bandwidth to, say, ~5 KHz? Do we still need to do the L2CAL upgrade? Or, what if one could improve the L1MET at L1 with higher resolution, to reduce the MET+2JET rate?
If we had much more L2A bandwidth, much less work online for sure! Jet90, MET35, Jet40/60 etc could be saved this way … However, still much more work offline…. Top_multi_jets efficiency is inst. luminosity dependent due to jet merging MET+2JETs efficiency is inst. luminosity dependent due to jet merging The same is true for any trigger requiring multi-jets… Offline analysis on systematic could be a lot of fun, making many heroes… It doesn’t really minimize the work for ALL of us
Wait a second! Do we really have to do this?
(cont.)
OK, I got it. This L2CAL upgrade thing is all about survival then: to improve the purity of Jet/Met related triggers so that we have extra bandwidth for other triggers… No, it is not just about survival, it is really about how to live gracefully… L2Cone algorithm is much less sensitive to luminosity… the trigger efficiency is more robust against inst. Luminosity… much easier life offline… Finer resolution MET sharpens the turn-on curve: possibility to lower the threshold. L2CAL upgrade allows more sophisticated algorithms to be implemented in software… JET/MET can be made nearly equivalent to offline quality Many new opportunities for additional improvements in trigger purity and performance, most notably for Higgs and exotics triggers, with such information available at L2 as dijet mass, delta-Phi between jets or between a jet and MET, sum Et of the cluster, better jet-SVT matching for b-jets, improvements in tau triggers …. With the L2CAL Pulsar upgrade, we could really push the Higgs/exotics search sensitivity beyond
the baseline … to make sure we have the best shot for discovery Keep in mind: we are not alone in the Tevatron Universe … D0 has just upgraded its L1CAL with more sophisticated capability at L1…
“We need to have trigger datasets with broad bandwidth for discovery” --- Jaco Konigsberg, the CDF spoke, has spoken
Wait another second, Do we have enough man/woman
power to do this?
For long, I have heard horrible stories that nowadays it is very hard to find people to even take care of the CDF operation …
This is so not true with the L2CAL Pulsar upgrade There are people who HATE the L2CAL upgrade, for whatever reasons Still, there are MANY brave people who are highly motivated to work on
this upgrade It has been a “phase transition” in the past few months around the trigger
workshop on May 12th… all started from hall-way talking … no single bottle of beer was even involved
Pisa, Rockefeller, Purdue, UC, Acadmia Sinica, Madrid, Padova, Frascati …
I am TOTALLY impressed that there is still SO MUCH energy within CDF. All it takes is to inspire them…
The man/woman power:only people with > 20% time shown
Engineers: Marco Piendibene (50%): Pisa (Pulsar firmware expert) Lucas Rogondino (100%): Pisa engineer student Mircea Bogdan or Fukun tang, Harold Sanders (part time): UC engineers,
Pulsar hardware experts Richard Northrop (part time): UC mechanical engineer for cabling Davis Pantano (30%): Padoca technicain
Physicists: postdocs Laura Satori (100%): pisa Gene Flanagan (100%): Purdue Giorgio Cortiana (100%): Padova, available soon New Padova postdoc (100%): available end of year New U. Chicago postdoc from Henry (100%): end of year
Physicists: Ph.D students Miguel Vidal (100%): Madrid, available Sept. 2006 Michael Schmidt (30%): U. Chicago (Y.K. Kim), available Sept. 2006
So how much it would cost?
No Pain, No Gain…. As we all learned. We are not cheap… In order to interface with L1 DIRAC, we need to design a new mezzanine card, and we need 72 of them (I.e. build ~100 of them). The total cost is about $50K. We already have 92 Pulsar boards for CDF: 36 for SVT+9 for L2D + 6 for XFT
= 51. Still 41 left. However, L2CAL upgrade takes 24 Pulsars… so only 17 Pulsars left.
This is clearly not enough, given all the test stand needs for the official projects at different places, and the unofficial XFT project needs, hot spares for operation … etc
Also depends on how many other upgrade proposals will be approved… The last thing we want to see is to run out of Pulsar boards. I would strongly suggest that we order ~25 Pulsar boards if the L2CAL upgrade
is approved… This would cost 25 x $4K ~ $100 K, bring the total to $150K Plus ~30% contingency: $50K + $150K ~ $200K. This is the price tag for doing
the dirty work for CDF…
So how long it would take?
Strongly depends on the people involved
It could take > 1 year to just change a simple firmware… We have the best people on the case… with excellent track-record With good people involved, strong leadership is the key…. Laura will be in charge of the technical aspects Gene will be in charge of the physics trigger aspects Paola/Henry/Mary/Matthew/Ted/etc will help We expect ~ 6 months to finish the hardware installation. It typically takes
another few months to fully make use of the new L2CAL trigger capabilities in the official trigger table, based on past experiences.
However, we will seek ways to shorten this time, by parallel processing…
Tasks involvedLVDS signal splitting
Mezzanine card design
Firmware
Readout software
Online monitoring software
Clustering algorithm code optimization
Physics trigger studies
Final Commissioning
Half year, assume all the man/woman power in place
We are familiar with all the tasks…
The Lockyer’s committee-- it is now in your hands
Backup slides
A few on hardware configuration
A few on schedule and people power:
As we speak, there are still more people joining the effort. The schedule shown here does not include all of them. Some of them will join the effort in the new trigger studies and developments, some will help out
the hardware/software.
Mezzanine slotsAUX card
Pulsar Design modular/universal/self-testable
Pulsar
Custommezzanine
Bottom view
Since most of the upgrade proposals are using Pulsar boards,This slide is a backup slide.
S-LINKPCIGbE
worksup to
100MHz
Top view
What’s involved?-- only CAL related shown: concept
L1CAL
L2CAL
Calorimeter10 bits tower energy
10 bits tower energy
288 LVDS cables
Only 8 bits tower energy used by L1CAL
L2CPU
L2 Pulsar crate
L2CAL Pulsar crate
L2 CPU for commission
(1) A copy of input signal(2) New mezzanine: 4 cable/card(3) 18 Pulsars/AUX with new input firmware(4) 6 Pulsar/AUX SLINK mergers(5) Some simple online code(6) New clustering algorithm code
Pulsar Cluster (1 Pulsar: 4 mezzanine x 4 cable = 16) x 18 = 288 input cables total
Pulsar x9
Pulsar x9
Pulsar Crate 1
Pulsar Crate 2
144 cablesfrom DIRAC
one 40-bit word/cable
144 cablesfrom DIRAC
9 slink outputs
9 slink outputs
Pulsar Slink merger x6 PC
The average data transfer latency after L1A~ expected within in ~10 usNote: unlike other L2 paths, CAL data already available at L2 input upon L1A
Raw data size w/o suppression: 288x40/8 ~1.5KB per evt. With some overhead, < ~ 600 slink words maximum
w/ suppression, data size should be much less.
Mezzanine card design concept
FPGA
LVDS/TTL chips
Cable connector
Pulsarside
Rack door
Cost estimate:
72 needed ~ 100
< $300 per card
(dominated by FPGA)
< $30K for mezzanine
~2 weeks engineer time
25 AUX cards
+ long cables
Total cost of the project:
< ~ $50K
40
EM HAD EM HAD
10bits x4 =40 @ cdfclkinput data per cable
JTAG
Pulsar
front panel
How to copy the input signal?-- for parasitic running,
crucial for commissioning
In principle, one could design a special splitter board. Or can use LVDS multi-drop property to get a copy:
DIRACdriver
DCAS
receiver
Pulsar mezzanine Receiver (no termination)
100 ohm
Need to make longer LVDS cablesActual cabling needs to be clean: with help from JDL
DS90C031
New Cabling at trigger room
L2 CAL crates
L1CAL crates
Under the floor
Not terminatedon pulsar mezzanine
Possible new pulsar crate locations
L2 Decision
crate
Commissioning can be done in pure parasitic mode, using the spare decision CPU,along with a copy of all other L2 data paths information
Above the racks
The man/woman power:only people with > 30% time shown
updated on Aug. 3rd, 2006
Engineers: Marco Piendibene (50%): Pisa (Pulsar firmware expert). At FNAL now. Lucas Rogondino (100%): Pisa engineer student. Arrived FNAL Aug. 2nd. Mircea Bogdan or Fukun tang, Harold Sanders (part time): UC engineers, Pulsar
hardware experts. New mezzanine card design. Richard Northrop (part time): UC mechanical engineer for cabling Davis Pantano (30%): Padova technician
Physicists: postdocs Laura Satori (100%): Pisa Gene Flanagan (100%): Purdue Anadi Canepa (100% after Sept 1st): Upenn (N. Lockyer) Giorgio Cortiana (100%): Padova, arriving FNAL Aug. 2006 Massimo Casarsa (50%): Pisa, arrive fall. New Padova postdoc (100%): available end of year New U. Chicago postdoc from Henry (50%): end of year
Physicists: Ph.D students Miguel Vidal (100%): Madrid, available Sept. 2006 Michael Schmidt (30%): U. Chicago (Y.K. Kim), available Sept. 2006 Yanjun Tu (50%): Upenn (N. Lockyer), available Sept. 1st 2006 (after ACE shifts)
L2CAL upgrade team
Overall Project Leader: TBA Technical coordinator: Laura Physics Trigger coordinator: Gene
The rest of the slides are Ted’s initial thoughts on the schedule and people power… poor Ted doesn’t know much at all about project managing, let alone fancy Microsoft project managing tools… so this is in power point format. Project managing experts, please do laugh at me, but not too much…:)
Assume this project will be approved officially in Aug. 06
Basic Tasks involvedLVDS signal splitting
Mezzanine card design
Firmware
Readout software
Online monitoring software
Clustering algorithm code optimization
Physics triggers
Final Commissioning
Half year, with all the man/woman power in place
We are familiar with all the tasks… 20% effort on new stuff, 80% effort on testing…
Summary: L2CAL work plan draft
(from now to system commissioning)
Cabling: design
Northrop/JDL/TL/Dervin
Cable testing: students*/Northrop
Cable/crates installation: Dervin/BB/JDL/TL/Vadim
Aug. Sept. Oct. Nov. 06 - Feb 07
Mezzanine design: Mircea + Laura/Marco/TL
Firmware design/testing: Marco/Davis/Laura + help fromTL
L2CAL Pulsar Testing software: Lucas + Pulsar group/Vadim
Mezzanine/Pulsar testing: students* + Marco + Lucas
Readout software: Laura W/ help from DAQ
Online monitoring: Lucas/Giorgio+students*/Vadim/Wojtek
Algorithm code with isolation and two electron passes: Anwar/Gene/Mary Final overall code optimization: Laura + Yanju + Vadim/TL
Continue work to replace all existing triggers: Gene
New triggers for new L2CAL: Anadi/Giorgio/Massimo/etc/TDWG (could use more help here!)
Systemcommissioningwith parasitic running first (Everyone)
Key person in red. Students* = Yanjun/Miguel/Michael. Some details follow …
LVDS signal splitting
Cabling design: Rich Northrop with help from JDL/Dervin/TL
Aug. 2006 (initial work started in July)
Cable/connectors assembly and testing: Rich Northrop + students*
Sept 2006
Cable/crates installation: Dervin’s team/BB/JDL/Vadim/TL
Oct. 2006
*Students: Yanju/Miguel/Michael
Mezzanine card/firmware The mezzanine card is a simple design… with LVDS signals into a small FPGA,
running at 132 ns clock…
Mezzanine design: Mircea Bogdan + Laura/TLAug.-Sept 2006
Mezzanine card / Pulsar DataIO firmware (simple): Marco with help from Laura/TL
Aug. - Sept. 2006
Mezzanine card testing software: Lucas with help from Pulsar group Aug - Sept. 2006
Mezzanine card production testing: students* + Marco + Lucas Oct. 2006
*Students: Yanju/Miguel/Michael. ~25 AUX cards made during this time
Readout software
Mostly Identical to the current Pulsar VME software. Minor modifications to format the new bank exactly the same as the current DCAS bank.
Laura with help from DAQ group and TL
Aug.-Sept 2006
Commissioning the readout software: Lucas + Laura Oct. 2006
Online monitoring software &
system installation
Mostly Identical to the current PulsarMon software. Cut&paste with minor modifications to compare the new bank to the current DCAS bank bit by bit.
also new TL2D bank vs old TL2D bank…cut&paste with minor mods
Lucas/Giorgio + students* with help from existing Pulsar group (Vadim/Wojtek)
Oct. 2006
Commissioning the online monitoring software: Lucas/Giorgio + Laura + students* Nov. 2006
System installation*: * New Pulsar production on going between Sept. - Nov. No need to wait for them for system
installation… we can use the existing spares.* Testing of the new Pulsars can be done at PREP, like before. They will be the new spares.
Clustering algorithm code
The core code (clustering/Met) already in good shape (optimized for timing)…
Add 2 electron passes and isolation: Anwar/Gene/Mary/TL Aug.-Sept. 2006
Pushing algorithm code optimization for speed: Laura
Oct. - Nov. 2006
Continue work to replace the existing triggers with the new triggers: Gene
Oct. - Dec. 2006
Developing new triggers for the new L2CAL system: Anadi/Giorgio/+ others from physics groupsSept. - Jan. 2006 (to get the best physics out of the new system. Help from TDWG)
System commissioning
People: ALL
Commissioning period: with help from TDWG and existing Pulsar group Nov. 2006 - Feb. 2007
Hardware/software commissioning: Laura/Lucas/Marco/Gene/Yanju/Miguel Offline trigger validation: Gene/Anadi/Giorgio/Massimo/Yanju/Miguel/Michael
System maintenance long term:
Students: Yanju/Miguel/Michael (first - second year grads. More coming)Postdocs: Laura/Gene/Anadi/Giorgio/Massimo…
Some new people(/blood) will be available ~end of 2006 (during final commissioning)