““L1 farm: some naïve L1 farm: some naïve consideration”consideration”

Gianluca Lamanna (CERN) & Riccardo Fantechi (CERN/Pisa)

The L1 farmThe L1 farm• The L1 farm can’t be avoidedcan’t be avoided:

– the NZS LKr must be read at 100 kHz100 kHz

– We are looking for a rare rare decaydecay: most of the events passing the L0 are junkjunk, avoid to build a huge and complex L2 farmL2 farm to fully reconstruct events that have to be reject

– Some detector (i.e. GTKGTK) needs additional information to reduce the readout window readout window (and then the bandwidth)

• It’s better to avoid the L1-L2 L1-L2 PCs mixingPCs mixing:– Characteristics of the data

links (protocolprotocol, raterate, etc.)– Specialization of the L1 PCs

with respect to the sub-detector

How to calculate the number of PC in How to calculate the number of PC in the L1 farm?the L1 farm?

• Two different “type”:“type”:– Detector participatingparticipating to the L1 trigger L1 trigger decision

number of PC driven by computing power needed to calculate the primitives

– Detector not participating not participating to the L1 trigger L1 trigger decision number of PC driven by data rate and bandwidth

CEDARCEDAR

GTKGTK

CHANTICHANTI

LAVLAV

STRAWSSTRAWS

RICHRICH

CHODCHOD

MUV3MUV3

L0TPL0TP

L1 CEDARL1 CEDAR

L1 GTKL1 GTK

L1 CHANTIL1 CHANTI

L1 LAVL1 LAV

L1 STRAWS

L1 STRAWS

L1 RICHL1 RICH

L1 CHODL1 CHOD

L1 LKrL1 LKr

L1 MUV3L1 MUV3

L1TPL1TP

SWI TCH

SWI TCH

L2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARM

L2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARM

L2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARM

L2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARML2 FARM

320 Mb/s

18 Gb/s

320 Mb/s

336 Mb/s

5.4 Gb/s

1.6 Gb/s

160 Mb/s

172 Gb/sLKr r/oLKr trigger

Not participating to L1Not participating to L1• Two relevant parameter:– BandwidthBandwidth: with 10 Gb 10 Gb links only the GTK needs 2 ports (2

PCs?) – Maximum data rateMaximum data rate: 1 MHz 1 MHz packet rate isn’t sustainable

from an ethernet card (max 200 kHz??) more events in more events in one packetone packet

• The number of event per packet depends on the size of the event: for instance assuming standard ethernet packet (1500 B1500 B) and RICH event size of 300 B300 B , a factor of 55 can be applied to the max rate (1MHz/5=200 kHz1MHz/5=200 kHz) – Gain by using Jumbo frame Jumbo frame (9000 B9000 B): buy switches capable

to manage Jumbo frames

Not participating to L1Not participating to L1• But the maximum frequency of

packets in the cable (1 Gb/s1 Gb/s) depends on the length of the packet: 1500 B 1500 B 12 us 12 us 80 80 kHzkHz– Increase the number of PC to

cope with high rate – RICH: 5 events 5 events per packet 80 80

kHzkHz 10 ports 10 ports (2-3 PCs ?? ), multicore (100 kHz of interrupts per core??)

• To evaluate the correct number of PC for each subdetector we need to know:– Events size– Maximum PPSPPS of the NICNIC– Maximum interrupts rate interrupts rate per

core– Number of events per packet

Participating to L1Participating to L1• Some detector should produce primitives for

L1TPL1TP • In this case the number of PC should be increased

to allow the online data processing: 1 MHz 1 us NN (number of cores) = MM (actual time of the algorithm)

• For instance: if the RICHRICH algorithm to reconstruct ring will take 100 us 100 us 100 cores 100 cores (10-12 PC10-12 PC)

• Using GPUsGPUs this number could be highly reduced: RICH 2-32-3 us 1 PC 1 PC with 2 video card 2 video card (probably less)

Final messageFinal message

• The dimension of the L1 farm L1 farm should be evaluated with care

• The real dimension depends on actual rate actual rate and computingcomputing needed : it’s very important to have the possibility to increaseincrease and decreasedecrease the number of PCs in each sub-L1farm sub-L1farm include small switches for each sub-L1farmsub-L1farm (or assure that the L0->L1 farm L0->L1 farm are large enough for future upgrade) it’s convenient to use 10 Gb to 1 Gb 10 Gb to 1 Gb switches instead of 10 Gb to 10 Gb 10 Gb to 10 Gb ???

Time measurementTime measurement• We are trying to measure transit time sending data from one pc to one pc to

another another using the oscilloscope.• Signals on LPTLPT• JitterJitter of the method

measured (under control… few usfew us)

• Study of time performance as a function of the interrupts frequency interrupts frequency and data ratedata rate.

• Future step: replace the “gun” “gun” with a TELL1/TEL62TELL1/TEL62 (with a signal produced by the firmware on the test connector)

PC1: gun

PC2: target

Interface to LPT