1

DAQ Update

MEG Review Meeting, Feb. 17th 2010

2

DAQ Status

• DAQ modifications

• Replace DRS2 by DRS4 boards

• Added synchronization signal between boards

• Added slow control equipment into DAQ allowing remote control

• Overview of DAQ performance

• Average total trigger rate 6.4 Hz

• ~5 minutes runs with 2000 events

• Inter-run time: 6.8 s

• Event size:9 MB (raw) 2.6 MB (zero suppressed)

• Data rate 16.6 MB/s, 1.4 TB/day

• Offline compression ÷ 2 38 TB taken in ~54 days

• DAQ issues during routine running

• Lost sync signal: twice a day run restart (10 s)

• Run startup problem: every few hours retry start (4 s)

3

OfflineCluster64 CPUs

150 TB disk

DAQ scheme

TRG1 TRG2 TRG3 TRG9 DRS4 DRS5 DRS6 DRS7 DRS8

trigger & sync & trigger type & event # LSB

busy

internal trigger & busy

SYSTEM01 SYSTEM02 SYSTEM03 SYSTEM04 SYSTEM05 SYSTEM06 SYSTEM07 SYSTEM08 SYSTEM09

Event Builder

SYSTEM

Logger

Archiver

4

Examples of remote control

5

“Ghost pulse” problem

R

“Ghost pulse”2% @ 2 GHz

“Ghost pulse”2% @ 2 GHz

After sampling a pulse, some residual charge remains in the capacitors on the next turn and can mimic wrong pulses

After sampling a pulse, some residual charge remains in the capacitors on the next turn and can mimic wrong pulsesSolution: Clear before write

write clear

Fixed in DRS4 chip

6

Problems with DRS2

• Ghost pulse effect (previous slide)

• Clock crosstalk (1-2 mV)

• Temperature dependenceoffset and gain

• Complicated calibration DAQ rate was limited by front-end computer software calibration

• Offset shift of last 64 bins

• Offset shift with readoutfrequency

Redesign of DRS chip and replacement of complete

DAQ electronics

Redesign of DRS chip and replacement of complete

DAQ electronics

7

New DRS4 Mezzanine Board

clock chip

4 x DRS4 chips (was 2 x DRS2)

8

On-chip PLL

PLL

ReferenceClock

Vspeed

Internal PLLInternal PLL

•On-chip PLL locks sampling speed to external clock•fclk = fsamp / 2048

•On-chip PLL locks sampling speed to external clock•fclk = fsamp / 2048

9

Old vs. New Synchronization

DRS2

signal

clock

trigger

DRS4

signal

Channel 0

Inverter Chain

Channel 1

Channel 2

Channel 3

Channel 4

Channel 5

Channel 6

Channel 7

Channel 8

PLL

Skipping Channel 8 during readout reduces dead time by 11%Skipping Channel 8 during readout reduces dead time by 11%

10

Global Synchronization

masterquartz

fan

out

VME Board

DRS chips

Clock dividerand jittercleaner

(LMK03000)

20 MHz

20 MHz

0.7

8 M

Hz

0.7

8 M

Hz

11

Synchronization of clock chips

1.2 GHz

0.78 MHzChip 1

0.78 MHzChip 2

n * 0.83 ns

SYNC & 20 MHz

• SYNC has to arrive on all board within 50 ns trigger bus• 20 MHz MEG clock has to arrive on all boards within 0.83 ns

• SYNC has to arrive on all board within 50 ns trigger bus• 20 MHz MEG clock has to arrive on all boards within 0.83 ns

20 MHz

50 ns

12

Problems with synchronization

• Synchronization pulse has to arrive within 50 ns on all boards optimize trigger bus cabling

• Clock divider chips need several configuration cycles

• Re-timing with global clock required firmware modification due to bug in original version syncing did not work initially

• Time frame changed (global clock edge vs. bin #0 time) modification in analysis and calibration database

• Due to these problems, the deployment of the DRS4 boards were not as smooth as anticipated

13

Timing Limitations

• Requirements: « 100 ps

• Current timing resolutions measured with split signal:

• 15 ps (same chip)

• 30 ps (different chips same mezzanine board)

• 130 ps (different VME boards)

• Worse than for DRS2 chips ( Ryu’s talk)

• Inter-board timing limits our experiment

• Immediate changes

• “Bypass wire”

• Keep DRS3 for timing counters

• Possible causes

• Global Clock Jitter

• Jitter inside DRS4 boards

• Jitter added by active splitter

• ~10 MHz 1 mV noise on top of signal

clock

fan

-ou

t

PMT splitter

trigger

+

=

threshold

14

Clock Distribution

LMK03000

< 20 ps period < 20 ps period (was 120 ps with old resistor)

LMK03000

< 32 ps relative

mezz

anin

em

ezz

anin

e

added “bypass wire” to sample original clock jitter of on-board clock distribution eliminated

15

Aperture Jitter inside DRS chip

3.2 GHz

1.6 GHz

rtUu

t

slope

Intrinsic DRS aperture jitter 1/fsampling

if power supply (U) is constant

Intrinsic DRS aperture jitter 1/fsampling

if power supply (U) is constant

16

Plan to improve timing

• Increase sampling speed 1.6 GSPS 3.2 GSPS

• only firmware modification required

• goal: end of March

• Disentangle different contributions to timing

• Re-measure global clock jitter between crates

• Measure split pulse jitter before/after active splitter

• Investigate noise situation in area(XEC timing improved in ’09, e+- timing decreased, TC uses same electronics (DRS3) noise in TC signals ???)

• Optimize DRS4 timing calibration algorithm can also be applied to old data

• Lab tests showed ~10 ps accuracy with sampling technique(DRS4 [PSI], SAM [Saclay], BLAB [U.Hawaii])

• Goal: Electronics should not limit capabilities of experiment

17

Daisy-chaining of channels

Channel 0 – 1024 cells

Channel 1 – 1024 cells

Channel 2 – 1024 cells

Channel 3 – 1024 cells

Channel 4 – 1024 cells

Channel 5 – 1024 cells

Channel 6 – 1024 cells

Channel 7 – 1024 cells

Domino Wave Generation • DRS4 can be partitioned in 4x2048 cells

• Running with 2048 cell channels allow sampling speed of 1.6 GHz * 2 = 3.2 GHz with current trigger latency internal timing accuracy should improve ~2x

• Deal with increased data rateby on-board averaging

• DRS4 can be partitioned in 4x2048 cells

• Running with 2048 cell channels allow sampling speed of 1.6 GHz * 2 = 3.2 GHz with current trigger latency internal timing accuracy should improve ~2x

• Deal with increased data rateby on-board averaging

1.6 3.2 averaging

18

• Reduce dead time

• Use various binning regions in waveforms

• Double-buffering in front-end electronics

• Use multi-threading in run start/stop sequencing(6.8 s inter-run gap 3-4 s = 1% dead time improvement)

VME Transfer

Further plans

Rebinning 2:1, 4:1, …

Chip readout

0.5 ms

VME Transfer

25 ms

Chip readout VME Transfer

now

reduceddata size

multi-eventbuffers

Chip readout

ready fornext event

19

2nd Level Trigger

• Drift Chamber wires are connected to trigger, but too slow to be included in trigger decision (400 ns)

• Would be possible with 2nd level trigger scheme

• Only firmware modifications (spare bits on trigger bus)

• If trigger rate ÷2, dead time would be ~÷2

1st LT

400 ns 25 ms

event

stop DRS chips

2nd LT

1 us

Chip readout VME Transfer

trigger

DRS boards

restart DRS chips