daq overview

MICE DAQ and Controls Review Jean-Sebastien Graulich

Slide 1

DAQ OverviewDAQ Overview

o Terminology

o Requirements

o Front End Electronics

o Trigger System

o DAQ Hardware and Software

o Event Building and Data Flow

o Interfaces: Monitoring, Controls, G4MICE, and Configuration DB

o Project Plan: Human resources and Time Line

Jean-Sebastien Graulich, Geneva


Slide 2

DAQ Terminology

ISIS / MICE / DATE Jargon Isis Cycle:

The injection and acceleration cycle of ISIS. It is 20 ms long (50 Hz). Machine Start or MS:

This is the pulse used for the ISIS synchronization Spill or Spill-Gate:

The time window during which the MICE Target is crossing the ISIS beam. The Spill cycle is driven by the cycle of the MICE target

Burst: The ~100 ns time window during which muons can be expected in the MICE

detectors. It corresponds to the time it takes for a proton bunch in ISIS to cross through the MICE target

DAQ-Trigger: Signal triggering the readout of the FE-electronics modules of the MICE detectors.

By definition, one DAQ-Trigger corresponds to one DAQ-Event Particle-Trigger:

Signal generated when the desired Trigger Condition is met. It is distributed to the sub-detectors Front End Electronics and initiates the digitization of the data therein.By definition, a Particle-Trigger corresponds to one Particle-Event


Slide 3

RF Cycle

1 secPULSE !

RF Pulse1 ms

20 ms

Isis Beam Radius

MS

RF Trigger

Target Position

SPILL !

SoS

DAQ gate

EoS~1 ms

DAQ Trigger

DAQ Event !

Particle Events !

“GoodEvent”

(TOF0xTOF1xTOF2)


Slide 4

100 ns

Digitisation Dead time

Particle Trigger

Bursts ! IsisMicrostructure

224 nsShould be ~500 ns

1700 ns on average

Next Particle Trigger

SoS

DAQ gate

EoS~1 ms

DAQ Trigger

DAQ Event !

Particle Events !

“GoodEvent”

(TOF0xTOF1xTOF2)

RF Pulse1 ms

20 ms

Isis Beam Radius

MS

RF Trigger

Target Position

SPILL !


Slide 5

Requirements

See the full list of system requirement in MICE Note 222.

The main requirement is on the rate:

3.2.1 The DAQ system should allow acquiring data for up to 600 Particles in a 1 ms spill repeating at 1 Hz.

Important consequencesDetector data Readout must be performed at the end of the spill

Data has to be buffered in the FEE

This is because the readout of 1 particle event takes several 100 This is because the readout of 1 particle event takes several 100 µµs…s…


Slide 6

Digitization dead time4.3.4 The DAQ dead time after a Particle

Trigger should not exceed one burst after the one that generated it.

Average Time between 2 muons is 1.7 µs Conversion time for conventional ADC > ~3 µs

Critical for EMCal and TOF Solution: Flash ADC after Signal stretching

Even if the event buffer was large enough, conventional ADC

can not collect 600 muons/ms

Requirements (2)


Slide 7

Charge Measurement in MICE (except Tracker)

100 MHz Flash ADCs (CAEN 1724)

Advantages No Splitter, no delay cable, no discri, no TDC ! Cabling very easy

Need custom made shaper Strong impact on data size

2 ns rise time

ShaperVthr

tthr >30 ns rise time

By fitting the rising edge, time resolution is much better than the 10 ns of the sampling rate (~1 ns obtained).


Slide 8

Requirements (3)

Event Size DAQ Event Size is quite large because it

corresponds to 600 muons

3.2.3 The DAQ system should be able to deal with DAQ event size up to 60 MB (up to 10 MB per sub-event)

Means ~ 60 MB/s data transfer rate (readout between spills)


Slide 9

Requirements (4)

Stability and Reliability Flexibility Partitioning

Following MICE Installation plan

Allow calibration events between spills

AND maintainable by a single person…


Slide 10

Front End ElectronicsFront End Electronics

o TOF

o KL

o CKOV

o Tracker

o EMR

o Target Position and Beam Loss


Slide 11

TOF FEE Sensor: PMT (Hamamatsu R4998)

Signal transmission: Single ended, 50 Ohm, coax cable (RG213)

Number of Channels 40 (TOF0) + 28 (TOF1) + 40 (TOF2) = 108 ch

Main Constrain Time Resolution -> Time-walk correction

TDC CAEN V1290, 32 ch Large Event Buffer ECL input

Discriminator Lecroy 4415 16 Channels ECL output Twisted Pair Input (110 Ohm)

Need Signal splitting for charge measurement


Slide 12

KL FEE

Sensor: PMT (Hamamatsu R1355) Signal transmission: Differential, 120 Ohm, Twisted

pair cable

Number of Channels 42 ch

Main Constrain Charge measurement Time ~ 1 ns

Flash ADC (WFD) CAEN V1724 100 MS/s, 14 bits Best commercial deal Single Ended, 50 Ohm

Input


Slide 13

Shaper development

Splitter designed in Geneva (P. Bene) Passive splitter, shielded, impedance matched (minimal signal loss) 16 x Lemo, 50 Ohm Input 2 twisted pair outputs, 34 contacts connectors,

110 Ohm to Discriminator120 Ohm to Shaper

Shaper designed in Sofia (I. Rusinov) 120 Ohm Twisted Pair Input, 34 contacts connectors 16 Lemo, 50 Ohm, single-ended outputs to fADC Single-Ended or Differential mode jumper selectable

P-Trg Signal

TOF PMT

DISCRI

ShaperSE Mode

TDC

Trigger LogicSplitter

fADC

BNC Cable50 Ohm

KL PMTTwisted pair Cable

ShaperDiff Mode

fADC


Slide 14

Shaper and Splitter


Slide 15

F(t) = offset – norm (0/(0-)) [ (e-(t-t0)/-e-(t-t0)/0) + (t-t0)/ e-(t-t0)/]

Shaper Output

V1724 - Seq

norm 264.6 2236 0.839 6.177 0.321 1.666t 0.33 27.48offset 7.1 8194

0 20 40 60 80 100

8200

8400

8600

8800

9000

9200

9400

Need for better tuning of baseline restorerUsed for individual

baseline evaluation

- Signal shape well understood- Signal shape well understood- Time resolution better than 1 ns- Time resolution better than 1 ns

Time (sample)


Slide 16

CKOV FEE

Sensor: PMT (8”) Signal transmission: Single ended, 50 Ohm,

coax cable (RG58) Number of Channels

4 + 4 = 8 ch Main Constrain

Rate (no segmentation) Small charge

Flash ADC (WFD) CAEN V1731 500 MS/s, 8 bits Single Ended, 50 Ohm

Input No Shaper !


Slide 17

Tracker FEE

4 VLSB Boards in VME Crate

= Data Buffer

MICE HALL

1 Cryo-Cooler

= ½ Tracker

4 AFE2-t

Boards

Fibers from VLPCs

Digital Signal

MIL1553(Control)

Read out PC

In Control Room

Optical Fiber

Custom Made Digital Data Buffer VLSB = VME LVDS CERDES Buffer MICE defined Data format Measure Both discriminated time and Zero

suppressed Charge 4096 + 4096 = 8198 channels


Slide 18

EMR FEE

Still in prototyping phase 50 layers x 59 bars = 2950 channels Digital information only

only -> 2950 bits (100 words = 400 bytes) In custom made VME buffer (similar to

tracker)

Charge measurement per layer only 50 fADC channel Direct connection, similar to CKOV


Slide 19

Target Monitoring FEE

The Target system records Target dip trajectory ISIS Beam loss monitors analog signals

Duplicate data flow CAM DAQ

Max 8 channels Read out once per spill

Nuclear Instrument PCI I/O card PCi6254

in target PC Marginal in term of data size


Slide 20

Data Size TOF TDC: Maximum 108 hits, 4 bytes/hit

432 bytes/pt TOF fADC: 60 samples per channel

13 kBytes/pt KL fADC: 60 samples per channel

6 kBytes/pt CKOV fADC 300 samples per channel, 1

byte/sample 2.4 kB/pt

Tracker: (w/o zero suppression) 5536 bytes per tracker/pt

10.8 kB/pt TOTAL w/o EMR:

~33 kB/pt 16.5 MB/Spill Maximum 16.5 MB/s

Electron Muon Ranger (Coming up after spring 2010) Digital: 0.4 kB/pt or 2 kB/pt if TDC information fADC: 300 samples/ch, 1 byte/sample: 15kB/pt TOTAL for EMR: 21 kB/pt -> 10.5 MB/Spill -> 10.5 MB/s

TOTAL at completion ~ 30 MB/s


Slide 21

Trigger SystemTrigger System

o Particle Trigger

o Timing

o Logic

o DAQ Trigger

o Spill Gate


Slide 22

Particle Trigger Triggers the digitization of the signals arriving at

the Front End Electronics Should arrive a few s around the signal Distributed to all FEE boards We expect about 500 Particle triggers for 1 DAQ trigger

The timing of the trigger should be given by the burst

Delay TOF0, TOF1 TOF2 such that they arrive approximately at the same time in the trigger logic

Make the TOF logic pulses ~200 ns long Make the Burst Gate narrow and Delay it such that it

arrives more than 100 ns after the TOF signals All single raw time distribution will be ~ 100 ns

wide Problem for beam commissioning !

Protons have very different timing Will use TOF0 timing until stable muon running is

established


Slide 23

Particle Trigger timing

TOF1

Burst Gate

TOF0

TOF0 TOF1 Burst Gate

200 ns

GVA1 hit

Burst Gate (made narrow)


Slide 24

TOF0 LogicTOF0 Logic

TOF0 has 40 TOF0 has 40 channelschannels

10 horizontal slabs 10 vertical slabs 2 PMts per slab

Because of hardware constraints,We also defineTOF0_H: OR over 8 central H slabsTOF0_V : OR over 8 central V slabsTOF0_EXT: OR over the 4 frame slabs

TOF0 signal is the OR over all the slabs of the AND of the two pmts of each slab


Slide 25

TOF0 Logic TOF0 Logic implementationimplementation

TOF1 will be included in the trigger in a similar way


Slide 26

Trigger ElectronicsTrigger Electronics

Crate LayoutCrate Layout DAQ Trigger distribution

Scalars and particle trigger NIM Logic

TOF discriminators and trigger CAMAC Logic

Interfacew/Target &Spill Gate

ShapersGVA Discri,KL cosmics trgTOF & CKOV

KL

MICE Ready

Target Ready

RF Ready

DAQ Ready

Gated Machine Start

Spill Request

Target Trigger

Protons on target

RF Trigger

RF Power

DT Gate

DAQ Trigger

Target Delay

RF Delay

DT Delay

20 ms Extraction

Validated Machine Start

MICE Systems synchronization


Slide 28

DAQ Trigger

Four types of DAQ Four types of DAQ triggerstriggers

- PHYSICS- PHYSICS

- CALIBRATION- CALIBRATION

- Start of Spill- Start of Spill

- End of Spill- End of Spill

Event Type is tagged in Event Type is tagged in the data headerthe data header

Used to impose a well-Used to impose a well-defined sequence of defined sequence of events and online events and online check of check of synchronisationsynchronisation

Fixed delay ~ few ms

Fixed delay

Software Check: No overlap with SOS busy (otherwise stop with error)

Depends on Data Size ~ 1 s

VMS

Particle Triggers

Fixed width ~ 1 ms

Physics Event DAQ Trigger

Fan out to LDCs

Fan out to LDCs

Fan out to LDCs

EOS Trigger

SOS Trigger

DAQ Idle = DAQ Ready

DAQ trigger distribution

Target Trg

Trigger ReceiverConnections

V977

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Common toall LDCs

Central trigger only

All PE Busy

All CALIB Busy

All SOS Busy

All EOS Busy

DT Gate

PE DAQ Trigger

CALIB Trigger

SOS trigger

EOS Trigger

PE Local Busy

CALIB Local Busy

SOS Local Busy

EOS Local Busy

DAQ Idle

2 spares

In Out

Ireg0

Oreg0 = Ireg1

Oreg1

Reset by software as soon as the trigger is seen

Reset by software as soon as the readout is done

CAEN V977

Channels 0-7 configured as Flip-Flop, reset by software


Slide 31

DAQ Spill GateDAQ Spill Gate

Detailed documentation of the procedure to adjust Detailed documentation of the procedure to adjust the DAQ Spill Gate is availablethe DAQ Spill Gate is available

MSD

R1-N1-M3-P1 End Mkr

Particle TriggerRequests

Acceleration

DAQ Spill Gate

Target DepthInjection

Extraction

R1-N1-M4-P1 End Mkr

Adjust to tune the DAQ Spill Gate width

Normaly constant delay (DAQ Spill timing)

Machine start after delay in the target system

Spill Gate is always aligned to extraction


Slide 32

Machine

StartDAQ Gate ~ 3.5ms

Hits in GVA1

DAQ Spill Gate SyncDAQ Spill Gate Sync


Slide 33

DAQ Hardware and Software

o Hardware overview

o Software Framework

o Event Building

o Data flow


Slide 34

D-DAQ Hardware Overview

RAL Net

MICE Main 1GB Switch

Control & Monitoring

100 MB Switch

MICE DAQ 1GB Switch

User PCs and Wi-Fiin Local Control Room

EPICS network for MICEControl and Monitoring (MCM)

Trigger distribution

Trackers EMCal TOF Trigger + Ckovs

Event Builder(GDC)

Run Control

Dedicated Link to Remote Mass Storage

VME Crates

Optical links

Linux PCs(LDCs)

MCMReadout

Data Saver

Online Monitoring

MICE Private Subnet

RAL Net

MICE Main 1GB Switch

Control & Monitoring

100 MB Switch

MICE DAQ 1GB Switch

User PCs and Wi-Fiin Local Control Room

EPICS network for MICEControl and Monitoring (MCM)

Trigger distribution

Trackers EMCal TOF Trigger + Ckovs

Event Builder(GDC)

Run Control

Dedicated Link to Remote Mass Storage

VME Crates

Optical links

Linux PCs(LDCs)

MCMReadout

Data Saver

Online Monitoring

MICE Private Subnet


Slide 35

Software Framework MICE has decided to use the ALICE DAQ

software framework The ALICE DAQ software framework, DATE, will

provide us with the necessary EventBuilder tool:Subevents collected by different processors have to be synchronized and put together before storage

DATE has much more functionalities than what MICE needs

Previous Experience (from HARP) Experiment could be used

Agreement with ALICE collaboration included 6 months hands on training with real contribution to the software (2006)

Currently using DATE version 6.05 (20/06/2007)

Historical reasons Upgrade in July and then froze it until the end of the

experiment


Slide 36

DATE Vocabulary

LDC : Local Data Concentrator The PC connected to the VME crate via the PC-

VME Interface

GDC : Global Data Collector Event Builder

Event DATE Event = DAQ Event !!!

It contains data for several Particle Events (~500)

Event Type Tag attached to the event depending on which

trigger receiver‘s input has been used


Slide 37

DATE Readout Process

Two processes running in each LDC The readout process waits for a trigger, reads out the

front-end electronics, and fills a FIFO buffer with the sub-event data

The recorder process off-loads the FIFO and sends the sub-event data to one (or several) GDC over the network

Each LDC contains a set of Equipments Equipment =~ 1 Vme board (in MICE) Each equipment has its own set of routines for its

initialization and readout. Adding an equipment is done without recompiling all

DATE Equipment configuration data is saved in MYSQL

database (but not archived)


Slide 38

DATE Readout Algorithm

5 user routines have to be implemented

(XXX is the name of the equipment) ArmHwXXX

Executed at the beginning of the Run

Allows initialization of the board AsynchReadXXX

Executed constantly even when there is no trigger

Don’t use ! EventArrivedXXX

Used only if the equipment needs to trigger the readout ( Trigger Receiver)

ReadEventXXX That is the readout itself

DisArmHwXXX Executed at the end of the Run

General algorithm for equipment readout:


Slide 39

Readout Code

Available and tested for TDC V1290A (TOF) FADC V1724 (TOF and KL) FADC V1731 (CKOV and EMR) Scaler V830 Trigger Receiver I/O V977 NI I/O PCi 6254 (Target) VLSB (Tracker) Trailer (special equipment handling the

release of the busy)

Todo EMR Front End Board


Slide 40

DATE Data Format The data sent by the equipment is just wrapped with a

LDC header (+ a GDC header if used)

The data format in the payload is defined by the manufacturer of the equipment ! (we will stick to 32 bits words)

DATE Header format defined in a header file event.hThis file contains all the information the offline codes needs to know

about DATE

Data from the equipment ->


Slide 41

Interfaces

o Online Monitoring

o Reconstruction

o Controls

o Configuration Data Base


Slide 42

Online Monitoring

Interface is based on DATE monitoring facility

A simplified version independent from DATE (running only from files) exists

Online monitoring process produces histograms and makes then available on demand on a ROOT socket

Socket handling implemented has a thread to avoid clashes with DATE monitoring semaphore system

Online Monitoring GUI is just a ROOT macro allowing the user to request histograms

See Linda’s talk


Slide 43

Reconstruction

Unpacking has to take into account the DATE format

Common library for online monitoring and Reconstruction

Dependency limited to the event.h file

All implemented in C++


Slide 44

Control And Monitoring

Two ways Interface CAM should know the DATE status A summary of CAM data should be inserted in the

online data stream The run should stop automatically when the CAM

goes in severe alarm state An EPICS Client uses the DIM layer of DATE

to make the status available A special DATE equipment will be

implemented to readout some CAM data identified as relevant for Offline Analysis

Currently under development The same equipment will stop the run if a

dedicated CAM variable is set More details in James’ and Pierrick’s talks


Slide 45

Configuration Database

The Equipment Parameters (configuration of the VME boards) will be stored in the configuration database

A run configuration file (in xml format) will be saved in the CDB each time a run is started

The CAM monitors DAQ status When the run starts, the CAM calls an API

function saving the configuration file automatically

More details in David’s talk


Slide 46

Project Plan

o Organization and Resources

o Time Line


Slide 47

OrganizationOrganization

Online GroupOnline Group ResponsibilitiesResponsibilities

Detector DAQ : J.S Graulich (80%) (Group Leader) Control and Monitoring: J. Leaver (50%)

P. Hanlet (20%) Online Reconstruction: L. Coney (20%) MLCR Manager: C. MacWaters (20%)

Other ContributorsOther Contributors Vassil Verguilov (DAQ) David Forrest (Config DB) Henry Nebrensky (GRID – Data Flow) Mike Courthold (Network Supervisor)


Slide 48

Time LineTime Line Project started from scratch in July 2005 First data taken acquired in March 2008 The DAQ is running stable with some limitations

Trigger selection by hardware Limit on DAQ Event Size is currently 30 MB

July 2009: Integration of Tracker in DAQ Include CAM Data in Online data stream (at the Spill level)

Current Priority: Problem with DAQ trigger distribution system

Cause Event building desync Need to include that in the Online Monitoring

Include EMR Include RF System !


Slide 49

The Cat only grinned when it saw Alice. It looked good- natured, she thought: still it had VERY long claws and a great many teeth, so she felt that it ought to be treated with respect.

`Cheshire Puss,' she began, rather timidly, as she did not at all know whether it would like the name: however, it only grinned a little wider. `Come, it's pleased so far,' thought Alice, and she went on. `Would you tell me, please, which way I ought to go from here?'

`That depends a good deal on where you want to get to,' said the Cat.

Ì don't much care where--' said Alice.

`Then it doesn't matter which way you go,' said the Cat. `--so long as I get SOMEWHERE,' Alice added as an explanation.

Òh, you're sure to do that,' said the Cat, ìf you only walk long enough.

' Alice felt that this could not be denied, so she tried another question.

`What sort of people live about here?' Ìn THAT direction,‘ the Cat said, waving its right paw round, `lives a Hatter: and in THAT direction,' waving the other paw, `lives a March Hare. Visit either you like: they're both mad.'

`But I don't want to go among mad people,' Alice remarked.

Òh, you can't help that,' said the Cat: `we're all mad here. I'm mad. You're mad.‘

`How do you know I'm mad?' said Alice.

`You must be,' said the Cat, òr you wouldn't have come here.'

daq overview

Documents

daq dead time

particle event

daq system

mice detectors

mice targetdaqtrigger

particle trigger

ns time window

sconversion time