gueorgui antchevprague 3-7 september 2007 1 the totem front end driver, its components and...

1Gueorgui ANTCHEV

Prague 3-7 September 2007

The TOTEM Front End Driver, its Components and Applications in the TOTEM Experiment

G. Antchev a, b, P. Aspell a, D. Barney a, S. Reynaud a, W. Snoeys a, P. Vichoudis a

a CERN, 1211 Geneva 23, Switzerland

b INRNE-BAS, 1784 Sofia, Bulgaria

2Gueorgui ANTCHEV


TOTEM Electronic System Architecture

RPS1 RPS2 RPS3 RPS4

GEM GEMCSC CSC

FED

FED FED FED FEDFED FED

CMS DAQ System

FRL

S-Link

FRL

S-Link

FRL

S-Link

FRL

S-Link

FRL

S-Link

FRL

S-Link

FRL

S-Link

3Gueorgui ANTCHEV


- The TOTEM Front End Driver (TOTFED) has to receive and handle trigger building and tracking data from the TOTEM detectors;

- Interface to the data acquisition and global trigger system;

- Modular, based on the VME64x standard;

- Very flexible and programmable to deal with the different TOTEM sub-detectors

- possible evolution of the data treatment and trigger algorithms over the duration of the experiment

- Acquire on-detector data from up to 36 optical links and:

- perform fast data treatment (data reduction, consistency checking, etc...);

- transfer to the next level of the system;

- store data on request for slow spy readout via VME64x or USB2.0.

- Compatible with CMS:

- permit TOTEM to run both standalone and together with CMS

General Requirements

4Gueorgui ANTCHEV


Similarities:- Common input (GOL)- Common output (Slink mezzanines to the CMS DAQ)- Similar architecture (VME system, spy memories on board)- Problems (use of ECAL DCC seems inconvenient for Preshower / not possible, for different reasons)

Differences:- 1 S-Link64 per 9 GOLs (TOTEM) vs. ~ 1 S-Link64 per 36 GOLs (Preshower)- Data Reduction on-board for Preshower- Possible Data taking for TOTEM before the other experiments @ low trigger rate (few kTriggers/sec) – USB2.0 can be used for readout

CMS Preshower: - Has the experience in gigabit optical receiving systems and in USB2.0 interface

TOTEM: - Has the experience in VME64x designs, Memory Interfaces and complex system design

Result:- Design of the TOTEM FED in such a way that can be used also for CMS Preshower - CMS Preshower supports the development of the gigabit optical receiver and USB parts.

TOTFED and CMS Preshower DCC - motivation

5Gueorgui ANTCHEV


TOTFED – Components

VME64x Host Board:- 9U VME64x Slave Module- Accepts several mezzanine modules

OptoRX12:- Custom 12 Channels Optical Receiver Module for GOH- Accepts CMC Transmitter Module

CMC Transmitter Module:- CMS DAQ Standard parallel data transfer module- Using S-Link64 interface

6Gueorgui ANTCHEV


TOTFED – Block Diagram and I/O Interfaces TOTFED – Block Diagram and I/O Interfaces

Gigabit Optical INPUT

640Mb/s/fiber x 12 = 7.68Gb/s

VME64x OUTPUT

40MB/s BLT

Data Bandwidth – by definition

S-Link64 OUTPUT

480MB/s 64bit@60MHz

USB2.0 OUTPUT

480Mb/s – high 12Mb/s – full320Mb/s – effective

TOTFED has:INPUT - 3 x OptoRX -> 3 x 7.68Gb/s OUTPUTS - 4 x S_Link64 -> 4 x 480MB/s

- 4 x USB2.0 -> 4 x 320Mb/s- 1 x VME64x -> 40MB/s

TOTEM ExperimentTrigger Rate - 1 kHz (TOTEM alone)Event Size - 40 kBytes

S_Link64 OUTPUT

480MB/s 64bit@60MHz

Block Diagram

VM

E64x

VME64xInterface

MAIN 1

Loca

l Bus

Spy 1Memory

OpRX12+

CMC Transmitter

MAIN 2

Spy 2Memory

MAIN 3

Spy 3Memory

CCS/ TTSOptional

To S-Link64

J TAG

CLOCK

CLOCK

USB 1

USB 2

USB 3

TTCrxQPLL

MergerFPGA

Spy 4Memory192bits 64bits

32bits

64bits

32bits

32bits

32bits

16bits

16bits

16bits

USB 4

Local Bus

16bits

Buffers

OpRX12+

CMC Transmitter

OpRX12+

CMC Transmitter

VM

E64x

VME64xInterface

MAIN 1

Loca

l Bus

Spy 1Memory

OpRX12+

CMC Transmitter

MAIN 2

Spy 2Memory

MAIN 3

Spy 3Memory

CCS/ TTSOptional

To S-Link64

J TAG

CLOCK

CLOCK

USB 1

USB 2

USB 3

TTCrxQPLL

MergerFPGA


32bits

64bits

32bits

32bits

32bits

16bits

16bits

16bits

USB 4

Local Bus

16bits

Buffers

OpRX12+

CMC Transmitter

OpRX12+

CMC Transmitter

7Gueorgui ANTCHEV


VME64x Host Board

Main Characteristics:

- 9U VME64x Slave;- Local Bus Master 32bits/40MHz;- 18MB Spy Memory:

- 3x 6MB per 12 optical channels (1 OptoRX), 96bits at 80MHz;

- 3x Main Bridge + Memory Controller;- 3x USB 2.0 Interface;- Merger + 6MB Memory + 4th USB;- TTCrx, QPLL and CCS clocks (optical or electrical);- TTS copper link;- JTAG controller on board;- Connectors for: - 3x OptoRX; - 1x DRM or Trigger Mezzanine; - 3x S-Link64 + 1x S-Link64 on rear at 200MB/s;- 2x Optional Board Connectors;

- Board t0 measurements VME64x Host Board

8Gueorgui ANTCHEV


OptoRX12 and S-Link64

OptoRX12:

- 12-channel digital optical receiver by NGK;- Altera Stratix GX EP2SGX60 device with embedded hardware de-serializers for up to ~3.2Gbps;- Connect via five 64pin connectors to the VME board;- Has connectors for “CMC Transmitter” module

CMC Transmitter:

- CMS DAQ standard;- 64bit electrical transmitter using S-Link64 protocol;- Altera Acex family device

9Gueorgui ANTCHEV


TOTFED in Readout System

TOTFED is set of:

“VME64x Host Board”;- 3 x “OptoRX12” modules;- 3 x “CMC Transmitter” modules- 2 x “Optional” modules

TOTFED in the Readout Crate

Readout CrateCAEN

TTCVI

TOTFED

FEC

TOTFED

TOTFED

10Gueorgui ANTCHEV


TOTFED Data Format

VFAT2 Data Format

“CMC Transmitter” CMS Data Format

11Gueorgui ANTCHEV


TOTFED in Trigger System

VM

E6

4x

VME64xInterface

MAIN 1

Local B

us

Spy 1Memory

12chOpRX 1

MAIN 2

Spy 2Memory

12chOpRX 1

MAIN 3

Spy 3Memory

12chOpRX 1

CCS/ TTSOptional

To S-Link64

J TAG

CLOCK

CLOCK

USB 1

USB 2

USB 3

TTCrxQPLL

MergerFPGA


32bits

64bits

32bits

32bits

32bits

16bits

16bits

16bits

USB 4

Local Bus

16bits

Buffers

VM

E6

4x

VME64xInterface

MAIN 1

Local B

us

Spy 1Memory

12chOpRX 1

MAIN 2

Spy 2Memory

12chOpRX 1

MAIN 3

Spy 3Memory

12chOpRX 1

CCS/ TTSOptional

To S-Link64

J TAG

CLOCK

CLOCK

USB 1

USB 2

USB 3

TTCrxQPLL

MergerFPGA


32bits

64bits

32bits

32bits

32bits

16bits

16bits

16bits

USB 4

Local Bus

16bits

Buffers

“Trigger Board” - 36 Optical Inputs; - Algorithms inside Main 1-3 and Merger FPGA; - Data on P2 rear private to “Global Trigger Board”

“Global Trigger Board” - On P2 rear inputs from several Trigger Boards; - Mezzanine to send LV1A

36 O

pti

cal In

pu

ts

Electrical Outputs

To “Global Trigger Board”

Based on VME64 Host Board

Trigger Crate

CAEN

GLOBAL

TRIG

TRIG

TRIG

TRIG

TRIG

TRIG

LV1A

12Gueorgui ANTCHEV


Applications - TOTEM experiment “TOTFED” – Front End Driver; “TOTFEDTC” – FED Tester Board; “TOTRIG” – Trigger Board; “GLTOTRIG” – Global Trigger Board

- CMS experiment “ES-DCC” CMS Preshower Data Concentrator Card;

PCB Information on EDMS: see the links: –> “VME64 Host Board”

–> “OptoRX12” –> “CMC Transmitter”

Technical Information: - Documentation; - Firmware; - Software; - Boards Status;WEB site on preparation

Applications and Documentation

13Gueorgui ANTCHEV


TOTFED - Status

Status - Produced and Mounted - 6 pcs; - Under Test - 3 pcs; - Explore final production possibilities - about 90 pcs. total

Results - Up to now no major issues, few errors were fixed; - Power Supply Voltages - 3.3V, 2.5V and 1.5V are O.K. - Internal Clock Distributions System for 40MHz and 80MHz works; - External Clock Distributions System works; - From on-board JTAG Devices (FPGA’s, PROM’s and TTCrx) all 18 are accessible via JTAG connector; - “VME64x Slave Controller” FPGA – tested: Internal

Registers (Board ID, Control, etc.) and Local Bus W/R access and BLT W/R; - “MAIN Controller” FPGA – programmed and tested: - “CMC Transmitter” on rear tested; - CCS/TTS copper link tested (CMS Preshower group);

Basic Test Lab VIEW VI using CAEN1718 USB-VME

14Gueorgui ANTCHEV


Summary

VME64x Host Board: - Needs to cover several different requirements for TOTEM and CMS;

- Based on modular principle, therefore: - Easy to develop and test separate mezzanines;

- Create not fully equipped unit (cost, performance);- Easy to upgrade, replace/redesign mezzanines

- Interconnectivities, Connectors and Plug-in

TOTFED:

- Complex system; - Successfully built;- Progress in testing and using it

TOTFED Applications:

- TOTEM - Data Readout and Trigger Systems;- CMS Preshower - Data Readout System “ES-DCC” module built and tested in Test Beam 07;- Interest from other experiments in using it

gueorgui antchevprague 3-7 september 2007 1 the totem front end driver, its components and...

Documents