Upgrade to the Read-Out Driver for ATLAS Silicon Detectors

Atlas Wisconsin/LBNL GroupJohn Joseph

March 21st 2007

ATLAS Pixel B-Layer Upgrade Workshop

2

Control Path:

ROD Controller FPGAMaster DSP

Program Reset Manager FPGA

Operation, Commands, Triggers

96

Bac

k o

f Cra

te C

ard

/ F

ron

t End

Ele

ctro

nic

s SL

ink

Ca

rd

Clo

ck

Fan

ou

t

Triggers and Event ID Data

Tim

ing

Inte

rfac

e

Mo

du

le

BOCSetup Bus

Serial Input Links

Xon/Xoff

ROD Busy

DSP Farm (4)

Event Trapping, Histograms

Data Path:

Formatter, Event Fragment Builder & Router FPGA,Diagnostic Memory, S-Link Interface & SDSP DMA

DMA 4

...

S-Link

48

System Clock

Serial Output Links

VME64x Signals

VM

E

Bu

s

40

DMA 1

ClocksHost Port

Silicon Read Out Driver (SiROD)

3

256 MB SDRAM

VME Slave FPGA

Configuration Controller

ROD BUSY Histogram

128Mb Flash CNFG & BOOT

Memory

ROD CONTROLLER

FPGA

XC4VFX20-10 Power PC Core

FORMATTER FPGA

XC3S4000-8

2x

EFB /ROUTER FPGA

XC4VLX60-10

DSP FARM TI 320C6713

220MHz 384MB SDRAM

4xDIAGNOSTIC

FIFOs

VME Bus

Back Of Crate (BOC)

CNFG DATA to All FPGAs

RESET

ROD BUSYD32

A32/D32

ROD Control/VME Path

D16

ADDR

D8

D48

1.5 GB SDRAM

ADDR

ARB

MOVE HPI INTERFACE

TO VME CONTROLLER

BOOT SLAVE DSP

FROM FLASH

4

ROD CONTROLLER

FPGA

XC4VFX20

FORMATTER FPGA

EFB/ROUTER FPGA

XC4VLX60

DSP FARM TI 320C6713

220MHz 384MB SDRAM

4x

XoffBOC

FORMATTER FPGA

INPUT FIFO 32K Deep

S-Link

Event Fragments

ROD BUSY

TTC

EVENT ID

READOUT

TIM

MDSP SP

BOCFE CMD (48)

XC3S4000

SiROD DATA PATH

Xoff

Xoff

DATA

DATA

STATUS

5

COMPONENT COMPARISONS: FORMATTER

• RevF SiROD

• Spartan2E Series: XC2S600-6FG456

• Cost: $100 x 8 = $800

• Available resources: 6912 Slices, 288K RAM Bits per FPGA

• Used resourses (1) : 2850 Slices (41%), 268K RAM Bits (93%)

• Used resourses (8) : 22.8K Slices (41%), 2140K RAM Bits (93%)

• Module FIFOs: 32 – 2048 x 32 Dual Port RAM Blocks

• Next Generation SiROD

• Spartan3 Series: XC3S4000-8FG900

• Cost: $220 x 2 = $440

• Available resources: 54K Slices, 3456K RAM Bits per FPGA

• Required resources (2) : 23K Slices, 1072K RAM Bits

• Module FIFOs: 32 - 4096 x 32 Dual Port RAM Blocks

6

COMPONENT COMPARISONS: EFB/ROUTER

• RevF SiROD

• Spartan2E Series: XC2S400-6FG676/XC2S400-6FG456

• Cypress 6 - 16K x18 15ns FIFOs

• Cost: $115 + $85 + $210 = $410

• Available resources: 9600 Slices, 1856K RAM + FIFO Bits

• Used resourses : 6080 Slices (63%), 1824K RAM + FIFO Bits

• Trap FIFOs: 4 1024 x 32 Dual Port RAM Blocks

• Next Generation SiROD

• Virtex4 Series: XC4LX60-10FG1152

• Cost: $558

• Available resources: 26K Slices, 2880K RAM Bits

• Required resources: 6080 Slices, 1824K RAM Bits

• Trap FIFOs: 4 - 4096 x 32 Dual Port RAM Blocks

7

COMPONENT COMPARISONS: CONTROLLER

• RevF SiROD

• Spartan2E Series: XC2S600-6FG676 / TI320C6201-200 DSP

• Cost: $130 + $100 = $230

• Available resources: 6912 Slices, 288K RAM + FIFO Bits

• Used resourses : 5130 Slices (74%), 188K RAM Bits (65%)

• DSP Processor

• Next Generation SiROD

• Virtex4 Series: XC4FX20-10FG

• Cost: $228

• Available resources: 8544 Slices, 1224K RAM Bits, Power PC Core

• Required resources: 5130 Slices, 188K RAM Bits, uProcessor

• Conversion from TI DSP to Xilinx Power PC core:

• Risk Evaluation Required

8

Silicon RevE(F) SiROD

MasterDSP

ControllerFPGA

FormatterFPGAs

Router FPGA

EventFragmentBuilderFPGA

SlaveDSPs

9U VME

VME Interface

Clock Distribution

9

ROD CONTROLLER

FPGA

XC4VFX20

128MB SDRAM

EFB/ROUTER FPGA

XC4VLX60

DSP FARM TI 320C6713

220MHz 384MB SDRAM

VME

J1

J2

INP

UT

FIF

O

32K

Dee

p

FORMATTER FPGA

XC3S4000

SiROD New Layout

BOC

J3

FORMATTER FPGA

XC3S4000

VME CONTROLLER

FPGA

XC3S400

DSP FARM TI 320C6713

220MHz 384MB SDRAM

DSP FARM TI 320C6713

220MHz 384MB SDRAM

DSP FARM TI 320C6713

220MHz 384MB SDRAM

128Mb FLASH

JTAG CONNECTOR ON FRONT PANEL TO PROGRAM VME CONTROLLER FPGA ROM

10

WHERE DO WE GO FROM HERE?

• Preliminary Design Study

• Recommendations:

• Purchase 2 Xilinx Virtex4 Evaluation Boards

• One for the LX series

• One for the FX series

• Combine the EFB/EventMemory/Router Designs to simulate and implement the proposed changes for verification

• Simulate and implement the Formatter as proposed for verification

• Simulate and implement the Controller to learn how to use the Power PC core and to plan a transition from the TI 6201 to the Xilinx PC core.

• Make the new design transparent to the DAQ system.

• Simulate the Control path to understand the limitations of VME BW

11

Additional Slides

12

RODBus Interface

Control & Status Registers (32)

RODBus Data (16)

RODBus Address

RODBus Control

Link Input MUX

&

Half Clock

Counters

IN0

ModeBits FIFO 512x24

Link FIFO 2048 x 32

Link Decoder: 40/80/160MHz

Read Out Controller

DATA OUT (40)

TOKEN

FMT MB (12)

Link FIFO 2048 x 32

Link Decoder: 40/80MHz

Link FIFO 2048 x 32

Link Decoder: 40MHz

Link FIFO 2048 x 32

Link Decoder: 40MHz

IN1

IN2

IN3

IN4

IN5

IN6

IN7

IN8

IN9

IN10

IN11

FMT Type

FMT ID

HT LIMIT

ROD BUSY

Trailer DetectTRAILER (12)

PIXEL FORMATTER FPGA (8x)

13

EVENT FRAGMENT BUILDER FPGA

EventMemA

FIFO16Kx48

Engine 0

Engine 1

EventMemC To/F

rom

Ro

ute

rTo/F

rom

Fo

rmat

ter

To/F

rom

Fo

rmat

ter

To/F

rom

Co

ntr

olle

r

Output FIFO Data (43)

Header and Trailer Data (32)

Halt Output

Halt Output

Header/TrailerGenerator

Error Summary Word

Error Summary Word

Fo

rma

t &

Co

un

tF

orm

at

& C

ou

nt

L1

/BC

ID

Ch

ec

kL

1/B

C I

D C

he

ck

Err

or

Ch

ec

kE

rro

r C

he

ck

INC

/DE

C L

1ID

L1 & BC IDs

Event Data& Trigger Type

Dynamic Mask

Link Num. (6 bits)

Link Num. (6 bits)

Time-out Error

Time-out Error

L1ID Error

L1ID Error

BCID Error

BCID Error

Data (32 bits)

Data (32 bits)

Formatter Num. (2 bits)

Formatter Num. (2 bits)

Link Num. (4 bits)

Link Num. (4 bits)

Data (32 bits)

Data (32 bits)

Data Valid

Data Valid

Time Out

Time Out

INC

/DE

C L

1ID

Purpose: Collect Formatter output, check L1 and BC

IDs, count errors, generate Event Header & Trailer

FIFO Controller Xoff

EventMemB

FIFO16Kx48

14

ROD Router (FPGA) & Slave DSPs

SDSP 3SDSP 2SDSP 1SDSP 0

Read, formatand direct the

event data

2 TrapsEvent Type: ATLAS,

ROD or TIM

2 TrapsEvent Type: ATLAS,

ROD or TIM

2 TrapsEvent Type: ATLAS,

ROD or TIM

2 TrapsEvent Type: ATLAS,

ROD or TIM

DMA Transfer Engine1024 32-bit FIFO

DMA Transfer Engine DMA Transfer Engine DMA Transfer Engine

S-L

INK

To

/Fro

m R

OS

To/F

rom

Eve

nt

Fra

gm

ent

Bu

ild

er

XOn / XOff S-Link

S-Link DataDsp Halt Output

Halt Output

Error Format Data

Data Valid

Output FIFO (43b)

Event Header andTrailer Data (32b)

Texas Instruments 6713 floating point DSPs running at 220 MHz for monitoring and calibration histogramming

Purpose: Route formatted data to Level-2 and/or Slave DSPs (histogram)

1024 32-bit FIFO 1024 32-bit FIFO 1024 32-bit FIFO