atlas wisconsin/lbnl group john joseph march 21 st 2007
DESCRIPTION
Upgrade to the Read-Out Driver for ATLAS Silicon Detectors. Atlas Wisconsin/LBNL Group John Joseph March 21 st 2007. ATLAS Pixel B-Layer Upgrade Workshop. Silicon Read Out Driver (SiROD). ROD Control/VME Path. Back Of Crate (BOC). BOOT SLAVE DSP FROM FLASH. 128Mb Flash CNFG & BOOT - PowerPoint PPT PresentationTRANSCRIPT
Upgrade to the Read-Out Driver for ATLAS Silicon Detectors
Atlas Wisconsin/LBNL GroupJohn Joseph
March 21st 2007
ATLAS Pixel B-Layer Upgrade Workshop
2
Control Path:ROD Controller FPGA
Master DSPProgram Reset Manager FPGA
Operation, Commands, Triggers
96
Bac
k of
Cra
te C
ard
/ Fro
nt E
nd E
lect
roni
cs
SLi
nkC
ard
Clo
ck
Fano
ut
Triggers and Event ID Data
Tim
ing
Inte
rfac
e M
odul
e
BOCSetup Bus
Serial Input Links
Xon/Xoff
ROD Busy
DSP Farm (4)
Event Trapping, Histograms
Data Path:Formatter, Event Fragment Builder & Router FPGA,Diagnostic Memory, S-Link Interface & SDSP DMA
DMA 4
...
S-Link
48
System Clock
Serial Output Links
VME64x Signals VM
E B
us
40
DMA 1
ClocksHost Port
Silicon Read Out Driver (SiROD)
3
256 MB SDRAM
VME Slave FPGA
Configuration ControllerROD BUSY Histogram
128Mb Flash CNFG & BOOT
Memory
ROD CONTROLLER
FPGAXC4VFX20-10
Power PC Core
FORMATTER FPGA
XC3S4000-82x
EFB /ROUTER FPGA
XC4VLX60-10
DSP FARM TI 320C6713
220MHz 384MB SDRAM
4xDIAGNOSTIC FIFOs
VME Bus
Back Of Crate (BOC)
CNFG DATA to All FPGAs
RESET
ROD BUSYD32A32/D32
ROD Control/VME Path
D16
ADDR
D8
D48
1.5 GB SDRAM
ADDR
ARB
MOVE HPI INTERFACE
TO VME CONTROLLER
BOOT SLAVE DSP
FROM FLASH
4
ROD CONTROLLER
FPGAXC4VFX20
FORMATTER FPGA
EFB/ROUTER FPGAXC4VLX60
DSP FARM TI 320C6713
220MHz 384MB SDRAM
4x
XoffBOC
FORMATTER FPGA
INPUT FIFO 32K Deep
S-Link
Event Fragments
ROD BUSY
TTC
EVENT ID
READOUTTIM
MDSP SP
BOCFE CMD (48)
XC3S4000
SiROD DATA PATH
Xoff
Xoff
DATA
DATA
STATUS
5
COMPONENT COMPARISONS: FORMATTER• RevF SiROD
• Spartan2E Series: XC2S600-6FG456• Cost: $100 x 8 = $800• Available resources: 6912 Slices, 288K RAM Bits per FPGA• Used resourses (1) : 2850 Slices (41%), 268K RAM Bits (93%)• Used resourses (8) : 22.8K Slices (41%), 2140K RAM Bits (93%)• Module FIFOs: 32 – 2048 x 32 Dual Port RAM Blocks
• Next Generation SiROD• Spartan3 Series: XC3S4000-8FG900• Cost: $220 x 2 = $440• Available resources: 54K Slices, 3456K RAM Bits per FPGA• Required resources (2) : 23K Slices, 1072K RAM Bits• Module FIFOs: 32 - 4096 x 32 Dual Port RAM Blocks
6
COMPONENT COMPARISONS: EFB/ROUTER• RevF SiROD
• Spartan2E Series: XC2S400-6FG676/XC2S400-6FG456• Cypress 6 - 16K x18 15ns FIFOs• Cost: $115 + $85 + $210 = $410• Available resources: 9600 Slices, 1856K RAM + FIFO Bits• Used resourses : 6080 Slices (63%), 1824K RAM + FIFO Bits• Trap FIFOs: 4 1024 x 32 Dual Port RAM Blocks
• Next Generation SiROD• Virtex4 Series: XC4LX60-10FG1152• Cost: $558• Available resources: 26K Slices, 2880K RAM Bits• Required resources: 6080 Slices, 1824K RAM Bits• Trap FIFOs: 4 - 4096 x 32 Dual Port RAM Blocks
7
COMPONENT COMPARISONS: CONTROLLER• RevF SiROD
• Spartan2E Series: XC2S600-6FG676 / TI320C6201-200 DSP• Cost: $130 + $100 = $230• Available resources: 6912 Slices, 288K RAM + FIFO Bits• Used resourses : 5130 Slices (74%), 188K RAM Bits (65%)• DSP Processor
• Next Generation SiROD• Virtex4 Series: XC4FX20-10FG• Cost: $228• Available resources: 8544 Slices, 1224K RAM Bits, Power PC Core• Required resources: 5130 Slices, 188K RAM Bits, uProcessor• Conversion from TI DSP to Xilinx Power PC core:
• Risk Evaluation Required
8
Silicon RevE(F) SiROD
MasterDSP
ControllerFPGA
FormatterFPGAs
Router FPGA
EventFragmentBuilderFPGA
SlaveDSPs
9U VME
VME Interface
Clock Distribution
9
ROD CONTROLLER
FPGAXC4VFX20
128MB SDRAM
EFB/ROUTER FPGA
XC4VLX60
DSP FARM TI 320C6713
220MHz 384MB SDRAM
VMEJ1J2
INPU
T FI
FO
32K
Dee
p
FORMATTER FPGA
XC3S4000
SiROD New Layout
BOCJ3
FORMATTER FPGA
XC3S4000
VME CONTROLLER
FPGAXC3S400
DSP FARM TI 320C6713
220MHz 384MB SDRAM
DSP FARM TI 320C6713
220MHz 384MB SDRAM
DSP FARM TI 320C6713
220MHz 384MB SDRAM
128Mb FLASH
JTAG CONNECTOR ON FRONT PANEL TO PROGRAM VME CONTROLLER FPGA ROM
10
WHERE DO WE GO FROM HERE?• Preliminary Design Study
• Recommendations:• Purchase 2 Xilinx Virtex4 Evaluation Boards
• One for the LX series• One for the FX series
• Combine the EFB/EventMemory/Router Designs to simulate and implement the proposed changes for verification
• Simulate and implement the Formatter as proposed for verification
• Simulate and implement the Controller to learn how to use the Power PC core and to plan a transition from the TI 6201 to the Xilinx PC core.
• Make the new design transparent to the DAQ system.• Simulate the Control path to understand the limitations of VME BW
11
Additional Slides
12
RODBus Interface
Control & Status Registers (32)
RODBus Data (16)RODBus AddressRODBus Control
Link Input MUX
&Half
Clock Counters
IN0
ModeBits FIFO 512x24
Link FIFO 2048 x 32
Link Decoder: 40/80/160MHz
Read Out Controller
DATA OUT (40)
TOKEN
FMT MB (12)
Link FIFO 2048 x 32
Link Decoder: 40/80MHz
Link FIFO 2048 x 32
Link Decoder: 40MHz
Link FIFO 2048 x 32
Link Decoder: 40MHz
IN1IN2IN3IN4IN5IN6IN7IN8IN9
IN10IN11
FMT TypeFMT ID
HT LIMIT
ROD BUSY
Trailer DetectTRAILER (12)
PIXEL FORMATTER FPGA (8x)
13
EVENT FRAGMENT BUILDER FPGA
EventMemA
FIFO16Kx48
Engine 0
Engine 1
EventMemC To/F
rom
Rou
terTo
/Fro
m F
orm
atte
rTo
/Fro
m F
orm
atte
rTo
/Fro
m C
ontr
olle
r
Output FIFO Data (43)
Header and Trailer Data (32)
Halt Output
Halt Output
Header/TrailerGenerator
Error Summary Word
Error Summary Word
Form
at &
Cou
ntFo
rmat
& C
ount
L1/B
C ID
Che
ckL1
/BC
ID C
heck
Erro
r Che
ckEr
ror C
heck
INC
/DEC
L1I
D
L1 & BC IDs
Event Data& Trigger Type
Dynamic Mask
Link Num. (6 bits)
Link Num. (6 bits)
Time-out Error
Time-out Error
L1ID Error
L1ID Error
BCID Error
BCID Error
Data (32 bits)
Data (32 bits)
Formatter Num. (2 bits)
Formatter Num. (2 bits)
Link Num. (4 bits)
Link Num. (4 bits)
Data (32 bits)
Data (32 bits)
Data Valid
Data Valid
Time Out
Time OutIN
C/D
EC L
1ID
Purpose: Collect Formatter output, check L1 and BC
IDs, count errors, generate Event Header & Trailer
FIFO Controller Xoff
EventMemB
FIFO16Kx48
14
ROD Router (FPGA) & Slave DSPs
SDSP 3SDSP 2SDSP 1SDSP 0
Read, formatand direct the
event data
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
DMA Transfer Engine1024 32-bit FIFO
DMA Transfer Engine DMA Transfer Engine DMA Transfer Engine
S-LI
NK
To/
From
RO
S
To/F
rom
Eve
nt F
ragm
ent B
uild
er
XOn / XOff S-Link
S-Link DataDsp Halt Output
Halt Output
Error Format Data
Data Valid
Output FIFO (43b)
Event Header andTrailer Data (32b)
Texas Instruments 6713 floating point DSPs running at 220 MHz for monitoring and calibration histogramming
Purpose: Route formatted data to Level-2 and/or Slave DSPs (histogram)
1024 32-bit FIFO 1024 32-bit FIFO 1024 32-bit FIFO