consideration of the lar ldps for the mm trigger processor
DESCRIPTION
Consideration of the LAr LDPS for the MM Trigger Processor. Kenneth Johns University of Arizona Block diagrams and some slides are edited from those of Guy Perrot (LAPP). Overview. - PowerPoint PPT PresentationTRANSCRIPT
Consideration of the LAr LDPS for the MM Trigger Processor
Kenneth JohnsUniversity of Arizona
Block diagrams and some slides are edited from those of Guy Perrot (LAPP)
2
• The LAr Calorimeter upgrade reduces the L1 EM object trigger rates by using finer granularity information (SuperCells versus Towers) as input to the L1 Calo Trigger system– Exploits differences in shower shape between signal and
background
• Data flow– SuperCell signals are digitized on-detector at 40 MHz (320 SC
per LTDB (LAr Trigger Digitizer Board)– Data is optically transmitted to the Back-end electronics at 5.12
Gbps (using the LOCx2 and LOCld)– There, the ADC data is converted to energy and time and sent to
the L1 Calo Trigger System and TDAQ using the LDPS (LAr Digital Processing System)
Overview
• LTDB = ADDC– LAr Trigger Digitizer Board = ART Data Driver Card
• LOCx2+LOCld = GBT• LDPS = MM Trigger Processor
– LAr Digital Processing System = MM Trigger Processor
LAr to Muon Dictionary
MM Trigger Processor
4
ADDC(ARTData
DriverCards)
ART Data @ 40 MHz
MM Trigger Processor Blades
L1 Muon Sector Logic @ 40 MHz
GBT Links
ADDC configuration and monitoring, TTC
TTC Partition
Partition Master
PCATCA System
Manager
10/40GbE via fabric
TTC
ROS PCsTDAQ To
HLT
Data Monitoring
PCs
DCS
FELIX
To shelf managers and Base interface (GbE)
FE
FELIX
TTC
DCS
GBT LinksVia Zone 3
Busy
ADDC Config & MonitoringBE
Configuration & Monitoring
MM Trigger Processor programming,configuration, monitoring
• LAr LDPS blade– Input: 40 (48) x 4 optical fibers at 5.12 Gbps– Output: 40 (48) x 4 optical fibers at ~10+ Gbps to L1 Calo– Output: Input data and results to TDAQ– Output: Input data and results to private Monitor PCs
• MM Trigger blade– Input: 32 (36) x 4 optical fibers at 4.48 Gbps (Wide Bus mode)– Output: 1 x 4 optical fibers at 4.48? Gbps– Output: Input data and results to TDAQ– Output: ?? Input data and results to private Monitor PCs
LAr and Muon Backend I/O
6
• ADDC to AMC data flow• AMC to Muon Sector Logic data flow• GbE Base Interface
– Board Configuration– Board Monitoring– Histogramming
• 10GbE Fabric interface– Data monitoring (Fast)
• GBT links– TTC/Busy– TDAQ Data
• JTAG• IPM Bus
– Only ATCA management!
• Power
MM Trigger Processor Communications
MM Trigger Processor
7
AMC
AMC
AMC
AMC
1/10 GbE Switch
Zone 2 : Base Zone 1
Zone 2 : FabricZone 3: RTM
Ch1 1GbE
Ch21GbE
ATCAPower400W
IPMC
1 GbE
TTC
1 GbE
TTC
1 GbE
TTC
1 GbE
TTC
XAUI 10Gbps (PCIe?)
ATCA Switches
(AMC Reprogramming,AMC Configuration,AMC Monitoring)
ATCA Shelf ManagerI2C IPMB
Ch210GbE
Ch1 10GbE 4
4
1
14
FPGA16
1
1
4
4
4
4
1
4
4
4
4
16
RJ45IPML,JTAG,Ref Clks…
XAUI (PCIe?)
1 GbE
IPML, JTAG, I2C
SensorsI2C
GbE (Debugging)
TTC
µ POD Tx (x4)
µ POD Rx (x4)
1 Tx links
µ POD Tx (x4)
µ POD Rx (x4)
µ POD Tx (x4)
µ POD Rx (x4)
µ POD Tx (x4)
µ POD Rx (x4)
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
4
Up to 4 GBT
connectors
1
1
1
FELIXGBT
GbE
ATCA Shelf Power 48VPower
(Data Monitoring)
ATCA Switches
(ATLAS Event-TDAQ)(TTC)
10/100MbE
32 Rx links
1 Tx links
32 Rx links
1 Tx links
32 Rx links
1 Tx links
32 Rx links
GBT
1GbE
(ATCA Management, Monitoring)
MMC
MMC
MMC
MMC
4
GBT
GBT
GBT
GBT
10GbE
1
11
XAUI 10Gbps (PCIe?)
XAUI 10Gbps (PCIe?)
XAUI 10Gbps (PCIe?)
1
4GBT
1 GbE
JTAG JTAG JTAG
I2C
IPML,JTAG,Ref Clks…
IPML,JTAG,Ref Clks…
IPML,JTAG,Ref Clks…
Ref Clks(TTC, XAUI…)
1
1
JTAGMuon Sector Logic
Data @ 40 MHz
Muon Sector Logic
Data @ 40 MHz
Muon Sector Logic
Data @ 40 MHz
Muon Sector Logic
Data @ 40 MHz
ADDC
Data @ 40 MHz
ADDC
Data @ 40 MHz
ADDC
Data @ 40 MHz
ADDC
Data @ 40 MHz
8
• ADDC to AMC data flow• AMC to Muon Sector Logic data flow• GbE Base Interface
– Board Configuration– Board Monitoring– Histogramming
• 10GbE Fabric interface– Data monitoring (Fast)
• GBT links– TTC/Busy– TDAQ Data
• JTAG• IPM Bus
– Only ATCA management!
• Power
MM Trigger Processor Communications
• Design (Stony Brook) and layout (BNL) are complete• Fabricated AMC board expected before 12/1• All parts in hand except MicroPOD optics, optical
cables, 40 MHz clock and front panels• Firmware support (Arizona) in progress
– Perhaps available to users May 2014
• Longer term, LAPP holds responsibility for ATLAS AMC design and production– Unclear if LAPP design will follow AMC Demonstrator design
AMC Demonstrator Status
• Carrier demonstrator cards (including IPMC) with Altera FPGA have been produced by LAPP and will be made available to Stony Brook and Arizona soon
• LAPP Carrier demonstrator card design currently being translated into Stony Brook PCB design software
• Stony Brook will design a second Carrier card demonstrator with a GbE switch and increased power for AMC slots– Perhaps available May 2014
• Longer term, Stony Brook, BNL, Arizona hold responsibility for ATLAS Carrier design, production and firmware
AMC Carrier Card Status
11
AMC Parts Layout (top)
Xilinx Vertex-7
4 micro-POD arrays2x12 receivers and2x12 transmitters each
DDR3 RAM chips
12
AMC Parts Layout (bottom)
Clock chips
13
• 16 layers total with thru vias• Material N4000-13EP
AMC Routing (top layer)
14
• The LAr LDPS is a developed candidate for the MM Trigger Processor– Pros
• Takes advantage of an existing R&D program to develop AMC cards using high density Avago MicroPODs and FPGAs for Serdes and supporting Carrier cards
• Though not final, data paths are defined for TDAQ, TTC, configuration, monitoring, etc.
• AMC and Carrier card demonstrators well along and both AMC and Carrier cards will be available for users by May 2014
– Cons• Output fibers for Muon Sector Logic << Output fibers for L1 Calo
trigger• Monitor stream (10 GbE out) probably overkill• Unknown compatibility with sTGC Trigger Processor
Conclusions
15
Backup
LAr LDPS
16
LTDBs
ADC Data @ 40 MHz
LDPBs
L1 Calo Data @ 40 MHz
GBT Links
LTDB Configuration & Monitoring, TTC
TTC Partition
Partition master PCATCA System
Manager
10/40GbE via fabric
TTC
ROS PCsTDAQ To
HLT
Data Monitoring
PCs
DCS
FELIX
To shelf managers and Base interface (GbE)
FE
FELIX
TTC
DCS
GBT LinksVia
Zone 3
Busy
LTDB Configuration & MonitoringBE
Configuration & Monitoring
LDPB programming,configuration, monitoring
LDPB Block Diagram
17
(e/jFEX Data@40MHz)
AMC
AMC
AMC
AMC
1/10 GbE Switch
Zone 2 : Base Zone 1
Zone 2 : FabricZone 3: RTM
Ch1 1GbE
Ch21GbE
ATCAPower400W
IPMC
1 GbE
TTC
1 GbE
TTC
1 GbE
TTC
1 GbE
TTC
XAUI 10Gbps (PCIe?)
ATCA Switches
(LDPB Reprogramming,LDPB Configuration,LDPB Monitoring)
ATCA Shelf ManagerI2C IPMB
Ch210GbE
Ch1 10GbE 4
4
1
14
FPGA16
1
1
4
4
4
4
1
4
4
4
4
16
RJ45IPML,JTAG,Ref Clks…
XAUI (PCIe?)
1 GbE
IPML, JTAG, I2C
SensorsI2C
GbE (Debugging)
TTC
µ POD Tx (x4)
µ POD Rx (x4)
48 Tx links
(ADC Data@40MHz)
(e/jFEX Data@40MHz)
LTDB
µ POD Tx (x4)
µ POD Rx (x4)
µ POD Tx (x4)
µ POD Rx (x4)
µ POD Tx (x4)
µ POD Rx (x4)
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
FPGA(DSPSOC)
48
48
4
Up to 4 GBT
connectors
1
1
1
FELIXGBT
GbE
ATCA Shelf Power 48VPower
(Data Monitoring)
ATCA Switches
(ATLAS Event-TDAQ)(TTC)
10/100MbE
48 Rx links
48 Tx links
48 Rx links
48 Tx links
48 Rx links
48 Tx links
48 Rx links
L1Calo
(ADC Data@40MHz)
LTDB
L1Calo
(ADC Data@40MHz)
(e/jFEX Data@40MHz)
LTDB
L1Calo
(ADC Data@40MHz)
(e/jFEX Data@40MHz)
LTDB
L1Calo
GBT
1GbE
(ATCA Management, Monitoring)
MMC
MMC
MMC
MMC
4
GBT
GBT
GBT
GBT
10GbE
1
11
XAUI 10Gbps (PCIe?)
XAUI 10Gbps (PCIe?)
XAUI 10Gbps (PCIe?)
1
4GBT
1 GbE
JTAG JTAG JTAG
I2C
IPML,JTAG,Ref Clks…
IPML,JTAG,Ref Clks…
IPML,JTAG,Ref Clks…
Ref Clks(TTC, XAUI…)
1
1
JTAG
18
• 124 LTDB’s total – 320 SuperCell channels per LTDB– 8 channels per fiber– 40 fibers per LTDB– 1 LTDB per AMC
• 124 AMC cards total– 4 AMC per Carrier Card– 31 Carrier Cards spread over 3 ATCA shelves
LAr Accounting
19
• ADDC to AMC– AMC pigtails– Connection to patch panel ?
• LDPB to Muon Sector Logic– Connector choice
• Base Interface: GbE (2 ports)– What: Reprogramming, Control, Monitoring, Histos?– How: switch between base interface and all components (AMC’s FPGA,
Carrier FPGA)?
• Fabric Interface: 10GbE/40GbE? (2 ports)– What: Data monitoring by PC farm.– Where is it connected to: Carrier FPGA, switch and AMCs?– If connected to Carrier FPGA, what is between AMCs and Carrier FPGA?
Interfaces (1)
20
• GBT links– What: TTC, Busy, Data from AMC to TDAQ (FELIX)– Where to: AMC, Carrier FPGA– How many?– TTC clock extraction– TTC command transmission & Decoding.
• IPM Bus– Only ATCA functions?
• Power– Amount per Carrier, per AMC
Interfaces (2)
21
• IPML Bus /ATCA signals– Anything special?
• JTAG Bus– Keeping the chain when no AMC present?
• TTC/Busy ?– Clock– Commands– Busy
• GbE• Fast links
– 10 GbE, PCIe…
• GBT• Power
AMC/Carrier Interface
AMC
22
1GbE
IPM-L
JTAG
TTC
XAUI (PCIe?)
4FPGA
DDR3
FLASH12
12
12
12
12
12
12
12
4x12 Optics Tx
4x12 Optics Rx
DC/DC;LDO
Oscillators
MMC
SensorsSDRAM
Optical fibres AMC connector
1
Power : 12V3.3V-IPMI
PLL
L1Calo e/jFEX Data @40MHz
GBT1
LTDBADC Data@40MHz
Ref Clks(TTC, XAUI…)
Reset
4
LDPS TDR
23
LTDBs
ADC Data@40MHz
LDPBs
L1Caloe/jFEX Data@40MHz
GBT Links
LTDB Configuration & Monitoring, TTC
TTC Partition
PM PCATCA System
Manager
Data Monitoring
TTC
To TDAQ
PC Farm
DCS
FETTC
DCS
GBT Links
Busy
LTDB Configuration & Monitoring
BE
ATLAS Event-TDAQ, TTC
10/40GbE Network
ATLAS Event-TDAQ
LDPB Reprogramming & Configuration & Monitoring
TDAQ Network
TDAQ Network
GbE NetworkShelf
Managers
ATCA Management & Monitoring DCS
FELIX
FELIX
Custom Links
Custom Links