sPHENIX Cost and Schedule Review 1

Calorimeter Digitizer Electronics

Cheng-Yi ChiColumbia University

Nov 9-10, 2015

Cheng-Yi Chi 2

Off-Detector Calorimeter Digitizer electronics

• Function• Past experience• System block diagram• ADC board block diagram, cable test result,

differential receivers, ADC choice. Comments on trigger primitives

• Progress on the Prototype• Issue/Concern

3/25/2015

sPHENIX Cost and Schedule Review 3

Calorimeter Digitizer electronics function

• Digitizing the signal after the on-detector electronics with single scale 14 bit ADC.– Try to maintain the best dynamic range as good

we can do with single scale ADC. We are looking for 12 bits dynamic range.

• Interface with PHENIX DAQ system– Receive beam clock, L0 timing and L1 trigger– Provide the 5 L1 trigger events buffer.– Provide the functionality to generate L1 trigger

primitives.

Nov 9-10, 2015

Cheng-Yi Chi 4

Past Experience

FEM Board

Differentialreceiver

ADC(TI’s ADS5272)

ALTERAFPGA

HBD Pedestal Run

Width s (ch)

Width s (ch)

Mean (ch)

Mean (ch)

Channel #

Channel #

Built PHENIX Hadron Blind Detector (HBD) off-detector digitizer (12 bits, 60 MHz)

also been use for the MPC readout electronics and Sphenix test beam readout

MicroBoone TPC readout (BNL+Nevis) and PMT readout (similar shaper+ADC like HBD, 64 MHz)

HBD ADC module

3/25/2015

Cheng-Yi Chi 5

Controller

clockmaster

GTMJSEB II

PC

ADC ADC XMIT

Triggerout

Triggerout

DCM II

crate

(rack based)

Slow control/readback

Beam clock,L0, L1 trigger

PHENIX DAQ System

detector

Slow control/readback bus, L0, L1 trigger

Token passingdataway

Token passingdataway

Clock are fanout point to point through the backplane

ADC System Block Diagram

Crate based system.

Signals are cable from the on-detector electronics.

Digitized with 14 bit ADC.

Receive timing information the PHENIX Granule Timing Module (GTM) Generate L1 trigger primitives(not in the baseline scope)

Receive L1 trigger and send out L1 triggered event data to Data Collection Module II (DCMII).

Provide buffer for both the 40 beam crossing L1 delay buffer and 5 L1 triggered events

4 ADC XMIT

3Gbit/secoptical link 1.6Gbit/sec

optical link

3/25/2015

7X ADC clock

FRAME

Serialized AD

C DATA De-serializer

Serialcommand

L1DelayBuffer

5 L1Accepted

Eventbuffer

TokenPassing

Data

ALTERA Arria V GX BB1D4F35

1152 pins FPGA

Analog Device

AD92578 channel

14bits65 MHz

ADC

ClockFanout

Differentialreceiver

6 X beam clock

2 mmHard Metricor mini-SAS Connector

Analog Device

AD92578 channel

14 bits65 MHz

ADC

2 mmHard Metric or min-SASConnector

Differentialreceiver

Serialized AD

C DATA

FRAME

7X ADC clock

64 channel ADC board

6x beam clockBeam phase,

init, L1

Serialized Token

Passing

command/offline data readDe-serializer

Bused commandSerial data

+- 3.5V, -2.5VDifferential

receiver power

+1.8V analogADC power

+1.8V digitalADC power

DC/DCswitch regulator

+4V, -3.5V, +2.5VPower

+12VPower

DC/DC switch regulator

1.5VLDO 1.1V

core

4V

2.5V FPGA I/O

3.3VLDO

3.3VLDO

DPU

two transceivers

two transceivers

sPHENIX ADC Module Block Diagram

L1 trigger primitives transceiver out

3/25/2015

Cheng-Yi Chi6

RHIC beam clock 9.6MHz

Cheng-Yi Chi7

Analog device AD9249 16 channel 14 bits ADC. Maximum sampling rate 65 MHz SNR 75db 1.8v technology. 58mw per channel at 65 MHz -> 1 W per chip. 144 pins package. 1cm X 1cm BGA pipeline latency 16 clocks. Analog Device AD9257

8 channel 14 bits ADC Maximum sampling rate 65 MHz SNR 75.5 db1.8v technology. 55mw per channel at 65 MHz

65 pins LFCSP package. 0.9mm by 0.9mm.pipeline latency 16 clocks.

Texas instrument ADS52948 channel 14 bits ADC. Maximum sampling rate 80 MHz SNR 75.5db1.8v technology. Per channel 58mw at 50 MHz, 77mw at 80 MHz.

1-wire only interface only for below 50 MHz sampling 80 pins QFP package. 12mm by 12mm

included digital processing block ( only after digitization)pipeline latency 11 clocks for 1 wire interface.

Linear Technology LTM9008-148 channel 14 bits ADC. Maximum sampling rate 65MHz SNR 73 db1.8v technology. Per channel 88mw at 65 MHz.140 pins BGA. 11.25mm X 9mm pipeline latency 6 clocks.

The limit of ADC LVDS serializerseems to be less than 1Gbits/sec 65 MHZ ADC

The FPGA does not have 128 LVDS De-serializer 1 LVDS outputper ADC channel

JSED204B’s ADC need the transceiver to receive data Limits number of ADC can be connected to the reasonable price FPGA

The Choice of ADC

3/25/2015

sPHENIX Cost and Schedule Review 8Nov 9-10, 2015

The ADC system is 14 bits (0- 16383 ADC counts)

The ADC input is differential, need to supply both + and – inputs ( peak-peak range is 2V)

(122 mV per ADC counts)+1V on the positive side and –1V on the negative side8192 ADC count happened when V+, V- difference is zeroOur signal only swing one side

To get full range we need to offset the signals. (the point to introduce the noise)

Cheng-Yi Chi9

2mm HM Input

connector

Differential receivers

AnalogDevice AD925714 bits

ADC

Analog Power

TriggerPrimitives

output

SerialCommand

output

SerialCommand

input

Clock inL0/L1

Digital Power

TokenPassing

Data In/out

sPHENIX ADC board

64 channel inputsMini-SAS / 2mm HM connectors160mm by 190mm (6U board)

The 8 channel 14 bits ADC will be running at 6x beam crossing rate ~ 60 MHz

Altera Arria 5GX FPGA de-serialize data, provide 5 events buffer, 40 beam crossing L1 delay, token passing data, L1 trigger primitives output

Will do a version of 2mm HM connector 1st choice.

ALTERA Arria 5GX

FPGA

3/25/2015

Cheng-Yi Chi 10

Thoughts on the Trigger Primitives.

FEM can do rough gain correction We will do on board 2x2 sum firstIf trigger primitive output will be 8 bits, for 40 Gev top scale, the least count will be ~156 Mev if the scale is linear.

If we output 8 bits per channels, the optical bandwidth will be 10 ( 8b/10b encoding) *64 * 10 MHz = 6.4 Gbits/sec. if one only output 2X2 sum, we will only have 1.6 Gbits/sec. With formatting, the output will be 2 Gbits/sec. ( 1 frame marker + 1 header + 8* (2* 8) data words.)

if trigger sum is 10 bits, we will have ( 1 frame marker + 1 header + 10 16 bits data words.), The opticalBandwidth will be 20 (bits)*12 words* 10 MHz = 2.4 Gbits/sec

The FPGA has a high speed transceiver port to the backplane with high speed LVDS repeater. A rear mounted optical transceiver could be used to send out L1 trigger primitives.

3/25/2015

sPHENIX Cost and Schedule Review 11

Prototype Progress

• ADC Module, Crate Controller and Backplane prototype modules have be build.– This set is enough for us to test analog function.• Can we safely built 64 channel ADC module with

reasonable performance?• Will offset circuit introduce more noise into the

system?

– The XMIT module is on the way

Nov 9-10, 2015

3/25/2015

Cheng-Yi Chi12

Sphenix ADC backplane

Sphenix ADC m

odule

Sphenix crate controller

3/25/2015

Cheng-Yi Chi13

ADC Board Performance (baseline, no signal)

Channel 1 without offset

adc

sigma

Channel number

Channel number

nhitsnhits

nhitsnhits

adc

adc

adc

adc

Ch 0

Ch 1

Ch 2

Ch 3

Baseline(average ADC)

Baseline(sigma)

3/25/2015

Cheng-Yi Chi14

adc

adc

adc

Baseline(average ADC)

Channel number

Channel number

Baseline(sigma)

sigma

Sample number

Sample number

Channel 0

Channel 1

ADC noise test with old detector HBD preamp

sPHENIX Cost and Schedule Review 15

Cross Talk Study

Nov 9-10, 2015

adc

adc

adc

adc

Sample number

Sample number

Sample number

Sample number

Channel 9

Channel 10

Channel 11

Channel 12

Cross talk with HBD preamp(similar rise time)

sPHENIX Cost and Schedule Review 16

Production Flow

• Last prototype cycle of the electronics should dress production needs.– Testing, yield etc.

• Parts and Printed Circuit Boards (PCB) procurements.– Work with distributors on getting the parts.– Work with prototype PCB vendors.

• Board assembly.– Have a preproduction run.– Work with prototype board assembler.– Schedule the board delivery to match with testing flow.

• Feedback to assembler.

• Testing– Set up at least 2 test stands.

• One for testing. The other one for debugging.

– Probably can do at least 10 boards a day.

Nov 9-10, 2015

sPHENIX Cost and Schedule Review 17

Cost of the system

• The design of the digitizer is trying to minimize the cost of the system.– Compact 6U design.

• Save space reduce the PCB cost. Saving the rack space. Easy handling.

– Try to use low cost FPGA to all ADC data, event data flow and Level 1trigger primitives.

• Except for the clock master module to interface with GTM system, we have design for reset of the system excluding the trigger portion. – We have ask our vendors for budgetary quotes.

Nov 9-10, 2015

sPHENIX Cost and Schedule Review 18

Status and Outlook

• Prototype modules of ADC, Controller and Backplanes has been build. – Testing ongoing. So far it is O.K.– XMIT module will be done before end of the year.

• Prepare to use the new system for upcoming testing beam– readout system with 2 ADC modules

• Need to test trigger primitives output.• Need to address the test system for the ADC analog

inputs.• Some modification is probably necessary for the 2nd round

prototype. – Enhance testing features.

Nov 9-10, 2015

sPHENIX Cost and Schedule Review 19

Backup Slides

Nov 9-10, 2015

Cheng-Yi Chi20

out-out+

Input +

Input -

Differential Receiver

The ADC module will receive the signal from the On detector module.

The signal will be AC coupled. The blocking capacitor will be on the frontend.

The ADC’s Vcommon is at 1V. With the signal swing +-0.5v among Vcommon. To get full rangeof the ADC, the signal need to swing in both direction. The signal from the detector only swing in one direction.

We offset of the differential receiver to push the baseline to the lower ADC range.

The resistors are discrete components. Supply voltages are adjustable through resistors.

V+

V-

3/25/2015

500

Cheng-Yi Chi21

2mm HM cable 10 meters10 meter Amphenol mini-SAS extend distance cable

SPHENIX cable cross talk study

~0.25% ~0.05%

3/25/2015

Cheng-Yi Chi22

clock in

Serialized L0, L1 Data in

USB 3.0 connectors

Optical link fromJSEB II (3 Gbits/sec)

readback

Slow control/download

Power

OfflineClock

crystal

Crate Controller

Receive clock and L0 (init, test etc), L1 trigger.

Receive slowdown load from DAQ system. Write ADC’s module FPGA boot code to

the EPROM. Write ADC, XMIT & Controller modules

running parameters. Download varies table. Download fake data Initialize system. Set online/offline system

Provide offline clock/L0 data, L1 trigger. (standalone crate running)

Provide readback from ADC module to computer.

3/25/2015

Cheng-Yi Chi23

Analog Power Section (ADC power, +- differential receiverpower)

Digital Power Section

Slow control/download bus

Readback bus

Token Passing Dataway Token Passing Dataway

Crate Controller

Point to Point clock fanout

Crate Backplane

3/25/2015

Cheng-Yi Chi24

Digital Power

Clock in

TokenPassing

Data In/out

TokenPassing

Data In/ token

out

SerialCommand

input

XMIT Module

Optical transceiver To DCM II

(1.6 Gbits/sec)

. Receive data from FEM

. Issue token after receive all data transfer

. Format FEM data ( header and checksum)

. Error handling

Altera Cyclone IV

FPGA

3/25/2015