firmware and software architecture for pixel

Xiangming Sun 1PXL Sensor and RDO review – 06/23/2010

STARSTAR

XIANGMING SUN

LAWRENCE BERKELEY NATIONAL LAB

Firmware and Software Architecture for PIXEL

L. Greiner,

H. Matis

T. Stezelberger

M. Szelezniak

C. Vu

H. Wieman

… …


STARSTAR Outline

• Requirements for event readout mode• Event readout mode• Other firmware functions• Firmware and readout software structure• Summary


STARSTARChip set

sensor output frequency

Mimostar2 analog 50MHz

Mimostar3 analog 50MHz

Phase1/2 4 LVDS pairs and analog

160MHz

Ultimate 2 LVDS pairs 160MHz


STARSTAR

Individual sensor test - A single sensor is connected. Probe test - The sensors are put on probe station and the power supply is controlled. Infrastructure Test Board (ITB) test - 10 sensors are connected in parallel, long cables are used. Beam test - 4 sensers are built into a telescope. The trigger is included. Phase-2 sector test - More than 1 sector is connected. The trigger is included. Final version - 40 sensors are connected. Long cables are used. The trigger is included.

Firmware structure is designed to handle all these tests. The set of firmware modules for the various tests will be very similar with only minor task dependant differences.

Test set


STARSTAR Rrequirement for event readout mode

40 sensors per readout board( Each sensor has 2 LVDS pairs. 80 LVDS pairs in 160MHz).

Sensor digitizes signal and records hit address in on-chip memory, then transfer hit address serially.

Trigger is STAR DAQ1000(about 1KHz).

DAQ DDL assumed throughput is 80MB/s on average. (DDL link rated for 200MB/sec).

requirement :mass terminition board

mother board

xilinx V5 PRO

SIU/DDL

DAQ PC


STARSTARData is buffered in FPGA

STAR and DAQ policy: 1 trigger per event

DDL peak speed(2Gb/s) is lower than data generation peak speed (12.8Gb/s).

TPC can buffer 8 triggers when DDL is not available. PIXEL should buffer ≥8 triggers

frame1frame1 frame2frame2 frame3frame3

Trigger time Event data


STARSTAREvent format

7

For sector

header

ender

Data blockFor ladder1




Header token

Trigger info

system status

temperature

Firmware version

Hardware ID

…….10 words reserved

Block start Data length

…Hit address

end token

sensor ID is encoded in four unused bits


STARSTAR

Ladder#1 Ladder#2 Ladder#3 Ladder#4

Event Builder #1

(head token added)

Event Builder #2

Event Builder #3

Event Builder #4(end token

added)

Event fifo

SIU

Event mode Readout structure for one sector


STARSTAREvent Builder

Serial to parallel conv Serial to parallel conv… 10 sensors

data memory

Trigger time memory

start frame memory

Builder state machine

To event fifo

circular bufferSensor 1

output1 output2


STARSTARSerial to parallel converter

Serial to parallel conv Serial to parallel conv

… 10 sensors

datamemory

Serial-to-parallel converter1, convert 160MHz 1 bit data to 10MHz 16bit parallel data.2, add sensor ID into the data.

Data memory scans Serial-to-parallel converter one by one every 10 ns (100MHz) . It starts from sensor 1 and returns to sensor 1 after reaching sensor 10.

From sensor1From sensor1


From sensor…From sensor…


From sensor…From sensor…


Sensor 1

output1 output2



STARSTARCircular buffers

datamemory

Trigger time memory

start frame memory

frame counter when trigger comes

index of start frame memory

index of data memory for the first frame

index of data memory for the second frame

64K words

512 words

512 words

trigger token and buffer full

frame1

frame2


STARSTARData check out

datamemory

Trigger time memory

start frame memory

frame1

frame2

get trigger timeand start frame index

get buffer index for 2 frames

Check out region where hit address bigger than trigger time for frame1 and smaller for frame2

Release unused memory.


STARSTARBuffering capability

• 19K words/trigger (684kb).( buffer length in chip is assumed to be 500 words and 150 word for inner and outer ladder from simulation ) . Current FPGA has 3Mb for event mode. XC5VLX330 has 8Mb.

• 256 continous triggers can be stored (data memory may be full already)• bottleneck is DDL. 39MB/s for trigger DAQ1000

device Total room Room used usage

sensor 2000word 500 / 150 25% / 7.5%

FPGA data 8Mb 684kb*2 17%

FPGA trigger 512 word 2word*2 1%

DDL 80MB/s 39MB/s 48%

In normal condition, memory and DDL usage

10 triggers can be buffered when the DDL link is busy.


STARSTAROther firmware functions

• JTAG function (all sensors)

• Analog full frame readout (Mimostar3 and Phase1)

• Digital full frame readout (Phase1 and Ultimate)

• I2C device control (Phase1 and Ultimate)

• System monitoring (Phase1 and Ultimate)


STARSTARFirmware structure

15

System control can access every module

DDL/USB PC

sensorsensor


STARSTAR JTAG function (all sensors)

JTAG is used for initializing the sensor.

JTAG is done by the FPGA in Virtex5 development board.

We reserve 5 FPGA pins as JTAG pins which connect to sensor’s JTAG pin serially. The JTAG data is sent from PC to FPGA SRAM. The JTAG module in FPGA will read the data from FPGA SRAM and transmit the data to sensor by JTAG pins.

sensorsensor


STARSTARAnalog full frame readout

analog signal is digitized in ADCADS5272

sensor

ADS5272300MHz, IOdelay is used

data is buffered in SRAM


STARSTAR IO delay for digital data alignment

1, set sensor to pattern mode, a single pulse is sent from sensor for each row.2, scan IO delay setting from 0 to 63. readout data in event mode.3, search for the pulse in data taken. plot the relation between pulse address and IO delay setting.4, find the transition region in above plot. set IO delay by shifting a half clock.

Digital signals pass 7 meters before arriving FPGA.How the digital signals from all sensors are aligned?

Scan steps0 63

NN+1

half clock

pulse

1 clock cycle

Transition region

Iodelay set


STARSTAR Digital full frame readout

sensorsensor


STARSTARI2C voltage control and measurement


STARSTARSystem monitoring

We have a separate data channel to monitor the system in physics run.

This channel includes a information builder module in FPGA, USB data channel and a monitor PC. If something wrong happens, we can easily see it from monitor PC.

sensorsensor


STARSTAREvent mode


STARSTARReadout software structure

command: generate commands in windows and linuxusb.exe :upload commands to FEE in windowsdownload_data_block_to_FEE: upload commands to FEE in windows getdata:get frame data in windows and linuxrorc_receive: get event data in linuxIO delay package: tune IOdelay for each digital channelI2C package: control I2C device

Design Goal:portable between Linux and Windowsevery test can be scripted


STARSTARSystem Control

command Hex file Hex file

usb.exedownload_data_block_to_FEE0x0402fffd0x1d82ff3f0x1502ffcf0x2642ffff0x2642fdff0x2202feff0x0c03fff00x1547ffff0x1547ffff0x1547ffff0x1547ffdf0x0cc7ffff0x0cc7ffff………….

LinuxPC

LinuxPC

rorc_receive getdata

WindowsPC

WindowsPC


STARSTARSummary

• The design meets the requirements as stated.

• The full set of expected operation modes are addressed.


STARSTAR

backup


STARSTARI2C voltage control

firmware and software architecture for pixel

Documents