Data acquisition system on

Advanced TCA

M. Nomachi and S. Ajimura

Osaka University, Japan

CAMAC – FASTBUS– VME / Compact PCIWhat’s next?

ATCA JAN-2006 2

Rack

Crate

Module

Chip

MemoryMapped

I/O

CPU

Network

DAQ architecture & Downsizing

MemoryMapped

I/O

CPU

Network

CPU

Network

ATCA JAN-2006 3

CPU: merits and demerits

• Merits– Scalable, easy to communicate– Data base access

• Demerits– Boot– Overhead– Maintenance

ATCA JAN-2006 4

Rack

Crate

Module

Chip

MemoryMapped

I/O

CPU

Network

DAQ architecture

We choose memory mapped I/O in a crate.

It makes easy to maintain CPU.There were many bad experiences on embedded CPUs.

ATCA JAN-2006 5

産業界の動向

• VME– 6U Euro card DIN connector– BUS

• Compact PCI– 6U Euro card metric connector– BUS

• ATCA– 8U Euro card metric connector– Serial link

ATCA JAN-2006 6

From Schroff

ATCA JAN-2006 7

ATCA JAN-2006 8

ATCA JAN-2006 9

ATCA JAN-2006 10

There are 4 ports for one channel. One port has one output and one input.

In order to make signal extraction easy,2 ports are used. (SpW uses 2 ports)The other 2 ports are to extend band width. It makes possible to use 4-layers PCBs.

ATCA JAN-2006 11From Schroff

ATCA JAN-2006 12

ATCA JAN-2006 13

Advanced TCA• Features

– -48V DC power supply and on board DC converter for any voltages required.

– 8U x 280 mm– Dual Star point to point differential connection– No definite protocol.

• VME back plane is for VME protocol. While, ATCA back plane is only defined as 100 ohm differential. Any protocol can use ATCA back plane. ( for example, ATCA for physics instrumentation)

ATCA JAN-2006 14

Read outmodule

Triggermodule

Front-endmodule

DAQ system

Front-end

Pipe-linebuffer

readoutbuffer

Trigger logic

Trigger control

Globaltrigger

Readout control

networkSecondlevel

trigger

CPU

To event builder

ATCA JAN-2006 15

Dual star connection

Triggermodule

Read outmodule

Front-endmodule

Front-endmodule

Front-endmodule

Front-endmodule

Front-endmodule

Asynchronous signals(4 lines)•Trigger/Reset•Local trigger/Busy

SpaceWire link(4 lines)

CPU(T-kernel)

ATCA JAN-2006 16

Serial data transfer

フロントエンドで使用するためには• フロントエンドの小規模なＦＰＧＡに実装できる簡単なプロトコル。多くのシリアル転送ではクロックの再構築のためのＰＬＬが必要であるが、フロントエンドで使用するにはＰＬＬがなくても動作するプロトコルが望ましい。

• 高速レスポンスを実現するための短い遅延時間。パイプライン処理を行わないフロントエンドでは遅延時間がそのまま Dead Timeとなってしまう。このため余計な処理を行わない単純なプロトコルが望ましい。

• ノイズ源とならない信号レベル。デジタル信号の遷移はアナログ系のノイズ源となる。このため、ＬＶＤＳ等、振幅の小さい信号レベルが望ましい。

ATCA JAN-2006 17

ATCA JAN-2006 18Steve Parkes 2003

ATCA JAN-2006 19Steve Parkes 2003

非同期に動作するため、時間情報を保存して送ることが可能

ATCA JAN-2006 20Steve Parkes 2003

ATCA JAN-2006 21Steve Parkes 2003

ATCA JAN-2006 22Steve Parkes 2003

ATCA JAN-2006 23Steve Parkes 2003

ATCA JAN-2006 24Steve Parkes 2003

ATCA JAN-2006 25Steve Parkes 2003

ATCA JAN-2006 26Steve Parkes 2003

ATCA JAN-2006 27Steve Parkes 2005

ATCA JAN-2006 28Steve Parkes 2005

ATCA JAN-2006 29Steve Parkes 2005

ATCA JAN-2006 30Steve Parkes 2005

ATCA JAN-2006 31Steve Parkes 2005

ATCA JAN-2006 32Steve Parkes 2005

ATCA JAN-2006 33

Write Operation

Write Request

Write DataRequest

WriteCommand

WriteReply

Write CompleteConfirmation

Source Destination

Write DataAuthorisation

Write Data

Write DataIndication

Steve Parkes 2005

ATCA JAN-2006 34

Read Operation

Read Request

Read DataRequest

ReadCommand

ReadReply

Read DataConfirmation

Source Destination

Read DataResponse

Steve Parkes 2005

ATCA JAN-2006 35

SpaceWire

• Remote Memory Access Protocol (RMAP)– RMAP is a protocol on the SpaceWire

standard.– Register access has large overhead.– Block transfer works almost full speed.– It meets the requirements. Good enough.– PCI express and the other protocol may also

meet the requirements. However, they may use too much logic (x10) for front-end FPGAs.

ATCA JAN-2006 36

Interrupt on VME bus

InterrupterInterrupthandler

Interrupt RequestBroadcast

Read Vector

Communication between strangers.

ATCA JAN-2006 37

Userprogram

Device

Send return address

Device access

Wake up message

RMAP read

processing

Response in 10~100 micro sec

ATCA JAN-2006 38

Interrupt register(event flag)

Return address

timer

Wait event

Wake up message

Read wake up message -1 : Busy. Another process is already waiting. 0: Event already exist. n>0 : Event occurs after “n” tick.

Write interrupt register 0 : cancel

Hardware implementation

ATCA JAN-2006 39

ATCA crate

One crate holds 96ch of 500MHz FADCs. Total power is expected to be 260 W.

ATCA JAN-2006 40

Virtual CAMAC on serial data link

One crate holds 96ch of 500MHz FADCs. Total power is expected to be 260 W.

Crate controller

Serial data way

ATCA JAN-2006 41

Trigger module

16 LVDS in16 LVDS out8 NIM in8 NIM out

Power consumption is about 10W

Cyclone EP1C12 for trigger logicCyclone EP1C6 for SpW100Mbps SpW

ATCA JAN-2006 42

500 MHz FADC

Cyclone EP1C6Cyclone EP1C12 for router100Mbps SpW (8~9 MB/s from the module)Readout buffer with 128Mb SDRAM waiting second level trigger

8 ch analog inputFADC mezzanine card is developed at KEK (FINESSE format)

Power consumption is About 20W

ATCA JAN-2006 43

Summary

• SpaceWire Remote Memory Access Protocol provides compact and flexible interconnection in a module and inter-module connection.

• Advanced TCA provide dual star LVDS connections. They are good to be applied for DAQ system

• Downsizing may continue.– We might have another solution in the

future.