Data acquisition system on
Advanced TCA
M. Nomachi and S. Ajimura
Osaka University, Japan
CAMAC – FASTBUS– VME / Compact PCIWhat’s next?
ATCA JAN-2006 2
Rack
Crate
Module
Chip
MemoryMapped
I/O
CPU
Network
DAQ architecture & Downsizing
MemoryMapped
I/O
CPU
Network
CPU
Network
ATCA JAN-2006 3
CPU: merits and demerits
• Merits– Scalable, easy to communicate– Data base access
• Demerits– Boot– Overhead– Maintenance
ATCA JAN-2006 4
Rack
Crate
Module
Chip
MemoryMapped
I/O
CPU
Network
DAQ architecture
We choose memory mapped I/O in a crate.
It makes easy to maintain CPU.There were many bad experiences on embedded CPUs.
ATCA JAN-2006 5
産業界の動向
• VME– 6U Euro card DIN connector– BUS
• Compact PCI– 6U Euro card metric connector– BUS
• ATCA– 8U Euro card metric connector– Serial link
ATCA JAN-2006 10
There are 4 ports for one channel. One port has one output and one input.
In order to make signal extraction easy,2 ports are used. (SpW uses 2 ports)The other 2 ports are to extend band width. It makes possible to use 4-layers PCBs.
ATCA JAN-2006 13
Advanced TCA• Features
– -48V DC power supply and on board DC converter for any voltages required.
– 8U x 280 mm– Dual Star point to point differential connection– No definite protocol.
• VME back plane is for VME protocol. While, ATCA back plane is only defined as 100 ohm differential. Any protocol can use ATCA back plane. ( for example, ATCA for physics instrumentation)
ATCA JAN-2006 14
Read outmodule
Triggermodule
Front-endmodule
DAQ system
Front-end
Pipe-linebuffer
readoutbuffer
Trigger logic
Trigger control
Globaltrigger
Readout control
networkSecondlevel
trigger
CPU
To event builder
ATCA JAN-2006 15
Dual star connection
Triggermodule
Read outmodule
Front-endmodule
Front-endmodule
Front-endmodule
Front-endmodule
Front-endmodule
Asynchronous signals(4 lines)•Trigger/Reset•Local trigger/Busy
SpaceWire link(4 lines)
CPU(T-kernel)
ATCA JAN-2006 16
Serial data transfer
フロントエンドで使用するためには• フロントエンドの小規模なFPGAに実装できる簡単なプロトコル。多くのシリアル転送ではクロックの再構築のためのPLLが必要であるが、フロントエンドで使用するにはPLLがなくても動作するプロトコルが望ましい。
• 高速レスポンスを実現するための短い遅延時間。パイプライン処理を行わないフロントエンドでは遅延時間がそのまま Dead Timeとなってしまう。このため余計な処理を行わない単純なプロトコルが望ましい。
• ノイズ源とならない信号レベル。デジタル信号の遷移はアナログ系のノイズ源となる。このため、LVDS等、振幅の小さい信号レベルが望ましい。
ATCA JAN-2006 33
Write Operation
Write Request
Write DataRequest
WriteCommand
WriteReply
Write CompleteConfirmation
Source Destination
Write DataAuthorisation
Write Data
Write DataIndication
Steve Parkes 2005
ATCA JAN-2006 34
Read Operation
Read Request
Read DataRequest
ReadCommand
ReadReply
Read DataConfirmation
Source Destination
Read DataResponse
Steve Parkes 2005
ATCA JAN-2006 35
SpaceWire
• Remote Memory Access Protocol (RMAP)– RMAP is a protocol on the SpaceWire
standard.– Register access has large overhead.– Block transfer works almost full speed.– It meets the requirements. Good enough.– PCI express and the other protocol may also
meet the requirements. However, they may use too much logic (x10) for front-end FPGAs.
ATCA JAN-2006 36
Interrupt on VME bus
InterrupterInterrupthandler
Interrupt RequestBroadcast
Read Vector
Communication between strangers.
ATCA JAN-2006 37
Userprogram
Device
Send return address
Device access
Wake up message
RMAP read
processing
Response in 10~100 micro sec
ATCA JAN-2006 38
Interrupt register(event flag)
Return address
timer
Wait event
Wake up message
Read wake up message -1 : Busy. Another process is already waiting. 0: Event already exist. n>0 : Event occurs after “n” tick.
Write interrupt register 0 : cancel
Hardware implementation
ATCA JAN-2006 39
ATCA crate
One crate holds 96ch of 500MHz FADCs. Total power is expected to be 260 W.
ATCA JAN-2006 40
Virtual CAMAC on serial data link
One crate holds 96ch of 500MHz FADCs. Total power is expected to be 260 W.
Crate controller
Serial data way
ATCA JAN-2006 41
Trigger module
16 LVDS in16 LVDS out8 NIM in8 NIM out
Power consumption is about 10W
Cyclone EP1C12 for trigger logicCyclone EP1C6 for SpW100Mbps SpW
ATCA JAN-2006 42
500 MHz FADC
Cyclone EP1C6Cyclone EP1C12 for router100Mbps SpW (8~9 MB/s from the module)Readout buffer with 128Mb SDRAM waiting second level trigger
8 ch analog inputFADC mezzanine card is developed at KEK (FINESSE format)
Power consumption is About 20W
ATCA JAN-2006 43
Summary
• SpaceWire Remote Memory Access Protocol provides compact and flexible interconnection in a module and inter-module connection.
• Advanced TCA provide dual star LVDS connections. They are good to be applied for DAQ system
• Downsizing may continue.– We might have another solution in the
future.