lnl 1 sadirc2000 resoconto 2000 e richieste lnl per il 2001 l. berti 30% m. biasotto 100% m. gulmini...
TRANSCRIPT
1
LNL
SADIRC2000SADIRC2000
Resoconto 2000 eResoconto 2000 eRichieste LNL per il 2001Richieste LNL per il 2001
L. BertiL. Berti 30% 30%M. Biasotto 100%M. Biasotto 100%M. GulminiM. Gulmini 50% 50%G. MaronG. Maron 50% 50%N. TonioloN. Toniolo 30% 30%
Le percentuali sono condivise con INFN-GRIDLe percentuali sono condivise con INFN-GRID
2
LNL
40 MHz
105 Hz
102 Hz
100 Tbyte/s
100 Gbyte/s
100 Mbyte/s
Level 1Trigger
Event Manager
Detector Frontend
Event Builder
Computing Services
Controls
Readout
Filter
Collision rate 40 MHzLevel-1 Maximum trigger rate 100 kHzAverage event size 1 MbyteNo. of In-Out units (200-5000 byte/event) 1000Event builder (512-512 switch) bandwidth 500 Gbit/sEvent filter computing power 5 106 MIPSData production Tbyte/dayNo. of readout crates 250No. of electronic boards 10000
Collision rate 40 MHzLevel-1 Maximum trigger rate 100 kHzAverage event size 1 MbyteNo. of In-Out units (200-5000 byte/event) 1000Event builder (512-512 switch) bandwidth 500 Gbit/sEvent filter computing power 5 106 MIPSData production Tbyte/dayNo. of readout crates 250No. of electronic boards 10000
Units
Units
Future DAQ SystemsFuture DAQ Systems
3
LNLSadirc2000 Event Builder DemonstratorsSadirc2000 Event Builder Demonstrators
• Square Event Builder (15x15): Square Event Builder (15x15): June 2000 (June 2000 (OKOK))
• Asymmetric Event Builder (4x40):Asymmetric Event Builder (4x40): November 2000November 2000
4
LNLHardware components for the Square EBHardware components for the Square EB
SysKonnect SK-9821 1000BaseT adapter
Foundrynet FastIron II+ 1000BaseT switch Supermicro PIIIDMEmotherboard
Intel 840 chipset
Pentium III 600 Mhz
64/66 PCI BUS
5
LNLSoftware components for the Square EBSoftware components for the Square EB
• vxWorks real time o/s
• C++ software for emulation of the Event Builder components ( EVM, RU, BU )
• Standard vxWorks driver for the SysKonnect adapter
6
LNLError recoveryError recovery
• Error recovery procedure implemented
• See: Samim Erhan, Wolfgang Schleifer, Nick Sinanis ‘System Error Analysis’ TriDAS Review 11 May 2000
7
LNLError recoveryError recovery
BU
send
RUsend timer
start
cache
send( retry )
start
cancel
timeout
cache
BU
allocate
EVMallocate timer
start
confirm
allocate start
cancel
timeout
confirm
BU – EVM communication BU – RU communication
8
LNLSquare Event builder layoutSquare Event builder layout
• RUs and BUs distributed in all the switch slots
• Part of the traffic localized in the slot
• Minimize the switch backplane utilization
1 15141312111098765432
1 15141312111098765432
EVM
RUs
BUs
Slot 1 Slot 2 Slot 3 Slot 4
9
LNLSwitch packets loss in special conditionSwitch packets loss in special condition
First packet forwarded correctly
Some packets lost
( 10-30 packets )
All the following packetsforwarded correctly
• The switch lose some packets in particular conditions
• When the transmission restart after a pause >60 Secs
• Probably related to the MAC address table management
• Problem solved by the Error recovery procedure
10
LNLTest conditionsTest conditions
• BDN+BCN in the same network• No Readout commands from EVM to RU• Clear/Allocate optimization: the clear
command is sent together an allocate for a new event
• No command or event aggregation: each packet transport a command or a data frame relative to a single event
11
LNLTest conditionsTest conditions
• Full data transfer: data moved from/to the pc memory
• Tests performed in the 400-4000 Bytes range
18
LNLConclusions for the Square Event BuilderConclusions for the Square Event Builder
• Event Builder Demonstrator 15x15 based on copper Gigabit Ethernet interfaces tested
• Good performances and scalability
• Actual limit is in the Readout Unit (?)
• Error recovery work fine, systematic tests need
19
LNLNNext Steps for 2000ext Steps for 2000
• Asymmetric Event Builder (4x40)
• Event Builder Protocol Simulations
21
LNLInfiniBand (II)InfiniBand (II)
Legacy host architectureLegacy host architecture
The Infiniband ModelThe Infiniband Model
22
LNLInfiniBand (III)InfiniBand (III)
• What?What?– Initial single link signaling rate of 2.5Gbaud Initial single link signaling rate of 2.5Gbaud
• Means unidirectional transfer rate of 250MB/sec with a Means unidirectional transfer rate of 250MB/sec with a theoretical full duplex rate of 500MB/sectheoretical full duplex rate of 500MB/sec
– Initial support for single, 4, and 12 wide link widthsInitial support for single, 4, and 12 wide link widths
– Point to point switched fabricPoint to point switched fabric
– Message based with multicasting supportMessage based with multicasting support
MultiMultiStageStageSwitchSwitch
LinksLinksTCATCA
I/O ControllerI/O Controller
MemoryMemoryControllerController
PCI-X PCI-X HostHost
BridgesBridges
CPUCPU CPUCPU CPUCPU CPUCPU
HCAHCALinkLink
Fibre ChannelFibre Channel
SCSISCSI
Gig. EthernetGig. Ethernet
MemoryMemory
HCAHCA- Host Channel Adapter- Host Channel AdapterTCATCA - Target Channel Adapter - Target Channel Adapter
23
LNL SADIRC Requests for 2001SADIRC Requests for 2001
• Simple test system (4 servers + Storage Area Network + Simple test system (4 servers + Storage Area Network + Network) forNetwork) for 20012001 is possibleis possible
• Early access to the productsEarly access to the products
• Test BedsTest Beds
• Requests 2001 (valuations) Requests 2001 (valuations) • 4 servers4 servers 50 Ml50 Ml• 1 IBA Switch1 IBA Switch 20 Ml20 Ml• IBA AdaptersIBA Adapters 10 Ml10 Ml