sebastian robert bablok (department of physics and technology, university of bergen, norway)

Normal text - click to edit

CHEP 2007 – Victoria

[email protected]

Alice HLT Online Calibration framework 1/17

High Level Trigger Online Calibration framework in ALICE

CHEP 2007 – Victoria (02.09. – 07.09.2007)

Sebastian Robert Bablok(Department of Physics and Technology, University of Bergen, Norway)

for the ALICE HLT Collaboration(see last slide for complete list of authors)



[email protected]


Outline

High Level Triger (HLT) system

HLT Interfacesto the Experiment Control System (ECS)

to Offline

to the Detector Control System (DCS)

to event visualization (AliEve)

Synchronization Timeline

Summary & Outlook



[email protected]


Purpose:Online event reconstruction and analysis

1 kHz event rate (minimum bias PbPb, pp)

Providing of trigger decisions

Selection of regions of interest within an event

performance monitoring of the ALICE detectors

Lossless compression of event data

Online production of calibration data

High Level Trigger system

DAQ HLT DCS Trigger

ECS

raw eventdata

analyzed events /trigger decisions

Mass Storage



[email protected]


High Level Trigger systemHardware Architecture:

PC cluster with up to 500 computing nodeAMD Opterons 2GHz (dual board, dual core)

8 GByte Ram

2 x 1GBit ethernet

Infiniband backbone

infrastucture nodes for maintaining the cluster

8 TByte AFS file server

dedicated portal nodes for interaction with the other ALICE systems

cluster organization matches structure of the ALICE detectors



[email protected]


High Level Trigger systemSoftware Architecture:

dynamic software data transport framework (Publish-Subscriber)

Cluster management (TaskManager)

Analysis Software (AliRoot)



[email protected]


OFFLINE

DCS(Detector Control System)

Shuttle

Processed events

Calibration data

Control

FEE(Front-End-Electronics)

Eventdata

New calculated calibration data

HLT - Interfaces

HLTcluster

OCDB(Offline

Conditions DB)

FEP

DDL

ECS-proxy

OFFLINE-portal(Shuttle, Taxi)

ECS (Experiment Control System)

DAQ(Data Acquisition)

DCS-portal(Pendolino, FED)

DCSvalues

HOMER

AliEve(ALICE Event Monitor)

Online event monitoring

FEP = Front-End-Processor; DDL = Direct Data Link; HOMER = HLT Online Monitoring Environment including ROOT

Calculatedvalues

DAQ(Data Acquisition)

Triggerdecisions



[email protected]


HLT ECS interfaceECS Proxy:

Steering of the HLT via the Experiment Control System (ECS)

Synchronizes the HLT with the other online and offline systems in ALICE

Provides the HLT with upcoming run setup (run number, type of collision, trigger classes, …)

Consists of a state machine states (implemented in SMI++)well defined states and transition commands

TaskManager

ECS-proxy

ECSHLT

clustercontrol

states andtransition

commands



[email protected]


OFFLINE HLT interface (Taxi)

Taxi portal:Fetches latest available calibration settings from the OCDB

pre-produced or in former runs calculated calibration objects enveloped in ROOT-files (can be directly used by ALICE analysing framework - AliRoot CDB access classes transparent access for analysis code if running online or offline)

Caches calibration data locally in the HLT (HLT Conditions DataBase - HCDB)

Retrieval is performed asynchronously to run

HCDB distributed to nodes with the Detector Algorithms (DAs)copied locally to the DA nodes before each run fixed version per run



[email protected]


Cluster node(Analysis)

OFFLINE

TaskManager


DADA

1.Taxi

Taxi-HCDB

portal-taxi

DA_HCDB

ECS-proxy

OCDB

current run number

DADA

AliRoot CDB access classes

DA_HCDB

updated before each run

OFFLINE HLT interface (Taxi)

Detector responsibility

Frameworkcomponents

DA = Detector Algorithm

intern

extern

AliRoot CDB access classes



[email protected]


DCS HLT interface (Pendolino)

Purpose:Requests current run conditions (like temperature, voltages, …)

permanently measured by the Detector Control System (DCS)

frequently request of “current” values (three different time intervals)

providing of latest available values to the Detector Algorithms (DAs)

preparation / enveloping of the data transparent access to data for the DAs (online and offline)



[email protected]


DCS

portal-Pendolino

Pendolino

ArchiveDB

DA_HCDB

PendolinoHCDB


DADA


Frameworkcomponents


intern

extern

frequent update

during run

DCS HLT interface (Pendolino)

PVSS

preparation /enveloping



[email protected]


DCS

FED-portal

FEDAPI

ArchiveDB

PVSS


DADA

HLT DCS interfaces (FED)intern

extern

FED portal:collects produced data inside the HLT during the run

writing back data to the DCS (TPC drift velocity, …)

uses the Front-End-Device (FED) APIcommon among all detectors in ALICE

PVSS panels to integrate data into DCS


Frameworkcomponents




[email protected]


HLT OFFLINE interface (Shuttle)

Purpose:Collects online calculated calibration settings from the DAs inside the HLT

calibration objects stored in a File EXchange Server (FXS)

meta data stored in a MySQL DB (run number, detector, timestamp, …)

Feeding back new calibration data to the OCDB

OFFLINE

Cluster node

OCDB

Cluster node

DADA

FXS MySQL

Shuttle

DADA

TaskManager

ECS-proxy

ECS

notifies: “collecting

finished”

synchronisation

portal-shuttle(FXS-Subscriber)



[email protected]


Homer

DADA

HLT AliEve interface

AliEve

(Alice Event visualization)

Visualization

of results

• integrates the HLT in existing visualization infrastructure of AliEve (M. Tadel)

• connection to the HLT from anywhere within CERN network

• ONE visualization infrastructure for all detectors



[email protected]


Synchronization timeline

Sequence:

Initializecluster

Init

...

HLT

ECS-proxy

SoR EoR Complete

Distribute HCDB

collect datafor Shuttle

FEE-data

DCS-Pendolino

outstreamto DAQ

DCS-FED

AliEve-Homer

OfflineTaxi

(asynch)

OfflineShuttle

analyze events / calculate calibration data

contact with the various other systems in ALICE

Before RunAt

Start-of-Run

During Run After End-of-Run



[email protected]


Summary & Outlook

HLT consists of a large computing farmmatching the structure of the ALICE detectors.

Interfaces with the other systems of ALICE (online and offline) enables data exchange with the connected parts.

HLT allows for detector performance- and physics monitoring and calibration online.

With the presented mechanism the HLT will be able to provide all required conditions and calibration settings to the Analysis Software for first physics in ALICE 2008.



[email protected]


ALICE HLT Collaboration Sebastian Robert Bablok, Oystein Djuvsland, Kalliopi Kanaki, Joakim Nystrand, Matthias Richter,

Dieter Roehrich, Kyrre Skjerdal, Kjetil Ullaland, Gaute Ovrebekk, Dag Larsen, Johan Alme(Department of Physics and Technology, University of Bergen, Norway)

Torsten Alt, Volker Lindenstruth, Timm M. Steinbeck, Jochen Thaeder, Udo Kebschull, Stefan Boettger, Sebastian Kalcher, Camilo Lara, Ralf Panse

(Kirchhoff Institute of Physics, Ruprecht-Karls-University Heidelberg, Germany)

Harald Appelshaeuser, Mateusz Ploskon(Institute for Nuclear Physics, University of Frankfurt, Germany)

Haavard Helstrup, Kirstin F. Hetland, Oystein Haaland, Ketil Roed, Torstein Thingnaes(Faculty of Engineering, Bergen University College, Norway)

Kenneth Aamodt, Per Thomas Hille, Gunnar Lovhoiden, Bernhard Skaali, Trine Tveter(Department of Physics, University of Oslo, Norway)

Indranil Das, Sukalyan Chattopadhyay (Saha Institute of Nuclear Physics, Kolkata, India)

Bruce Becker, Corrado Cicalo, Davide Marras, Sabyasachi Siddhanta(I.N.F.N. Sezione di Cagliari, Cittadella Universitaria, Cagliari, Italy)

Jean Cleymans, Artur Szostak, Roger Fearick, Gareth de Vaux, Zeblon Vilakazi(UCT-CERN, Department of Physics, University of Cape Town, South Africa)



[email protected]


Backup slides



[email protected]


ECS:Controls the HLT via well defined states (SMI++)Provides general experiment settings (type of collision, run number, …)

DCS:Provides HLT with current Detector parameters (voltages, temperature, …) Pendolino, (DCS HLT) Provides DCS with processed data from the HLT (TPC drift velocity, …) FED-portal, (HLT DCS)

OFFLINE:Interface to fetch calibration settings from the OCDB Taxi, (OFFLINE HLT) Feeding back new calibration data to OCDB Shuttle-portal, (HLT OFFLINE)

HOMER:HLT interface to AliEve for online (event) monitoring

FEE:stream in of raw event data via H-RORCs

DAQ:stream out of processed data and trigger decisions to DAQ-LDCs

HLT interfaces



[email protected]


Synchronization timelineTimeline (1)

Before Run (continuously):Taxi requests latest available calibration settings from the OCDB (asynchronously to actual run)

At Start-of-Run (SoR) - Initial Setting:ECS HLT: initialization of the HLT cluster (run number, beam type, trigger classes, …)

fixating the HCDB for a run, distribution of the HCDB to the DA nodes

Initializecluster

Init

...

HLT

ECS-proxy

SoR EoR Complete

Distribute HCDB

collect datafor Shuttle

FEE-data

DCS-Pendolino

outstreamto DAQ

DCS-FED

AliEve-Homer

OfflineTaxi

(asynch)

OfflineShuttle

analyze events / calculate calibration data

contact with the various other systems in ALICE



[email protected]


Timeline (2)During Run (after SoR):

DCS HLT: current environment/condition values fetched via the Pendolino

update of the HCDB on the DA nodes with the new DCS data

HLT DCS: processed, DCS relevant values are written back via the FED-portal (e.g. TPC drift velocity)

HLT DAQ: analyzed events go to permanent storage (includes a certain time period after the End-of-Run)

HLT AliEve: online event monitoring in the Alice Control Room

After End-of-Run (EoR):HLT OFFLINE:

collecting of fresh produced calibration objects in the FXS of the Shuttle portal

OFFLINE Shuttle fetches these objects from the HLT (synchronization via ECS)

Synchronization timeline



[email protected]


OFF

INITIALIZING<< intermediate state>>

DEINITIALIZING<< intermediate state>>

INITIALIZED

CONFIGURED

READY

RUNNINGRUNNING

CONFIGURING<< intermediate state>>

ENGAGING<< intermediate state>>

DISENGAGING<< intermediate state>>

INITIALIZE

implicit transition

SHUTDOWNCONFIGURE

+ params

RESET

ENGANGE

DISENGAGE

START

STOP

implicit transition

implicit transition

implicit transition

implicit transition

COMPLETINGCOMPLETING

implicit transition

Preparing of HCDB for DAs

Collecting data for Shuttle

Portal

Offline Shuttle can fetch data

Event processing

Pendolino fetches DCS data for DAs

ECS Proxy

ECS interface



[email protected]


Pendolino-portal

TaskManager

DCS Archive DB

Pendolino

ECS-proxy

DA

DA

HLT - cluster

OCDB

AliEn

TAXI-portal

TAXITAXI-HCDB

HCDB(read-only)

HCDB

Amanda

Predictionprocessor

TAXI-HCDB

(read-only)

Init HCDB script

releaseTAXI-HCDBbefore SOR

release HCDBafter update

set links fromTAXI-HCDB

to HCDB

links from TAXI-HCDB

to HCDB DA

notify PubSub of

new release

inform aboutnew release

ship update infothrough chain

AFS

AFS

controlPendolino



[email protected]


Introducing the HLT system

HLT counting room in

ALICE pit



[email protected]


HLT overview

● Central Pb+Pb collisions ● event rates: ~200 Hz (past/future protected)● event sizes: ~75 Mbyte (after zero-suppression)● data rates: ~ 15 Gbyte/sec

ALICE data rates (example TPC)

TPC is the largest data source with 570132 channels, 512 timebins and 10 bit ADC value.

TPC data rate alone exceeds by far the total DAQ bandwidth of 1.25 Gbyte/sec

Detectors

DAQ HLT

Mass storage

● Event selection based on software trigger

● Efficient data compression



[email protected]


Architecture



[email protected]


Architecture

DAQ HLT DCS Trigger

ECS

raw eventdata

analyzed events /trigger decisions

Mass Storage

sebastian robert bablok (department of physics and technology, university of bergen, norway)

Documents