07/10/99 focus meeting1 review of application software services for lhc era lucia silvestris lucia...

07/10/99 Focus Meeting 1

Review of application software services for Review of application software services for LHC eraLHC era

Lucia SilvestrisLucia Silvestris

Cern/CMC Cern/CMC -Infn/Bari-Infn/Bari

07/10/99 L.Silvestris Focus Meeting 2

Sources and Contributions

Presentations at last RD45 workshop Presentations at the LHC++ meetings Presentations at last LCB Workshop Experiments’ Web pages

Focus on LHC experiments’ prototypes IT services for LHC


ALICE - Simulation and Reconstruction Framework

Simulation and reconstruction framework fully integrated in ROOT (ALIROOT)

ALICE AlgorithmsROOT data structures

ALICE AlgorithmsROOT data structures

RootVisual

RootVisual

GeomDataBase

GeomDataBase

GEANT3transport

GEANT4transport

FLUKAtransport

RootI/O

RootI/O Objy

I/O

ObjyI/O

RootUI

RootUI

Recons Analysis


ALICE - Mass Storage Project (ALICE/IT)

Will be used in Test Beams (2000) Mockup Data Challenge done: 7 TB in seven

days Use HPSS and/or CASTOR for file

management

GbitEther

9x5MB/sHPSS

3(5)x5.5MB/s

LDCLDCLDCLDCLDCLDCLDCLDCLDC

Experiment

Computer Centre

LDC local data collectorGDC global data collector

GDC | ROOT GDC |

DATE


ATLAS

Used Objectivity in several test-bed applications:

HCAL test-beam Serve as a testbed for ATLAS software strategies and

technologies Planned since March 1999; Testbeam period was 7-21

July 1999

ATLFAST++ porting of ATLFAST++ algorithms into LHC++

environment provide to MONARC Project an easy way to populate

database with events similar at those expected at LHC


ATLAS - 1TB Milestone (ATLAS/IT)

1TB Milestone (HPSS used as MSS) Wrote 1 TB of simulated raw data (jet production

digits) to Objy databases stored in HPSS Learning from this: Setting up a system to “stage”

Zebra data sets into Objy databases for people to work with (Performances optimization need to be done)

Plan to use Objectivity in future test-beams and MonteCarlo reconstruction

The application framework will provide a “database” independent interface


CMS - Software Components


CMS - Analysis and Reconstruction Framework

Objectivity is fully integrated in the Analysis and Reconstruction Framework (CARF) CARF manages transactions, physical clustering and

the whole persistent object structure and its relations with the transient structure

users access persistent objects through C++ pointers

CARF takes care of pinning

Prototype using Objectivity during ‘97 and ‘98 Production using Objectivity in 1999

Test Beam DAQ Monte Carlo (GEANT3) reconstruction


CMS - Raw Data Server and DB Populator

VMEReadout

E’NetReadout

PCIReadout

OtherReadout

GenericFarm

InterfaceSt

anda

rdiz

ed in

terf

ace

Raw Data Server

DAQ-Control System Server

Backend Process

Test Beam Specific CMS wide Generic part (with user specific adaptations)

OODBMS

DataStore

Thread

DataFilter

Thread

EventTokenQueue

Workers


CMS - Objectivity Central Data Recording (CMS/IT)

DB1DB1 DB2DB2 DB3DB3 DBnDBnDB1DB1 DB2DB2 DB3DB3 DBnDBn

RunDBRunDB

LogDBLogDB

BConfDBBConfDB Prod Prod FDFD

Prod BootProd Boot Offline - cmsc01Offline - cmsc01

Clone FDClone FD

OnlineOnlineProd BootProd Boot

Prod Prod FDFD

LogDBLogDB

BConfDBBConfDB

RunDBRunDB


CMS - Data Taking and Data Analysis

During 1999 we have stored a total of ~800 GB into Objectivity/HPSS. The average data transfer is ~900 KB/sec.

Now it’s running a Tracker Testbeam and we will collected ~ 150GB.

In the next weeks will start also the first Reconstruction Production for the Muon and Jets Analysis Groups (~500 GB).

The G3 and the TestBeam reconstruction is done using CARF. For the Analysis we use hbook and n-tuples.


LHCb

LHCb application framework (GAUDI) is independent from persistent technology Several technologies should be available

simultaneously: Objy, ROOT (TestBeams), ZEBRA (MC), ...

Peaceful coexistence: freedom without anarchy Manage its own application caches (data

services) specialized in event data, detector data, statistical data

Provides abstract interface for user provided converters


LHCb - Event Persistent Data

OutputStream

Transient Event Store

Event DataService

PersistencyService

Sicb dataFiles

AlgorithmAlgorithm

SicbCnvSvc

RootCnvSvc

Root dataFiles

ConverterConverterConverter

ConverterConverterConverter

Sic

b/Z

ebra

Root

I/O

OutputStreamAppManager


LHCb - Detector Database

timeVersion AVersion B

Calibration

Version BVersion A

AlignmentVersion C

Version BVersion A

GeometryVersion C

Snapshot for detector data items valid for the currently processed event.(Simplified view)

Environment

“Calibration Database” project based on BaBar’s conditions database HEPODBMS/conditions DB


LHCb - Statistical Data

Manages histograms (HTL based) and histogram directories

Output currently to HBOOK RZ file Data conversion fairly trivial

Adopt what is needed for the commonly used viewers

JAS PAW ROOT ...


COMPASS

Farm core Data servers

Input/Output lines Online + CDR

byte-stream Disk Pool

byte-stream (from online via CDR)

Objectivity/DB CPU clients

via servers (AMS) File transfer

Tape servers HSM

1

2

3

4


Database Production Service - What is available?

Production Lock and Federation Servers a given host can only run one lock or AMS server

though these support multiple federations. A client accesses a federation through a local bootfile which points to the lock server and location of the federation master file.

The IT DataBase Production Service provides small but secure hosts as production lock and federation servers for Atlas, CMS-Ecal, Compass and NA45. The servers are identical Sun Ultra 5 with 64MB memory and mirrored external 9GB disks for storing the server files.



CDR data services Different experiments have chosen different

architectures CMS testbeams will create database files in

federations local to the DAQ systems. Files will be copied via rfcp to Unix file systems on central Sun data servers and added into local federations from where they will be analysed by local processes (no AMS usage). The files (a total of 2TB is expected over 6 months) are copied into HPSS via rfcp. Later recall will be via the AMS/HPSS to other CMS servers.



Atlas tilebeam will probably repeat last year where primary CDR is to zebra files later converted to Objectivity databases on ATLOBJ02. Total volume will only be 100GB. However they are interested in WAN access to USA so will explore either remote connection to a CERN AMS or remote replication (probably ftp rather Objectivity WAN functionality).

Compass small amount of data this year (maybe 1TB) during 3

week run in September but extensive testing to be done.


Database Production Service - What is missing?

Transparent non-blocking interface with MSS

User capability to: export, extract, replicate data and schema manipulate data and schema outside

production database and while accessing data and schema from production database

Fully functional, reliable high-quality database system including VLDB support (>>1PB) management tools


Database Production Service - Objectivity/DB V5.2

V5.2 contains main HEP enhancements: HPSS interface multi-threaded AMS (plus numerous SLAC enhancements)

security hooks, AMS collectives, defer protocol, opaque protocol etc.

VLDB (postponed) Originally foreseen for December 1998 Release schedule (end of September 1999)


Licensing and Distribution for LHC++

Licenses valid at CERN and outside institutes for work on CERN Research Programme

For outside sites http://wwwinfo.cern.ch/asd/lhc++/distribution.html

(over 100 sites now registered)

The installation procedure (for the outside sites) is well described in the web pages.

The procedure for the outside sites could be easier if it’s possible access the libraries directly from the afs tree.


Licensing and Distribution for LHC++

The LHC++ Release is done every month. The last release is “Sept99” and includes:

Objectivity ObjectSpace HepODBMS HTL (Histogram Template Library) Gemini/HepFitting CLHEP NAG_C Why Geant4 is not included in the official LHC+

+ release?

07/10/99 focus meeting1 review of application software services for lhc era lucia silvestris lucia...

Documents

event data

detector data

data analysisduring

zebra data sets

mockup data challenge

average data transfer

reconstruction production

testbeam reconstruction