CHEP 2000 Session F:

Data Analysis:Technology and Presentation

Convenors: Francois Etienne - IN2P3, Atlas, Ruth Pordes, FNAL

Thank You to all the Speakers for Providing me with their Transparencies to use. 29 Oral presentations; 1 Poster

Outline

• Quick Review of Topics/Titles

• More Detail for Each Category

• Summary

Numerical Library Software - 2 papers

• Development of Numerical Library Software in Java – KEK – Initial Work

• Scientific Toolkit for Java – Phenix – Really an Analysis Framework

• (poster paper not in this session on COLT - Cern/IT)

• All in Java

Electronic Document Management systems - 4 papers from 2 systems

• Java-based User Interfaces for CERN's Engineering Data Management System

• Integrated Information Management for TESLA

• Asset Management for Cavities and Computers at DESY

• Experience with Document Management at DESY

Singletons:

• Bringing Mathematical Formatting to the Web - An Overview of the Technology and SLAC's Experience

• Multi-threaded, Discrete Event, Simulation of Distributed Computing Systems – CMS

• CORBA Scripting in HEP and Beyond.

• ATLAS DAQ back-end Software Integrated toolset, mainly Open Source Second use of this will be an interesting test. Defining deliverables of “pre-design” phase. How to move to an Open Source development environment for critical

piece of Experiment architecture?

Event Displays - 3 talks

• Atlantis an Event Display for ATLAS: When displaying 6000 hits recorded by only 7 layers, can we recognize tracks?

• WIRED - World-Wide Web Interactive Remote Event Display

• HepRep: a Generic Interface Definition for HEP Event Display Representables

General Analysis Packages/Frameworks

• Java Analysis Studio

• ROOT at RUN II

• Open Scientist

• A Comparison of Data Analysis Packages (Root and IDL)

Experiment Analysis Packages

• A Graphics Sampler for the D0 Experiment

• The CMS IGUANA Project

• Object Oriented Data Analysis in the DELPHI Experiment

• Distributed BELLE Analysis Framework

• New Data Analysis Environment in H1

General Analysis Components

• Abstract Interfaces for Data Analysis – AIDA – Cern/IT

• A Histogramming package in Java - KEK

• Lightweight Histogram Interface Layer – from NERSC/LBL

Software Process and Configuration Management - 5 talks all from Europe..

• An Evaluation of Tools for Static Checking of C++ Code

• Impact of Software Review and Inspection

• C++ Code Analysis: an Open Architecture for the Verification of Coding Rules

• CMT: a Software Configuration Management Tool

• Use of a Configuration Management Tool in LHCb Software

What there was Not

• Software Development Methodologies and Case Tools.

• 3-D and Virtual Reality.

Overall Impression

• Continuation of directions and initiatives from last Chep.

• Situation today (finally): method for defining a mathematical expression in terms of

its presentation and semantics MathML is a standardized instance of XML

• Used “IBM TechExplorer Hypermedia Browser” A Web browser plug-in Dynamically formats and displays documents coded with

TEX/LATEX and/or MathML Beta version applied to existing Spires TeX files works.

• Try it and feed back comments ...

F19 MathML Interface for Abstracts in SLAC Spires

F19 - Tool for Simulation of Complex Distributed Data Handling Systems -

• Technology – in house developed JAVA toolkit. Process driven not event driven architecture Full presentation and easy editing of parameters

encourage many different models and users Immediate publication of results on the web

Enable physicists to read, analyse and feedback.

• Alternative architectures and designs can be commented with quantitative and analytic input

• Technology is ready to be tested for “Simulator Friendliness” …

Volunteers to model the existing Babar and Run II systems would indeed be useful.

F359, F65, F66, - DESY Integrated Information Management

• Reminded us of the importance of need for and time required to Peoples Use and Access Patterns change after they

have experience with a working system Current and Future Business Processes significantly

affect design Users expect latest Technology (currently video,

mobile computing etc)

• Demonstrated Success of collaborative development of vendor tool for HEP needs

Current Tuovi Status & Statistic

• 20 projects with:- 6 GB data vault- 2400 project nodes- 18600 files- 3300 registrated user

Tuovi 1.2 Up- and Download Statistic in 1999

1

10

100

1000

10000

100000

Month

Nu

mb

er

of

Ac

tio

ns

upload

download

Cern EDMS Extensions

• Import/Export Interface - define an exchange data format in XML. Conversion of metadata to and from various

formats e.g. Excel, Html, Databases, Text Processors etc.

• in use by several LHC groups: LHC Baseline and ABS ATLAS: Import/Export of ABS Structures CMS: ECAL ABS ALICE: Documents and Files check-in Documents Management in the LHC Design office (in

progress)

Histogramming Packages

• Histogrammer in JAVA - Successful interfacing of Java and C++ objects for in a simple Histogrammer application

• Lightweight C++ Histogram Interface - Break between Statistical Data Gathering, from analysis and visualization tasks. Define Standard API and Support Object Structures. How does it relate to transient Cern/HTL

implementation

Analysis and Event Display Toolkits and Interfaces

• WIRED

• HepRep

• AIDA

• Java Analysis Studio

• Scientific Toolkit for Java

• Open Scientist

• ROOT

• IDL - through comparison with ROOT

Wired

• Rewrite since Last CHEP

• In Use by CHORUS, BaBar, DELPHI etc.

• Lacks a Scripting Language.

• BaBar Use shows up Need to Think before Defining what Data gets Transported between User Client and Data Server – don’t want a “Packet per Point”

HepRep

• Experience with Wired at BaBar led to need to layer interface between Client and Display so as to

reduce granularity in getting remote information

increase ability to for Client/Users to easily add objects and views

• HepRep defines a general API for serving Representables, not Physics Objects

• “Serving Representables keeps the detailed reconstruction code, swimmers and detector models on the server side where they belong.

• Spatial information is assembled and shipped in an efficient manner, avoiding the overhead of too many individual method calls.”

General Comments

• Transport the Physics Analysis Code to the Data - JAS – No Answer Yet on Whether this will Work Well.

• WIRED and JAS both Talking about Scripting Languages – no decision yet.

• Once Package in Use and Responding to User Use Needs, More Core Development More Difficult

• Integration with C++ Analysis Algorithms and Data Adaptors Not Here Yet.

AIDA

• Part of Next LHC++ Analysis Toolkit – the Start.

• HepVis ’99 established Collaboration for Definition of Abstract Interface for Data Analysis

• Interface Architected to Support Scripting and GUI Equivalently.

• Goal of Adaptors/Interfaces to HepTuple, JAS, Openscientist, Root etc.

JAS

• Extensions since last CHEP: JEdit Editor Bean Using Swing JEditorPane JAS now supports HTML display XML support - JASHist bean directly supports reading/writing

XML Servlet Support

• Continued use in BaBar Online and LCD Simulations

• Packaged and made Available in OpenSource Paradigm - Contributions starting e.g. 3D Classes from Fermilab.

Scientific Toolkit for Java

• Explained was a Separate Project to “Learn Technology”. Use of JavaHelp Based onJava 2D (JDK 1.2)

• Generic Swing Console for those Interpretive Languages not supported by Swing.

• Anticipate Discussing Collaboration with / Contributions to JAS.

• Starting New Development Allowed use of Latest Generation Technologies

Open Scientist

• Work Continuing for Experiments – LHCb, Virgo, Auger etc.

• Concept of “Hub” as Integration Layer. What is boundary/layering between Frameworks and Utilities - Do we need More Concepts Here?

• Worklist includes Integration of CINT and Extensions to Free Open Inventor Implementation.

ROOT - Run II Perspective

• Progress since Run II Adoption of ROOT with Caveats.

• Priorities from 1999 Workshop been in Large Part Addressed – Robustness, CINT Extensions, Pre-compilation of user classes.

• Tutorials and Education Popular with Run II Users.

• To meet Local Needs currently Building 23 Flavors.

• Collaboration Works with Technical Commitment

• Second ROOT Workshop last week at Cern resulted in New Worklist - now posted – main extension Self Describing Data format.

Comparison of ROOT and IDL:

• IDL command language Concatenate Sets of Files of ntuples - chain

command. Read in and Use Variables from an “ntuple”File -

Local Extension. Event Selection Cuts on Several Variables Plot - Static or “Liveplot”.

• Large ntuple problem addressed by “mapped files”.

• Needs test drive for “large scale analysis”.

Experiment Specific

Event Displays and Analysis

Mature Experiments Retooling Technology for Analysis – Delphi and H1 reported

• Issues include: Maintenance – Technical and Resources; Support of Multiple Generations of Technology.

• OO, ROOT, C++ - Speed and Usability Benefits

• Encourage New Collaborators

• Allow and Provide for Curation of the Data

Atlas Event Display - Atlantis

• Applying Hans Drevermanns Algorithms to New/Future Experiment “V-plot” Z finder Hit filtering

• Plan Recode from Fortran to Java.

Experiment Event Displays – D0

• Suite of Different Tools based on Legacy and New Packages.

• New Tools based on Open Inventor. Will Evolve.

• Finding very useful for algorithm development and starting for analysis of processed Monte Carlo data

Open Inventor

HEPVis

Mesa

D Graphics

OpenGL

Rest of D0 software

Exceed

3D

Xi Graphics

D Graphics Sampler

CMS - IGUANA

• On the way to being a Fully Featured Data Presentation and Manipulation Framework.

• Will be used in CMS Simulation and Test Beam Analysis this Year.

• Pragmatic Approach to getting Prototypes and Early Production Releases.

• Based on Open Toolkits and Open Inventor.

• “Sharing experience may be more important than sharing code”- aligned with LHC++ strategy.

IGUANA Objectivity

tag analyser

(tag is sim

ilar to ntuple)

6(d) Qwt &

Qt

6 (b) H

EPInventor

& OpenInventor,

Motif

IGUANA Prototypes: Objectivity Browsers and Persistent Histograms and Tags Plotters

• 6(c)

SciPlot

• (Xt / Qt)

6(a) HepExplorer &

IRIS Explorer

IGUANAObjectivitybrowsing

BELLE – BASF Analysis Framework

• Currently supports Single SMP Parallel Processing of Events for Users.

• dBASF Extension being Designed to Support Network Connected Clusters of PCs. Package of Daemons for Job Management Data I/O Resource Management Monitoring.

• Resource Managers Dynamically Reconfigures – Increase, Decrease - Number of Computing Servers Running Jobs.

HEP Configuration Management Tools

• Report from 2 users of CMT – Build and Release System.

• Poster papers on SRT Rewrite, NT Extensions

• From Marjorie’s Plenary Talk: “Everyone now using them, can’t we use the same one…”

jcmt

brow

ser

edit

orm

onit

or

Package areas

Packages & versions

Free action onto the package

Requirements editor

Configuration monitor

F63 - C++ Code Rules and Checking

• C++ Coding Standard – 180 Rules for Naming, Coding and Style of Varying Severity for Each Experiment.

• Evaluation of Commercial tools for Code Rule Checking. Tested on Geant4 and Atlas Libraries.

• Out of 5 tools considered 2 Meet the Basic Needs – about 75% of total: CodeWizard and QA C++.

• Now Traditional Questions on: “What does it take to Decide?” “Will IT/CD Support It?”

F361 - C++ Code Rules and Checking - Alice

• Development of Code Analysis Tools - Parser, Formatter, Checker

• Limitations in Checking Complete C++ syntax e.g. Templates, Namespace

• Successfully used on ALICE code

F331 - Atlas Software Review and Inspections

• Movement in HEP from Fagan Inspections (Fermilab SDSS/DART/Run II) to Gilb Inspection Method

• Bodes Well that this is Part of Formal Development of Proposed OpenSource Project..

F119 - Applications using CORBA Script – continuation of work reported at Chep ‘98.

• Interpreted Scripting Language which can interact with CORBA objects through IIOP and has dynamic access to any CORBA object.

• OMG Standard

• Interface Definitions Stored in Repository

• Applications in Babar Data Handling System

Summary – TechnologyJava, XML, CORBA• JAVA

Mainstream for New Presentation Applications.

Not Mainstream for Data Analysis and Reconstruction.

Work Not yet Done to Integrate with C++ Algorithm Modules and Frameworks.

Work only just Beginning on HENP Utility Libraries.

Summary – Physics Analysis Packages

• Not much development using Commercial Analysis Tools.

• ROOT most Generally Used; Extensions still being requested to meet Experiment Long Term Needs. Open Source Collaboration without Open Source licence?

• JAS, OpenScientist, WIRED, progressing steadily.

• Question still Open as to How or Whether any of the Existing Analysis Technologies will Successfully meet Needs of >500 Physicists doing Analysis of Babar and Run II over the next 10 Years.

Summary – Development Environment Technology

• Experiments looking for more and better tools for Configuration Management but no widespread adoption yet of single solution no work yet on significant extensions to the services

provided no new work to look at commercial tools

• Attendees very interested in Code Quality and Checking Tools and Process – but if the number of talks is an indication of the relative investment..

Summary - misc

• Mature Experiments Taking Time to Upgrade Analyses to New Technologies to Support Ongoing Analysis and Provide Tools for Newcomers –

Reminder this will Happen for Current and Future Experiments.

• (Where did Virtual Reality go?)