dqm services: status and plans dqm workshop (cms week) – 28 february 2007 christos leonidopoulos...

DQM Services: Status and Plans

DQM Workshop (CMS week) – 28 February 2007

Christos LeonidopoulosCERN − PH

for the EvF/DQM group

DQM Services – 28 Feb 2007 Christos Leonidopoulos 2

“What is DQM?”

• DQM is:– A monitoring project originally created for the Event Filter Farm

– A set of monitoring tools that can be used either on- or off-line: Tree-like structures w/ histograms, profiles, scalars, strings Quality tests that produce warnings, errors, alarms Visualization tools Transfer of monitoring information to remote nodes

– A wrapper around ROOT objects that offers the above functionality

• DQM is not:– A validation suite (just like ROOT is not a physics analysis)

– A set of “out-of-the-box” templates that are ready to use


“What can DQM do for me?”

• Main reasons for using DQM:– Transfer of monitoring information over network

Creation/updates of histograms/profiles/etc transparent to user Suitable for distributed systems

– Library with quality tests, alarm generation/storage/distribution Often updated w/ “algorithm donations” from users

– Bonus: Extended usability by CMS helps w/ bug discovering/fixing

(EvF, OnSel, all subdetector groups, beam-tests, MTCC, …)

• Ongoing development over last 2 years:• Emilio Meschi (original design, run-control, mentoring)

• CL (core functionality, rules & alarms library, tech support)

• Dimitrios Tsirigkas (Web interface), Giulio Eulisse, Andrea Carboni (Qt-GUI)

• Ilaria Segoni (specialized clients, coordination with detector groups)

• Kurt Biery (DQM data & Storage Manager)


DQM: On-line mode

On-line operating mode: Monitoring producers → server → monitoring consumersUseful when:• Need to access info from multiple monitoring producers• Need clear separation of production (sources) and processing (client) of monitoring information (e.g. Filter Farm)

Monitoring producers

Monitoring consumersSee Emilio’s s

lides

See Emilio’s s

lides


DQM: On-line mode #2

“Clients”

“Super Clients”

CERN Network

Non-CERN NetworkWhat if you want something simpler?

See Emilio’s s

lides

See Emilio’s s

lides


DQM: Off-line (standalone) mode

Merge source & client; eliminate collectorUseful when one wants to:• First access/produce monitoring information• Then process it• In the same program!

Equivalent to “off-line” or “validation” modeEquivalent to a “standalone” ROOT macro

Monitoring producer

Monitoring consumer

Single program/node


DQM from a user’s perspective

Client

“DQM”

Monitoringinformation

• Configuration• Reference objects• Historic plots• Etc…

• “Comparison-to-reference”• Collation of similar objects

Database Tools

“Alarm”

“System ok”

• Create monitoring information• Process: Visualize, analyze, determine quality, store (à la ROOT or kumac macro)

Possible today, but notmanaged by DQM


2

Core Features #1: The Basics

• Support for all the “usual stuff”: static and dynamic sets of objects 1,2,3-D histograms, 1,2-D profiles, integers, floats, strings (ROOT objects behind the scenes) • Support for tree-like directory structures

1

3


Core Features #2: Quality Tests

• Library with “rules” for assigning “quality” value to tests− Comparison to reference (χ2, Kolmogorov tests)− Contents within range ([xmin, xmax], ([ymin, ymax])− Exact match− Mean of (e.g. gaussian) distribution “near” expected value− Check for dead channels− Check for noisy channels− Mean & RMS of bins within allowed range

− Quality tests run automatically by DQM− Results “attached” to Monitor Elements− Can be saved in ROOT file for “off-line” use

• “Alarm” library Warnings & error messages propagating to all clients downstream


1

Quality Tests: Examples

2

3


Core Features #3: Output/Input

• Support for ROOT-files Create and save ROOT-files w/ (any subset of ) monitoring structure on the fly (including results of quality tests)

Can also read back file (“playback”), create MonitorElements, “attach”quality test results (1_2_0 feature)


Web interface v2.0


Customized SiStrip Web interface


DQM Iguana Qt-GUI #1


Cosmic test data forcalorimeter detector(reading from file)

DQM Iguana Qt-GUI #2


Recent highlights

• Added the ability to group MEs from different folders/nodes: “Tags”

Similar to google mail’s “labels”

• Improved interface for accessing MEs

• Can open ROOT files in “playback” mode

• Fit functions/results can be transferred to remote nodes

• Improvements in code stability


“Historic” Plots & DQMhttps://twiki.cern.ch/twiki/bin/view/CMS/DQMHistoricPlots


Open items & to-do lists (savannah)DQM support requests:

DQM bugs:

DQM tasks:


DQM documentation

• Release notes for latest developmentshttp://cmsevf.web.cern.ch/cmsevf/DQM_doc/Release_Notes_v033.txt

• Instructions for getting startedhttp://cmsevf.web.cern.ch/cmsevf/DQM_doc/DQM_instructions.html

• Archive with presentations from DQM group:http://cmsevf.web.cern.ch/cmsevf/DQMMeetings.html

• DQM status for subdetector groupshttps://twiki.cern.ch/twiki/bin/view/CMS/DQMSubDetectors

• Twiki page with discussions, notes, manuals, linkshttps://twiki.cern.ch/twiki/bin/view/CMS/DQMInfrastructure

dqm services: status and plans dqm workshop (cms week) – 28 february 2007 christos leonidopoulos...

Documents

dqm services

dqm slide

dqm results

functionality dqm

historic plots dqm https

christos leonidopoulos

feature slide

file dqm iguana qtgui