spi report for the lhcc comprehensive review

SPI Report for the

LHCC Comprehensive Review

Stefan Roiser

for the SPI project

19. Nov. 2007 LHCC Comprehensive Review - SPI 2

SPI services

• Software process support– Software testing, configuration & release

• Documentation support– Code documentation, web services

• Collaboration tools– Hypernews, Savannah

• Hardware infrastructure– Build servers, special purpose machines

Software

19. Nov. 2007 LHCC Comprehensive Review - SPI 4

Time to user

• The time from a bug report / feature request until it is used by the physicists in the experiments– Especially important to minimize this time when

running with “live” data.

19. Nov. 2007 LHCC Comprehensive Review - SPI 5

Nightly Builds

• Recommendation of the 2006 internal review• Motivation

– Early integration into experiment software– Fast feedback to LCG/AA developers of changes

• Usage– Building with own experiment nightlies on top– Developers testing their code on LCG/AA stack

• Used by– LHCb– Atlas

19. Nov. 2007 LHCC Comprehensive Review - SPI 6

Nightly Builds (ctd.)

5 configurations

LCG/AA projects

13 possible platforms

Builds & Tests

test evolution

19. Nov. 2007 LHCC Comprehensive Review - SPI 7

Operating System

eg. Athena, CMSSW, Gaudi

Linux (slc) 32/64 bit, Mac OSX, Windows

External Software

LCG / AA software

Experiment software

Operating System

eg. Athena, CMSSW, Gaudi

Linux (slc) 32/64 bit, Mac OSX, Windows

External Software

LCG / AA software

Experiment software

Software release

56 packages, eg. Boost, Python, Qt, etc…

COOL, CORAL, POOL, RELAX, ROOT, SEAL

LCG release

• rubberstamps all package versions• provides several products

• afs installation• web pages• src tar balls• bin tar balls• documentation

19. Nov. 2007 LHCC Comprehensive Review - SPI 8

Software processes

mysql

externs/w

scramcmt install.mgr.

cron /pythontar

ballsxml /

python

python

nightlybuilds

projects/w

websites docu

xml

historicallygrown

19. Nov. 2007 LHCC Comprehensive Review - SPI 9

Software processes new

externs/w

install.mgr.

tarballs

nightlybuilds

projects/w

websites docu

Python /LCGCMT

19. Nov. 2007 LHCC Comprehensive Review - SPI 10

LCGCMT

• Is a CMT based project • Contains the configuration information

– How to build software packages from source• Using sh/bat scripts

– Extra information • Eg. homepage, package maintainer, description, etc.

– In addition for other CMT based projects• How to use/build against the packages• Runtime environment

19. Nov. 2007 LHCC Comprehensive Review - SPI 11

Example: package build

package gsl

use LCG_BuildPolicy v* LCG_Builders

macro gsl_build_config_version "$(GSL_config_version)"set LCG_tarfilename "gsl-$(gsl_build_config_version).tar.gz"

apply_pattern buildscripts_local

#!/bin/sh

cd ${LCG_builddir}tar xvfz ${LCG_tarfilename}cd gsl-${LCG_package_config_version}./configure --prefix=${LCG_extdir}/GSL/${LCG_package_config_version} /${LCG_CMTCONFIG}

#!/bin/sh

cd ${LCG_builddir}/gsl-${LCG_package_config_version}make

#!/bin/sh

cd ${LCG_builddir}/gsl-${LCG_package_config_version}make install

lxplus:~ >setenv CMTCONFIG slc4_amd64_gcc34lxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_getlxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_configlxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_makelxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_install

19. Nov. 2007 LHCC Comprehensive Review - SPI 12

Benefits of new s/w process

• Avoid duplication of information• Easier maintenance / automation• All parties use the same info (nightly builds,

release, users rebuilding the software)• Speedup time to user

Current configuration

LCG 53f has been

built with these tools

Communication

19. Nov. 2007 LHCC Comprehensive Review - SPI 14

Hypernews

• Bulletin board system used by LHC experiments

• Moved to new IT provided hardware– 1 production machine, 1 hot standby

• Helping with instance maintenance – Managing user DB, setup of for a

• Atlas and CMS heavy users– Also used in Alice and Totem– LHCb is rethinking their usage

19. Nov. 2007 LHCC Comprehensive Review - SPI 15

Savannah

• Savannah is the main bug reporting system in LCG/AA

• Moved to new IT provided hardware• Used by many other parties (IT, Grid, etc.)• Reasonable system / user maintenance

0

50

100

150

200

250

aug'02feb'03aug'03feb'04aug'04feb'05aug'05feb'06aug'06feb'07aug'07

Active Projects

0

500

1000

1500

2000

2500

3000

aug'02feb'03aug'03feb'04aug'04feb'05aug'05feb'06aug'06feb'07aug'07

Registered Users

0

50

100

150

200

(days)

aug'02feb'03aug'03feb'04aug'04feb'05aug'05feb'06aug'06feb'07aug'07

Bug Average Lifetime

05000

1000015000200002500030000

aug'02feb'03aug'03feb'04aug'04feb'05aug'05feb'06aug'06feb'07aug'07

Submitted Bugs

0

500

1000

1500

2000

2500

aug'02feb'03aug'03feb'04aug'04feb'05aug'05feb'06aug'06feb'07aug'07

Submitted Support requests

Hardware

19. Nov. 2007 LHCC Comprehensive Review - SPI 17

Hardware

• LCG / AA build servers– 4 linux machines (managed by IT/FIO)– 2 mac osx machines (one ppc)– 2 windows machines

• Special purpose machines– Web / database servers

• Recycling old Hypernews machines (located in CC)

– New machines for• Savannah, Hypernews

Staff & Program of Work

19. Nov. 2007 LHCC Comprehensive Review - SPI 19

Staff

• Change of project leader– Andreas Pfeiffer left to CMS– Stefan Roiser took over

• 1 staff left, 1 fellow joined– Manuel Gallas left for Atlas– Piotr Kolet joined

• 1 summer student– Luana Micallef

• Currently 3.2 FTE

19. Nov. 2007 LHCC Comprehensive Review - SPI 20

Program of work

• Completion of the integrated software configuration / testing / release – Web pages derived from LCGCMT– Integrate remaining release steps

• Consistency checking• Production of distribution kits• Documentation generation

– Completion of integration with nightly builds– Integration of installation manager

19. Nov. 2007 LHCC Comprehensive Review - SPI 21

Program of work (ctd.)

• Nightly build system– Integrate Geant4 builds

• Using IT infrastructure for– Setup of common LCG/AA web services– Setup of database services– Virtual windows machines for s/w build

19. Nov. 2007 LHCC Comprehensive Review - SPI 22

Program of work (ctd.)

• Hypernews– Investigate usage of IT provided SSO– Find solution for longterm support

19. Nov. 2007 LHCC Comprehensive Review - SPI 23

Conclusion

• Current focus on LCG/AA s/w stack– Optimize release process– Facilitate and unify processes for

• Nightly builds• Release procedure• User rebuilds

• Outsourcing of maintenance for other services, but still a lot of SPI manpower involved– Savannah– Hypernews– Hardware