Short report on TSTG ITA INFN Activities

• Who : people

• What : install and config tests

• How : structure of the testing suite

• Where : contact info for people,code,doc

• What else : future work

Overview

• Group composition

• Install and Config test scripts– Basic assumptions– Site customization– Open issues

• Where to find people and code

People ( nobody is 100% on this one subject)

• Alessandro Cavalli - INFN CNAF Bologna

• Antonio Forte - INFN Torino

• Gennaro Tortone - INFN Napoli

• Livio Salconi - INFN Pisa

• Francesco Taurino - INFN Napoli

• Mario Reale - INFN CNAF Bologna

What did we do up to now:• Wrote down machine requirements for each

profile : UI, RB, CE, SE, WN• Designed structure of basic install and config

tests• Set up a CVS server for code and doc

– http://cvs.infn.it/cgi-bin/cvsweb.cgi/testgrid/

• Set up a reference web site for doc and links– http://cern.ch/infn-testing

• Releasing (this week ) first version (1.0 ) of the software (RPM) and the related reference document

Requirements• Split into a common part and a machine-

specific part• Common part includes:

– Check OS required services– Check the presence of mandatory files– Check the correct install of a list of reference RPMs after

downloading this list from the marianne CVS repository– Check mandatory users– Check running services (daemons)– Mandatory parameters is /etc/globus.conf– ENV variables checking– Port occupation checking– Mount point checking– GDMP components check ( mandatory files, libs)– GRID Info Systems components : check files and daemons (GRIS)– Grid Security and CA mandatory files check

Structure of the install & config test suite: basic assumptions

• /bin/bash used everywhere• split between core functions and machine-specific

test scripts• used our own function return code convenctions

– 0 = OK

– 1 = failure

– 2 = no mandatory parameter passed

• Reference EDG release : 1.2

Core Functions

• ceinfoprovcheck.sh

• checkbrokerinfo.sh

• checkenvvariable.sh

• checkglobusconfparam.sh

• checkmultipleservices.sh

• checknfsmount.sh

• checkopenport.sh

• checkservice-ps.sh

• checkservice.sh

• checkuser.sh

• checkuserenv-script.sh

• checkuserenv.sh

• common.sh

• echofunc.sh

• extractglobusconfparam.sh

• fileexist.sh

• finaloutput.sh

• gdmpuserconf.sh

• getFileParam.sh

• getfileparamhead.sh

• getfileparamtail.sh

• getglobusconfparam.sh

• getportnumber_etc_services.sh

• getportnumber_netstat.sh

• getportnumber_rb_conf.sh

• getrpmlist.sh

• grisqueryce.sh

• grisqueryse.sh

• layout.sh

• rpm-checklist-script.sh

• rpmchecker.sh

• rpmcheckernilist.sh

• seinfoprovcheck.sh

• sourcer.sh

Testing Shell scripts (1/2)• One comprehensive shell script for each

EDG main machine profile:– UI, RB, WN, CE, SE

• Each test script follows the requirements and checks them by means of the core functions

• A site-customization script sets preliminary environment for the main test script to be correctly executed and performe sensible tests according to specific site config setup

Testing shell scripts (2/2)• Currently the global test is successful only

if all individual tests are passed

• Each test is described in the std-output and the result is visualized at each step

• At the end of the execution, the summary of the total number of success and failures is reported and the final test result is presented

Site customization• A script (layout.sh) sourced by all testing script

sets the variables needed to customize the test suite for a particular site:– SEhostname=grid007g.cnaf.infn.it– CEhostname=testbed001.cnaf.infn.it– LCFGhostname=lcfgsrv– DataAccessProtocol="file rfio gridftp"– SupportedVOs="tutor alice lhcb atlas cms iteam wpsix

biome eo"– lcfgserveron=yes– LRMStypes="pbs lsf bqs"

An Example : CE test (1/3)

##############################

#### RUNNING SERVICES ######

##############################

checkservice edg-crl-upgraded

[ OK ]

checkservice edginfo-mds

[FAILED]

checkservice globus-gatekeeper

[ OK ]

checkservice globus-gsi_wuftpd

[ OK ]

####################################

#### MANDATORY FILES CHECK ####

####################################

fileexist /etc/globus.conf ...

[ OK ]

getting GRIDMAP from /etc/globus.conf and check file

[ OK ]

getting X509_GATEKEEPER_CERT from /etc/globus.conf and check file

[ OK ]

getting X509_GATEKEEPER_KEY from /etc/globus.conf and check file

[ OK ]

An Example : CE test (2/3)################################# RPM check procedure #################################verify RPM list installation

################################################### downloading the list from datagrid.in2p3.fr....##################################################

--16:47:09-- http://datagrid.in2p3.fr:80/autobuild/rh6.2/rpmlist/RB-All-v1_2_0.html => `CE-v1_2_0.html'Connecting to datagrid.in2p3.fr:80... connected!HTTP request sent, awaiting response... 200 OKLength: 49,035 [text/html]

0K -> .......... .......... .......... .......... ....... [100%]16:47:10 (123.42 KB/s) - `CE-v1_2_0.html' saved [49035/49035]verify RPM list installationrpmchecker(): ca_CERN-0.8-1 NOT INSTALLED !rpmchecker(): ca_CESNET-0.8-1 NOT INSTALLED !rpmchecker(): ca_CNRS-0.8-1 NOT INSTALLED !rpmchecker(): ca_CNRS-DataGrid-0.8-1 NOT INSTALLED !rpmchecker(): ca_CNRS-Projets-0.8-1 NOT INSTALLED !rpmchecker(): ca_DOESG-0.8-1 NOT INSTALLED ! [FAILED]

###############################

#### MDS / GRIS QUERY ####

###############################

grisqueryce

Analizyng ceId: ceid=testbed001.cnaf.infn.it:2119/jobmanager-pbs-short

Analizyng closeSE: closese=grid007g.cnaf.infn.it

Analizyng ceId: ceid=testbed001.cnaf.infn.it:2119/jobmanager-pbs-medium

Analizyng closeSE: closese=grid007g.cnaf.infn.it

Analizyng ceId: ceid=testbed001.cnaf.infn.it:2119/jobmanager-pbs-long

Analizyng closeSE: closese=grid007g.cnaf.infn.it

Analizyng ceId: ceid=testbed001.cnaf.infn.it:2119/jobmanager-pbs-infinite

Analizyng closeSE: closese=grid007g.cnaf.infn.it

The GRIS for the Computing Element is up

and data returned is complete

[ OK ]

An Example : CE test (3/3)

##################################################

# FINAL TEST RESULT : NOT PASSED #

##################################################

[FAILED]

##################################################

# TOTAL NUMBER OF SUCCESSES = 32

# TOTAL NUMBER OF FAILURES = 11

##################################################

What else ? Future developments

• Integrate this work in the EDG TSTG testsuite scheme

• Complete and integrate documentation• Include uncovered testing features, in close

collaboration with EDG TSTG (functionality tests, stress tests….)

• Collect feedback from users and correct/improve testsuite accordingly