towards an information system product team
DESCRIPTION
Towards an Information System Product Team. David Horat EGEE’09, Barcelona, Spain 24 th September, 2009. Objectives. Where we are today Where we might be tomorrow Managing the transition Software planning. Where we are today. Where we are today. JRA1. SA3. Middleware Engineering. - PowerPoint PPT PresentationTRANSCRIPT
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
Towards anInformation System
Product Team
David Horat
EGEE’09, Barcelona, Spain
24th September, 2009
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Objectives
1. Where we are today
2. Where we might be tomorrow
3. Managing the transition
4. Software planning
Towards an Information System Product Team 2
• Today • … • Tomorrow
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Where we are today
Towards an Information System Product Team 3
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Where we are today
Towards an Information System Product Team 4
JRA1 SA3Middleware Engineering Integration Testing
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 Towards an Information System Product Team 5
Where we are today
gLite CVS
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 Towards an Information System Product Team 6
Where we are today
Savannah
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 Towards an Information System Product Team 7
Where we are today
Twiki
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 Towards an Information System Product Team 8
Where we are today
ETICS
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667 Towards an Information System Product Team 9
Where we are today
SA3
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Where we are today
Towards an Information System Product Team 10
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Where we might be tomorrow
Towards an Information System Product Team 11
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Where we might be tomorrow
1. Self Contained Software Unit– Self sufficient– Interface (What we deliver and how?)
2. Same as now with minor changes– Plus certification– Interface (What we deliver and how?)
Towards an Information System Product Team 12
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Managing the transition
Towards an Information System Product Team 13
Information System Product Team• Areas to consider
– Hosting options– Building system– Certification– Release management
• Current products– BDII– GLUE 2.0– GIP– Etc.
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Hosting options
Towards an Information System Product Team 14
[1] It is really an internal messaging system rather than a standard mailing list[2] You can use Google Sites.[3] You can use Google Groups.[4] You can use Google Groups.[5] CVS and SVN are imported into Bazaar format. No native support.[6] It is really an internal blog rather than a standard mailing list
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Hosting options: CERN SVN
Towards an Information System Product Team 16
• CVS SVN• Savannah + Twiki Trac
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Building system
Towards an Information System Product Team 17
• Best practices in Free Software?
• Get closer to the OS?
• Include components in OS distributions?
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Building system
A possibility …
The Fedora Build System
koji build <repo_tag> <source><repo_tag>: The repository to use for dependencies<source>: CVS URL, SVN URL, source RPM, etc.
• Dependencies resolved using the spec file
Towards an Information System Product Team 18
Koji
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Building system
Towards an Information System Product Team 19
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Building system
• Pros– Community supported– Best practices implementation– Multiple source types– One command– Process driven from the spec file– Tag based approach
• Cons– RPM based– Difficulty to build debs
Already being discussed in the community
– Other possibilities? The Debian Build System
Towards an Information System Product Team 20
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification
Towards an Information System Product Team 21
• Improve automatic tests• Virtual machine images• Automatic setup• The testbed
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: Automatic tests
• Bash scripts– Exit value 0: OK– Exit value != 0: ERROR
Towards an Information System Product Team 22
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: VM images
Towards an Information System Product Team 23
• VNode: XEN virtual machines– SL4-32-DH– SL4-64-DH– SL5-64-DH
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: Automatic setup
Towards an Information System Product Team 24
1. Update the system
2. Select repository
3. Install nodes
4. Install patches
5. Configure the system
6. Configure the node using YAIM
7. Run basic tests
And now we can start certifying …
./yaimgen.sh <input file> <log file>
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: The testbed
Towards an Information System Product Team 25
TopBDII
SiteBDII
ResourceBDII
ResourceBDII
SiteBDII
ResourceBDII
ResourceBDII
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: The testbed
Towards an Information System Product Team 26
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Certification: The testbed
Towards an Information System Product Team 27
TopBDII
SiteBDII
ResourceBDII
ResourceBDII
SiteBDII
ResourceBDII
ResourceBDII
BDII
Web Config
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release management
Towards an Information System Product Team 28
• Announcements• Bug tracking• Project portal• Code browser• XML-RPC API
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: Announcements
Towards an Information System Product Team 29
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: Bug tracking
Towards an Information System Product Team 30
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: Bug tracking
Towards an Information System Product Team 31
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: Project portal
Towards an Information System Product Team 32
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: Code browser
Towards an Information System Product Team 33
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: XML-RPC API
• Web Service• XML-RPC• Doc: http://trac-hacks.org/wiki/XmlRpcPlugin
• Manage• Tickets• Wiki• Search
Towards an Information System Product Team 34
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Release man.: XML-RPC API
import xmlrpclib
url = "https://user:[email protected]/no_sso/trac/gridinfo/login/xmlrpc"
server = xmlrpclib.ServerProxy(url)multicall = xmlrpclib.MultiCall(server)
for ticket in server.ticket.query("owner=horat"): multicall.ticket.get(ticket) print map(str, multicall())
Towards an Information System Product Team 35
Retrieving my tickets in Python
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Software planning
• Software released– BDII– BDII File Editor– GStat – Deployment statistics
• Software in which we collaborate– GLUE Schema– Yaimgen
Towards an Information System Product Team 36
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
Software planning
• To come …– GLUE 2.0 roll out– BDII maintenance
Performance and scalability improvement Investigate a more decentralized model
– GStat 2.0 release Validation probes Web application
– Yaimgen More and more automation
Towards an Information System Product Team 37
EGEE-III INFSO-RI-222667
Enabling Grids for E-sciencE
www.eu-egee.org
EGEE and gLite are registered trademarks
David Horat
http://cern.ch/horat
Thank you!