ccsm software engineering
DESCRIPTION
CCSM Software Engineering. CCSM Software Engineering Group Model Porting and Performance. Documentation and Training. Software Engineering Practices. Lawrence Buja CCSM Software Engineering Group. CSEG. CCSM Software Engineering Group. Purpose - PowerPoint PPT PresentationTRANSCRIPT
Community Climate System Model www.ccsm.ucar.edu 1CCSM
CCSM Software Engineering
CCSM Software Engineering GroupModel Porting and Performance.Documentation and Training.Software Engineering Practices.
Lawrence BujaCCSM Software Engineering Group
Community Climate System Model www.ccsm.ucar.edu 2CCSM
CSEG
Purpose
Core group responsible for CCSM software engineering development
PositionsCSEG ManagerCSEG Quality Assurance LeadCCSM Component Liaisons
CCSM Software Engineering Group
Community Climate System Model www.ccsm.ucar.edu 3CCSM
CSEG Member PositionTony Craig CSEG ManagerLawrence Buja QA LeadErik Kluzek ATM LiaisonNancy Norton OCN LiaisonMariana Vertenstein LND LiaisonJulie Schramm ICE LiaisonBrian Kauffman CPL Liaison
Community Climate System Model www.ccsm.ucar.edu 4CCSM
Role of CCSM SE ManagerSoftware decision making within CCSM and single point of contact for all CCSM SE Issues
"allow the scientists to do science"
• Prioritize and assign SE tasks• Track activities across CCSM Groups • Coordinate internal and external developers• Model release coordination and closure• Represent SE interests/issues on SSC• Create and Maintain SE plans• Schedule SE training
Community Climate System Model www.ccsm.ucar.edu 5CCSM
CCSM PerformanceOCN
ICE
LND
ATM
CPL
8 3 7
8 37
7 4
3 7 2 14
41secs
53 secs/day total (~130 years/month)
NERSC T42/gx1v3 (128Pes)
32
8
8
32
48
Critical path
Community Climate System Model www.ccsm.ucar.edu 6CCSM
CCSM Ports
CCSM currently being tested/run on:• NCAR IBM SP• NCAR SGI Origin 2000• NCAR Compaq ES-40• LANL SGI Origin 2000• NERSC IBM SP
Community Climate System Model www.ccsm.ucar.edu 7CCSM
CCSM DocumentationCCSM Documentation Project:
- User and Developers Guide for each component
- Quick Start and User Guide for CCSM
- Internal Interface documentation
- Growing online library of CCSM SE documents
Community Climate System Model www.ccsm.ucar.edu 8CCSM
CCSM TrainingQuick-start tutorial June 2001
- Half-day tutorial on building and running the CCSM
Week long workshop Fall 2002
- Theory and practice of the CCSM
Community Climate System Model www.ccsm.ucar.edu 9CCSM
XP CX1 SEI/CMMExtreme Construx CapabilityProgramming CXone Maturity Model
Modern SE Practices
• CCSM becoming increasing complex• Many new players in the development• Target architectures continue to evolve
Software Engineering Models
Why?
Too Light Just Right Too Heavy
Community Climate System Model www.ccsm.ucar.edu 10CCSM
SE Coordination Plan
• Change Review Board (CRB) • Repository • Testing • Status Accounting • Documents • Planning and Prioritization • Management • Other Issues
Community Climate System Model www.ccsm.ucar.edu 11CCSM
Roadmap to JuneDec Jan Feb Mar Apr May Jun
CCSM2 Release
Work Shop
CCSMFreeze
CCSM2 Documentation
CPL6 beta
PaleoCCSM
CCSM2 Experiments
CCSM2 Control Runs
Community Climate System Model www.ccsm.ucar.edu 12CCSM
Fini
Community Climate System Model www.ccsm.ucar.edu 13CCSM
Roadmap to June
CCSM2 beta
CCSM2 alpha tests
CCSM2
Freeze
Oct Nov Dec Jan
CCM
CSIM
POPLSM
Components, interfaces, gridsshould be close to final state
SSCOct 17
NASA CAN due Nov 20
NASA CAN Selections Feb 5
Community Climate System Model www.ccsm.ucar.edu 14CCSM
Freezing CSM-2
• Finalizing Component Configurations• Component Acceptance• Validation Procedure
Community Climate System Model www.ccsm.ucar.edu 15CCSM
CCSM Issues
•Code Development Process (this meeting)• Who defines process?• Who coordinates the efforts?
• Integration with ACPI and NASA CAN efforts• Insufficient Software Education• Implementation of Software Engineering Plan• Testing/Validation procedures to be formalized
Community Climate System Model www.ccsm.ucar.edu 16CCSM
Role of SW MGR/COOD1. Prioritization and assignment of the SE Tasks2. Tracking activities across CCSM Groups (interface mgmt)3. Coordination (external outside CCSM)
To obtain shared codesTo track + help coordinate research codesTechnology tracking in generalAcross outside Collaborators (ACPI/NASA)
4. Software Decision Making within CCSM5. Release Coordinating/accountability/closure6. Represent SE interests/issues on SSC7. Single point of contact for other groups8. Maintaining SE plans9. SE training10 responsible for setting SE protocols
Community Climate System Model www.ccsm.ucar.edu 17CCSM
Action Items1. Erik write up CM procedure, Put in Developers Guide2. Internal meeting on building/testing. Bring desired features3. ACPI meeting. Extra day to iron out building/testing
Recommendations:1. SEI training for tech support2. Incrementally build up protocols and documents with goal of
better organizing the software process for ccsm.3. SE coordinator/manager added to present SSC
Community Climate System Model www.ccsm.ucar.edu 18CCSM
Functions Prioritization Closure Coordination
Scientific SE's working in "research mode"Collaborate with/ reviewed by Scientists Software/Utility SE's working on infrastructure developmentCoordinated by Software Professional
Negotiation of time between scientists and coordinatorCoord has power to make decisions about how things are done.
Coordination/Management
Community Climate System Model www.ccsm.ucar.edu 19CCSM
CCSM ORG ChartSSC
ATM OCN PALEO Software
scientist
programmer
research infrastructure
Coordinator
Community Climate System Model www.ccsm.ucar.edu 20CCSM
ORG Chart
research In fras tru c tu re
sc i p rog ram m er
S c ien tis t
A TM O S P H E R E O C E A N H A R D W A R E softw are
S S C coord in a to r
Move infrastructure work
Community Climate System Model www.ccsm.ucar.edu 21CCSM
THE PLAN: NCAR will organize the composition and management of its large modeling efforts to maximize the effectiveness of multidisciplinary teams.
THE PLAN: NCAR will ensure that the organizational structure of its major modeling projects integrates the expertise of both physical and computer scientists.
THE PLAN: NCAR and UCAR will develop a coordinated strategy for developing simulation models that allows the institution to be agile in allocating resources for the
development and evolution of all aspects of the end-to-end simulation environment.THE PLAN: Management of major software projects will
accommodate the broad constituency of the team effort, facilitated by project managers who have authority commensurate with their responsibility
NCAR Strategic Plan
Community Climate System Model www.ccsm.ucar.edu 22CCSM
THE PLAN: NCAR will adopt software engineering practices that promote high performance and the efficient development of large simulation models and software infrastructure.
• Establish formal software design procedures.
• Develop layered software
• Use software development tools.
• Object-oriented techniques and frameworks.
NCAR Strategic Plan
Community Climate System Model www.ccsm.ucar.edu 23CCSM
Computer = processor and memory layoutcomplexity = computation speed + data motionPerformance obtained by minimizing complexity for a particular
architecture and exploiting parallelism
Memory access is quantized in cache lines (4-16 words)Pay cache load cost whether use 1 value or all values in line
Registers fastest (on chip)L1 Cache (on chip, 2-3 clock cycles away)L2/L3 Cache (off chip)Local/remote memory
Scalar Processor Optimization
Community Climate System Model www.ccsm.ucar.edu 24CCSM
CCSM Issues
•Code Development Process (this meeting)• Who defines process?• Who coordinates the efforts?• What design docs are necessary?
• Integration with ACPI and NASA CAN efforts• Insufficient Software Education• Implementation of Software Engineering Plan•Testing/Validation procedures to be formalized
Community Climate System Model www.ccsm.ucar.edu 25CCSM
ACPI Update (rosinski)• Short duration DOE initiative to speed up climate models
• Oak ridge, LLNL, ANL, NASA DAO NCAR,
• 2-d decomp of ccm and modularize physics/dynamics• Initial focus on CPL and ATM (see drake web page)• Numerous targets completed (truesdale doing config manage)• 2-d specific problems: Transposition logic (FFT• Oak Ridge doing Eulerian spectral and Semi-lagrangian
spectral parallelization• LLNL (rotman/lin/mirin) doing Lin-Rood parallelization• 5.5 Years/day on 64 Winterhawk II processors standalone• Little impact from NetCDF output•CPL: two tracks: Dec freeze track w/ classic cpl
NGC track: In prototyping phase
Community Climate System Model www.ccsm.ucar.edu 26CCSM
NASA CAN (deluca) Increasing Interoperability and performance of Grand Challenge Applications in the earth, space, life and microgravity sciences
Testing/Validation crucial. Allows decision making and schedulingUp Front Design is very important.
Structure: things that people deal with day-to-dayProcess: How we do things. New stuff
Concentrate on structure things that help us do our job better.
Community Climate System Model www.ccsm.ucar.edu 27CCSM
NASA CAN (deluca) Increasing Interoperability and performance of Grand Challenge Applications in the earth, space, life and microgravity sciences
Testing/Validation crucial. Allows decision making and schedulingUp Front Design is very important.
Structure: things that people deal with day-to-dayProcess: How we do things. New stuff
Concentrate on structure things that help us do our job better.
Community Climate System Model www.ccsm.ucar.edu 28CCSM
Summary
Community Climate System Model www.ccsm.ucar.edu 29CCSM
NASA CAN (deluca) Increasing Interoperability and performance of Grand Challenge Applications in the earth, space, life and microgravity sciences
Testing/Validation crucial. Allows decision making and schedulingUp Front Design is very important.
Structure: things that people deal with day-to-dayProcess: How we do things. New stuff
Concentrate on structure things that help us do our job better.1st start with pieces, then bring it all together.
Community Climate System Model www.ccsm.ucar.edu 30CCSM
Misc items1. Configuration management2. Coding Convention3. Build procedures and user support4. Test scripts and validation5. Coupler interfaces
Community Climate System Model www.ccsm.ucar.edu 31CCSM
Supervision1. Many Ses continuing to report to scientists2. Some report directly to se coord/mgr3. coord/mgr reports to maurice/ssc