www.nasa.gov

National Aeronautics Space Administration

Using a Systems Engineering Approach to Develop NASA Engineering Talent

Mr. John Marinaro and Dr. James MayNASA Safety Center

2

What does NASA’s future hold?

3

One Thing is Certain…

NASA faces an Engineering Challenge!Of the 11,216 NASA Engineers, 60% will be eligible to retire in the next 10 years. Of the remaining engineering population, 33% have less than five years of experience. These statistics indicate that NASA will face an engineering knowledge drain over the next decade.

This is not a Center-specific challenge, but an Agency and Leadership Challenge.

NASA Engineering Population

Eligible for Re-tirement

<5 Yrs Experience

>5 Years Expe-rience

4

The Data to Support the Challenge (2011)

Eligible to retire in the next 10 years with full benefits: 6710 Percent of the NASA Engineering population: 60%

NASA 08xx Series (All Centers)

 Service\Age Under 20 20 to 24 25 to 29 30 to 34 35 to 39 40 to 44 45 to 49 50 to 54 55 to 59 60 to 64 65 to 69 70 or

older Total

Under 5 6 347 399 246 197 156 163 136 57 16 5 0 1728

5 to 9 0 1 257 326 191 186 232 153 69 34 6 3 1458

10 to 14 0 0 3 214 194 208 274 189 85 47 18 7 1239

15 to 19 0 0 0 4 86 147 133 118 50 26 9 2 575

20 to 24 0 0 0 0 7 529 1081 596 304 145 47 17 2726

25 to 29 0 0 0 0 0 13 583 940 250 103 45 9 1943

30 to 34 0 0 0 0 0 0 21 469 343 121 48 17 1019

35 to 39 0 0 0 0 0 0 0 14 175 112 22 9 332

40 or more 0 0 0 0 0 0 0 0 6 46 72 72 196

Total 6 348 659 790 675 1239 2487 2615 1339 650 272 136 11,216

5

Looking 10 Years Forward

NASA 08xx Series (All Centers)

 Service\Age Under 20 20 to 24 25 to 29 30 to 34 35 to 39 40 to 44 45 to 49 50 to 54 55 to 59 60 to 64 65 to 69 70 or

older Total

Under 54400 new Engineer hires in the next 10 years!

4400

5 to 9

10 to 14 6 347 399 246 197 156 163 136

15 to 19 0 1 257 326 191 186 232 153

20 to 24 0 0 3 214 194 208 274 189

25 to 29 0 0 0 4 86 147 133 118

30 to 34 0 0 0 0 7 529 1081 596

35 to 39The Senior Tier Retires – ~4400 Engineers

40 or more

Total 6 348 659 790 675 1226 1883 1192 11,216

The year is 2021…

6

What Does this Data Mean to You?

10 years from now, if you are NASA Engineer and are not retired

– GET READY!

ARE YOU READY TO LEAD THIS CHANGE – while managing a challenging and demanding NASA Project?

You should be, this is an opportunity that affects all NASA Organizations…

This presentation describes how the NASA Safety Center used the Engineering Lifecycle Model to achieve project success.

http://nsc.nasa.gov/7

How the NASA Safety Center is Tackling the Safety & Mission Assurance Challenge

Project Fundamentals and SMA Engineering Technical Excellence

8

NASA Safety Center

In 2005, the NASA Safety Center was established in Cleveland, OH

NASA Headquarters Strategic Guidance -- Build systems to improve NASA SMA knowledge, information and capability

Three primary impetus factors: CAIB Report Demographics Need for engineering professionals to be trained in SMA

9

NASA Safety Center (NSC)

First and foremost, the NSC is not the NESC or the NSSC

The NSC is comprised of four primary Directorates: Technical Excellence Knowledge Management Systems Audits & Assessments Mishap Investigation Support

10

SMA and STEP

Safety and Mission Assurance Technical Excellence Program (STEP) Agency’s Professional Development Systems for SMA professionals – from

fresh-out to Subject Matter Expert

Six major SMA Disciplines: System Safety, Quality Engineering, Reliability & Maintainability, Operational Safety, Software Assurance, and Aviation Safety

Career-oriented and competency-based

Appropriate for SMA, Engineering, and Project Managers

Heavy emphasis on web-based training via SATERN

11

STEP’s Overnight Success

In less than 3 years, STEP has become recognized Agency-wide as the way SMA trains

750 SMA Civil Servants have voluntarily completed Level 1 (75% of the SMA population)

113 SATERN courses developed (650 hours of engineering-oriented training)

45,000+ hours of training completed

Highly successful NASA engineering-oriented training program

12

Success Contributors

How did we accomplish so much so quickly? Inspiring leadership Dedicated Civil Servant/Contractor team Technical Discipline Teams with representatives from each Center SMA

organization (60+ total members)

and, most importantly, a solid process:The NASA Systems Engineering Project Lifecycle Model (NPR 7120.5)

13

Treat it like a NASA Engineering Challenge / Project

This lifecycle model works for Shuttles, Satellites, Airplanes, Automobiles, Ships, etc.

It also works for Training Program Development – from Formulation to Implementation

The NASA Systems Engineering Project Lifecycle Model (NPR 7120.5)

14

The NASA Engineering Lifecycle Model

Five Primary Phases:A. Concept & Technical DevelopmentB. Preliminary Design and Technology CompletionC. Final Design & FabricationD. System Assembly, Integration, Test and RolloutE. Operations and Sustainment

15

NASA Safety Center’s STEP as an Engineering Professional Development Model

That’s exactly what we did for the NASA SMA community which accounts for 10% of the NASA Engineering Workforce…

We analyzed the problem / challenge

Then: We conceived,

We designed,

We fabricated,

We tested,

We operated and sustained, and

(when necessary…) We will closeout.

This model and process helped us evolve NASA’s SMA career-oriented, professional development system – STEP

16

We Started with a Vision

My VisionCreate the NASA University for Safety and ultimately become the Harvard and MIT of NASA Safety.

Vision Tip

Your Vision should fit on the back of your Business Card (clear and concise) – Professor Bart Timm, Georgetown University Executive Leadership Program

17

We Conceived

In Conceiving, we: Looked at successful technical professional development models Benchmarked NASA and Industry Safety Training Programs Spent time with our customers and stakeholders (NASA SMA and Engineering) Engaged NASA Human Resources and the NASA Shared Services Center (NSSC) Considered the career life-cycle of our SMA Engineers Built six Technical Discipline Teams with representatives from each NASA

Center SMA organization Conceived a common framework Focused internally on team development and cohesion Developed a robust development and implementation schedule with Key

Decision Points, Milestones, Phased Rollout Strategies, and built-in contingency slack for two of the key events

Developed a robust formulation to implementation cost analysis

18

Engineering Professional Development

Engineers Council for Professional Development (1979)"Engineering is the profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind.”

19

Concept Result

Career-oriented Professional Development Program (duration: 8 – 10 years)

Six major Discipline Programs System Safety Software Assurance Quality Engineering Reliability and Maintainability Operational Safety Aviation Safety

Components Four Qualification Levels 488 hours of Academics 1000 hours of OJT Comprehensive Test Peer-review

Attributes Comprehensive and Credible Competency-based Engineering-oriented

Safety and Mission Assurance Technical Excellence Program (STEP)

20

We Designed

In Designing, we: Explored off-the-shelf and advanced technological solutions Used Bloom’s Taxonomy (Hierarchy of Learning) for each Discipline Developed major competencies, minor competencies, learning and

performance objectives for each minor competency (~250 minor competencies)

Documented, documented, documented… Peer-reviewed, peer-reviewed, peer-reviewed… Assembled NASA HQs independent-review team for the major Milestone

events Developed test strategies and included time in the schedule Developed minimum success criteria Orchestrated Agency SMA organizational momentum strategies Briefed every Center SMA Director face-to-face

21

Design Results – Competency Wheels

System Safety

6 Disciplines – over 250 Engineering Competencies

For each competency, detailed objectives were written to describe exactly what performance is expected at each level. (Typical performance objectives shown.)

Design Results – Performance Objectives

23

Design Results – Curriculums

04/10/2023

System Safety Level 2 Core Training Discipline Training Readings and Resources Domain Training OJT Enrichment Activities

24

We Developed

In Developing, we: Located off-the-shelf, best practice NASA and Industry tutorials and Subject-

matter Experts focused on our courses and learning/performance objectives Bought a video integration tool (Mediasite) to capture live courses and

simultaneously integrate video, Powerpoint, and audio into web-based solutions for SATERN and webcasting

Reshaped our team to integrate new skill-sets and capabilities Rapid-prototyped new course solutions (Build a little / Test a little) Collaborated and pooled resources and needs with APPEL and other Center

SMA and Engineering initiatives Created licensing agreements with commercial providers to capture SATERN-

ready courses and lectures for unlimited NASA use by Civil Servants and Contractors

Created Virtual Instructor-led Courses that are webcasted live training events

Development Results – STEP Course Delivery Methodology

Levels 2-4 Courses offerings are: Web-based e-Learning (WBT) via

SATERN

Existing NASA instructor-led classroom training (ILT)

Videotaped ILT lectures using Sonic Foundy’s Mediasite technology

Virtual ILT courses using Webex and telecon

External WBT and ILT courses (Industry/Commercial Providers)

25

26

We Assembled and Tested

In Assembly and Test, we: Performed subsystem tests on the individual courses and learner performance

tests Simulated the operational in environment through internal (NSC) and external

(GRC & MSFC SMA) full Beta-test runs of the Level 1 & Level 2 Curriculums in SATERN

Refined the courses and implementation based on Beta-test results and feedback

Implementation/Operations Preparation: We planned and simulated a full-scale rollout that included an Agency-wide

SMA webcast that was attended by 1000 participants The Program Manager personally visited every NASA Center and HQs in the

four months prior to Program rollout and commissioning and met with each Center SMA Director and their senior leadership team

27

We Conducted Operations

In Operations from 2009-2011: Provided over 45,000 hours of web-based and instructor-led NASA-

oriented training

Level 1 (Graduated/Enrolled): 2143 / 2246 (95% of the Active Learners)

Level 2 (Graduated/Enrolled): 29 / 331 (9%)

Level 3 Graduated (CS): 1

Civil Servants Active in Levels 2 – 4 (2011): 193

Civil Servants Hours Levels 2 – 4 (2011): 18,708.5 (includes OJT)

STEP - A NASA SMA Engineering Transformation

SMA Instructor-Led & Web-Based Learners (July 2006 – December 2010)

Pre-STEP STEP

- 4400% increase in web-based training in the first operational Quarter of STEP Level 1!- 550% increase in NASA’s overall Safety training usage

Impact of Web-Based Training

29

In Summary

The quest for NASA Technical Excellence is never ending and the Organization’s Professional development system must be continuously assessed from both the capability and strategic application perspectives…

We strategically apply the STEP Program to the SMA Community We continuously measure performance We listen to the voice of the customer (Learners) and stakeholders

(SMA Leaders)

30

Parting Thoughts to Ponder

As a NASA Project and Engineering Leader…Are you using APPEL and other Engineering Professional Development

capabilities from a strategic perspective (do you have learning and development goals for each team member associated with team goals) or are team members choosing their own destiny (which may or may not be where the team needs them to be)?

Is your organization truly a driven and optimized learning organization?

Are your engineers using techniques from the 1980’s or are they using today’s industry and government best practices?

http://nsc.nasa.gov/31

I’m interested in your thoughts and some discussion…