DHS SCIENCE AND TECHNOLOGY

Developing Leaders for Test and EvaluationITEA T&E Symposium

Test Area ManagerOffice of Test and EvaluationScience and Technology Directorate

Mr. Wayne Dumais

Panel Members• Mr. James Cooke, Army Evaluation Command• Mr. Tomas Simms , DASD T&E• Mr. Ken Stefanek , Defense Acquisition University• Dr. Robin Poston , University of Memphis• Mr. Omar Merced , Federal Aviation Administration

DHS Science and Technology Directorate | MOBILIZING INNOVATION FOR A SECURE WORLD 2

U.S. ARMY EVALUATION CENTERMr. James C. CookeAEC Executive Director3 October, 2017

ATEC Employee Development Portfolio

Examples of Acquisition Workforce Development and Retention Incentives

• Student Loan Repayment Program - annually funded repayment of federally insured student loans

• Continuing Education/Degree Programs - School of Choice, Naval Postgraduate School Master of Science in PM or Systems Engineering, University of Tennessee Aerospace and Defense MDA Program

• Acquisition Tuition Assistance Program - permanent AAW members whom meet eligibility requirements may pursue required business hours or acquisition/ business related courses towards completion of either a bachelors or masters degree at an accredited college or university

• Recruitment Incentive, Defense Acquisition Workforce Development Fund (DAWDF) - available for AAW members appointed to a critical/hard to fill acquisition position

• Career Program (CP) Management – supports all facet of workforce (CP 16,17, 34, 35, 36, & 51) development, training, and education up through Doctorate degree (Advanced Degree Training).

Distribution Statement A – Approved for public release by DOPSR. Case # 17-S-2630 applies. Distribution is unlimited.

Acquisition & T&E Workforce Panel

Improving the DoD T&E Workforce

Prepared for 34th Annual ITEA SymposiumMr. Thomas Simms

Deputy Director, T&E Competency and Development October 3, 2017

Distribution Statement A – Approved for public release by DOPSR. Case # 17-S-2630 applies. Distribution is unlimited.

T&E Workforce Developmental Road Map FY 2018 – FY 2020

FY 2018 Annual Certification - Update DAU T&E Courses- Update Probability & Statistics CLM

FY 2019 Annual Certification - Update DAU T&E Courses- Add CLM on STAT - Add CLM on Intro to T&EAutonomous Systems

FY 2020 Annual Certification - Update T&E curriculum, as needed

• Improve efficiency of T&E activities• Advance skills of T&E Workforce• Emphasize critical thinking skills• Improve acquisition outcomes

FY 2018

FY 2019 Input:• FY 2018 NDAA• Acquisition alternative pathways• Defense Acquisition Guidebook (Update)• Intro to T&E of Autonomous Systems CLM• T&E Cybersecurity Guide (Update)• Scientific Test Analysis Techniques (STAT) CLM

FY 2019FY 2020

FY 2018 Input:• FY 2017 NDAA• DoDI 5000.02 (Update) • DoDI for Defense Business

Systems• JCIDS Cybersecurity Requirement• T&E Interoperability (Update)• T&E Cybersecurity (Update)

FY 2020 Input:• FY 2019 NDAA• Big Data Analytics

2

Acronyms:Defense Acquisition University (DAU)Continuous Learning Module (CLM)Joint Capabilities Integration Development System (JCIDS)

Distribution Statement A – Approved for public release by DOPSR. Case # 17-S-2630 applies. Distribution is unlimited.

Intern

DAWIA T&E Cert Level I

Core Plus Guide and Continuous Learning Modules (80 CLPs required every two years)

Defense Civilian Emerging Leaders Program

Acquisition Leadership Challenge Program (Beginning, Level I, II, III)

Student Loan Repayment Program / Acquisition Tuition Assistance Program

T&E Planning, Test Preparation, Test Execution, Analysis, Evaluation, Reporting, Professional

Career Timeline NH-02/3 / GS 5-11 / O-1/2/3 NH-02/3/4 / GS 12-15 / O-3/4/5 NH-03/4 / GS 13-15 / O-4/5/6

DOD T&E Acquisition Workforce

8

Functional Experience Broadening Strategic Leadership

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30Years of Service

DAWIA T&E Cert Level II DAWIA T&E Cert Level III

Harvard Leadership Communication Program

Senior Service College Fellowship Program

Rotational Training Assignments

Critical Acquisition Positions / Key Leadership Positions

Acquisition Corps Membership

Trai

ning

Req

.Tr

aini

ng

Opp

ortu

nitie

sPo

sitio

nC

ompe

tenc

ies

Skill

Senior Executive

Chief Developmental Tester

White House Leadership Development Program

Supervisor Development Course (New Supervisors & Refresher)

Subject Matter Expert

Practitioner

Defense Senior Leader Development Program

Acquisition Coded Positions

Mr. Ken Stefanek

TST 102 and T&E-Related CLEs Demographics – FY2017(As of Sep 8)

Note: “Unknown” refers to students who are not part of the Defense Acquisition Workforce (DAW). Additionally, career field data on DAW members will be reported as "Unknown“ if that information is missing from the student profile in the system.

Total inputs to date: 3,012

Engineering590 (20%)

Program / S&T Management

281 (9%)

Logistics171 (6%)

Test and Eval858 (28%)

Business Related (Cost, Fin Mgmt)

19 (>1%)

Contracting29 (>1%)

Unknown928 (31%)

Industrial Prop Mgmt/Facilities

5 (>1%)

IT51

(2%)

PQM80 (3%)

Total inputs to date: 8,229

Test and Eval3321 (40%)Engineering

1906 (23%)

Contracting66 (>1%)

Logistics226 (3%)

Unknown1727 (21%)

IT186 (2%) PQM

216 (3%)

Program / S&T Management

487 (6%)

Business Related (Cost, Fin Mgmt)

45 (>1%)

Industrial Prop Mgmt/Facilities

49 (>1%)

TST 204 and TST 303 Demographics – FY2017(As of Sep 8)

Total inputs to date: 287

Test & Eval216 (75%)

Engineering33 (11%)

Program Management

23 (8%)

Unknown12 (4%)

IT3 (1%)

Total inputs to date: 530

PQM1 (>1%)

Program Management

26 (5%)

IT6 (1%)

Test & Eval432 (82%)

Unknown12 (4%)

Engineering33 (11%)

Logistics4 (>1%)

Note: “Unknown” refers to students who are not part of the Defense Acquisition Workforce (DAW). Additionally, career field data on DAW members will be reported as "Unknown“ if that information is missing from the student profile in the system.

Preparing the T&E Workforce

STEP and the Science of Testing Approach

System Testing Excellence Program

A Partnership for Advancing the Science of Testing

A Combat Support Agency

13

From the Beginning…STEP and FedExWorkforce Development based on the

Science of Testing

• Collaboration started in 2006 to strengthen science of testing and industry best practices

• STEP explores the interdisciplinary boundaries of computer engineering, computer science and information systems

• T&E is not just a stage after coding but ubiquitous

• STEP research and training partner with FedEx

Expanding the Science of Testing

• Industry-based research in software testing

• Annual international research workshop on advances and innovations in software testing

• Foundational and Advanced level training certification

• Emphasizing research-based industrial best practices

Thank you!

Find out more: step.memphis.edu

Acquisition & T&E Workforce Development

Presented to: 34th Annual ITEA SymposiumBy: Omar Merced, Electronics Engineer – Test and Evaluation DivisionDate:October 3, 2017

18

Current Acquisition & T&E Workforce Core acquisition & T&E workforce consists

of approximately 1,830 federal employees.LeadershipProgram / Project ManagementResearch and Engineering / Systems Engineering

Business - Financial ManagementContractingRealty Specialist – Real Estate Contracting Officers/Specialist (RECO)

Contracting Officer’s Representative (COR)Integrated Logistics SupportTest and EvaluationAcquisition LawSpecialized Support

19

Acquisition & T&E Workforce Potential Future Losses

Attrition Forecast: September 2017: Aprox. 600,000 (31 percent) of the current federal workforce will be

eligible to retire*. September 2017: 29 percent of the acquisition workforce will be eligible to retire.**

*Federal Workforce: Recent Trends in Federal Civilian Employment and Compensation”, General Accountability Office, January 2014, GAO-14-215.

20

Acquisition & T&E Workforce Development –Balancing Act

Meeting Needs of Current Workforce Building/ Retaining Workforce of the Future

Acquiring/Utilizing the Right Technologies Enabling Right Processes and Procedures

FAA Technical Center’s T&E Mission

TECHNOLOGIES PROCESSES AND PROCEDURES

PEOPLE

21

Enabling Complexity Consciousness CultureOperational Integration of Technology, People, and Procedures

Current Efforts- Introducing Agile Testing to T&E Efforts- Lessons Learned Modifications to T&E

Handbook

FAA Technical Center’s T&E Mission

TECHNOLOGIES PROCESSES AND PROCEDURES

Current Efforts- Pathways Internship Program- Tuition Assistance Program- Structured Aviation STEM Program- Establish Professional Certifications Programs

- Strategy 1: Maintain core acquisition staffing levels

- Strategy 2: Continue to strengthen workforce capability

- Strategy 3: Sustain cross-agency focus on the acquisition workforce

Current Efforts

- Distributed Simulation Capabilities -Trajectory Based Operations (TBO)

- Transition System Testing to Capabilities Testing

- Laboratory Integration Across Multiple Domains and with External Stakeholders

PEOPLE

22

DHS SCIENCE AND TECHNOLOGY

Developing Leaders for Test and EvaluationITEA T&E Symposium

Test Area ManagerOffice of Test and EvaluationScience and Technology Directorate

Mr. Wayne Dumais

T&E in a time of Risk & Change

• Risk• Resources• Workforce recruitment and retention

• Staying sharp

• Change• Technology

• Need for complex products and systems of systems at the tip of technology development

• Users and PMs want it now

DHS Science and Technology Directorate | MOBILIZING INNOVATION FOR A SECURE WORLD 23

What is DHS doing?• Developing two new T&E courses• T&E managers of major acquisition programs must be certified• Monthly T&E Working-level Integrated Product Team (WIPT)

• Share T&E news across DHS Components• Monthly T&E Learning Seminars

• Reliability and T&E• MOEs and MOSs• Introduction to Agile Design and Requirements• Human Systems Integration

• T&E Workshops• Cybersecurity T&E• Certified ScrumMaster• Intro to STAT• Software Reliability

DHS Science and Technology Directorate | MOBILIZING INNOVATION FOR A SECURE WORLD 24

New DHS T&E Courses• Developing new T&E courses for FY18

• T&E for IT Systems (a future requirement for Level III T&E Certification)• Specific topics to help T&E Managers conduct T&E for IT Systems

• IT systems as defined by DHS• Risk Management Framework (RMF) overview• Cybersecurity and Cybersecurity T&E

• Inputs to IEF• T&E strategy & resources

• Cybersecurity T&E facilities• Agile development and T&E in support of Agile

• OTD• DHS-specific course for Operational Test Directors • Must have Level II T&E Certification to attend• Specific topics to assist OTDs

• Updating and aligning DHS TEV 200 and 300 courses

DHS Science and Technology Directorate | MOBILIZING INNOVATION FOR A SECURE WORLD 25