Symposium Committee

Schedule-at-a-GlanceOct 2 & 5 - Symposium Tutorials (separate fee required)Oct 3-5 - Symposium Plenary and Technical Sessions

Program DescriptionThe current government climate can pose a change to the defenseacquisition system. It is important to understand ways to improvecommunications and leverage existing investments made by bothDoD and commercial firms to preserve U.S. technological superior-ity. How will the current military budget, new administration, and thedefense industry adjust in a time of risk and change? Is your organization ready?

We encourage you to help us explore these ideas and others relatedto “T&E in a Time of Risk and Change” as part of our 34th AnnualInternational Symposium.

Featured Speakers

REGISTER ONLINE AT: www.itea.org/symposium

Thank You to Our Sponsors!Gold Level Sponsors Silver Level Sponsors Bronze Level Sponsor

Mr. Dave DumaActing Director, Operational Test andEvaluation (DOT&E)

Steve Hutchison, Ph.D.Director of T&E, Department of HomelandSecurity (DHS)

Mr. Derrick HintonActing Director, Test ResourceManagement Center (TRMC),Office of the Under Secretaryof Defense for Acquisition,Technology and Logistics

J. Brian Hall, Ph.D.Deputy Assistant Secretaryof Defense, Office of theUnder Secretary of Defensefor Acquisition, Technologyand Logistics

Maj. Gen. Matthew MalloyCommanding General,Air Force Test and EvaluationCenter (AFOTEC)

Mr. Paul MannPrincipal Deputy, Test Resource Management Center(TRMC)

Arun A. Seraphin, Ph.D.Professional Staff Member,Senate Arms Services Committee

Jaime FigueroaDeputy Director, WJHughes FAA TechnicalCenter, Federal AviationAdministration (FAA)

Symposium ChairMr. Erwin SabileBooz Allen Hamilton

Technical Program Co-Chairs Mr. Tom Treakle, DellEMCBruce Einfalt, AppliedResearch Laboratory/PSU

International Panel ChairMs. Gloria Deane, DOT&E

Awards Committee ChairMs. Stephanie Clewer, PAE

Exhibits and Sponsorship ChairMs. Lena Moran, TRAX International

Mr. Kevin GatesProfessional Staff MemberHouse Armed Services Committee

34th International Test and Evaluation SymposiumHyatt Regency • 1800 Presidents Street, Reston, VA 20190 • Ph: 703-709-1234 Fx: 703-925-8244

Special Panels

> Acquisition & T&E Workforce Panel

Chair: Terry MurphyDeputy Director, Policy and Workforce Development, DHS

Mr. Omar MercedExecutive Engineer, FAA

Mr. Ken StenfanakDAU

Mr. Thomas SimmsDeputy Director, Policy and WorkforceDevelopment, DASD DT&E

Robin Poston, Ph.D.Professor and Dept Chair, BusinessInformation & Technology, and Director Systems Testing ExcellenceProgram, University of Memphis

Mr. James C. CookeExecutive Director,U.S. Army EvaluationCenter (AEC)

> OTA Commanders Panel

Chair: Catherine Warner, Ph.D.Science Advisor and Director Operational Test and Evaluation(DOT&E), Office of the Secretary ofDefense (OSD)

RADM Paul SohlCommander,Operational Test &Evaluation Force(COMOPTEVFOR)

MG John W. CharltonCommanding GeneralUS Army Test andEvaluation Command,(ATEC)

Maj. Gen. Matthew H. MolloyCommander,Air Force OperationalTest and EvaluationCenter (AFOTEC)

COL Mark T. BrinkmanDirector, Marine Corps OperationalTest and Evalaution (MCOTEA)

> Test Ranges Panel

Ms. Leslie TaylorNaval Air WarfareCenter Aircraft Division / DeputyAssistant Commanderfor Test and

Evaluation Naval Air Warfare CenterAircraft Division (NAWCAD)

Brigadier GeneralEric L. Sanchez Commanding General White Sands MissileRange (WSMR)

Brigadier GeneralCarl E. SchaeferCommander, 412th TestWing, Edwards AirForce Base

Mr. Eric SpigelNUWC Newport,Code 70, Head of TheRanges, Engineeringand Analysis Dept;SSTM USW T&E; NUWCDiv Newport T&E

> Modeling and Simulation Panel

Chair: I D (Dai) Morris MA, D.Phil,M.Inst.P, C.Phys, Cert. IoDHead of Capability(Weapons, Evaluationand Capability

Assurance), UK Ministry of Defence

Lt. Gen. Robert WalshDeputy Commandantfor Combat Develop-ment and IntegrationMarine Corps CombatDevelopment Command

Colonel Robert H.EpsteinCommander, Air ForceAgency for Modelingand Simulation(AFAMS)

Colonel Joseph NolanDeputy DirectorUS Army Modelingand Simulation Office

Jim Clifton, Ph.D.PM JSF Joint Simulation Environment

Syed Mohammad, Ph.D.S&T, Department of Homeland Security(DHS)

> Cybersecurity Panel

Chair: Mr. William Redmond, SESExecutive Director, AFOTEC

Paul Waters, Ph.D.AFTC

Mr. John Garstka (Inivited)Deputy Director for Cyber Programswithin the Office of the Deputy Assistant Secretary of Defense forCommand, Control, Communications,Cyber & Business Systems; Office ofthe Under Secretary of Defense forAcquisition, Technology, and Logistics

Mr. James WellsDeputy Director, Cyberspace & Enterprise Programs, DHS

Mr. Rick QuadeN94B, OPNAV

Maj Gen Denker (Invited)Director, NSA

> Space Panel

Chair: Mr. Steven KremerAdvanced Projects Office,NASA/GSFC’s Wallops Flight Facility

Robert E. Lindberg Jr., Eng.Sc.D.Vice President, Products and Programs, Vector Launch

Robert L. Bayt, Ph.D.Verification and Validation Manager,NASA Commercial Crew Program

Mr. Daniel HicksCEO, Spaceport America

> Sponsorship Information

ITEA is a 501(c)(3) professional educationassociation dedicated to the education andadvancement of the test and evaluation pro-fession. Registration fees, membership dues,and sponsorships are tax deductible.Your sponsorship dollars help defer the cost

of the Symposium and support the ITEA schol-arship fund, which assists deserving studentsin their pursuit of academic disciplines relatedto the test and evaluation profession. Sponsor-ship and related benefits will become effectiveon receipt of payment.

NOTE: This ITEA event is a non-competitiveenvironment meant for a free exchange ofideas and information.

If you would like more information on sponsor-ing at this event, please contact Mr. JamesGaidry at symposium@itea.org

All Symposium Sponsorships provide the following recognition:• On the ITEA Web site - Your logo and ahyperlink to your Web site posted on theSymposium Web page.

• On signage throughout the Symposium• In all Symposium Collateral Materials• In the Symposium Program Guide• In the March, June, and September 2017issues of The ITEA Journal of Test and Evaluation

• Reserved seating at the Symposium Plenarysessions

PREMIUM SPONSORSHIP LEVELS

PLATINUM LEVEL SPONSORSHIP* - $10,000• Additional recognition as a Platinum Sponsor– Your Company’s logo will be placed on thehotel room key cards that will be given to allSymposium attendees staying at the hosthotel.

• Six (6) Complimentary Symposium Registra-tions. NOTE: Tutorial registration is separateand not included.

GOLD LEVEL SPONSORSHIP* - $5,000• Four (4) Complimentary Symposium Registra-tions. NOTE: Tutorial registration is separateand not included.

SILVER LEVEL SPONSORSHIP* - $2,500 • Two (2) Complimentary Symposium Registra-tions. NOTE: Tutorial registration is separateand not included.

BRONZE LEVEL SPONSORSHIP* - $1,000• Available only for Small Organizations ofunder 50 employees.

• One (1) Complimentary Symposium Registra-tion. NOTE: Tutorial registration is separateand not included.

ADDITIONAL SPONSORSHIP OPPORTUNITIES(Offered on a “first come, first served” basis)

• $15,000 – TUESDAY EVENING NETWORKINGRECEPTION* (One Sponsorship available).Your logo will be displayed as the Wednes-day Evening Networking Reception’s spon-sor on signage at each food station.

• Includes 150 drink tickets with company logoand Eight (8) Complimentary SymposiumRegistrations.

• $10,000 – SYMPOSIUM POCKET GUIDE*(One Sponsorship available). The SymposiumPocket Guide is a handy tool for the Sympo-sium attendees. It provides them with an at-a-glance look at the events for theSymposium as well as a floor map of thehotel’s meeting space. Your logo will beprinted on the Symposium Pocket Guide thatwill be distributed throughout the Sympo-sium and Exhibit Hall. Includes Six (6) Complimentary Symposium Registrations.

• $10,000 – WEDNESDAY AWARDS LUNCH-EON* (One Sponsorship available). Your logowill be displayed as the Wednesday AnnualTest and Evaluation Professional AwardsLuncheon’s sponsor on signage during theluncheon. Includes Six (6) ComplimentarySymposium Registrations.

• $5,000 – TUESDAY LUNCH BUFFET* (TwoSponsorships available). Your logo will bedisplayed as the Sponsor of the TuesdayLunch Buffet on signage throughout theExhibit Hall. Includes Four (4) ComplimentarySymposium Registrations.

• $5,000 – TUESDAY, WEDNESDAY, andTHURSDAY BREAKS* (Four Sponsorshipsavailable). Your logo will be displayed as theSponsor of the Breaks on signage through-out the Break area. Includes Four (4) Compli-mentary Symposium Registrations.

• $5,000 – WIRELESS HOTSPOT* (One Spon-sorship available). Recognition of your Sponsorship of the Wireless Hotspotthroughout the Exhibit Hall that attendeeswill visit to check e-mail messages and surfthe Web. Includes Four (4) ComplimentarySymposium Registrations.

• $1,000 – PLENARY SPECIAL PANEL (FiveSponsorships available). Recognition of yourSponsorship of a Plenary Panel Session withleading authorities and visionaries from theT&E Industry. Includes One (1) Complimen-tary Symposium Registration.

• $500 – 25 DRINK TICKETS FOR CASH BAR ATTHE TUESDAY EVENING NETWORKINGRECEPTION (Multiple Sponsorships avail-able) – Drink tickets imprinted with your logofor you to pass out to your valued customers,prospects, and guests.

Continuing Education Units (CEUs)Each 4-hour Tutorials provide 4 contact hours of instruction (4 CEUs) that are directly applicableto your professional development program, including the Certified Test and Evaluation Profes-sional Credential (CTEP).In addition to the Pre-Symposium Tutorials, the Annual Symposium provides 4 contact hours of

instruction (4 CEUs) for each half-day, 8 contact hours of instruction (8 CEUs) for each full-day, or20 contact hours of instruction (20 CEUs) for attending the full Symposium, that are directly appli-cable to your professional development program, including the Certified Test and Evaluation Pro-fessional Credential (CTEP).

ITEA has a room block available to all of our attendees andwe are truly pleased to offer our attendees the prevailinggovernment per diem rate (exclusive of taxes).

*Includes one (1) year ITEA Corporate Membership for First Time Sponsors of an ITEA Event

REGISTER ONLINE AT: www.itea.org/symposium

34th International Test and Evaluation SymposiumHyatt Regency • 1800 Presidents Street, Reston, VA 20190 • Ph: 703-709-1234 Fx: 703-925-8244

2017 Symposium TutorialsNOTE: These tutorials require a separate fee from the Symposium. See Registration Information.

Tutorials AbstractsCybersecurity Test & EvaluationInstructor: Pete Christensen, CTEP – Cyber Support to OSDPrograms, The MITRE Corporation

Now more than ever, Program Managers (PM) mustensure that cybersecurity be given careful considerationthroughout the system life cycle. Specifically this includesidentifying cybersecurity requirements early in the acqui-sition and systems engineering life cycle. Initiating a focuson cybersecurity earlier will provide PMs the opportunityto give careful consideration, upfront, to related cyberse-curity testing activities that can be integrated into theengineering planning and design phases. Results of infor-mal cybersecurity testing can then be applied to influencedesign and development efforts and to posture programsfor success in Developmental Test (DT) and OperationalTest (OT). The Deputy Assistant Secretary of Defense(DASD) Developmental Test and Engineering (DT&E) hascollaborated with key systems engineering stakeholders todevelop disciplined processes that will assist ProgramManagers (PM) in implementing an incremental and itera-tive phased approach to develop cyber secure systems.The National Cyber Range (NCR), under the purview of theTest Resource Management Center (TRMC), is a resourcethat can be leveraged by PMs to support cybersecuritytesting. This presentation will provide an overview of thecybersecurity test and evaluation phased approach andthe NCR.

Data Science and Its Relationship to Test &EvaluationInstructor: Mark Kiemele, Ph.D. – President, Air AcademyAssociates

In a data-driven economy, industry and government lead-ers rely increasingly on skilled professionals who can seethe significance in data and use data analytic techniquesto properly collect data, solve problems, create newopportunities, and shape change. Data science can bedefined as the art and science of solving problems andshaping decisions through the precise collection andanalysis of data. This tutorial is intended for executives,leaders, managers, and practitioners who need to knowhow their critical thinking can be impacted by such thingsas Big Data, Predictive Analytics, Design of Experiments(DOE) and other tools in the Data Science toolkit. Thistutorial will cover the need for critical thinking as well as ahigh-level view of a variety of data analytic tools that canbe used to enhance critical thinking. Even if one neverdesigns a test or evaluates its results, this tutorial partici-pant will be able to explain the uniqueness of DOE andwhy big data and predictive analytics are needed to gener-ate the analytical capability every organization needs.

How to Build a Reliability Growth ProgramInstructors: Shawn Brady and Wayne Martin, AMSAA,Center for Reliability Growth

Reliable systems are more likely to be fielded sooner, morelikely to be available when the Soldiers need them, andmore likely to reduce maintenance costs over the system'slife cycle. Unfortunately, many programs in Department ofDefense (DoD) fail to produce reliable systems. As a mem-ber of the defense acquisition community, are you armedwith the knowledge and tools needed to help the DoDdevelop, test, and field more reliable systems? This tuto-rial will help you answer “yes”.

For the past decade, the Army’s Center for ReliabilityGrowth at the Army Materiel Systems Analysis Activity(AMSAA) has been working to improve Army and DoD reli-ability by providing policy, guidance, standards, methods,tools, and training. This tutorial provides an overview ofthe latest methods and tools that DoD analysts should con-sider when managing or supporting a reliability program.Specific topics include adequately contracting anddesigning for reliability, identifying and mitigating reliabilityrisks early using AMSAA’s Reliability Scorecards, deter-mining appropriate reliability test durations, building arealistic reliability growth plan using AMSAA’s PlanningModel Based on Projection Methodology (PM2), and pro-jecting the anticipated improvement in reliability using theAMSAA Maturity Projection Model (AMPM).

Identifying Requirements and Vulnerabilitiesfor Cybersecurity; Or How We Learned to StopWorrying and Love the Six-Phase Cybersecurity T&E ProcessInstructors: Mike Lilienthal, Electronic Warfare Associatesand Patrick “Preacher” Lardieri, Lockheed Martin

Many Service acquisition, System Engineering (SE), andTest and Evaluation (T&E) teams are starting to move theirprograms from “checklist information assurance or com-pliance” cyber security approach to a proactive, iterativerisk management process with the goal of ensuring per-sonnel can still carry out their duties in a cyber contestedenvironment. Many people are struggling to formulate apractical and effective approach to develop requirementsand a plan to incorporate cyber security into their SE andT&E activities using the recent spate of cybersecurity poli-cies and guidelines released by the Office of the Secretaryof Defense (OSD). This tutorial will step through the use ofthe Navy’s Cyber Table Top (CTT) Wargaming Process andthe National Cyber Range’s cyber security evaluation test-ing process as an approach to gain actionable cyberthreat understanding. The tutorial will also show how the

use of the CTT and the NCR support execution of DOT&E’sSix Phase Cybersecurity T&E process.

The CTT (which has been adopted by the Navy) is a rigor-ous, intellectually intensive and interactive data collectionand analysis process that introduces and explores thepotential effects of cyber offensive operations on thecapability of a system to carry out its designed functions. Itproduces a prioritized list of actionable recommendationsto support more informed decisions and tradeoffs in a fis-cally constrained environment. Personnel using theprocess are better able to identify threat vectors, under-stand the vulnerabilities and mission risks of their systemunder development, and understand cyber threat conse-quences categorized by their impact and their likelihood ofsuccessful attacks. This helps scope the cyber securitytesting done at the NCR and other places. The tutorial willalso show how the use of the cyber wargaming process inconjunction with the NCR will inform systems engineers ontradeoffs and potential workarounds to prevent or mini-mize cyber effects. The tutorial is based on the lessonslearned from using the process and the NCR to supportNAVAIR and SPAWAR acquisition programs. It is intendedfor use by Acquisition Program Management Offices, Sys-tems Engineers, Chief Developmental Testers, and LeadDevelopmental Test and Evaluation (DT&E) Organizations.In short, this tutorial will introduce how the cyberwargame and the NCR iteratively support the developmentof systems that will be more resilient and survivable inhostile cyber threat environments.

Planning and Executing Cyber Table Tops,Facilitator TrainingInstructor: Sarah Standard, Cybersecurity/InteroperabilityTechnical Director, OSD AT&L, DASD DT&E

The primary objective of the Cyber Table Top (CTT) Facilita-tor Training Workshop is to build the knowledge, skills andabilities that will allow trainees to successfully construct,coordinate, organize, and execute a Cyber Table Top (CTT)exercise. The primary audience for this training are thosepersonnel who will facilitate and moderate CTT’s for theirprogram, command. The training will include tips, tools,and resources for CTT facilitators as well as a practicalexample of the process and outputs.

Processes for Testing with International PartnersInstructors: Gloria Deane and Mitchell Dossett, DOT&EInternational Programs

Defense budgets are shrinking; requirements for complexsystems and systems–of–systems are increasing; and

> MONDAY - October 2, 2016

8:00 a.m. – 12:00 p.m. Morning Tutorials

• Cybersecurity Test & Evaluation - Pete Christensen, Cyber Support to OSD Programs,The MITRE Corporation

• How to Build a Reliability Growth Program -Shawn Brady and Martin Wayne, AMSAA, Center for Reliability Growth

• Processes for Testing with International Partners - Gloria Deane and Mitchell Dossett,DOT&E International Programs

• T&E 1-2-3, The Fundamentals – Matt Reynolds,President, Test and Evaluation Consulting

1:00 p.m. – 5:00 p.m. Afternoon Tutorials

• The Art of Planning Preview T&E: AustralianTechniques for Early Test Strategies for

Technical Maturation and Risk Reduction -Group Captain Keith F. Joiner, Ph.D., (Royal Australian Air Force, Ret’d), CSC, University ofNew South Wales, Australia

• Data Science and Its Relationship to Test &Evaluation - Mark J. Kiemele, Ph.D., Air Academy Associates

• Identifying Requirements and Vulnerabilities forCybersecurity; or How We Learned to Stop Worrying and Love the Six-Phase CybersecurityT&E Process - Michael Lilienthal, Ph.D., CTEP,Director of Cyber and Navy Programs, ElectricWarfare Associates, and Mr. Patrick Lardieri,Lockheed Martin Corporation

• Software Assurance - Bob Martin, SeniorSecure Software & Technology Principal Engineer, MITRE

• Test and Evaluation Across the Acquisition LifeCycle - Michael Flynn, PhD, CTEP, DefenseAcquisition University

• Using TENA and JMETC to Reduce Risk, SavingTime and Money – Gene Hudgins, KBRWyle

> THURSDAY - October 5, 2016

1:00 p.m. – 5:00 p.m. Afternoon Tutorials

• Planning and Executing Cyber Table Tops, Facilitator Training - Sarah Standard, Cyberse-curity/Interoperability Technical Director, Officeof the Secretary of Defense, AT&L, DASD(DT&E).

• Real-World DOE and Modern Design and Analysis Methods - Thomas A. Donnelly, Ph.D.,CAP, SAS Institute Inc.

• Test and Evaluation Science and Technology -George Rumford, Deputy Director, Major Initiatives and Technical Analyses, DoD TRMC,AT&L, and Program Manager for the T&E/S&TProgram

REGISTER ONLINE AT: www.itea.org/symposium

34th International Test and Evaluation SymposiumHyatt Regency • 1800 Presidents Street, Reston, VA 20190 • Ph: 703-709-1234 Fx: 703-925-8244

interoperability with allies is becoming the norm by neces-sity. These are challenges all nations are facing. Duplica-tive testing is inefficient for all nations, so sharing of “testresources” is highly desirable. “Test resources” includestest facilities, open air ranges and operating areas, labora-tories, equipment, expertise, methods, data, and funds.Upon making the decision to test, participants must com-plete certain administrative actions to implement a test pro-gram. To test with an international partner an internationalagreement must be in force. To test under such an agree-ment, the partnering nations must negotiate and approve aproject arrangement. The laws of sovereign nations governsuch activity and DOD has developed administrativeprocesses to ensure statutory compliance. The Office ofthe Director, Operational Test and Evaluation (DOT&E) willoffer a tutorial to inform members of the test community ofthe capabilities and limitations of the international Test andEvaluation Program and how to develop project arrange-ments with an individual and with multiple partneringnations. Speakers will be representatives from the Office ofthe Director, International Cooperation in the Office of theUndersecretary of Defense for Acquisition, Technology, andLogistics, the International Test and Evaluation team withinDOT&E, and international partners with whom the DOD testcommunity has worked for many years.

Speakers will be representatives from the Office of theDirector, International Cooperation in the Office of theUndersecretary of Defense for Acquisition, Technology,and Logistics, the International Test and Evaluation teamwithin DOT&E, and international partners with whom theDOD test community has worked for many years.

Real-World DOE and Modern Design andAnalysis MethodsInstructor: Thomas A. Donnelly, Ph.D., CAP, SAS Institute, Inc.

Part 1: Custom DOE – Making Your Design Fit the ProblemThis tutorial will present solutions to real-world Design ofExperiment (DOE) problems. You will learn how to treat in-combination, factors of the following types:continuous/quantitative, categorical/qualitative, discretenumeric, mixture, covariate, blocking, and hard-to-change.It will demonstrate how to constrain design regions. Algo-rithmic custom DOE is the most efficient way to developaccurate and useful models of complex real-worldprocesses.

Part 2: Using Definitive Screening Designs to Get MoreInformation from Fewer TrialsLearn to use the new Definitive Screening Design (DSD)method of Design of Experiments. DSDs not only efficientlyidentify important factors but can often support second-order predictive models. For the same number of factorsthree-level DSDs are often smaller than popularly used 2-level fractional-factorial (FF) designs yet yield more infor-mation especially about curvature for each factor. A casestudy will be shown in which a 10-factor process is opti-mized in just 24 trials. In cases where too many factors aresignificant and the design can't collapse into a one-shotdesign, existing trials can economically be augmented tosupport a response-surface model in the important factors.

Part 3: Strategies for Analyzing Modern Screening Designof ExperimentsThe new Definitive Screening Designs (DSD) provide cleanestimates of all main effects and squared effects for thedesign factors. This leads to saturated or nearly saturatedmodels and the potential to falsely identify lower powersquared terms as important. Effective strategies for ana-lyzing these designs are reviewed to build a consensusmodel from the data. Plus, a newly developed (2015)method for robustly determining the most likely model willbe featured. In this tutorial, we examine several strategiesfor analyzing DOE data sets. Actual vs. Prediction plotswith checkpoints can be used to help choose models.

Software AssuranceInstructor: Bob Martin – Senior Secure Software & Technology Principal Engineer, MITRE

This introductory tutorial provides an overview of theCyber domain while providing the DoD and industry T&Epractitioner the necessary information, perspectives,

understanding, and tools to work effectively in this space.This tutorial provides an introduction to the domain ofCyber, including its concepts, relevant systems (defensiveand offensive), and testing considerations. Discussionswill cover key differences in testing cyber systems versusconventional military systems and test methodologies.Also, we will identify current and future implications forthe T&E community. Additional topics include a generaloverview of cyber warfare, vulnerability analysis, mal-ware, and associated threat vectors pertaining to systemtesting.

T&E 1–2–3, The FundamentalsInstructor: Matt Reynolds – Test and Evaluation Consulting

This tutorial is designed to describe the evolution of T&Eover the course of the last several decades, as well as toexplain the timeless concepts and precepts that apply toall testing. The literature on T&E is replete with policiesand practices that have served the needs of specific gen-erations of systems, of technologies and of acquisitionstrategies. But little has been published that describes theuniversal principles that underlie those policies. An under-standing of these principles and smart implementation ofthem are critical to the success of complex T&E programs.The primacy of thorough planning, contingency strategies,statistics-based test design, enterprise level thinking, anda thorough understanding of customer requirements (bothstated and unstated) will be addressed, and will be rein-forced by lessons learned from the past programs.

Test and Evaluation Across the AcquisitionLife CycleInstructor: Michael Flynn, Ph.D., CTEP - Defense Acquisition University

This tutorial will focus on the latest DoDI 5000.02 guidancefor defense acquisition process from a Test and Evaluationperspective with emphasis on the involvement in the Sys-tems Acquisition Life Cycle and T&E's relationship to theSystems Engineering processes used throughout the lifecycle of major acquisition programs from requirementsgeneration, through Post Milestone C. Coverage willinclude the relationship between the Test and EvaluationMaster Plan (TEMP), and Systems Engineering Plan (SEP)as they proceed through each of the major Milestonephases. Focus will be on the major events that occur dur-ing each phase of acquisition, required documentation,and expected entrance and exit criteria for successfullyachieving approval. The intended audiences are engi-neers, program managers, and industry for an understand-ing of DoD acquisition in relationship to T&E’s involvement.

Test and Evaluation Science and TechnologyInstructor: Mr. George Rumford, Deputy Director, MajorInitiatives and Technical Analyses, DoD TRMC, AT&L, andProgram Manager for the T&E/S&T Program

The T&E/S&T Program develops test technologies that willenable future test capabilities to characterize and optimizethe performance of emerging warfighting systems, beingdeveloped to advance the third offset strategy. Technologyareas of focus include autonomy, electronic warfare,cyber warfare, future computing, micro-electronics, hyper-sonics, directed energy amongst others. This tutorial pro-vides the key attributes of a successful test technologydevelopment project. Attending this session will help thoseunfamiliar with the T&E/S&T Program develop test tech-nology solutions that satisfy T&E needs. The course willalso discuss how to structure a test technology project toassess technology maturation from concept exploration,through engineering, integration and experimentation, andultimately reaching technology transition.

The Art of Planning Preview T&E: AustralianTechniques for Early Test Strategies for Technical Maturation and Risk ReductionInstructor: Group Captain Keith F. Joiner (Royal AustralianNavy Ret.), Ph.D., CSC, University of South Wales, Australia

This four-hour tutorial will benefit anyone who is involvedin planning or conducting early T&E to de-risk and shapemore successful projects. Such participants are likely tohave been part of such planning processes before, but this

workshop is an opportunity for them to examine a freshsystematic approach and see where their previousprocesses and personal master test planning skills mightbe made more robust. Western governments continue tofind an unacceptable proportion of projects fail to deliverthe capability sought and that inadequate early T&E or tri-aling is a significant factor in the risks not being deter-mined early enough for them to be mitigated. In a Senateinquiry into Defence procurement (2012, especially Ch. 2 &12) this was found to be some ten percent of projects byvalue. A more recent report on broader Australian govern-ment public project failings (Shergold Report, 2015) foundsystemic inability to identify and plan early trialing as partof scoping projects. New Defence T&E policy was imple-mented in Australia from 2013-14 to systematically planand conduct de-risk or preview T&E (See Dr. Joiner articleITEA Journal, Dec 2015). Focused workshops ensure pre-view T&E is driven by significant technical and operationalrisk into a program of key confirmatory demonstrations,configuration audits and user trials. Within the U.S. DoD,such early T&E would typically occur during the TechnicalMaturation and Risk Reduction (TMRR) life cycle phaseand thus would be planned and funded in the Analysis ofAlternatives (AOA) phase at Milestone A. The Australianplanning technique has now been confirmed in DefenceT&E policy updates (2016) and is taught at the leadingDefence university in Australia, University of New SouthWales Canberra, as part of all postgraduate master pro-grams in system engineering and project management.Workshop participants will be given an overview of theworkshop process and use a hypothetical capabilityrequirement to role-play the workshops, so as to deter-mine indicative outcomes of each phase of the hypotheti-cal project. Two Australian examples will then be coveredwhere such planning was positively used and anotherwhere it was comparatively ignored in order to contrastthe benefits to de-risking projects through such early T&E.At the end of the workshop, students will have a chance toreflect back to the group on their own possible chances tohave previously used such processes.

Using TENA and JMETC to Reduce Risk, Saving Time and MoneyInstructor: Gene Hudgins – TENA and JMETC User Support Lead, KBRWyle

Together, TENA and JMETC enable interoperability amongranges, facilities, and simulations in a timely and cost-effi-cient manner. TENA provides for real-time system interop-erability, as well as interfacing existing range assets,C4ISR systems, and simulations; fostering reuse of rangeassets and future software systems. JMETC is a distrib-uted, LVC capability which uses a hybrid network architec-ture; the JMETC Secret Network (JSN), based on theSDREN, is used for secret testing and the JMETC MultipleIndependent Levels of Security (MILS) Network (JMN) isthe T&E enterprise network solution for all classificationsand cyber testing. JMETC provides readily-available con-nectivity to the Services’ distributed test and trainingcapabilities and simulations, as well as industry resources.This tutorial addresses using the well-established TENAand JMETC tools and capabilities to reduce risk in an oftenuncertain environment; regularly saving ranges time andmoney in the process.

Exhibit InformationThe Exhibition area will be filled with the industry's leading organizations demonstrating their technology solutions, products, and services specifically tailoredfor the test professional. Reserve your booth today toensure maximum visibility for your organization!For Exhibit Applications visit: www.itea.org/symposium

EXHIBIT HALL HOURS• Monday, October 2 - 1 pm to 6 pm, Exhibitor Set-up• Tuesday, October 3 - 9:30 am to 7:00 pm• Wednesday, October 4 - 9:30 am to 4 pm• Wednesday, October 4 - 4 pm, Exhibitor Move-out

If you would like more information on exhibiting at thisevent, please contact Mr. James Gaidry atsymposium@itea.org

Thank you to our Exhibitors!

ITEA has a room block available to all of our attendees andwe are truly pleased to offer our attendees the prevailinggovernment per diem rate (exclusive of taxes).