Judges Score Card Overview

Mark TurnerNASA Ames Research Center

Scorecard 2

Ground Tournaments

• Purpose is to provide interim monetary awards and feedback to Teams

• Four Ground Tournaments (GT4 is required for EM-1 launch) six months apart

• Evaluation, analysis and simulations will be performed by ARC’s Mission Design Center, WFF’s Mission Planning Lab and Subject Matter Experts

January 7, 2015

3

GT Concept of Operations

January 7, 2015

Teams Prepare Data

40% - Probability of Mission Success 60% - Compliance with SLS Interface Requirements & Specific Challenge Rules

Judges

Simulations and Review byARC-MDC WFF-MPL SMEs

Review by SLS PIM SLS S&MA SLS Safety Panel

Judges

Centennial Program Office

Scores & Evaluations

Judges Final Scoring & Ranking

Awards

Questions for clarification Safety Hazard Report action items

Scores (and feedback) to Teams

ARC- MDC: Ames Mission Design CenterWFF-MPL: Wallops Mission Planning Laboratory SMEs: Subject Matter Experts

Scorecard 4

Purpose of the Score Card

• Provide a method to objectively judge each team to a set of clearly defined metrics

• Provide to each team a clear picture of what data they will need to provide for each Ground Tournament (GT)

• Provide to Teams a clear picture of how they will be evaluated for each GT

January 7, 2015

Scorecard 5

Score Card Development

January 7, 2015

• Subject Matter Experts from ARC, WFF and GSFC developed and reviewed the Score Card over several months.

• ARC-MDC and WFF-MPL developed detailed inputs necessary from Teams to run simulations.

• Two judges reviewed the Score Card.

Scorecard 6

Scorecard Evaluation Criteria

• Probability of Mission Success – 40% of total score– Likelihood of Achieving Communications Goals– Likelihood of Achieving Orbit or Distance Goals – Likelihood of Meeting Longevity Goals– Systems Design Maturity Relative to GT

• Compliance with SLS Interface Requirements & Specific Challenge Rules - 60% of total score– Specific Challenge Rules– SLS SPDS IDRD requirements (beginning with GT2)– Safety Panel Hazard Report

• GT1 identify potential safety hazards• GT2 & 3 Identify how you plan to address these hazards• GT4 Provide data to prove hazards have been mitigated

January 7, 2015

Scorecard 7

Inputs from Teams

• Each category identifies REQUIRED inputs and RECOMMENDED inputs (as noted in the tabs on the Score Card)

• REQUIRED inputs identify data critical for judges to evaluate each Team– Required data is the minimum acceptable data required for judging the

competition– By minimizing the amount of required inputs, each team can allocate its

resources as needed to develop a successful mission design• RECOMMENDED inputs identify data NASA would typically develop for in-

house missions– Given that teams are operating with limited resources, each team can decide

which recommended data they will provide– Recommended data provides additional insight to the judges as to how

robust and complete the proposed mission design is

January 7, 2015

Scorecard 8

Scorecard Evaluation Criteria

• Probability of Mission Success – 40% of total score– Likelihood of Achieving Communications Goals– Likelihood of Achieving Orbit or Distance Goals – Likelihood of Meeting Longevity Goals– Systems Design Maturity Relative to GT

• Compliance with SLS Interface Requirements & Specific Challenge Rules - 60% of total score– Specific Challenge Rules– SLS Secondary Payload Deployment System (SPDS) IDRD requirements

(beginning with GT2)– Safety Panel Hazard Report

• GT1 identify potential safety hazards• GT2 & 3 Identify how you plan to address these hazards• GT4 Provide data to prove hazards have been mitigated

January 7, 2015

Scorecard 9

Expectations

• Teams will reach higher fidelity of designs as the series of GTs progress. Less fidelity is expected at GT1 and high fidelity is expected at GT4.

• Judges’ grading will become more stringent as the series of GTs progress.

January 7, 2015

Scorecard 10

Evaluation Criteria for GTs

• GT 1 – Evaluate the mission architecture and concepts to meet Challenge Goals and

evaluate plans to meet Rules & Interface Requirements.• GT 2

– Evaluate plans and designs to meet Challenge Goals, Rules & Interface Requirements.

– Determine if the integrated design is appropriately mature to continue with final designs and fabrication.

• GT 3– Evaluate the readiness to begin system Assembly, Integration & Testing and

progress toward meeting Challenge Goals, Rules & Interface Reqts. • GT 4

– Evaluate the readiness for launch, likelihood of meeting Challenge Goals and ensure final compliance to Rules & Interface Requirements.

– Three Flight selections shall be made at GT4 with one additional team as backup

January 7, 2015

Scorecard 11

Schedule

January 7, 2015

O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D

Vehicle Related EventsSPUG AvailableCube Quest Summit (7th & 8th)Payload ~PDR (maturity check)IDRD AvailablePayload ICD CompletePhase 0/I Safety ReviewPayload ~CDR (maturity check)Phase II Safety ReviewVerif. Testing CompletePhase III Safety ReviewKSC Safety ReviewCOFRIntegrated Payload Turn OverPayload Battery ChargingLaunch, EM-1

Centennial Challenge EventsChallenge Released to Public XGT #1 - Rank Top 5 20GT #2 - Rank Top 5 30GT #3 - Rank Top 5 30GT #4 - Select Flight Group 20

Notes: All safety packages must be delivered 14 days prior to the scheduled review.Depending on the number of payloads, some milestones may be shifted to reduce technical loads on review panels.

FY2018 FY2019

Centennial Challenge EM-1 CubeSat Milestone Schedule

Payload MilestonesCY2014 CY2015 CY2016 CY2017 CY2018

FY2015 FY2016 FY2017

Point of no return

SPUG: SLS Secondary Payload User’s GuideIDRD: Interface Definition Requirement Document (IDRD)

Scorecard 12

Scorecard Examples

January 7, 2015