pilot validation methodology for agent-based simulations workshop

Pilot Validation Methodology for Agent-Based Simulations

Workshop

02 October 2007

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Administrative Remarks

02 October 2007

Agenda

• Presentation by Dr. Andreas Tolk (Old Dominion University): “Constraints for V&V of Agent Based Simulation: First Results”

• Presentation by Mr. Paul Wehner (MITRE): “ABMs and the Validation Hurdle - An Illustration”

• Introduction to Topic #2: Invalidation Tools and Techniques

• Topic #2 Breakout Discussions• Lunch• Topic# 3: ABS-Application Pairings• Presentation - TBD• Wrap-Up

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Constraints for V&V of Agent Based Simulation: First Results

Dr. Andreas Tolk

Old Dominion University

01 October 2007 (0905 to 0950)

Pilot Validation Methodology for Agent-Based Simulations

Workshop

ABMs and the Validation Hurdle - An Illustration

Dr. Alfred Brandstein

Mr. Paul Wehner

MITRE

01 October 2007 (0950 to 1035)

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Introduction to Topic #2: Invalidation Tools & Techniques

Ms. Lisa Jean Moya

WernerAnderson, Inc.

01 October 2007 (1035 to 1055)

Invalidation Techniques– Executable compared to concept/referent

• Results validation• Mini-analysis• Accuracy

– Theoretical model compared to concept/referent

• Assumption testing• SME Review

– Mathematical model compared to theoretical model

• Boundary analysis • Algorithm review• Spreadsheet Modeling

– Software code compared to mathematical model

• Existence of required outputs• Symbolic debugger• Code walk through

– Data compared to executable

• Existence of required inputs

• Comparison to other models• Turing test• Intuition

• Intuition • Functionality assessment

• Completeness assessment• Formal Methods• Algorithm review

• Input range validation• Control parameter review• Component testing

• SME validation

Pilot Validation Methodology for Agent-Based Simulations

Workshop

BREAK

1055 - 1105

Please Assemble in Breakout Rooms By 1105

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Topic #2: Invalidation Tools & Techniques

1105 - 1220

Takes Place in Breakout Rooms

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Wrap-Up for Topic #2: Invalidation Tools & Techniques

1230 - 1300

Takes Place in Auditorium

Pilot Validation Methodology for Agent-Based Simulations

Workshop

LUNCH BREAK

1300 - 1400

Please Return To Breakout Rooms After Lunch

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Topic #3: ABS-Application Pairings

Dr. Michael Bailey

Operations Analysis Division

Marine Corps Combat Development Command

01 October 2007 (1035 to 1055)

Pilot Validation Methodology for Agent-Based Simulations

Workshop

BREAK

1515 - 1530

Return to Breakout Rooms after Break

Pilot Validation Methodology for Agent-Based Simulations

Workshop

Open Discussion/Overall Wrap-Up

1615 - 1645