episode 3 / caats ii joint dissemination event lessons learnt episode 3 - caats ii final...
TRANSCRIPT
Episode 3 / CAATS II joint dissemination event
Lessons Learnt
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
Philippe LeplaeEUROCONTROLEpisode 3
Brussels, 13 & 14 Oct 2009
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
2
Content
• Project objectives• Lessons learnt method• Lessons learnt in:
management validation concept clarification result integration validation techniques
• Conclusion
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
3
Project objectives
• Provide detail on key concept elements in SESAR• Initial operability and performance assessment• Initial technical impact assessment• Analysis of available tools for SESAR validation• Report on validation methodology……
• Hence specific emphasis on lessons learnt
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
4
Lessons learnt method
• 2 workshops, Madrid and Brétigny• 9 different sessions on 5 main topics involving more than
30 participants Validation strategy and project DOW Project management Concept clarification Integration and Consolidation EP3 validation techniques and levels of maturity
• Most project members also working in SESAR• Strong desire to provide feedback to SESAR
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
5
High level lessons learnt
• System view should drive validation activities Need to adapt DOW and management to required
flexibility Need time to develop system driven project We have developed artefacts to drive it
• Many different techniques tried on Episode 3 Suited to low maturity level concept elements Guidelines on when and how to use it are produced
• Importance of support tools Good strategy useless without tools to support it Tools also require investment in training This provides harmonized working methods, key to make
distributed project work
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
6
Lessons learntin management
• Our challenges: Wide range of disseminated partners Crossed responsibilities and thinly spread effort Innovative tasks High quality requirements Tight deadlines
• Our answers: Development of tools for distributed work Training on the tools, and specific delivery process A lot of co-ordination activities
• Follow-up actions: Build more flexibility in the programme (DOW and contract)
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
7
Lessons learntin validation
• Our challenges: Maturity lower than expected at programme onset E-OCVM V2 not well adapted to concept wide validation Lack of E-OCVM expertise inside team Wide range of results
• Our answers: We built a framework with guidelines, principles, templates. Developed assumptions management Performance framework and DOD’s provided at start Guidance provided through system WP and co-ordination cell Thorough review of experimental plans
• Open issue: Maturity assessment at start of each validation activity
WP6
Increasing Level of Concept Maturity
V1 V2 V3
Overall Validation Strategy
Updated Overall
Validation Strategy
Updated Overall
Validation Strategy
Operational Context
Validation Strategy
Updated Operational
Context Validation Strategy
Updated Operational
Context ValidationStrategy
Target Concept
Operational Context
Concept Level
OIs Validation Exercises
EPISODE 3
WP5WP4
WP3
WP2
WP6WP6
Increasing Level of Concept Maturity
V1V1 V2V2 V3V3
Overall Validation Strategy
Updated Overall
Validation Strategy
Updated Overall
Validation Strategy
Operational Context
Validation Strategy
Updated Operational
Context Validation Strategy
Updated Operational
Context ValidationStrategy
Target Concept
Operational Context
Concept Level
OIs Validation Exercises
EPISODE 3
WP5WP4
WP3
WP2
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
8
Lessons learntin concept clarification
• Our challenges: Multiple sources from SESAR (CONOPS, OI’s,etc…) Lack of details for validation Multiple sources to update DOD’s
• Our solutions: Propose a structure, around process model and DOD’s Develop different modes of descriptions
- DOD’s - Operational scenarios - Story boards
Develop collaborative working methods to update documents
• Follow-up: Establish a recognised authority to select among options Use hot topics to drive validation
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
9
Lessons learntin result integration
• Our challenges: Exercises not providing consistent results Different granularity of results Various degree of validity of quantitative data
• Our solutions: Influence diagrams help understanding benefit mechanism
and identify where performance is improved Influence model has been developed ready to be fed with
results Management of assumptions to interpret results
• For subsequent coverage: Steer validation activities from the performance framework Use of models developed in EP3 to provide more quantitative
data to performance model
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
10
Lessons learnt in validation techniques
• Successful used of different techniques Prototyping Macro-modelling Various sorts or gaming Expert groups
• Work packages integrated different techniques Combine results of very different nature Various sequences of activities tried Expert groups key to success
• Benefits of industry assessment: “Reality check” Should run as a complementary process to operational
validation
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
11
Validation techniques:Expert Groups
• Lessons learnt Useful at all maturity levels Need good facilitation techniques by somebody
with OPS expertise Need current operation experts + concept
experts Storyboards good method to structure EG
debates
• Our input to SESAR Storyboards technique Various facilitation techniques Hot topics Guidelines document
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
12
Validation techniques:Gaming
• Lessons learnt To be associated with Expert Groups and modelling Suited to clarify roles and responsibilities for all maturity levels More flexible than prototyping as not in real time (interruptions
feasible) Progression from paper-based gaming to platform gaming
• Our input to SESAR Different platforms
- CHILL - DARTIS- ACCES platform
Paper based gaming and associated web based tool
Guidelines document
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
13
Validation techniques:Prototyping
• Lessons learnt: Not a cheap RTS! Allowed to fail Concept clarification-no quantitative
data Successive runs with same team
of OPS staff
• Our input to SESAR First results on:
- PRNAV, CDA in descent phase.- Acceptability of 4D trajectory to
controllers Guidelines document
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
14
Validation techniques:Modelling
• Lessons learnt: Use of process modelling and
network modelling Process modelling complement
gaming Need to work on precise concept
elements Macro-modelling
• Our input to SESAR: 3 innovative tools developed in EP3
- One abandoned (unsatisfactory results)- One providing results when applied
to mature concept elements- One promising for network wide assessments
Guidelines document
Node type I (airport)
Node type II (congestion zone)
Link: air traffic flow
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
15
Conclusion
• Episode 3 is the first ATM-wide concept validation programme
• We had to develop tools and methods adapted to this new challenge
• Another 2 months to complete our work (WP and final report)
• We provide to SESAR: New tools New methods An integrated approach First validation results… See www.episode3.aero
• And our most precious output: A team with the expertise and concrete experience already
working on the SESAR projects:
Brussels, 13 & 14 Oct 2009
Ep
isod
e 3
- C
AA
TS
II Fin
al D
isse
min
ati
on E
vent
16
Thank you