NASA Langley Research Center - 1Workshop on UQEE

Prediction of Computational Qualityfor Aerospace Applications

Michael J. Hemsch, James M. Luckring, Joseph H. MorrisonNASA Langley Research Center

Elements of Predictability WorkshopNovember 13-14, 2003

Johns Hopkins University

NASA Langley Research Center - 2Workshop on UQEE

Outline

• Breakdown of the problem (again) with a slight twist.

• The issue for most of aerospace is that non-computationalists are doing the applications computations.

• What are they doing now? What can we do to help?

NASA Langley Research Center - 3Workshop on UQEE

Breakdown of tasks

Off-lineComputation

Measuring themeasurement system

Measuring themeasurement system

Random errorcharacterizationusing standardartifacts

Discrimination testingof the measurementsystem

Discrimination testingof the measurementsystem

Systematic errorcharacterization

QA checks againstabove measurementsduring customertesting

QA checks againstabove measurementsduring customertesting

Process outputof interest

Calibration ofinstruments

Off-lineExperimentation

Traceableoperationaldefinition ofthe process

Calibration ofinstruments

Traceability tostandards

Verifying that thecoding is correct

Verifying that thecoding is correct

Off-line

Off-line

Measuring thecomputational process

Measuring thecomputational process

Off-lineCharacterizationof processvariation usingstandard problems

Model-to-model andmodel-to-realitydiscrimination

Model-to-model andmodel-to-realitydiscrimination

Off-lineSystematic errorcharacterization

QA checks againstabove measurementsduring computation forcustomer

QA checks againstabove measurementsduring computation forcustomer

Solutionverification

NASA Langley Research Center - 4Workshop on UQEE

The key question for applications:

“How is the applications person going to convince the decision maker that the computational process is good enough?”

NASA Langley Research Center - 5Workshop on UQEE

Our tentative answer based on observation of aero engineers trying to use CFD on real-life design problems is that it is the quantitative explanatory force of any approach that creates acceptance.

NASA Langley Research Center - 6Workshop on UQEE

• How can quantitative "explanatory force“ be provided?

• Breakdown to two questions:– How do I know that I am predicting the

right physics at the right place in the inference space?

– How accurate are my results if I do have the right physics at the right place in the inference space?

NASA Langley Research Center - 7Workshop on UQEE

Airfoil Stall Classification

NASA Langley Research Center - 8Workshop on UQEE

Boundaries Among Stall Types

NASA Langley Research Center - 9Workshop on UQEE

• The applications person needs a process that can be

ControlledEvaluatedImproved

(i.e. a predictable process)

NASA Langley Research Center - 10Workshop on UQEE

Creating a predictable process …

Controllable input(assignable cause variation)

Geometry,flight conditions,

etc.

Predictedcoefficients,

flow features,etc.

Process

Uncontrolled input from the environment(variation that we have to live with,

e.g. numerics, parameter uncertainty,model form uncertainty, users)

NASA Langley Research Center - 11Workshop on UQEE

Critical levels of attainment for a predictable process

• A defined set of steps

• Stable and replicable

• Measurable

• Improvable

NASA Langley Research Center - 12Workshop on UQEE

What it takes to have an impact ...

• Historically, practitioners have created their designs (and the disciplines they work in) with very little reference to researchers.

• Practitioners who are successfully using aero computations already know what it takes to convince a risk taker.

• If we want to have an impact on practitioners, we will have to build on what they are already doing.

NASA Langley Research Center - 13Workshop on UQEE

What is takes to have an impact ...

• Good questions:– Are researchers going to be an integral

part of the applications uncertainty quantification process or are we going to be irrelevant?

– What specific impact on practitioners do I want to have with a particular project?

– What process/product improvement am I expecting from that project?

NASA Langley Research Center - 14Workshop on UQEE

What is takes to have an impact ...

• We can greatly improve, systematize and generalize the process that practitioners are successfully using right now.

• The key watchwords for applications are:– practicality, as in mission analysis and design– alacrity, as in "I want to use it right now."– impact, as in "Will my customer buy in?" and "Am I

willing to bet my career (and my life) on my prediction?"

NASA Langley Research Center - 15Workshop on UQEE

Actions

• Establish working groups like the AIAA Drag Prediction Workshop (DPW)– Select a small number of focus problems– Use those problems

» to demonstrate the prediction uncertainty strategies » to find out just how tough this problem really is

• For right now …– Run multiple codes, different grid types, multiple models, etc.– Work data sets that fully capture the physics of the

application problem of interest.– Develop process best practices and find ways to control and

evaluate them.– Develop experiments to determine our ability to predict

uncertainty and to predict the domain boundaries where the physics changes.

NASA Langley Research Center - 16Workshop on UQEE

Breakout Questions/Issues

1. Defining predictability in the context of the application2. The logical or physical reasons for lack of predictability3. Possibility of isolating the reducible uncertainties in view

of dealing with them (either propagating them or reducing them)

4. The role of experimental evidence in understanding and controlling predictability

5. The possibility of gathering experimental evidence6. The role that modeling plays in limiting predictability7. Minimum requisite attributes of predictive models8. The role played by temporal and spatial scales and

possibilities mitigating actions and models