nsf grant dmi-9979711 2002 nsf-snl grantees meeting, 09/24/2002, albuquerque nm 0 protection against...

NSF Grant DMI-9979711

2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 1

PROTECTION AGAINST MODELING AND SIMULATION UNCERTAINTIES IN DESIGN

OPTIMIZATION

NSF GRANT DMI-9979711

Bernard Grossman, William H. Mason, Layne T. Watson, Serhat Hosder, and Hongman Kim

Virginia Polytechnic Institute and State University Blacksburg, VA

Raphael T. HaftkaUniversity of Florida

Gainesville, FL

Life-Cycle Engineering Program Meeting and Review24-25 September 2002

Albuquerque, NM



• Detection and Repair of Poorly Converged Optimization Runs

Statistical Modeling of Structural Optimization Errors due to Incomplete Convergence

Computational Fluid Dynamics (CFD) Simulation Uncertainties

Research Performed Under the Project



• Estimate error level of the optimization procedure

• Identify probabilistic distribution model of the optimization error

• Estimate mean and standard deviation of errors without expensive, accurate runs

Statistical Modeling of Optimization Errors

Objective



+ x

z Leading edge radius (fixed)

Outboard LE sweep (fixed)

x y

v1

v3

Location of maximum thickness (fixed)

v2

v4

Wing semi span (fixed)

v5: Fuel Weight

• 250 passenger aircraft, 5500 nm range, cruise at Mach 2.4

• Minimize take off gross weight (WTOGW)

• For this study, a simplified 5DV version is used

Test problem: High Speed Civil Transport (HSCT)



Structural Optimization and Modeling of Error

• Structural optimization performed a priori for many aircraft configurations to obtain optimum Wing Structural Weight (Ws)

• Multiple optimization results to construct response surface

• With multiple optimization results available, statistical techniques can be used to model the convergence error:

Error = Ws – (Ws)t

• Probabilistic modelError fit (expensive approach)

• Weibull Distribution

From high fidelity optimization



Previous Results

• Weibull Distribution models the convergence error of the optimization runs successfully

• Difference fit used to estimate the mean and standard deviation of errors without expensive, high fidelity runs

Multiple sets of low fidelity optimization results required for Difference Fit

Changing convergence criteria (previous results)

Changing initial design points (current results )



design

Op

timu

mw

ing

stru

ctu

ralw

eig

ht

(Ws)

5 10 15 2040000

60000

80000

100000

120000

140000

Case 1Case 2

Case 1 Case 2

Average error in Ws

5.51% 5.34%

For Case 2, the initial design points perturbed from that of Case 1, by random factors between 0.1 ~ 1.9

High fidelity runs used in error calculations

In average, Case 2 has the same level of error as Case 1

Change of the Initial Design Point



Errors of two optimization runs of different initial design point

s = W1 – Wt

t = W2 – Wt , (Wt : unknown true optimum, expensive to calculate)

The difference of s and t is equal to W1 – W2

x = s – t = (W1 – Wt) – (W2 – Wt ) = W1 – W2

We can fit distribution to x instead of s or t.

• s and t are independent • Joint distribution of g(s)

and h(t)

g(s; 1)

PDF

s, t

h(t; 2)

x=s-t

dsxshsgxf )()(),;( 21

• MLE fit for optimization difference x using f(x)

• No need to estimate Wt

Difference fit to estimate statistics of optimization errors



Cases Case 1 Case 2 Average of Abs(W1-W2) 5941

From data 4458 4321 Error fit

(discrepancy) 4207

(-5.63%) 3952

(-8.54%) Estimate of mean, lb. Difference fit

(discrepancy) 3804

(-14.7%) 3481

(-19.4%)

From data 8383 9799 Error fit

(discrepancy) 7157

(-14.6%) 7505

(-23.4%) Estimate of STD, lb. Difference fit

(discrepancy) 9393

(12.0%) 9868

(0.704%)

p-value of 2 test 0.5494

Estimated distribution parameters



Conclusions for Statistical Modeling of Optimization Errors

• Multiple simulation results enable statistical techniques to estimate the uncertainty level of the simulation error

• “Weibull distribution” successfully used to model the convergence error of the optimization runs

• Multiple starting points used to construct two sets of low fidelity optimizations

• “Difference fit” allowed the estimation of average errors without performing high fidelity optimizations



CFD Uncertainties

Drag polar results from 1st AIAA Drag Prediction Workshop (Hemsch, 2001)

Motivation



Objective

• Finding the magnitude of CFD simulation uncertainties that a well informed user may encounter and analyzing their sources

• We study 2-D, turbulent, transonic flow in a converging-diverging channel

• complex fluid dynamics problem• affordable for making multiple runs• known as “Sajben Transonic Diffuser”

in CFD validation studies



x/ht

y/h

t

-4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.00.0

0.5

1.0

1.5

2.0

2.5

y/h

t

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Transonic Diffuser Problem

y/h

t

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x/ht

y/h

t

-4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.00.0

0.5

1.0

1.5

2.0

2.5

0.3

0.4

0.5

0.6

0.7

0.8

0.9

P/P0i

P/P0i

Strong shock case (Pe/P0i=0.72)

Weak shock case (Pe/P0i=0.82)

Separation bubble

experiment CFD

Contour variable: velocity magnitude

streamlines



Uncertainty Sources (following Oberkampf and Blottner)

• Physical Modeling Uncertainty• PDEs describing the flow

• Euler, Thin-Layer N-S, Full N-S, etc.• boundary conditions and initial conditions • geometry representation • auxiliary physical models

• turbulence models, thermodynamic models, etc.• Discretization Error• Iterative Convergence Error• Programming Errors

We show that uncertainties from different sources interact



Computational Modeling

• General Aerodynamic Simulation Program (GASP)• A commercial, Reynolds-averaged, 3-D, finite volume

Navier-Stokes (N-S) code• Has different solution and modeling options. An

informed CFD user still “uncertain” about which one to choose

• For inviscid fluxes (commonly used options in CFD)• Upwind-biased 3rd order accurate Roe-Flux scheme • Flux-limiters: Min-Mod and Van Albada

• Turbulence models (typical for turbulent flows) • Spalart-Allmaras (Sp-Al)• k- (Wilcox, 1998 version) with Sarkar’s

compressibility correction



Grids Used in the Computations

Grid level Mesh Size (number of cells)

1 40 x 25

2 80 x 50

3 160 x 100

4 320 x 200

5 640 x 400

A single solution on grid 5 requires approximately 1170 hours of total node CPU time on a SGI Origin2000 with six processors (10000 cycles)

y/ht

Grid 2

Grid 2 is the typical grid level used in CFD applications



Uncertainty in Nozzle Efficiency

+ ++ + + +

+ + ++ + + + +

+ + + + +

+

+

x x x x x x x x x x x xx

xx

xx

xx

x

x

o

o

+

xo

++

x

x

Pe/P0i

ne

ff

0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.840.700

0.725

0.750

0.775

0.800

0.825

0.850

0.875

0.900

grid 1grid 2

grid 4grid 3

grid 5

Sp-Al



Uncertainty in Nozzle Efficiency

+ ++ + + +

+ + ++ + + + +

+ + + + +

+

+

x x x x x x x x x x x xx

xx

xx

xx

x

x

++ + + +

+ + + ++ + + + +

+ + +

+ + ++

++

x x x x x x x x x x x x x x x xx

xx

xx

x

x

o

o

o

o

+

xo

+

xo +

+

x

x

Pe/P0i

ne

ff

0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.840.700

0.725

0.750

0.775

0.800

0.825

0.850

0.875

0.900

grid 1grid 2

grid 4grid 3

grid 5

Sp-Al k-



Discretization Error by Richardson’s Extrapolation

)( 1 ppexactk hOhff

Turbulence model

Pe/P0i estimate of p (observed order of

accuracy)

estimate of (neff)exact

Grid level

Discretization error (%)

Sp-Al

0.72

(strong shock)

1.32 0.720

1 14.30

2 6.79

3 2.72

4 1.09

Sp-Al

0.82

(weak shock)

1.58 0.811

1 8.00

2 3.54

3 1.12

4 0.40

k- 0.82

(weak shock)

1.66 0.829

1 4.43

2 1.45

3 0.46

4 0.15

order of the method

a measure of grid spacing grid level

error coefficient



x/ht

y/h

t

-1.0 -0.5 0.0 0.5 1.0 1.5 2.00.98

0.99

1.00

1.01

1.02

1.03

1.04

1.05

1.06

1.07

1.08

1.09

modified experimentaldata points

upper wall contour obtainedwith the analytical equation

upper wall contour of the modified-wallgeometry (cubic-spline fit to the modified data points)

upper wall contour of the modified-wallgeometry (cubic-spline fit to the data points)

x/ht

y/h

t

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00

1.00

1.10

1.20

1.30

1.40

1.50 upper wall contour of the modified-wallgeometry (cubic-spline fit to the data points)

upper wall contour obtainedwith the analytical equation

Error in Geometry Representation



Downstream Boundary Condition

x/hty/

ht

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.00.5

1.0

1.5

2.0Extended geometry, Sp-Al, Van Albada, grid 3ext, Pe/P0i=0.7468

x/ht

y/h

t

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.00.5

1.0

1.5

2.0Extended geometry, Sp-Al, Van Albada, grid 3ext, Pe/P0i=0.72

x/ht

y/h

t

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.00.5

1.0

1.5

2.0Original geometry, Sp-Al, Van Albada, grid 3, Pe/P0i=0.72

x/ht

y/h

t

2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.51.0

1.1

1.2

1.3

1.4

1.5

Extended geometry, Sp-Al,Van Albada, grid 3ext, Pe/P0i=0.72

Original geometry, Sp-Al, Van Albada, grid 3, Pe/P0i=0.72

Extended geometry, Sp-Al, Van Albada,grid 3ext, Pe/P0i=0.7468

x/ht

y/h

t

2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.51.0

1.1

1.2

1.3

1.4

1.5

x/ht

y/h

t

2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.51.0

1.1

1.2

1.3

1.4

1.5



the total variation in nozzle efficiency 10% 4%

the difference between grid level 2 and grid level 4

6%

(Sp-Al)

3.5%

(Sp-Al)

the relative uncertainty due to the selection of turbulence model

9%

(grid 4)

2%

(grid 2)

the uncertainty due to the error in geometry representation

0.5%

(grid 3, k-)

1.4%

(grid 3, k-)

the uncertainty due to the change in exit boundary location

0.8%

(grid 3, Sp-Al)

1.1%

(grid 2, Sp-Al)

Uncertainty Comparison in Nozzle EfficiencyStrong Shock

Weak ShockMaximum value of



Conclusions for CFD Uncertainties• Based on the results obtained from this study,

• Informed users may get large errors for the cases with strong shocks and substantial separation

• Systematic uncertainty (discretization error and turbulence models) large compared to numerical noise

• Grid convergence not achieved with grid levels that have moderate mesh sizes

• Uncertainties from different sources interact, especially in the simulation of flows with separation

• We should asses the contribution of CFD uncertainties to design problems that include the simulation of complex flows



Publications 1. Hosder, S., Grossman, B., Haftka, R. T., Mason, W. H., and Watson, L. T., “Observations on CFD

Simulation Uncertainties,” Proceedings of the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Paper No. AIAA-2002-5531, Atlanta, GA, Sept. 2002.

2. Kim, H., Papila M., Haftka, R. T., Mason, W. H., Watson, L. T., and Grossman, B., “Estimating Optimization Error Statistics via Optimization Runs From Multiple Starting Points,” Proceedings of the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Paper No. AIAA-2002-5576, Atlanta, GA, Sept. 2002.

3. Kim, H., Mason, W. H., Watson, L. T., Grossman, B., Papila, M., and Haftka, R. T., "Protection Against Modeling and Uncertainties in Design Optimization," in Modeling and Simulation-Based Life Cycle Engineering, eds.: K. Chung, S.Saigal, S. Thynell and H. Morgan, Spon Press, London and New York, 2002, pp. 231-246.

4. Hosder, S., Watson, L. T., Grossman, B., Mason, W. H., Kim, H., Haftka, R. T., and Cox, S. E., “Polynomial Response Surface Approximations for the Multidisciplinary Design Optimization of a High Speed Civil Transport,” Optimization and Engineering, Vol. 2, No. 4, December 2001, pp.431-452.

5. Kim, H., Papila M., Mason, W. H., Haftka, R. T., Watson, L. T., and Grossman, B., “Detection and Repair of Poorly Converged Optimization Runs,” AIAA Journal, Vol. 39, No. 12, 2001, pp. 2242-2249.

6. Kim H., “Statistical Modeling of Simulation Errors and Their Reduction via Response Surface Techniques,” Ph.D Dissertation, Virginia Tech., July 2001.

7. Kim, H., Haftka, R. T., Mason, W. H., Watson, L. T., and Grossman, B., “A Study of the Statistical Description of Errors from Structural Optimization,” Proceedings of the 8th AIAA/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Paper No. AIAA-2000-4840-CP, Long Beach, CA, Sept. 2000.



Publications (continued)8. Kim, H., Papila M., Mason, W. H., Haftka, R. T., Watson, L. T., and Grossman, B., “Detection and

Correction of Poorly Converged Optimizations by Iteratively Re-weighted Least Squares,” Paper AIAA-2000-1525, 41st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Atlanta, GA, April 3-6, 2000.

nsf grant dmi-9979711 2002 nsf-snl grantees meeting, 09/24/2002, albuquerque nm 0 protection against...

Documents