1

MODEL REFINEMENT DECISIONS USING THE PROCESS PERFORMANCE INDICATOR

Matthias Messer INNOVATIONCENTER

Freudenberg Dichtungs- und Schwingungstechnik GmbH

(FDS) D-69465 Weinheim, Germany

Jitesh H. Panchal School of Mechanical and

Materials Engineering Washington State University

(WSU) Pullman, WA 99164, U.S.A.

Janet K. Allen1 & Farrokh Mistree2 School of Industrial1 & Aerospace

and Mechanical2 Engineering The University of Oklahoma

(OU) Norman, OU 73019, U.S.A.

2

ABSTRACT

Model refinement is crucial to managing complexity in the design of complex engineered systems,

particularly in the context of integrated product and materials design. In order to manage complexity, a

systematic approach for model refinement is presented in this paper. The approach is based on a value-of-

information-based Process Performance Indicator (PPI). The systematic approach and indicator can also be

used to make refinement decision using multiple models when, for example, models are refined through

model replacement. Both are particularly well-suited for integrated product and materials design, and other

scenarios where perfect knowledge of the true system behavior and error-bounds is not available

throughout the entire design space. In this paper, the proposed approach is applied to model refinement for

designing multifunctional Blast Resistant Panels (BRPs). It is shown that the proposed approach based on

the PPI is useful for model refinement particularly when accuracy of the prediction or the associated error

bounds are not known.

Key Words: Value-of-information, process design, complexity management, model refinement, blast

resistant panels, lightweight design

Word Count: ca. 10,000

NOMENCLATURE

PPI Process Performance Indicator EOP Expected Overall Payoff DVV Design Variable Value Pi EOP of model i PT,x* EOP of most truthful model x* Decision Point BRP Blast Resist Panel For BRP design problem specific nomenclature, please refer to Table 3.

1. FRAME OF REFERENCE – THE MODEL REFINEMENT PROBLEM 1 Corresponding author email janet.allen@ou.edu and phone +1-405-325-0412.

3

In the design of complex engineered systems, such as integrated design of materials and products [23],

designers are often challenged by high complexity. As proposed by Panchal and coauthors [29], a

promising approach to manage complexity in multilevel and multiscale design processes is step-wise model

refinement. From the decision-centric perspective [25, 26, 30] adopted in this paper, model refinement is

driven by improved decision-making in the context of a given design problem formulated in terms of often

interacting decisions and analysis models. Managing complexity with respect to models may thus involve

refining (i) interactions between coupled decisions and analysis models, addressed in detail in the literature

[6, 7, 12, 29], or (ii) individual analysis models. The focus in this paper is on the step-wise simulation-

based refinement of analysis models that can be formulated mathematically and solved computationally. In

this paper, model refinement includes the use of multiple models for refinement decisions when, for

example, models are refined through model replacement.

Value-of-Information (VoI) based metrics, derived from the field of information economics, i.e., “…

the study of choice in information collection and management when resources to expend on information

collection are scarce”, have been utilized to evaluate models or make model refinement decisions [2, 3, 30].

The VoI from model refinement is represented by an increase in the overall expected payoff or utility when

a refined model is used compared to the original model, as discussed in Section 3.1. However, existing

metrics for VoI [3, 15, 20], such as the Improvement Potential introduced by Panchal [29], do not account

for the refinement of simulation models when knowledge of either perfect information about system

behavior or its error bounds (deterministic or probabilistic) is not available globally, i.e., throughout the

entire design space. Therefore, we investigate the following question in this paper:

How much refinement of a simulation model is appropriate for a particular design problem, when

knowledge of prediction accuracy or its error bounds is not available throughout the entire design space?

Since engineering analysis is expensive and time consuming, designers usually seek the most

appropriate model. The most appropriate model is as complex as necessary and as computationally

efficient as possible from a decision-centric perspective. We believe that only an instance of the final

product may be considered an equivalent of truth [35]. Therefore, the most truthful model available can be

determined from experimental testing or computationally expensive simulations locally at specific points in

4

the design space. Thus, in order to overcome the limitation that existing metrics for VoI-based model

refinement are based on global information, our approach is to leverage local information at certain points

in the design space from the most truthful model available in order to facilitate model refinement.

A generally applicable approach based on a VoI-based metric, the Process Performance Indicator

(PPI), is described in Section 1.2. As an example for integrated product and materials design we consider

the design of multifunctional Blast Resistant Panels (BRPs), as introduced in Section 1.1.

1.1 Integrated Product and Materials Design Example: Multifunctional Blast Resistant Panels

As an example, consider the design of panels that resist blasts by absorbing large amounts of energy

per unit mass compared to solid plates. In one application, panels designed this way can be attached on the

outside of vehicles to protect them from explosions. In the Blast Resistant Panel (BRP) design problem

investigated in this paper, BRPs are to be designed in order to ensure specified performance with respect to

protection against blasts while minimizing overall system weight. The underlying multi-objective decision

formulation for robust design used in this work is the utility-based compromise Decision Support Problem

(cDSP) [25, 35], as described in detail in Section 3.

The design of multifunctional BRPs is a challenging example that involves decisions on both the

system and material level. On the system level, a decision has to be made on configuring a multilayer BRP

– for example featuring various panel concepts, ranging from monolithic to composite panels, or

unreinforced to stiffened to multilayer sandwich panels. Also, a designer is confronted with material level

decisions to better achieve performance requirements. For example, by selecting a sandwich structure to

configure the overall containment system, various microscale cellular material or truss structure core

configurations are designed to feature increased energy dissipation per unit mass and are robust to changes

in blast loads.

Various BRP concepts have been explored, ranging from simple compression, tension, bending,

torsion, splitting, flattening, crushing or inversion of structures to more complex foam, honeycomb, weave,

micromechanism or microtruss structures. A detailed review of existing BRP concepts is given in the

literature [24]. However, a key characteristic of BRP concepts should be a long, flat plateau of the stress-

strain curve at near-constant load to maximize energy dissipation truncated by a regime of densification in

5

which the stress rises steeply. Hence, materials constituting BRPs should have two dominant properties: the

energy dissipated per unit mass, and the stress at which this energy is dissipated [13]. In cases when energy

dissipation is to be maximized, a combination of high plateau stress and strain is required.

Based on these requirements, out-of-plane honeycomb core sandwich structures, as illustrated in

Figure 1, have been selected as the most promising BRP concepts [24]. The benefits of sandwich

construction mainly depend on the core topology. Core designs that afford simultaneous crushing and

stretching resistance, such as out-of-plane honeycomb cores, are preferred to those that are bending

dominated, such as foam cores. Additional benefits of sandwich construction derive from fluid/structure

interaction effects [14, 42]. Details on the systematic selection of out-of-plane honeycomb core sandwich

structures and a detailed concept exploration of various BRP concepts can be found in the literature [24].

The BRP design problem is summarized in the word formulation of the utility-based cDSP show in Table 1.

Figure 1. Geometry selected BRP concept.

Having selected a BRP concept, models of varying complexity can be used for concept exploration,

such as analytical models, static or dynamic Finite Element Analysis models, detailed multiscale analysis

models accounting for fluid-structure interactions, or even prototypes. However, modeling resources that

add more information must be well spent in real-world scenarios. A model used for designing should be

efficient to instantiate and execute, particularly when accounting for uncertainty in design decisions. If

efficiency is the only consideration, a designer would prefer to use the least complex model which

minimizes the instantiation and execution time.

Generally speaking, it is beneficial to refine models only if the corresponding decrease in efficiency is

justified. For example, while complex simulation models may lead to better designs, less complex ones are

less costly to instantiate and execute. However, simulation models inevitably incorporate assumptions and

hc

B

hf

hb

H

Front-Face-Sheet

Back-Face-Sheet

6

approximations that impact the accuracy of predictions as well as design decisions. The primary concern

treated in this paper is thus that of a trade-off for obtaining additional information to refine simulation

models, as discussed in the following section.

Table 1. Word formulation utility-based cDSP.

Given: • Uncertainty in impulse load and peak blast • Models for deflection and mass • Geometric parameters, and material properties • Utility functions for mean and variance of deflection and total mass Find: • Geometry variables: front face sheet thickness hf, back face sheet thickness hb, core thickness H, and core wall thickness hc • Values of deviation variables di

+ and di-

Satisfy: Constraints: • Mass per area, deflection, and relative density • Front face shear-off • Deviation variables must be greater-than/equal to 0 and multiply to 0 Goals: • Minimize deviation from mean and variance of deflection and weight Bounds: • Upper and lower bounds for system variables Minimize: • Deviation function

1.2 A Review of Value-of-Information Based Approaches to Model Refinement

Model refinement decisions and associated trade-off for obtaining additional information can be

facilitated through various metrics. However, often direct metrics [36], such as cost and accuracy, or

indirect [18] metrics, such as complexity and uncertainty, cannot be readily evaluated. Hence, information

economic tradeoffs and the derived VoI-based concept [2, 3, 30] first introduced by Howard [15] have been

investigated in the context of model refinement, as depicted in Table 2. Lawrence [19] provides a

comprehensive overview of the VoI-based concept and associated metrics.

7

Table 2. Assumptions in previous research efforts.

Research Effort Assumption VoI- or utility-based model selection [10, 11, 33]

Error characterizations or utilities for model cost and accuracy are available throughout the entire design space

Traditional VoI-based Decision Evaluation [15]

Uncertainty is described using exact probability distributions throughout the entire design space

P-box approach [2]

Uncertainty is characterized using imprecise (intervals of) probabilities throughout the entire design space

Improvement Potential [29]

Non-probabilistic uncertainty using intervals is available throughout the entire design space

As summarized in Table 2, it has generally been assumed that the precisely characterized joint and

conditional probability distributions are available [15]. If these probability distributions are not available,

approaches have been proposed to generate those through an educated guess that is based on the designers’

prior knowledge. To address the problem of lack of knowledge about the probability distributions,

Aughenbaugh and co-authors [2] present an approach of measuring the VoI based on probability bounds on

the VoI that are calculated by a designer during the information collection process using imprecise

probabilities in a p-box approach.

Ling and Paredis [20] present a VoI-based approach for model selection for the particular scenario in

which the systematic error in the models can be bounded by an interval. But, the underlying assumption is

that an interval-based characterization of the error in model output is truthful and available throughout the

design space. Furthermore, to argue about a model´s usefulness, Doraiswamy and Krishnamurty [11];

Radhakrishnan and McAdams [33]; and Cherkassky [10] consider cost-benefit trade-offs in selecting

models at various levels of abstraction. They present a utility-based framework in which a designer can

reason about model uncertainty and benchmarks assuming availability of a perfect model.

To overcome the assumption of having a perfect model available in the context of model refinement,

Panchal and coauthors [29] present a VoI-based metric called Improvement Potential to account for the

non-probabilistic uncertainty resulting from model refinement or simplification of interactions, as proposed

in the literature [6, 7, 12]. However, the Improvement Potential metric [29] is based on the assumption that

information about bounds on the output of the model (i.e., the lower and upper bounds on the overall

system outcome) is available globally, i.e., throughout the entire design space.

In various circumstances a designer has access to different kinds of analysis models that embody

8

different assumptions, but, the error bounds on those models are unavailable throughout the entire design-

space. Particularly, in the context of simulation-based integrated product and materials design, it is not

practically feasible to either know a model’s error or associated probability bounds throughout the entire

design space or generate those through an “educated guess” in terms of p-boxes. Also, a designer usually

does not know the behavior of a most truthful model throughout the entire design space. But, experimental

data is frequently available or relatively easy to generate for designers and can hence be leveraged as the

most truthful information to assess the value of a specific model to the decision maker by evaluating the

PPI only with respect to local information at specific points in the design space. Besides empirical data, this

local information can often also be readily obtained from computationally expensive simulation models.

In order to address the limitation of previous research efforts that perfect knowledge of the true system

behavior and error-bounds are available throughout the entire design space, an approach for refining

simulation models based on local information from the most truthful model available is proposed. Since it

is often not feasible to “afford” exploring the most truthful model throughout the entire design space due to

computational or experimental efficiency restraints, the approach is not dependent on the knowledge of

perfect information or the computationally/experimentally expensive knowledge of error bounds

throughout the entire design space. Furthermore, the systematic approach and indicator can also be used to

make refinement decision using multiple models when, for example, models are refined through model

replacement. However, the VoI-based approach for model refinement leveraging local information is

crucial to support a designer in answering the question of how appropriate a model is.

The threshold defining appropriateness is subject to a designer’s value judgment. The degree of model

refinement thus depends on specific characteristics of the decision at hand, including the preference of a

designer, the uncertainty in problem parameters, as well as the importance of the decision and the resulting

payoff or utility to a designer. Such decision characteristics affect the information economic tradeoffs that

are foundational to the proposed PPI, which is calculated in terms of a designer´s payoff or utility and

represents the key construct of the VoI-based approach to facilitate model refinement decisions, as

described in the following section.

9

2. SYSTEMATIC APPROACH FOR MODEL REFINEMENT BASED ON THE PROCESS

PERFORMANCE INDICATOR

Before addressing the VoI-based approach for model refinement in Section 3.2, the Process

Performance Indicator (PPI) is presented in Section 3.1.

2.1 The Process Performance Indicator

Refining models, i.e., increasing their accuracy, is generally accompanied by increased complexity.

But, increased complexity is neither a necessary nor sufficient condition for accuracy. In this paper we refer

to accuracy as an exact or careful conformity to truth [41]. A basic assumption that underlies our method is

that increased complexity leads to increased accuracy and therefore more truthful information [33].

However, models inevitably incorporate assumptions and approximations that impact the precision and

accuracy of predictions. Tightening assumptions and approximations may or may not improve design

decisions. Thus, the systematic strategy to model refinement presented in this paper is driven by a decision-

centric perspective, not the more scientific need to validate the models [21, 22]. A fundamental assumption

in this work is that the models used have been validated.

As discussed by Panchal and coauthors [29], a model is considered as a source of information. Thus,

refining a model is equivalent to adding more information for decision making so that the refined model

more accurately predicts a system´s behavior. Therefore, value of information from model refinement is

represented by an increase in the overall expected payoff or utility when a refined model is used as

compared to the original model. Hence, value-of-information-based metrics can be used by decision makers

to resolve trade-offs with respect to both refinement and execution of a more complex model as well as

reduced uncertainty.

Due to the ex-ante nature of decision making, the decisions about the information have to be made

before a state actually occurs, capturing the value-of-information by considering uncertainties in the

system, described by:

( ) ( )0,,),( axEaxEyxv xyyx ππ −= (1)

10

where Exf(x) is the expected value of f(x) and Ex|yf(x) is the expected value of f(x) given y. The symbols a0

and ay represent the actions taken by the decision maker in the absence and presence of information y and

π(x,a) represents the payoff achieved by selecting an action a, when the state realized by the environment

after the decision is x. However, as described by Panchal and coauthors [29], the use of value of perfect

information is impractical in real design scenarios. Hence, similar to the VoI-based Improvement Potential

metric [29], the PPI is based on a variation in the value of perfect information using a bound on the benefit

(payoff or utility) that can be achieved by obtaining perfect information derived from most truthful

information locally, as described in the following.

To illustrate the PPI, consider a scenario, shown Figure 2, in which the horizontal axis is a design

variable and the vertical axis is the corresponding expected overall payoff (EOP) that is achieved by

determining that specific design variable (DV). The design variable can be some physical dimension or

material property which the designer can control, whereas the EOP represents payoff or utility to a

designer. The EOP distributions predicted using model i refer to global information throughout the design

space. The dots are based on local information representing the EOP using the most truthful model at the

decision point x*. The decision point x* is the design variable value or the point in the design space that

solves the design problem, such as maximizes the payoff function or minimizes a deviation function, based

on the behavior predicted by the less complex model i.

As shown in Figure 2, the PPI refers to the difference in payoff between the outcomes of decisions

made using the most truthful model and the outcomes achievable using less complex models. In Figure 2,

we illustrate the refinement of an initial Model 1, resulting in Models 2 and 3. Model 3 is thus the most

refined model with the best possible PPI achieved so far. A mathematical definition of the PPI is presented

in the following.

11

EOP using model 1(global information)

EOP using refined model 2 (global information)

Px*2

x*1 x*2

Px*1PPI1

PT,x*i using most truthful model (local information)

PPI2

PPI3

EOP using refined model 3 (global information)

Px*3

x*3 DV

EOP

Figure 2. PPI evaluation (adapted from [24]).

Definition: The Process Performance Indicator PPIi for a given model i is defined as:

i*,*,1 xTxii PPPPIi−−= (2)

where Pi,x*i is the expected payoff in the design space calculated using model i and PT,x* is the expected

payoff of the most truthful model available at the decision point x*ì in the design space calculated using

model i. If the PPI is close to 1, the benefit of using a more complex model is negligible. This implies that

model i is appropriate from a decision-centric perspective. On the other hand, if its value is low relative to

the maximum value of 1, the model i evaluated should be refined. Besides the absolute PPI being close to

1, a relatively small gradient of subsequent PPIs obtained from less complex models shows that there is a

very low chance that adding of information, i.e., further refining the model, at increased complexity adds

value from a decision-centric perspective.

The same concept extends to higher dimensional problems where there are many design variables and

the EOP is determined by multiple conflicting criteria. In the case of multiple design variables, the curve is

replaced by a multidimensional surface. In the case of multiple design criteria that affect the EOP, the

12

criteria are combined into an overall EOP-function based on a designer’s preferences. However, calculation

of the PPI proceeds accordingly.

Also, multiple points in the design space can be used to refine the PPI evaluation for a specific model.

A designer has the freedom to choose the sampling strategy, i.e., number and locations at which the PPI is

evaluated, depending on the given design problem. We recommend leveraging strategies commonly used in

response surface generation for metamodeling (such as uniform or grid distributions, fractional factorials,

Latin Hypercube, or importance sampling) to sample the design space, as presented in the literature [26],

when using multiple points to refine PPI evaluations.

Using multiple points in the design space to refine PPI evaluations, an overall PPI can be calculated

based on various evaluation scenarios, for example the maximum absolute difference or average of the

difference in expected payoff achieved, depending on the given design problem and the designer’s attitude

towards risk. If a designer has a conservative attitude towards risk or the design problem at hand is

significantly characterized by uncertainty, a worst-case scenario is recommended. Alternatively, a designer

may choose to use the mean of calculated PPIs or specific points may be weighted differently, depending

on the characteristics of the underlying design problem. However, in this paper, we assume a conservative

mindset and evaluate the PPImin by referring to the maximum absolute difference in expected payoff or the

minimum PPI obtained:

( )ixTi PPPPI *,min max1 −−= (3)

It is important to note that the absolute value of the difference in EOP is chosen to represent the PPI.

Less complex models may over- or under-predict performance predictions resulting from the most truthful

model, even though only underprediction is illustrated in Figure 2. Also, the decision point at which the PPI

is evaluated does not necessarily need to be located at the global maximum of the expected payoff function,

as illustrated here. For example, the decision point could also be positioned at flat regions in the design

space when a designer prefers a more robust solution [8, 37]. Application of the systematic approach to

facilitate model refinement decisions based on the PPI must thus be adapted to the underlying payoff

function and decision formulation used to solve the given design problem.

In this paper, multi-attribute utility functions [17] based on a designer’s preferences are used as

expected payoff functions in the context of the utility-based compromise Decision Support Problem (cDSP)

13

[25, 35]. Preferences for achievement of goals are modeled as utility functions. Responses for a specific

design variable combination (potentially evaluated under statistical variability and imprecision) translate

into expected utility and are mapped into an overall deviation function which is to be minimized. The

mathematical formulation of the utility-based cDSP is presented in Figure 3. For further details on the

underlying utility-based cDSP decision template, the interested reader is referred to the literature [25, 35].

Given An alternative to be improved through modification. Assumptions used to model the domain of interest. The system parameters. n number of system variables p+q number of system constraints p equality constraints q inequality constraints m number of system goals gi(X) system constraint functions Ai(X) system goals ui(Ai(X)) utility function for each goal U(X) Overall, multi-attribute utility function = f[u1(A1(X)),u2(A2(X)),…, um(Am(X))] Find System variables X=X1, … , Xj j = 1,…, n Deviation variables di

-,di+ i = 1,…, m

Satisfy System constraints (linear, nonlinear) gr(X) = 0 r = 1,…,p gr(X) > 0 r = p+1,…, p+q System goals (linear, nonlinear) E[ui(Ai(X))] + di- - di+ = 1 ; i = 1,…,m Bounds Xj

min < Xj < Xjmax ; j = 1,…,n

di-,di

+ > 0 and di- . di

+ = 0 Minimize Deviation function: Additive Multi-attribute Utility Function

1

1 [ ( )] ( )m

i i ii

Z E U X k d d− +

=

= − = +∑

Figure 3. Mathematical utility-based compromise Decision Support Problem (cDSP) [25, 35].

It is important to realize that the PPI does not depend on the accuracy of a model only. It depends also

on the complete decision formulation that includes constraints, preferences, bounds, and goals. The VoI-

based PPI metric is developed considering only information about payoff or utility to a designer. Details of

a systematic approach for model refinement based on the PPI are described in the following section.

14

3.2 Systematic Approach for Model Refinement based on the PPI

The systematic approach based on the VoI-based PPI proposed in this paper is particularly well suited

for integrated product and materials design or the design of other complex engineered systems. For

example, a designer almost always seeks to avoid the cost of acquiring the most truthful information

available throughout the design space. Using the approach for evaluating and refining models presented in

this paper, a designer is now enabled to evaluate a least complex but valid model first, and, if necessary,

gradually refine this model until the most appropriate model is obtained. In order to do so, it is not required

to execute the model based on the most truthful model available throughout the whole design space (which

may be equivalent to developing a large number of prototypes), but, only at a few feasible points in the

design space (which then requires a few feasible prototypes only while leveraging the most appropriate

model to manage complexity), as discussed in Section 4 in the context of the BRP design problem.

An overview and specific steps of the proposed approach are presented in Figure 2. Step I relates to

formulating a design decision using utility-based cDSPs [25, 35]. Having determined a least complex initial

model in Step II, the decision formulation is solved for the decision point, i.e., the point in the design space

that minimizes the utility-based cDSP deviation function in Step III. Then, in Step IV, the most truthful

model is determined. In Step V, the PPI is determined to help determining whether to refine the model in

Step VI.

15

Step II) Determine Least Complex Initial

Model

Step I) Formulate Design Decision

Step III) Identify Decision Point(s)

Low Process Per-formance Indicator

Step V) Evaluate PPI

Step VI) Refine Model and Repeat Steps II

Through VStep IV) Determine

Most Truthful Model

Figure 4. Overview of the systematic approach to model refinement.

Step I: Formulate Design Decisions

The first step in the systematic approach to model refinement is to formulate the design decisions. The

decision-centric templates used in this paper are rooted in the Decision Support Problem Technique (DSP

Technique) [26], specifically the compromise Decision Support Problem, cDSP [25]. The utility-based

cDSP [35] is used here. It contains information about design variables, responses, analysis models used for

evaluating responses from design variables, a designer’s preferences, constraints, and goals. In an

environment such as ModelCenter® (from Phoenix Integration [31]), the instantiated computational objects

that can be manipulated are the constraints, variables, parameters, goals, response and analysis

computational templates. Preferences for achievement of goals are modeled as utility functions [17].

Step II: Determine Least Complex Initial Model

In the second step, a least complex but valid initial model is determined. The choice of an appropriate

initial model thus depends on specific characteristics of the design problem and decision at hand. However,

in this step, ensuring model validity to predict trends correctly is crucial. Based on the results of the

following steps, a designer decides whether to use the information produced by the model or refine this

model.

16

Step III: Identify Decision Point(s)

In Step III, the least complex initial model is evaluated based on the EOP. In this paper, EOP is

represented by the weighted deviation function given in the utility-based cDSP. Specifically, the point in

the design space that minimizes the utility-based cDSP deviation function, the decision point, is

determined. With respect to evaluating the least complex initial model, the sensitivity in main effects

should be analyzed using a sampling strategy and evaluation scenario tailored to the design problem

investigated, as described in Section 3.1. Acquiring additional information to analyze sensitivity is

beneficial, but its feasibility depends on the complexity and computational cost of the most truthful model

available as well as the design problem investigated.

Step IV: Determine Most Truthful Model

In Step IV, the most truthful model is determined involving experimental tests or relatively expensive

computations compared to the least complex initial model. It serves as a baseline for determining the PPI

locally at the specific decision point(s) identified in Step III.

Step V: Evaluate PPI

Having evaluated the most truthful model at the decision point(s), the PPI(s) is (are) calculated

according to Equation 2. If multiple decision points are used to calculate an overall PPI, a sampling strategy

and an evaluation scenario must be selected, depending on the decision maker’s attitude towards risk and

given design problem, as discussed in Section 3.1. However, having determined an overall PPI, a decision

is to be made whether to refine (Step VI) or use the information produced by the model. If the PPI is close

to 1 and the gradient of subsequent PPIs is sufficiently small, the benefit of refining the current model is

likely to be negligible from a decision-centric perspective.

Step VI: Refine Model and Repeat Steps III Through V

If the designer decides to refine the model evaluated, Steps III through V are to be repeated for a more

complex model. Step VI may be repeated until an appropriate model is obtained, which may also involve

using multiple models when, for example, models are refined through model replacement. However, the

17

best decision point(s) obtained previously is (are) to be included in evaluating the current model based on

the sampling strategy and evaluation scenario chosen initially.

3. DESIGNING MULTIFUNCTIONAL BLAST RESISTANT PANELS USING THE PROPOSED

APPROACH TO MODEL REFINEMENT

To demonstrate the use of the proposed approach to model refinement based on the PPI, it is applied to

the Blast Resistant Panel (BRP) design problem in this section. Details of the various simulation models

evaluated using the systematic strategy for model refinement based on the PPI are summarized in Table 3.

The steps of the systematic approach to obtain the refined models listed in Table 3 are discussed in the

following.

Table 3. Summary of simulation-based analysis models.

Model Mi Description (simulation times with respect to a Dell Dimension XPS_GEN_5 Intel®

Pentium® CPU 3.60GHz 2.00 GB of RAM) M1 Analytical analysis model (execution time ~ seconds) M2 Response surface analysis model of an ABAQUS explicit FEM model with a

mesh size of 0.02 using a central composite sampling method and full quadratic model fitting to the observed data (execution time ~ seconds)

M3 Response surface analysis model of an ABAQUS explicit FEM model with a mesh size of 0.02 using a central composite sampling method and quadratic stepwise regression model fitting to the observed data (execution time ~ seconds)

M4 Response surface analysis model of a nABAQUS explicit FEM model with a mesh size of 0.02 using a full factorial level 3 sampling method and quadratic stepwise regression model fitting to the observed data (execution time ~ 1 minute)

Most truthful

ABAQUS explicit FEM model with a mesh size of 0.005 (execution time ca. 20 minutes)

Step I: Formulate Design Decisions

The square honeycomb core sandwich panel BRP for which a design decisions in terms of utility-based

cDSPs is to be formulated consists of a front face sheet, cellular core with an out-of-plane honeycomb, and

back face sheet as shown in Figure 1. The front face sheet receives the initial pressure loading from the

blast. The topology of the core is designed to dissipate a majority of the impulse energy in crushing. The

back face sheet provides additional protection from the blast as well as a means to confine the core collapse

18

and store strain energy in bending and stretching. The responses of interest are the maximum deflection of

the back face sheet and the overall panel weight. Both must not exceed a maximum deflection and a

maximum mass per unit area.

Results presented in this paper are for the dynamic response to impulsive blast-type loads. In all cases,

a uniformly distributed impulse per unit area, I, is applied to the plate at time t = 0. For the sandwich panel

investigated in this paper, the impulse is applied only to the front face sheet towards the blast, again as a

uniform initial velocity, v = I/ρhf, where hf is the front face sheet thickness. The rationale for replacing the

pressure pulse acting on the panel by an initial impulse rests on the fact that the period of the blast pulse is

short compared to the response time of the panel – the period, t0, characterizing the leading portion of a

blast pulse is typically on the order of a tenth of a millisecond [43].

Following early work by Taylor [38], the time variation of the free-field pressure at any point in the

fluid engulfed by the pulse starting at t0 is characterized as follows [43]:

  0/0

ttepp −= (4)

where p is the pulse pressure, p0 is the peak pressure of free-field pulse, and t0 is the characteristic time of

the incident pressure pulse. Therefore, the momentum per area of the free field pulse becomes [43]:

 000 tppdtI ∫ ==

(5)

When the pressure pulse impinges on the panel it sets the plate in motion and is partly reflected. The time at

which the panel achieves its maximum velocity coincides with the onset of tensile stress (negative pressure)

at the interface between the fluid and the panel, leading to cavitation in the fluid shortly thereafter.

According to Xue and Hutchinson [42], the maximum possible moment, i.e., twice the free-field moment

impulse per area, 2I0, is transferred to the panel in air for a blast of short duration, unless the panel is

exceptionally thin. Hence, in case of sandwich panels for independent face sheet thicknesses and material

properties, the kinetic energy transferred from blast is [43]:

 

mIE sandkin

20

, 2= (6)

where m for sandwich structures equals the mass of the front face sheet, mf.

19

From Equation 6, it becomes obvious that there is a penalty associated with employing sandwich

construction versus solid panel construction. The initial kinetic energy must be dissipated by the structure.

Subject to the same initial momentum impulse, the ratio of initial kinetic energy per unit area imparted to

the sandwich front face sheet compared to that imparted to the solid plate is h/hf. Thus, if the design goal

for the sandwich panel is to sustain smaller deflections than the solid plate of equal mass when subject to

the same initial impulse, then the sandwich panel must absorb more than twice the energy as its solid

counterpart, as expressed in Equation 6. Muchnik [27] investigated and compared uniform and spherical

pressure waves in the context of blast resistant panels. The difference however has appeared to be

negligible in the early stages of design. Hence, in this paper, blast pressure waves with different pressure

over time distributions – assumed to be only uniformly distributed over a panel – are investigated.

The utility-based cDSP template is instantiated in the computational environment of ModelCenter®. In

an environment such as ModelCenter®, decision templates are computational objects that can manipulated,

as illustrated in Figure 5. However, during concept exploration using partially instantiated cDSP

formulations, response surfaces throughout the feasible design space are explored and extreme values are

identified first. Concept exploration using partially instantiated compromise DSP formulations is a

necessary activity to fully instantiate compromise DSP formulations. In particular, main effects are

identified to simplify the decision formulation by turning variables with minor effect on performance

prediction into constants.

As a result of various concept explorations using partially instantiated cDSPs, top panel height, bottom

panel height, core wall thickness, and core height have been identified as main effects. Therefore, the focus

in the following is on exploring design variables down to mesoscales, i.e., top-, core-, and bottom-panel

height as well as core wall thickness for the out-of-plane honeycomb sandwich panel BRP concept.

Material variables associated with smaller scale effects, in other words the elements constituting parts of

the containments system, are fixed as constants. Based on a systematic selection procedure [24] as well as

commonly known material properties (as for example summarized by Ashby [1]), a square honeycomb core

sandwich panel concept with aluminum front-face sheet, titanium core and (quasi-isotropic) polymer

carbon fiber composite back-face sheet is selected as the most promising BRP concept for maximum

energy dissipation and explored in the following.

20

In order to fully instantiate a utility-based cDSP template, conditional utilities for individual attributes

are elicited. Also, subjective probability functions for uncertain events are determined. Concept exploration

using partially instantiated cDSP templates has shown that results are by far most sensitive to uncertainty in

blast loading than material property uncertainties. Therefore, a subjective probability function with respect

to the most uncertain event – blast loading – is determined. Other uncertainties, such as material property

uncertainties, are neglected. However, in the context of designing BRPs, preferences are elicited for mean

and variance of deflection as well as mean and variance of total mass to find robust solutions [9, 37, 39],

based on uncertainty in blast loading.

Goals

Preferences

Variables

Parameters

Constraints

Response

Objective

Analysis

Driver

Fully Instantiated cDSP Template

Driver

BRP analysis modelResponse

corefront

backσ σ

εε

Blast

A

t

Other constraints, variables, parameters, goals, and preferences

Material properties

Objective function

Figure 5. Utility-based cDSP template instantiated in ModelCenter.

Blast peak pressure is specified to vary be between 190 and 280 MPa. Hence, beliefs regarding the

blast peak pressure are elicited through a series of elicitation questions. Answers to these questions then are

points on a cumulative distribution function of a designer’s subjective probability with respect to the blast

peak pressure. A maximum-likelihood-estimation [5] is conducted to estimate the subjective probability

distributions elicited. The triangular distribution provides the smallest estimated squared error of 0.0399.

This triangular distribution is conservative, tending towards the “worst case” blast peak pressure – a result

of the conservative attitude towards risk when eliciting beliefs. With that, the utility-based cDSP template

formulation can be fully instantiated as computational objects in ModelCenter, as specified in Table 4.

21

Table 4. Utility-based cDSP template formulation.

Given: Impulse load defined by uniformly distributed pressure, p [Pa], over time, t [s]

Time [s]

Bla

st [P

a]

[(t, p)] = [(0, 5.7628x107);(1x10-8, 2.8x108); (1x10-7, 5.7570x107);(1x10-6, 5.7055x107); (2x10-6, 5.6487x107);(5x10-6, 5.4817x107); (6x10-6, 5.4272x107);(1x10-5, 5.2144x107); (2x10-5, 4.7182x107);(1x10-4, 2.1200x107); (2x10-4, 7.7991x106);(3x10-4, 2.8691x106); (4x10-4, 1.0555x106);(5x10-4, 3.8829x105); (6x10-4, 1.4285x105);(7x10-4, 5.2550x104); (8x10-4, 1.9332x104);(9x10-4, 7.1119x103); (1x10-3, 2.6163x103)]

Uncertainty in peak blast, p0 [Pa] triangular distribution (2.6x108, 2.7x108, 2.8x108)

Vertical length full,-size panel LV [m] LV = 1 Horizontal length full-size panel, LH [m] LH = 1 Area, A [m2] A = 1 Core cell spacing, B [m] B = 0.1 Sandwich core cell shape square Shear-off resistance ΓSH ΓSH = 0.6 Core crushing and stretching factors for a square honeycomb λc = 1, λs = 0.5, Boundary conditions clamped on all edges Mesh size front face sheet, core, and back face sheet 0.02 Poisson’s ratio 0.3

Young’s modulus front face sheet, Ef [Pa] 88.5x109

Yield strength front face sheet, σY,f [Pa] 510x106 Density front face sheet, ρf [kg/m3] 2500 Young’s modulus back face sheet, Eb [Pa] 450x109 Yield strength back face sheet, σY,b [Pa] 4.x109 Density back face sheet, ρb [kg/m3] 1550 Young’s modulus core, Ec [Pa] 140x109 Yield strength core, σY,c [Pa] 1250x106 Density core, ρc [kg/m3] 4360

Utility function (preferences) for mean of deflection 31055.5

7

1007.61

1068.65

−

⋅

−

⋅+−

⋅−= − δδ

eU

Utility function (preferences) for variance of deflection 25.078.083.283.1 δδ

Δ−Δ ⋅+−= eU

Utility function (preferences) for mean of total mass WeW eU

21033.179.0

5.1−⋅+−−⋅=

Utility function (preferences) for variance of total mass 73.041049.112101210 WW eU Δ⋅−

Δ

−

⋅+−= Deviation variables: dδ

- = dW-= dΔδ

- = dΔW-= 0

Find: Geometry: - front face sheet thickness hf - back face sheet thickness hb - core thickness H - core wall thickness hc Value of deviation variables: dδ

+, dΔδ+, dW

+, and dΔW+

Satisfy: Constraints:

Mass per area w must not exceed 150 kg/m2 w < 150

22

Deflection must not exceed 10% of span, i.e., 0.1 m, for specified boundary conditions

δ < 0.1

Relative Density R must be greater than 0.07 to avoid buckling 07.02

2

2

≥−

=B

hBhR cc

SHfyffh

tpΓ≤

,

002σρ

Front face shear-off constraints

34

≤ff

c

hHR

ρρ

0,0 ≥∧=⋅ −+−+iiii dddd Deviation variables must be greater than or equal to zero and

multiply to zero for i = δ, Δδ, W, ΔW

Goals: Minimize deflection, δ

δδ Ud −=+ 1

Minimize variance of deflection, Δδ δδ Δ

+Δ −= Ud 1

Minimize weight, W WW Ud −=+ 1

Minimize variance of weight, ΔW WW Ud Δ

+Δ −=1

Bounds: Front face sheet thickness hf [m] 0.001 ≤ hf ≤ 0.05 Back face sheet thickness hb [m] 0.001 ≤ hb ≤ 0.05 Core thickness H [m] 0.01 ≤ H ≤ 0.1 Core wall thickness hc [m] 0.0001 ≤ hc ≤ 0.04 Minimize:

Deviation Function Z = kδdδ+ + kΔδdΔδ

+ + kWdW+ + kΔWdΔW

+ with kδ + kΔδ + kW + kΔW = 1

Step II: Determine Least Complex Initial Model

An existing analytical model, established by Fleck and Deshpande [14], as well as Hutchinson and Xue

[16], is used as the least complex initial model to predict performance, i.e., deflection and mass per area, of

square honeycomb core sandwich structures. The work of Hutchinson and Xue is based on a proper

treatment of underlying physical mechanisms. Also, it matches very well with experiments and extensive

finite element validations [16]. At the same time, it is easy to implement and computationally efficient

(simulation times on the order of seconds). Therefore, it is accepted as a valid and feasible analysis model

within the bounds of the design space defined in the utility-based cDSP formulation.

Following a three phase deformation theory, the impulse of the blast is received by the front face sheet

and momentum is transferred in a first phase. In the second phase, some of the kinetic energy is dissipated

through crushing of the core layer. The crushing strain is used to determine the crushed height of the core

layer and is derived by equating the plastic dissipation per unit area in the core to the loss of kinetic energy

23

per unit area. In the next phase, the remaining kinetic energy must be dissipated through bending and

stretching of the back face sheet. The equation for deflection is derived by equating the remaining kinetic

energy per unit area to the plastic work per unit area dissipated through bending and stretching. The

average plastic work per unit area dissipated is estimated by summing the dissipation from bending and

stretching. Blast pressure loading is modeled through an impulse load as specified in Table 4. The equation

for deflection δ is determined by equating the average plastic work per unit area dissipated to the kinetic

energy and is shown in Equation 7. The mass/area M is given in Equation 8. For details in deriving these

equations, the interested reader is referred to the literature [28].

( ) ( )( )

( ) ( )( )

20 , , ,2 2 2

, , 2

, , ,

20

,

313 9

2where 1

Y f f Y c c S Y b bY b b Y b b

f f c c b b

Y f f Y c c S Y b b

b b c cc

c c Y c f f f f b b c c

I h R H hh H h H

L h R H h

h R H h

I h R HH H H

R h h h R H

σ σ λ σσ σ

ρ ρ ρδ

σ σ λ σ

ρ ρε

λ σ ρ ρ ρ ρ

⎛ ⎞⎡ ⎤+ +⎛ ⎞ ⎣ ⎦⎜ ⎟− ± +⎜ ⎟ ⎜ ⎟+ +⎝ ⎠ ⎝ ⎠=⎡ ⎤+ +⎣ ⎦

⎛ ⎞+⎜ ⎟= − = −⎜ ⎟+ +⎝ ⎠

(7)

b b c c f fM h R H hρ ρ ρ= + + (8)

Step III: Identify Decision Point(s)

The utility-based cDSP given in Table 4 is solved by minimizing the weighted deviation function to

identify the decision points. With respect to evaluating this model in the following, a uniformly distributed

+/- 5 % sensitivity in the main effect, i.e., front-face sheet height as determined earlier, is analyzed –

referring to decision points 2 and 3.

Table 5. Decision points initial model M1.

Design Variables Responses x*i hf

[m] hb

[m] H

[m] hc

[m] δ

[m] Δδ [m]

W [kg]

ΔW [kg]

Z1,i [-]

1 0.015 0.03 0.052 0.038 0.097 0.004 225.47 10-21 0.467 2 0.01575 0.03 0.052 0.038 0.0997 0.004 226.32 10-13 0.466 3 0.01425 0.03 0.052 0.038 0.106 0.005 222.57 10-13 0.466

24

Step IV: Determine Most Truthful Model

Focusing on simulation-based design, an ABAQUS explicit FEM model, as illustrated in Figure 7, is

considered as the most truthful, functionally integrated model for concept exploration. The commercial

software package ABAQUS 6.6 CAE in its explicit version has been used to conduct simulations.

ABAQUS explicit is a well established tool in the materials domain to predict accurately the plastic

deformation. ABAQUS models have been transformed into scripts that can be read and executed by the

interpreter built into ABAQUS. The script mode allows to selectively iterate specific sections of the

underlying ABAQUS code if desired and to automate repetitive tasks.

The full-size blast resistant panel is assumed to be clamped on all edges and hence symmetrical in x

and y direction. Therefore, in order to reduce the simulation time, the full-size panel is divided into quarter

panels of which only one is simulated. As shown in Figure 6, quarter panel 1 and 2 are symmetric with

respect to the y axis. Hence, they are free to move and deform in the y direction but constrained in x

direction along the y axis. Also, they are free to rotate in x direction while rotationally constrained in y

direction. The same condition is applied on the bottom edge of panel 1 by only switching y and x

directions.

2 m

x

y

2 m

1 2

3

Figure 6. Full-size panel divided quarter panels.

For rapid concept exploration, a standard (uniform) mesh is used. If the mesh size is decreased, and

thereby the number of elements and nodes is increased, the accuracy of the FEA method could be further

increased. To achieve the best tradeoff between accuracy and simulation time, a finer mesh should be

assigned to some parts and coarser to others. For rapid concept exploration in this paper however, the

standard (uniform) mesh size can only be changed for major sections, such as front face sheet, core, or fill

25

materials as a whole. Of course, the use of physical models, prototypes or experiments as the most truthful

model would result in a more enhanced PPI evaluation. However, with respect to the focus on rapid

concept exploration and widespread use of FEM models to validate analysis models in the domain of

materials design, the scope of this work is limited to simulation-based design. Hence, the ABAQUS FEM

model validated in the work of Muchnik [27] is used as the most truthful model.

The ABAQUS FEM model is a computationally expensive model. Complexity increases with the

increase in mesh density. The key to selecting and evaluating a most truthful FEM analysis model is to

determine the mesh size at which discretization errors converge to zero. In this work, the Richardson

extrapolation [34] is used as the approach to evaluate discretization errors, i.e., errors that consist of the

difference between the exact solution of the governing equations and the corresponding solution of the

algebraic system. Discretization errors are primarily associated with both the mesh (structured,

unstructured, uniform, etc.) and the discretization method (Finite Elements, Finite Volumes, Boundary

Elements, Spectral methods, etc.) [40]. Assuming that the exact solution is smooth enough and that the

Taylor series expansion for the error is justified, the formal convergence (error) is of second order, and the

mesh spacing is sufficiently small such that the leading-order error term dominates the total discretization

error (i.e., the convergence is monotonic in the asymptotic range), the discretization error, εh, is estimated

using two different nested meshes, hi and hj (assuming that the refinement ration r = hi/hj > 1), as

12 −

−≅−=

ri

i

hhjhexact

φφφφε

(9)

where φexact is the “exact” solution of the governing equations at a given point x, φ h is the discrete solution.

Using the Richardson extrapolation error estimation in the context of the ABAQUS FEM analysis model, a

mesh size of 0.005 is determined as the mesh size at which discretization errors converge to zero.

26

Figure 7. Most truthful model.

Step V: Evaluate PPI

Evaluating both the initial and most truthful model at the identified decision points based on Equation 2

and the conservative worst-case evaluation scenario using the minimum of the PPIs determined at decision

points 1-3, the overall PPI1 for model M1 is determined. Results with respect to the least complex initial

model Z1,i as well as the most truthful model ZT,x*i are shown in Table 6. The overall PPI1 for the least

complex initial model M1 becomes 0.59.

Table 6. PPI1 initial model M1.

Design Variables x*i

hf [m]

hb [m]

H [m]

hc [m]

Z1,i [-]

ZT,x*i [-]

PPI1 [-]

1 0.015 0.03 0.052 0.038 0.467 0.055 0.588 2 0.01575 0.03 0.052 0.038 0.466 0.055 0.589 3 0.01425 0.03 0.052 0.038 0.466 0.055 0.589

Overall PPI1: 0.59

Step VI: Refine Model and Repeat Steps III through V

The PPI of about 0.59 of the least complex initial model M1 is not relatively high compared to the

maximum possible value of 1. Also, a more refined analysis models M2 can be derived from the most

truthful model with relative ease. Since the ABAQUS FEM model is a computationally expensive model,

performing design space exploration using this analysis model directly is difficult. Hence, instead of this

27

analysis model, less computationally expensive response surfaces are used. Using a central composite

sampling method and full quadratic model fitting to the observed data, Steps III through V are repeated for

model M2. Results are shown in Table 7. The overall PPI2 of this refined model M2 is 0.63 based on the

worst-case evaluation scenario.

Table 7. PPI2 model M2.

Design Variables x*i

hf [m]

hb [m]

H [m]

hc [m]

Z2,i [-]

ZT,x*i [-]

PPI2 [-]

4 0.0057 0.028 0.0356 0.034 0.0796 0.448 0.63 5 0.00599 0.028 0.0356 0.034 0.0797 0.446 0.63 6 0.0054 0.028 0.0356 0.034 0.1432 0.45 0.69

Overall PPI2: 0.63

Since the overall PPI2 of about 0.63 of this refined model M2 is still not high compared to the

maximum possible value of 1, a central composite sampling method and quadratic stepwise regression

model fitting to the data obtained from the ABAQUS FEM model is developed. Repeating Steps III through

V with this refined model M3, an overall PPI3 of 0.9 is calculated based on the worst-case evaluation

scenario. Results are shown in Table 8.

Table 8. PPI3 model M3.

Design Variables x*i hf

[m] hb

[m] H

[m] hc

[m]

Z3,i [-]

ZT,x*i [-]

PPI3 [-]

7 0.0054 0.031 0.0345 0.034 0.151 0.052 0.898 8 0.0057 0.031 0.0345 0.034 0.154 0.052 0.898 9 0.0051 0.031 0.0345 0.034 0.152 0.052 0.9

Overall PPI3: 0.9

An overall PPI3 of about 0.9 achieved with model M3 is relatively high compared to the maximum

possible value of 1. However, the gradient with respect to PPI2 and PPI3 is relatively high. Since a

conservative attitude towards risk is adopted in this paper, model M3 is further refined based on a more

exhaustive sampling method. Using a full factorial level 3 sampling method and quadratic stepwise

regression model fitting to the data obtained from the ABAQUS FEM model, model M4 is developed.

Repeating Steps III through V with this model M4, an overall PPI4 of 0.98 is calculated based on the worst-

case evaluation scenario. Results are shown in Table 9.

28

Table 9. PPI4 model M4.

Design Variables x*i hf

[m] hb

[m] H

[m] hc

[m]

Z4,i [-]

ZT,x*i [-]

PPI4 [-]

10 0.0077 0.0067 0.01 0.015 0.009 0.029 0.98 11 0.0081 0.0067 0.01 0.015 0.011 0.029 0.982 12 0.0073 0.0067 0.01 0.015 0.016 0.028 0.988

Overall PPI4: 0.98

Results of evaluating the various models in terms of their PPI are summarized in Table 10. Making the

trade-off between gathering more information to refine models for decision making, analysis model M4

yields a PPI4 of at least 0.98 at a relatively low gradient between subsequent PPIs based on the worst-case

evaluation scenario and results in the minimal deviation function value ZTx*i found of 0.028. Thus, model

M4, derived from the central composite sampling method and quadratic stepwise regression model fitting to

the data obtained from the ABAQUS FEM model, represents an appropriate analysis model for concept

exploration from a decision-centric perspective in the context of the specific BRP design problem

investigated. Details of the various simulation models evaluated using the systematic strategy for model

refinement based on the PPI are summarized in Table 3.

Table 10. Summary of results.

Design Variables Responses x*i hf

[m] hb

[m] H

[m] hc

[m] δ

[m] Δδ [m]

W [kg]

ΔW [kg]

ZT,x*i [-]

PPIi [-]

1 0.015 0.03 0.052 0.038 0.097 0.004 225.47 10-21 0.0553 0.59 4 0.006 0.028 0.036 0.034 4 10-5 0.003 145.3 0.025 0.448 0.63 7 0.0054 0.031 0.0345 0.034 3 10-8 6 10-24 145.08 2 10-14 0.052 0.9

10 0.0077 0.0067 0.01 0.015 0.097 1 10-24 80.82 7 10-15 0.028 0.98

Even though a PPI4 of 0.98 at ZTx*I of 0.028 is achieved, analysis models could still be refined by, for

example, increasing the number of elements in the FEM model, using a more exhaustive sampling method,

or leveraging more enhanced model fitting. As the analysis model is refined, the value of the PPI is likely to

increase. But, from a decision-centric perspective there is a very low chance that this additional information

gathering, in other words further refining analysis models, at increased computational expense results in

increased value to a designer.

29

Using model M4, a BRP design with the following specifications is determined as a result of the robust

concept exploration:

• 7.7 mm thick aluminum front face sheet,

• 6.7 mm thick polymer carbon fiber composite back face sheet thickness, and

• 1 cm thick titanium square honeycomb core with a core cell spacing of 1 cm and a core wall

thickness of 1.5 mm.

In this paper, a design specification for a multifunctional BRP has been obtained by applying the

proposed systematic approach and VoI-based PPI. However, a fundamental issue encountered when

executing various models is that the design space is highly nonlinear and not convex. This characteristic of

nonconvex partial-differential-equation-based problems that lack most analytical guarantees makes it

extremely unlikely that the global solution can be found with absolute certainty. So, one must settle for

local solutions, which need not be globally optimum solutions, but, must achieve the design goals set

initially. This, however, does not affect the general principles and steps of the systematic approach to

facilitate model refinement decisions presented in this paper.

4. CLOSURE

In response to the question of “How much refinement of a simulation model is appropriate for a

particular design problem, when knowledge of prediction accuracy or its error bounds is not available

throughout the entire design space”, a generally applicable approach for evaluating and refining models

based on the PPI is presented. The key to this systematic approach is in leveraging local information at

specific points in the design space obtained from experimentation or computationally expensive analysis

models to refine less complex models available globally from a decision-centric perspective based on a

VoI-based concept. It is well-suited to overcome the limitation of existing model refinement metrics, such

as the Improvement Potential [29], that perfect knowledge of the system behavior and error bounds is

available throughout the entire design space, particularly in scenarios of complex engineering systems or

integrated product and materials design involving refinement decisions using multiple models.

The advantage of the proposed systematic approach is that model refinement decisions are facilitated.

However, more than one refinement step may be required for identifying the most appropriate model. In the

30

worst case, multiple executions of models at increasing levels of refinement may result in computational

expenses that exceed the total cost of evaluating the most truthful model available directly. Also, if the most

truthful model is less expensive to execute or advanced computing capabilities are available, evaluation of

the most truthful model at multiple points using adaptive or sequential metamodeling techniques [32] is

recommended. Since the PPI is evaluated locally, it can only be determined exactly if it is evaluated at each

single point in the design space, which is not practically feasible. However, true exactness cannot really be

achieved with any of the existing metrics. Hence, there is a tradeoff that calls forth a designer’s judgment

based on the expected benefits from refining models through local evaluation derived from the most

truthful model available.

The main advantage of the metric is that it may be used to evaluate and quantify the impact of design

process decisions taking into account the overall design problem formulation and thereby help designers in

managing complexity and reducing modeling efforts, indicating when model refinement is necessary and

when an approximate model is appropriate from a decision-centric perspective. Evaluating results obtained

in the context of designing multifunctional BRPs, it has been shown to be advantageous to use the

generally applicable approach to evaluating and refining models based on the PPI proposed in this paper.

ACKNOWLEDGMENTS

We gratefully acknowledge the support and assistance given to us by The NSF I/UCRC Center for

Computational Materials Design, a joint venture between The Pennsylvania State University and Georgia

Institute of Technology.

REFERENCES

[1] Ashby, M.F., 1999, Materials Selection in Mechanical Design, Oxford, UK: Butterworth-Heinemann.

[2] Aughenbaugh, J.M., Ling, J., and Paredis, C.J.J., 2005, "The Use of Imprecise Probabilities to Estimate the Value of Information in Design." 2005 ASME International Mechanical Engineering Congress and RD&D Expo, Orlando, FL, USA. ASME, Paper Number: IMECE2005-81181.

[3] Aughenbaugh, J.M., and Paredis, C.J.J., 2005, "The Value of Using Imprecise Probabilities in Engineering Design." ASME 2005 Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Long Beach, CA, USA. ASME, Paper Number: DETC2005-85354.

31

[4] Box, G.E.P., Hunter, J.S., and Hunter, W.G., 2005, Statistics for Experimenters: Design, Innovation, and Discovery, New York, NY: Wiley.

[5] Bradley, S.R., and Agogino, A.M., 1994, "An Intelligent Real Time Design Methodology for Component Selection: An Approach to Managing Uncertainty," Journal of Mechanical Design, 116, pp. 980-988.

[6] Buede, D.M., 1999, The Engineering Design of Systems: Models and Methods, New York, NY: Wiley-Interscience.

[7] Chen, W., Allen, J.K., Mavris, D., and Mistree, F., 1996, "A Concept Exploration Method for Determining Robust Top-Level Specifications," Engineering Optimization, 26(2), pp. 137-158.

[8] Chen, W., Allen, J.K., and Mistree, F., 1997, "A Robust Concept Exploration Method for Enhancing Productivity in Concurrent Systems Design," Concurrent Engineering: Research and Applications, 5(3), pp. 203-217.

[9] Cherkassky, V., 2002, "Model Complexity Control and Statistical Learning Theory," Natural Computing, 1, pp. 109-133.

[10] Doraiswamy, S., and Krishnamurty, S., 2000, "Bayesian Analysis in Engineering MOdel Assessment." Proceedings of ASME Design Engineering Technical Conferences, Baltimore, NY. ASME, Paper Number DETC/DTM-21699.

[11] Eppinger, S.D., and Salminen, V., 2001, "Patterns of Product Development Interactions." International Conference on Engineering Design (ICED), Glasgow, August 21-23, pp 283-290.

[12] Evans, A.G., Hutchinson, J.W., and Ashby, M.F., 1999, "Multifunctionality of Cellular Metal Systems," Progress in Materials Science, 43, pp. 171-221.

[13] Fleck, N.A., and Deshpande, V.S., 2004, "The resistance of clamped sandwich beams to shock loading," Journal of Applied Mechanics, 71, pp. 386-401.

[14] Howard, R., 1966, "Information Value Theory," IEEE Transactions on Systems Science and Cybernetics, SSC-2(1), pp. 779-783.

[15] Hutchinson, J.W., and Xue, Z., 2005, "Metal sandwich plates optimized for pressure impulses," International Journal of Mechanical Sciences, 47, pp. 545–569.

[16] Keeney, R.L., and Raiffa, H., 1976, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, New York: John Wiley and Sons.

[17] Kusiak, A., Larson, N., and Wang, J., 1994, "Reengineering of Design and Manufacturing Processes," Computers and Industrial Engineering, 26(3), pp. 521-536.

[18] Lawrence, D.B., 1999, The Economic Value of Information, New York: Springer. [19] Ling, J., and Paredis, C.J.J., 2006, "An Information Economic Approach for Model Selection in

Engineering Design." ASME International Mechanical Engineering Congress and Exposition, Chicago, Il, U.S.A. ASME, Paper Number: IMECE2006-14535.

[20] Malak, R.J., and Paredis, C.J.J., 2004, "Foundations of Validating Reusable Behavioral Models in Engineering Design Problems." Proceedings of the 2004 Winter Simulation Conference, Washington D.C., pp. 420-428.

[21] Malak, R.J., and Paredis, C.J.J., 2007, "Validating Behavioral Models for Reuse," Research Engineering Design, 18(3), pp. 111-128.

[22] McDowell, D.L., 1998, "New Directions in Materials Design Science and Engineering." Report of a Workshop Sponsored by the U.S. National Science Foundation, Georgia Institute of Technology and Morehouse College, Atlanta, GA.

[23] Messer, M., 2008, "A Systematic Approach for Integrated Product, Materials and Design-Process Design," PhD Dissertation, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.

[24] Mistree, F., Hughes, O.F., and Bras, B.A., 1992, "The Compromise Decision Support Problem and the Adaptive Linear Programming Algorithm." In M.P. Kamat, Structural Optimization: Status and Promise pp. 251-290. Washington, D.C.: AIAA.

[25] Mistree, F., Muster, D., Shupe, J.A., and Allen, J.K., 1989, "A Decision-Based Perspective for the Design of Methods for Systems Design." Recent Experiences in Multidisciplinary Analysis and Optimization, Hampton, Virginia.

[26] Muchnick, H., 2007, "Robust Design of Multilevel Systems Using Design Templates," M.S. Thesis, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.

32

[27] Muchnick, H., Allen, J.K., McDowell, D.L., and Mistree, F., 2007, "A Template-Based Approach for Multilevel Blast Resistant Panel Design." ASME 2007 Design Engineering Technical Conferences and Computer and Information in Engineering Conference: Paper Number: DETC2007-35447.

[28] Panchal, J., Paredis, C.J.J., Allen, J.K., and Mistree, F., 2008, "A Value-of-Information Based Approach to Simulation Model Refinement," Engineering Optimization, 40(3), pp. 223-251.

[29] Panchal, J.H., 2005, "A Framework for Simulation-Based Integrated Design of Multiscale Products and Design Processes," PhD Dissertation, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA, USA.

[30] Phoenix Integration Inc. ModelCenter®, Version 5.0. 2004; Available from: http://www.phoenix-int.com/products/ModelCenter.html.

[31] Quian, Z., Seepersand, C.C., Joseph, V.R., Allen, J.K., and Wu, C.F.J., 2006, "Building Surrogate Models Based on Detailed and Approximate Simulations," Journal of Mechanical Design, 128, pp. 668-677.

[32] Radhakrishnan, R., and McAdams, D.A., 2005, "A Methodology for Model Selection in Engineering Design," Journal of Mechanical Design, 127, pp. 378-387.

[33] Richardson, L.F., 1910, "The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations with an Application to the Stresses in a Masonry Dam," Transactions of the Royal Society of London, Series A(210), pp. 307-357.

[34] Seepersad, C.C., Mistree, F., and Allen, J.K., 2005, "Designing Evolving Families of Products Using Utility-Based Compromise Decision Support Problem," Inaugural Issue of the International Journal of Mass Customization, 1(1), pp. 37-64.

[35] Simpson, T.W., Lautenschlager, U., and Mistree, F., 1998, "Mass customization in the age of information: the case for open engineering systems." In W. Read and A. Porter, The Information Revolution: Current and Future Consequences 49-71). Greenwich, CT: Ablex Publications.

[36] Taguchi, G., 1986, Introduction to Quality Engineering, New York: UNIPUB. [37] Thompson, S.C., Muchnik, H., Choi, H.-J., McDowell, D., Allen, J.K., and Mistree, F., 2006,

"Robust Materials Design of Blast Resistant Panels." AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Portsmouth, VA: Paper Number: AIAA 2006-7005.

[38] Vaz, M., and Gaertner, E.L., 2004, "Finite Element Simulation and Error Estimation of Polymer Melt Flow," Revista Materia, 9(4), pp. 453-464.

[39] Webster, N., 2000, Merriam-Webster`s Collegiate Dictionary, Springfield, MA: Merriam-Webster, Inc.

[40] Xue, Z., and Hutchinson, J.W., 2004, "A Comparative Study of Impulse-Resistant Metal Sandwich Plates," International Journal of Impact Engineering, 30, pp. 1283-1305.

[41] Xue, Z., and Hutchinson, J.W., 2005, "Metal Sandwich Plates Optimized for Pressure Impulses," International Journal of Mechanical Sciences, 47, pp. 545-569.

33

List of Tables Table 1 Word Formulation utility based cDSP Table 2 Assumptions in previous research efforts.

Table 3. Summary of simulation-based analysis models. Table 4. Utility-based cDSP template formulation. Table 5. Decision points initial model M1. Table 6. PPI1 Initial Model M1. Table 7. PPI2 Initial Model M2. Table 8. PPI3 Model M3. Table 9. PPI4 Model M4. Table 10. Summary of Results.

34

Table 11. Word formulation utility-based cDSP.

Given: • Uncertainty in impulse load and peak blast • Models for deflection and mass • Geometric parameters, and material properties • Utility functions for mean and variance of deflection and total mass Find: • Geometry variables: front face sheet thickness hf, back face sheet thickness hb, core thickness H, and core wall thickness hc • Values of deviation variables di

+ and di-

Satisfy: Constraints: • Mass per area, deflection, and relative density • Front face shear-off • Deviation variables must be greater-than/equal to 0 and multiply to 0 Goals: • Minimize deviation from mean and variance of deflection and weight Bounds: • Upper and lower bounds for system variables Minimize: • Deviation function

35

Table 12. Assumptions in previous research efforts.

Research Effort Assumption VoI- or utility-based model selection [10, 11, 33]

Error characterizations or utilities for model cost and accuracy are available throughout the entire design space

Traditional VoI-based Decision Evaluation [15]

Uncertainty is described using exact probability distributions throughout the entire design space

P-box approach [2]

Uncertainty is characterized using imprecise (intervals of) probabilities throughout the entire design space

Improvement Potential [29]

Non-probabilistic uncertainty using intervals is available throughout the entire design space

36

Table 13. Summary of simulation-based analysis models.

Model Mi Description (simulation times with respect to a Dell Dimension XPS_GEN_5 Intel®

Pentium® CPU 3.60GHz 2.00 GB of RAM) M1 Analytical analysis model (execution time ~ seconds) M2 Response surface analysis model of an ABAQUS explicit FEM model with a

mesh size of 0.02 using a central composite sampling method and full quadratic model fitting to the observed data (execution time ~ seconds)

M3 Response surface analysis model of an ABAQUS explicit FEM model with a mesh size of 0.02 using a central composite sampling method and quadratic stepwise regression model fitting to the observed data (execution time ~ seconds)

M4 Response surface analysis model of a nABAQUS explicit FEM model with a mesh size of 0.02 using a full factorial level 3 sampling method and quadratic stepwise regression model fitting to the observed data (execution time ~ 1 minute)

Most truthful

ABAQUS explicit FEM model with a mesh size of 0.005 (execution time ca. 20 minutes)

37

Table 14. Utility-based cDSP template formulation.

Given: Impulse load defined by uniformly distributed pressure, p [Pa], over time, t [s]

Time [s]

Bla

st [P

a]

[(t, p)] = [(0, 5.7628x107);(1x10-8, 2.8x108); (1x10-7, 5.7570x107);(1x10-6, 5.7055x107); (2x10-6, 5.6487x107);(5x10-6, 5.4817x107); (6x10-6, 5.4272x107);(1x10-5, 5.2144x107); (2x10-5, 4.7182x107);(1x10-4, 2.1200x107); (2x10-4, 7.7991x106);(3x10-4, 2.8691x106); (4x10-4, 1.0555x106);(5x10-4, 3.8829x105); (6x10-4, 1.4285x105);(7x10-4, 5.2550x104); (8x10-4, 1.9332x104);(9x10-4, 7.1119x103); (1x10-3, 2.6163x103)]

Uncertainty in peak blast, p0 [Pa] triangular distribution (2.6x108, 2.7x108, 2.8x108)

Vertical length full,-size panel LV [m] LV = 1 Horizontal length full-size panel, LH [m] LH = 1 Area, A [m2] A = 1 Core cell spacing, B [m] B = 0.1 Sandwich core cell shape square Shear-off resistance ΓSH ΓSH = 0.6 Core crushing and stretching factors for a square honeycomb λc = 1, λs = 0.5, Boundary conditions clamped on all edges Mesh size front face sheet, core, and back face sheet 0.02 Poisson’s ratio 0.3

Young’s modulus front face sheet, Ef [Pa] 88.5x109

Yield strength front face sheet, σY,f [Pa] 510x106 Density front face sheet, ρf [kg/m3] 2500 Young’s modulus back face sheet, Eb [Pa] 450x109 Yield strength back face sheet, σY,b [Pa] 4.x109 Density back face sheet, ρb [kg/m3] 1550 Young’s modulus core, Ec [Pa] 140x109 Yield strength core, σY,c [Pa] 1250x106 Density core, ρc [kg/m3] 4360

Utility function (preferences) for mean of deflection 31055.5

7

1007.61

1068.65

−

⋅

−

⋅+−

⋅−= − δδ

eU

Utility function (preferences) for variance of deflection 25.078.083.283.1 δδ

Δ−Δ ⋅+−= eU

Utility function (preferences) for mean of total mass WeW eU

21033.179.0

5.1−⋅+−−⋅=

Utility function (preferences) for variance of total mass 73.041049.112101210 WW eU Δ⋅−

Δ

−

⋅+−= Deviation variables: dδ

- = dW-= dΔδ

- = dΔW-= 0

Find: Geometry: - front face sheet thickness hf - back face sheet thickness hb - core thickness H - core wall thickness hc Value of deviation variables: dδ

+, dΔδ+, dW

+, and dΔW+

Satisfy: Constraints:

Mass per area w must not exceed 150 kg/m2 w < 150

38

Deflection must not exceed 10% of span, i.e., 0.1 m, for specified boundary conditions

δ < 0.1

Relative Density R must be greater than 0.07 to avoid buckling 07.02

2

2

≥−

=B

hBhR cc

SHfyffh

tpΓ≤

,

002σρ

Front face shear-off constraints

34

≤ff

c

hHR

ρρ

0,0 ≥∧=⋅ −+−+iiii dddd Deviation variables must be greater than or equal to zero and

multiply to zero for i = δ, Δδ, W, ΔW

Goals: Minimize deflection, δ

δδ Ud −=+ 1

Minimize variance of deflection, Δδ δδ Δ

+Δ −= Ud 1

Minimize weight, W WW Ud −=+ 1

Minimize variance of weight, ΔW WW Ud Δ

+Δ −=1

Bounds: Front face sheet thickness hf [m] 0.001 ≤ hf ≤ 0.05 Back face sheet thickness hb [m] 0.001 ≤ hb ≤ 0.05 Core thickness H [m] 0.01 ≤ H ≤ 0.1 Core wall thickness hc [m] 0.0001 ≤ hc ≤ 0.04 Minimize:

Deviation Function Z = kδdδ+ + kΔδdΔδ

+ + kWdW+ + kΔWdΔW

+ with kδ + kΔδ + kW + kΔW = 1

39

Table 15. Decision points initial model M1.

Design Variables Responses x*i hf

[m] hb

[m] H

[m] hc

[m] δ

[m] Δδ [m]

W [kg]

ΔW [kg]

Z1,i [-]

1 0.015 0.03 0.052 0.038 0.097 0.004 225.47 10-21 0.467 2 0.01575 0.03 0.052 0.038 0.0997 0.004 226.32 10-13 0.466 3 0.01425 0.03 0.052 0.038 0.106 0.005 222.57 10-13 0.466

40

Table 16. PPI1 initial model M1.

Design Variables x*i hf

[m] hb

[m] H

[m] hc

[m]

Z1,i [-]

ZT,x*i [-]

PPI1 [-]

1 0.015 0.03 0.052 0.038 0.467 0.055 0.588 2 0.01575 0.03 0.052 0.038 0.466 0.055 0.589 3 0.01425 0.03 0.052 0.038 0.466 0.055 0.589

Overall PPI1: 0.59

41

Table 17. PPI2 model M2.

Design Variables x*i hf

[m] hb

[m] H

[m] hc

[m]

Z2,i [-]

ZT,x*i [-]

PPI2 [-]

4 0.0057 0.028 0.0356 0.034 0.0796 0.448 0.63 5 0.00599 0.028 0.0356 0.034 0.0797 0.446 0.63 6 0.0054 0.028 0.0356 0.034 0.1432 0.45 0.69

Overall PPI2: 0.63

42

Table 18. PPI3 model M3.

Design Variables x*i hf

[m] hb

[m] H

[m] hc

[m]

Z3,i [-]

ZT,x*i [-]

PPI3 [-]

7 0.0054 0.031 0.0345 0.034 0.151 0.052 0.898 8 0.0057 0.031 0.0345 0.034 0.154 0.052 0.898 9 0.0051 0.031 0.0345 0.034 0.152 0.052 0.9

Overall PPI3: 0.9

43

Table 19. PPI4 model M4.

Design Variables x*i

hf [m]

hb [m]

H [m]

hc [m]

Z4,i [-]

ZT,x*i [-]

PPI4 [-]

10 0.0077 0.0067 0.01 0.015 0.009 0.029 0.98 11 0.0081 0.0067 0.01 0.015 0.011 0.029 0.982 12 0.0073 0.0067 0.01 0.015 0.016 0.028 0.988

Overall PPI4: 0.98

44

Table 20. Summary of results.

Design Variables Responses x*i hf

[m] hb

[m] H

[m] hc

[m] δ

[m] Δδ [m]

W [kg]

ΔW [kg]

ZT,x*i [-]

PPIi [-]

1 0.015 0.03 0.052 0.038 0.097 0.004 225.47 10-21 0.0553 0.59 4 0.006 0.028 0.036 0.034 4 10-5 0.003 145.3 0.025 0.448 0.63 7 0.0054 0.031 0.0345 0.034 3 10-8 6 10-24 145.08 2 10-14 0.052 0.9

10 0.0077 0.0067 0.01 0.015 0.097 1 10-24 80.82 7 10-15 0.028 0.98

45

List of Figures Figure 1. Geometry Selected BRP Concept Figure 2. PPI Evaluation (adapted from [24] Figure 3. Mathematically based compromise Decision Support Problem (cDSP) [25, 35]. Figure 4. Overview of systematic approach to model refinement.

Figure 5. Utility-based cDSP template instantiated in Model Center.

Figure 6. Full size panel divided quarter panels. Figure 7. Most truthful model.

46

Figure 1

Figure 1. Geometry selected BRP concept.

hc

B

hf

hb

H

Front-Face-Sheet

Back-Face-Sheet

47

EOP using model 1(global information)

EOP using refined model 2 (global information)

Px*2

x*1 x*2

Px*1PPI1

PT,x*i using most truthful model (local information)

PPI2

PPI3

EOP using refined model 3 (global information)

Px*3

x*3 DV

EOP

Figure 2. PPI evaluation (adapted from [24]).

48

Given An alternative to be improved through modification. Assumptions used to model the domain of interest. The system parameters. n number of system variables p+q number of system constraints p equality constraints q inequality constraints m number of system goals gi(X) system constraint functions Ai(X) system goals ui(Ai(X)) utility function for each goal U(X) Overall, multi-attribute utility function = f[u1(A1(X)),u2(A2(X)),…, um(Am(X))] Find System variables X=X1, … , Xj j = 1,…, n Deviation variables di

-,di+ i = 1,…, m

Satisfy System constraints (linear, nonlinear) gr(X) = 0 r = 1,…,p gr(X) > 0 r = p+1,…, p+q System goals (linear, nonlinear) E[ui(Ai(X))] + di- - di+ = 1 ; i = 1,…,m Bounds Xj

min < Xj < Xjmax ; j = 1,…,n

di-,di

+ > 0 and di- . di

+ = 0 Minimize Deviation function: Additive Multi-attribute Utility Function

1

1 [ ( )] ( )m

i i ii

Z E U X k d d− +

=

= − = +∑

Figure 3. Mathematical utility-based compromise Decision Support Problem (cDSP) [25, 35].

49

Step II) Determine Least Complex Initial

Model

Step I) Formulate Design Decision

Step III) Identify Decision Point(s)

Low Process Per-formance Indicator

Step V) Evaluate PPI

Step VI) Refine Model and Repeat Steps II

Through VStep IV) Determine

Most Truthful Model

Figure 4. Overview of the systematic approach to model refinement.

50

Goals

Preferences

Variables

Parameters

Constraints

Response

Objective

Analysis

Driver

Fully Instantiated cDSP Template

Driver

BRP analysis modelResponse

corefront

backσ σ

εε

Blast

A

t

Other constraints, variables, parameters, goals, and preferences

Material properties

Objective function

Figure 5. Utility-based cDSP template instantiated in ModelCenter.

51

2 m

x

y

2 m

1 2

3

Figure 6. Full-size panel divided quarter panels.

52

Figure 7. Most truthful model.