Managing approximation models in collaborative optimization

Download Managing approximation models in collaborative optimization

Post on 10-Jul-2016




1 download

Embed Size (px)


  • DOI 10.1007/s00158-004-0492-y


    Struct Multidisc Optim (2005) 30: 1126

    B.-S. Jang Y.-S. Yang H.-S. Jung Y.-S. Yeun

    Managing approximation models in collaborative optimization

    Received: 12 April 2004 / Revised manuscript received: 12 August 2004 / Published online: 7 January 2005 Springer-Verlag 2005

    Abstract Collaborative optimization (CO), one of the mul-tidisciplinary design optimization techniques, has been cred-ited with guaranteeing disciplinary autonomy while main-taining interdisciplinary compatibility due to its bi-level op-timization structure. However, a few difficulties caused bycertain features of its architecture have been also reported.The architecture, with discipline-level optimizations nestedin a system-level optimization, leads to considerably in-creased computational time. In addition, numerical difficul-ties such as the problem of slow convergence or unexpectednonlinearity of the compatibility constraint in the system-level optimization are known weaknesses of CO.

    This paper proposes the use of an approximation modelin place of the disciplinary optimization in the system-leveloptimization in order to relieve the aforementioned difficul-ties. The disciplinary optimization result, the optimal dis-crepancy function value, is modeled as a function of theinterdisciplinary target variables, and design variables ofthe system level. However, since this approach is hinderedby the peculiar form of the compatibility constraint, it ishard to exploit well-developed conventional approximationmethods. In this paper, neural network classification is em-ployed as a classifier to determine whether a design point

    B.-S. Jang (B)Structure/Shipbuilding & Plant R&D Institute, Samsung Heavy In-dustries, Koje-City, Kyungnam, 656-710, KoreaE-mail: beomseon.jang@samsung.comY.-S. YangDepartment of Naval Architecture and Ocean Engineering, SeoulNational University, San 56-1, Shillim-dong, Kwanak-gu, Seoul, 151-742, KoreaE-mail: JungRolling Stock Research Department, Structural Mechanics ResearchGroup, Korea Railroad Research Institute, 60-1, Woulam-Dong,Uiwang-City, Kyonggi-Do, 437-050, KoreaE-mail: YeunDepartment of Mechanical Design Engineering, Daejin University,San 11-1, Sundanri, Pochen, Kyonggi-do, 467-711, KoreaE-mail:

    is feasible or not. Kriging is also combined with the classi-fication to make up for the weakness that the classificationcannot estimate the degree of infeasibility.

    In addition, for the purpose of enhancing the accuracy ofthe predicted optimum, this paper also employs two approx-imation management frameworks for single-objective andmulti-objective optimization problem in the system-leveloptimization. The approximation is continuously updatedusing the information obtained from the optimization pro-cess. This can cut down the required number of disciplinaryoptimizations considerably and lead to a design (or Paretoset) near to the true optimum (or true Pareto set) of thesystem-level optimization.

    Keywords Approximation management framework Col-laborative optimization Neural network classification

    1 Introduction

    Most engineering design is carried out by multiple teamsthrough the process of decomposition into a set of tractabledesign problems for which a decision is made through an-alysis. These subproblems are hard to solve independentlysince they have interdependent (or coupled) relationships.The field of multidisciplinary design optimization (MDO)has emerged to develop approaches for efficiently optimiz-ing the design of such large coupled systems (Balling andSobieszcznskiSobieski 1996).

    Collaborative optimization (CO) was thereafter de-veloped to follow the multidisciplinary characteristics ofengineering design. It basically consists of a bi-level opti-mization architecture. It is the job of the discipline teamsto satisfy constraints while working to define a design onwhich all the teams involved can agree. The system teamis in charge of adjusting the target values so that suchagreement is possible while minimizing or maximizingthe system-level objective. This architecture is designed topromote disciplinary autonomy while maintaining interdis-ciplinary compatibility (Braun 1996a; Kroo and Manning2000). Due to this structure, CO is judged highly advan-

  • 12 B.-S. Jang et al.

    tageous in its applications to practical engineering designproblems.

    However, there is insufficient quantitative informationavailable to demonstrate its merit. Although its variousadvantageous features have been demonstrated in Braunet al. (1996b), Kroo et al. (1994), and Kroo (1997), CO isstill relatively immature and little experience in actual indus-trial environments is available yet.

    Some difficulties associated with the inherent features ofthe architecture have been reported. Numerical difficultiescaused by certain mathematical manipulations have beencited in Kroo and Manning (2000), Alexandrov and Lewis(1999, 2000). The use of quadratic forms for the system-level compatibility constraints means that changes in thesystem target variables near the solution have little effect onthe constraint values. Specifically, the gradient approacheszero, leading to difficulties for many optimizers, especiallythose that rely on linear approximations to the constraints.This leads to a slow rate of convergence of the system nearthe presumed solution. Several choices are possible for theforms of the disciplinary objective functions and the system-level compatibility constraints. These have been investigatedin a number of previous studies (DeMiguel and Murray2000; McAllister et al. 2000). In spite of such endeavors,in common with most multi-level schemes, the system-leveloptimization of CO may be sensitive to the selection ofdiscipline-level optimization parameters such as feasibilityor optimality tolerances.

    In addition, the price that must be paid for the advan-tages of decomposition is an increased computational time some studies have cited extremely large computationaltime. This unexpected cost is mostly caused by the fact thatthe architecture nests discipline-level optimization in thesystem-level optimization, that is, every disciplinary designshould be performed once in order to evaluate the compati-bility constraint of the system-level optimization. This prob-lem is one of the reasons that hinders the application of CO

    Fig. 1 Use of approximation in CO

    to engineering design, especially when a disciplinary designcannot be automated or when it requires time-consuming an-alysis.

    As an alternative to relieve the aforementioned prob-lems, the use of an approximation model has been proposedin place of the disciplinary design in CO (Sobieski et al.1998a). This models the result of the disciplinary design asa function of system-level design variables (i.e. di = f(z))as depicted in Fig. 1. The system-level optimizer uses thisapproximation instead of the disciplinary optimizations forestimating the interdisciplinary compatibility constraint.

    This concept was addressed initially in Sobiebski et al.(1998a). The paper uses the response surface method andsuggests two approaches to determine a quadratic fit tothe disciplinary optimal results: modeling directly the dis-crepancy function and the optimal interdisciplinary designvariables as a function of the target variables. This paper(Sobiebski et al. 1998a, and Sobiebski 1998b) uses a trustregion approach (Alexandrov et al. 1998) to refine the re-sponse surface sequentially.

    This concept is particularly appealing in CO for severalreasons. When used to model the results of disciplinary op-timization, the dimensionality of the approximation can bemuch smaller than would be required for fitting an inte-grated analysis system. Along with the usual approximationfeatures that aid in parallel execution and load balancing,this approach renders this very robust, but inefficient, opti-mizer acceptable. That is, the direct search method such asHooke and Jeeves method or the probabilistic search methodsuch as genetic algorithm or simulated annealing methodcan be used instead of the gradient-based method due to thenearly free computational cost of approximations. This canrelieve the convergence problem of CO (Kroo and Manning2000).

    What distinguishes the method of this paper from themethod of Sobiebski et al. (1998a) is that this paper adaptsglobal approximations instead of local approximations. The

  • Managing approximation models in collaborative optimization 13

    sequential response approaches, such as the trust regionmethod, sequentially isolate a small region of good designthat can be accurately represented by a low-order poly-nomial response surface model. However, this is not appro-priate for a multi-objective optimization problem becausethe response region of interest will never be reduced toa small neighborhood that is good for all conflicting ob-jectives. Finally the converged design of the trust regionmethod tends to depend considerably on its starting pointand move limit.

    However, there is a difficulty in the use of global ap-proximation. Because of the peculiar form of the compati-bility constraint, it is hard to use conventional approxima-tion methods, such as the response surface method, kriging,neural network, and so on, directly. In this paper, neural net-work classification is employed as a classifier to determinewhether a design point is feasible or not. Also, kriging iscombined with the classifier in order to estimate the degreeof infeasibility. Kriging, spatial correlation modeling, hasbeen asserted to be an approximation technique that showsgood promise for building accurate global approximations ofa design space (Cressie 1993; Simpson 1998b; Trosset andTorczon 1997).

    As an effort to reduce the inaccuracy due to the use ofapproximations, this paper also adopts two approximationmanagement frameworks for single-objective and multi-objective optimization problems. Their purpose is not onlyto reduce the computational cost but also to obtain a nearlytrue optimum or Pareto set by sequentially updating the ap-proximations. This framework was proven to reduce the re-quired function calls and to provide a more accurate optimalresult than a method just using approximation without suchan updating process (Yang et al. 2002).

    The paper is organized as follows: in Sect. 2, the com-bination of neural network classification and kriging is ex-plained. In Sect. 3, a brief description of the approximationmanagement strategy is provided. Its availability is demon-strated through illustrative examples in Sect. 4. Finally, con-clusions are laid out in Sect. 5.

    2 Approximation in system-level optimization in CO

    In this section, the characteristic of the discrepancy functionis described in comparison with a conventional constraintand the resultant difficulty in the modeling using conven-tional approximation techniques is cited in Sect. 2.1. Sec-tion 2.2 explains what extra-point information in CO is andhow to utilize it in the approximation. The employment ofneural network classification and the combination of it withkriging is discussed in Sect. 2.3. To begin with, two mathe-matical examples to assist the discussion of this section areintroduced.

    Mathematical example 1This problem was initially introduced in Simpson (1998a) asan illustrative example to demonstrate the usefulness of krig-ing. However, it is modified here into a constrained problemin order to reformulate it as a collaborative optimization

    problem. The definition as a standard optimization problemis described in Fig. 2.

    The problem can be formulated as a collaborative opti-mization problem as Fig. 3. The system-level optimizer hasone interdisciplinary target variable (y) and one interdisci-

    Fig. 2 The standard optimization problem definition for example 1

    Fig. 3 The collaborative optimization problem definition for ex-ample 1

    Fig. 4 Optimal solution of the system-level optimization problem ofexample 1

  • 14 B.-S. Jang et al.

    plinary compatibility constraint. A discipline only evaluatesthe objective function value ( f ), while, the other disciplineexecutes an optimization and passes the optimal discrepancyfunction value (d ) to the system level. The profiles of f andd depicted in Fig. 4 can be obtained by repeatedly perform-ing the discipline-level evaluation and the optimization forvarious values of y.

    Mathematical example 2This problem requires the solution of a coupled analysis toevaluate constraints and an objective function. As shown inFig. 5, there exists a link between the evaluation of y1 and

    Fig. 5 The standard optimization problem definition for example 2

    Fig. 6 The collaborative optimization problem definition for ex-ample 2

    Fig. 7 Discrepancy function profiles with varying system-level targetvariables

    y2, that is, one of them requires the other as input to itscomputation. In the collaborative optimization problem de-picted in Fig. 6, the system level coordinates the couplingbetween the coupling target variables, y12 and y21, which en-able two disciplines to compute their actual function valuesindependently and to determine the design variables, x1 andx2, separately . Figure 7 depicts the shape of the discrepancyfunction obtained from a series of discipline-level optimiza-tions varying the target variables.

    2.1 Difficulty in modelling the discrepancy function usingconventional approximation techniques

    In this subsection, the character of the discrepancy functionis explained and the difficulty in its approximation is demon-strated through the aforementioned examples.

  • Managing approximation models in collaborative optimization 15

    The compatibility constraint for system-level optimiza-tion (discrepancy function, di = 0) has a peculiar form com-pared with a conventional inequality constraint (gi 0). Itcan take on a non-negative value positive in the infeasibleregion and zero in the feasible region, while the conven-tional constraint may take a negative value in the feasibleregion. Therefore, there is a remarkable change in the trendof its response surface near the boundary between the feas-ible region and the infeasible region. This feature causes theslow convergence problem for CO and makes it difficult toemploy conventional approximation methods directly. Evena small fitting error of the approximation in the feasibleregion may lead to a misjudgment of a feasible design asinfeasible. This may eventually restrict the feasible region.This problem can be observed in the aforementioned mathe-matical example 1 and example 2, as shown in Figs. 4 and 7.

    Figure 8a and b depict the profiles of the discrepancyfunction approximated by kriging and neural network, re-spectively in comparison with the exact profile for ex-ample 1. The approximation models are fitted to three sam-ple points. Figure 9a, b, and c show three profiles of theexact shape, kriging and neural network for the discrepancyfunction 1 of example 2, respectively. In this problem, nine

    Fig. 8 Profiles of a...


View more >