title model-free controller design based on simultaneous ......title model-free controller design...

Title Model-Free Controller Design based on SimultaneousPerturbation Stochastic Approximation( Dissertation_全文 )

Author(s) Mohd, Ashraf bin Ahmad

Citation 京都大学

Issue Date 2015-03-23

URL https://doi.org/10.14989/doctor.k19125

Right 許諾条件により本文は2015/06/15に公開

Type Thesis or Dissertation

Textversion ETD

Kyoto University

Model-Free Controller Design

based on Simultaneous

Perturbation Stochastic

Approximation

Dissertation

Submitted in Partial Fulfillment of the Requirements

for the Degree of Doctor of Informatics

Mohd Ashraf bin Ahmad

Department of Systems Science

Graduate School of Informatics

Kyoto University

Abstract

Recently, with the rapid growth in science and engineering, most of the real-

world process plants have been built on a large scale and complex systems. As a

consequence, modeling of such systems may become very difficult and require a

lot of effort. Therefore, it is necessary to develop a control method that does not

depend on plant models, which is known as the model-free control approach. At

the same time, it is also worthy to consider an optimization tool for the model-

free approach that is simple to understand for engineers and can optimize a large

number of control parameters in a fast manner. So far, there have not been enough

literatures to discuss the application of model-free control schemes for the above

demands.

Motivated by the above background, a model-free control scheme is considered in

our study. Here, a simultaneous perturbation stochastic approximation (SPSA)

algorithm is suggested as a promising tool for the model-free control approach.

Then, this dissertation focuses on assessing the effectiveness of the SPSA-based

algorithm for various model-free control problems such as PID tuning of MIMO

systems, optimizing fuel consumption of hybrid electric vehicles, and maximizing

power production of wind farms.

Firstly, we present a performance comparison of SPSA-based methods for PID

tuning of MIMO systems. In particular, four typical SPSA-based methods, which

are one-measurement SPSA (1SPSA), two-measurement SPSA (2SPSA), global

SPSA (GSPSA), and adaptive SPSA (ASPSA) are examined. Their performances

are evaluated through extensive simulation for several controller design examples,

in terms of stability of the closed-loop systems, tracking performance, and compu-

tation time. In addition, the performance of the SPSA-based methods is compared

to the other stochastic optimization based approaches.

Secondly, we propose a model-free controller design for hybrid electric vehicle

systems. Here, a switching control scheme is adopted, where each sub-controller

is specified for each driving condition, in order to improve the fuel efficiency. An

SPSA-based method is utilized to optimize a large number of design parameters

in the switching controller. The design method is applied to the JSAE-SICE

benchmark problem, which is developed using GT-SUITE of Gamma Technologies,

Inc. and integrated with Simulink / MATLAB. The effectiveness of the proposed

i

ii

controller is evaluated in terms of the fuel efficiency improvement and driver’s

satisfaction, as compared to the sample controller of the benchmark problem.

Finally, we provide a model-free approach for maximizing power production of

wind farms. Based on the information on the wind farm configuration, such as the

turbine location and wind direction, we propose a multi-resolution SPSA (MR-

SPSA)-based method that can achieve fast model-free controller tuning. In order

to evaluate the effectiveness of our proposed scheme, a wind farm model with

dynamic characterization of wake interaction between turbines is used and then

the proposed method is applied to the Horns Rev wind farm. Furthermore, the

performance of the MR-SPSA-based method is also compared with other existing

model-free methods, in terms of maximum power production and convergence

time.

Acknowledgements

I would wish to convey my earnest gratitude to a number of people who have

taught, helped, stimulated and supported my study.

Firstly, I would like to express my sincere gratitude to Professor Toshiharu Sugie

for his excellent guidance, good motivation and dedicated support during my pe-

riod of study. Without his advice and encouragement, this dissertation could never

be completed. Once again, I really appreciate the training and guidance he have

passed on to me over the years.

Secondly, I would like to thank Associate Professor Shun-ichi Azuma for his valu-

able advice and informative discussion. I have learned a lot of skills and techniques

that he has presented to me during this period of study, especially in composing

a good technical article. I very appreciate for the time and effort that he spend

for me.

Thirdly, I would like to thank Dr. Ichiro Maruta for his expert advice and valuable

discussion, especially in performing the algorithm in Matlab. Also, I would like to

appreciate the senior, junior members and all the staff of the Mechanical System

Control Laboratory of Kyoto University for beneficial advices on my study and

daily academic life in Japan.

Fourth, I wish to express a special thanks and deep gratitude to the Ministry of

Education Malaysia and Universiti Malaysia Pahang for the scholarship that they

have provided for me and my family during the period of study.

Finally, I would like to express my heartfelt gratitude to my wife, Nor Aminah

binti Abdulah Suhaimi and my daughter for their great support and understanding

during this three and a half years of study. As well, special thanks to my entire

family for their moral support and warm encouragements.

iii

Contents

Abstract i

Acknowledgements iii

List of Figures vii

List of Tables ix

Symbols and Definitions x

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 What is model-free controller design? . . . . . . . . . . . . . 1

1.1.2 Why model-free controller design? . . . . . . . . . . . . . . . 2

1.1.3 Review on tools for model-free controller design . . . . . . . 3

1.1.4 Motivation of using SPSA as a tool for model-free controllerdesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Goals and Contributions . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Organization of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Model-Free Controller Design based on Simultaneous Perturba-tion Stochastic Approximation 11

2.1 Simultaneous Perturbation Stochastic Approximation . . . . . . . . 12

2.2 Convergence conditions of the SPSA algorithm . . . . . . . . . . . . 13

2.3 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4 Framework of Model-Free Controller Design based on SPSA . . . . 17

2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3 PID Controller Tuning of MIMO Systems 21

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.3 Model-Free PID Controller Design using Simultaneous PerturbationStochastic Approximation . . . . . . . . . . . . . . . . . . . . . . . 24

3.3.1 Simultaneous Perturbation Stochastic Approximation Meth-ods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

iv

Contents v

3.3.2 Model-Free Design . . . . . . . . . . . . . . . . . . . . . . . 27

3.4 Implementation and Results . . . . . . . . . . . . . . . . . . . . . . 27

3.4.1 Example 1 (Wind turbine) . . . . . . . . . . . . . . . . . . . 28

3.4.2 Example 2 (Distillation column) . . . . . . . . . . . . . . . . 31

3.4.3 Example 3 (Bell 201A-1 helicopter) . . . . . . . . . . . . . . 35

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4 Switching Controller Design for Hybrid Electric Vehicles 39

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2 Benchmark Problem: Optimizing Fuel Consumption of the HybridElectric Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.2.1 Hybrid Electric Vehicles System . . . . . . . . . . . . . . . . 41

4.2.2 Performance Evaluation . . . . . . . . . . . . . . . . . . . . 43

4.3 Switching Model-free Controller Design using Simultaneous Pertur-bation Stochastic Approximation . . . . . . . . . . . . . . . . . . . 44

4.3.1 Motivation of using Switching Controllers . . . . . . . . . . 44

4.3.2 Controller Description . . . . . . . . . . . . . . . . . . . . . 45

4.3.3 Model-Free Design . . . . . . . . . . . . . . . . . . . . . . . 47


4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5 Maximizing Power Production of Wind Farms 58

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.3 Multi-Resolution Simultaneous Perturbation Stochastic Approxi-mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.4 Model-Free Controller Design for Maximizing Power Production ofWind Farms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64


5.5.1 Wind Farm Model . . . . . . . . . . . . . . . . . . . . . . . 67

5.5.2 Horns Rev Example . . . . . . . . . . . . . . . . . . . . . . 69

5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6 Conclusion 81

A Parameters for the numerical tests of the PID tuning 84

B Convergence conditions of the 1SPSA, GSPSA, and ASPSA meth-ods 89

C Design parameters in the HEV controller 92

Contents vi

Bibliography 97

Published Papers 111

List of Figures

1.1 Model-free controller structure . . . . . . . . . . . . . . . . . . . . . 2

1.2 General flow of SPSA algorithm . . . . . . . . . . . . . . . . . . . . 7

2.1 Responses of the objective function f(θ(k)) in Example 1 . . . . . . 15

2.2 Responses of the objective function f(θ(k)) in Example 2 . . . . . . 16

2.3 Responses of the objective function J (κ(k)) . . . . . . . . . . . . . 18

3.1 PID control system . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2 Output responses of the wind energy conversion system . . . . . . . 31

3.3 Output responses of the binary distillation column system . . . . . 33

3.4 Output responses of the Bell 201A-1 helicopter system . . . . . . . 38

4.1 The architecture of split type hybrid powertrain . . . . . . . . . . . 41

4.2 Control system of the benchmark problem . . . . . . . . . . . . . . 42

4.3 Driving data of going to office on Monday . . . . . . . . . . . . . . 45

4.4 Driving data of going to office on Wednesday . . . . . . . . . . . . . 45

4.5 Driving data of returning to home on Monday . . . . . . . . . . . . 46

4.6 Driving data of returning to home on Wednesday . . . . . . . . . . 46

4.7 Diagram of Switching controller . . . . . . . . . . . . . . . . . . . . 47

4.8 Convergence of the objective function J(Pv, KSW ,R(t), (62007, 63699,..., 60519, 62007)) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.9 Responses of the engine torque Y7 and power Y8 for the drivingmode j = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

4.10 Responses of the motor 1 revolution speed Y9, torque Y10, and powerY11 for the driving mode j = 1 . . . . . . . . . . . . . . . . . . . . . 55

4.11 Responses of the motor 2 torque Y13 and power Y14 for the drivingmode j = 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.1 MR-SPSA optimization problem based on three resolution steps . . 63

5.2 The selection of the design parameter in each resolution . . . . . . . 66

5.3 The wake expansion in the Park model . . . . . . . . . . . . . . . . 69

5.4 Horns Rev wind farm layout . . . . . . . . . . . . . . . . . . . . . . 70

5.5 Convergence of the total power production P(υ1, υ2, ..., υ80) withwind direction 170◦ . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.6 Group selection in the first and second resolutions for wind direction220◦ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

vii

List of Figures viii

5.7 Group selection in the first and second resolutions for wind direction270◦ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.8 Four types of selected groups in the second resolution for wind di-rection 220◦ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.9 Non-static wind speed and direction during 10 h simulation time . . 77

5.10 Total power improvement of the MR-SPSA, SPSA, GT, and FS-MPPT-based methods for non-static wind speed and direction dur-ing 10 h simulation time . . . . . . . . . . . . . . . . . . . . . . . . 77

5.11 Layout of Horns Rev wind farm with five turbine failures . . . . . . 78

5.12 Convergence of the total power production P(υ1, υ2, ..., υ75) withfive turbine failures . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

C.1 Engine controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

C.2 ICE controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

C.3 Motor 1 controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

C.4 Motor 2 controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

C.5 Battery controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

C.6 General switching controller . . . . . . . . . . . . . . . . . . . . . . 94

List of Tables

2.1 The values of convergence time for different dimensions . . . . . . . 19

3.1 Design parameters of Example 1 . . . . . . . . . . . . . . . . . . . . 29

3.2 Statistical result of Example 1 . . . . . . . . . . . . . . . . . . . . . 30



3.5 Statistical result of Example 2 with time delay and stochastic dis-turbance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34



4.1 Fuel efficiency comparison for benchmark problem . . . . . . . . . . 40

4.2 The information of reference input, control input, and output ofHEV system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.3 Driving data on [0, 17595] . . . . . . . . . . . . . . . . . . . . . . . 46

4.4 Design parameters in the HEV controller . . . . . . . . . . . . . . . 49

4.5 The driving data for the switching controller design . . . . . . . . . 50

4.6 Simulation results of the total power for the engine, motor 1, andmotor 2 in three weeks driving data . . . . . . . . . . . . . . . . . . 53

5.1 Performance analysis of the total power production (MW) for theMR-SPSA, SPSA, and existing model-free approaches with differentwind directions, Std.: Standard deviation . . . . . . . . . . . . . . . 72

5.2 Performance analysis of the convergence time (h) for the MR-SPSA,SPSA, and existing model-free approaches with different wind di-rections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.3 Performance comparison of different group selections in the secondresolution for wind direction 220◦ . . . . . . . . . . . . . . . . . . . 76

5.4 Performance comparison of the MR-SPSA, SPSA, and existing model-free approaches with five turbine failures . . . . . . . . . . . . . . . 79

A.1 Parameters of the SPSA-based algorithms . . . . . . . . . . . . . . 88

C.1 Initial values of the design parameters . . . . . . . . . . . . . . . . 95

C.2 Optimal values of the design parameters . . . . . . . . . . . . . . . 96

ix

Symbols and Definitions

In this thesis, we use the following symbols and definitions. The symbols R and R+represent the set of real numbers and the set of positive real numbers, respectively.

The symbol Sn×n denotes the set of n×n positive definite matrices. The cardinalityof set S is denoted by | S |. The vector whose all elements are one is denotedby 1. For the vector θ, we use ‖ θ ‖ to express the standard Euclidean norm.For the random variable V , the probability of event V = a is represented by

P(V = a). The expectation of the random variable b is denoted by E(b). Forδ ∈ R+, satδ : Rn → Rn denotes the saturation function whose i-th element givenas follows:

The i-th element of satδ(θ) =

δ if δ < θi

θi if − δ ≤ θi ≤ δ−δ if θi < −δ

where θ ∈ Rn and θi ∈ R is the i-th element of θ.

x

Chapter 1

Introduction

1.1 Background

1.1.1 What is model-free controller design?

Model-free controller design is to find a controller using the input-output (I/O)

data of the systems, which are controlled tentatively, but without using explicit

or implicit information of the plants such that the performance of the control

objective is achieved. The conceptual structure of the model-free controller is

described in Figure 1.1. For example, the control objective is evaluated in terms

of the minimization of the measured error and input of the controlled system,

and minimization or maximization of the measured output (e.g., minimization

of the fuel consumption, maximization of the power production). The control

performance value of the given evaluation period is then used by the optimization

tool to obtain an updated tuning value for the controller at the next iteration.

This tuning process is iteratively performed until the final iteration.

The significant feature of this method is that both controller design and control

system analysis are simultaneously performed based on the measured I/O data.

In addition, since the controller is designed without using the plant model, this

method can directly handle a large class of plants (possibly nonlinear) as long as

the required measurement data is available.

1

Chapter 1. Introduction 2

Figure 1.1: Model-free controller structure

1.1.2 Why model-free controller design?

The motivation and necessity of model-free controller design have been discussed

in many literatures from the aspects of theory and applications. The main reasons

for implementing this approach are summarized as follows:

(i) Independence of the plant models:

Today, with the rapid development of science and technology, practical plants

in various fields, such as in the chemical industry, machinery, electronics,

electricity, transportation and logistics, have grown up to a large-scale pro-

duction and the processes have become more complex. As a result, modeling

of the plants using first principles or system identification may become more

difficult. Even if it is possible to develop such models, it may consume a lot

of time and require a significant amount of effort. Therefore, the model-free


control scheme is useful to overcome this situation, since this scheme does

not explicitly include any parts or the whole of plant models.

(ii) No gap between control theory and real application:

The model-free controller design has overcome the unmodeled dynamics

problem in the traditional model-based controller design, which may require

a robust control framework. In other words, its control performance does

not depend on the accuracy of the model. Finally, a huge gap between the

control theory and real application vanishes, since the model-free controller

design directly measures I/O data in real time from actual process plants.

(iii) Practical controller design:

From the perspective of practical applications, most of the industrial pro-

cesses require low-cost and easy-to-install control algorithms. In other words,

most of the engineers try to avoid complex mathematics and identification

theory (which is the heart of the traditional model-based controller design),

since it is hard to understand and requires additional effort and time. There-

fore, a model-free control scheme, which requires less complexity, contributes

great practical demands.

From the above reasons, model-free controller design is useful and it has become

one of the important topics to be explored in both theory and applications.

1.1.3 Review on tools for model-free controller design

The fundamental essence of the model-free controller design concerns data-based

optimization tools to determine the optimal controller. So far, there exist a large

number of data-based optimization methods. In general, they can be divided into

two classes of optimization methods:- multi-agent-based optimization and single-

agent-based optimization.

The multi-agent-based optimization methods include particle swarm optimiza-

tion [1], artificial bee colony [2], ant colony optimization [3], bacterial foraging

[4], genetic algorithm [5], differential evolution [6], artificial immune system [7],

and spiral optimization [8]. These optimization methods, which are also called

population-based searches, normally employ a huge number of agents over a large


set of feasible solutions. Then, these agents will interact with one another through

some specific mechanism and try to improve multiple candidate solutions. How-

ever, these methods perform well only for a small number of design parameters,

which is no more than 10. In order to handle a large number of design parameters, a

cooperative version of them has been introduced, e.g., cooperative co-evolutionary

[9], cooperative particle swarm optimization [10], cooperative genetic algorithm

[11], and cooperative artificial bee colony [12]. Most of these optimization meth-

ods are commonly invented by the computer science community and they have

tested their algorithms on artificial benchmark problems, e.g., Rosenbrock, Ras-

trigin, Griewank, Schaffer and Sphere functions. Therefore, the effectiveness of

their optimization methods in the actual control problems is not clear.

Recently, there have also been literatures that use these population-based searches

for model-free controller design such as in tuning the controller for robotic systems

[13–16], industrial and automation plant [17–20], renewable energy plant [21–23],

and transportation systems [24–26]. However, these optimization tools require

heavy computation time to achieve convergence due to the number of evaluated

objective functions per iteration being proportional to the number of agents. Thus,

a tool that requires less computation time is needed.

Meanwhile, the single-agent-based algorithms, which are also called single solution

searches, include random search [27], variable neighborhood search [28], simulated

annealing [29], stochastic gradient [30], tabu search [31, 32], simultaneous per-

turbation stochastic approximation (SPSA) [33], smoothed functional algorithm

[34], iterated local search [35], and greedy randomized adaptive search procedure

[36]. These approaches concentrate on modifying and improving a single candi-

date solution based on the random perturbation of its design parameter elements.

Therefore, they require less computation time in their design process than the

multi-agent-based searches. As a result, these optimization methods would be an

attractive tool for the model-free controller design.

Recent applications of these methods in the model-free control scheme mostly in-

clude the tuning of the parameters in the classical PID controller and intelligent

controllers such as fuzzy logics and neural networks. For example, a simulated an-

nealing has been used for the PID tuning in a multi-objective optimization prob-

lem. Their method has been evaluated in a super-maneuverable fighter aircraft

system [37]. A tabu search for tuning the PID controller in a simple process plant


has been introduced by [38]. Similar work has also been reported in [39] by per-

forming a comparison with Ziegler-Nichols, genetic algorithm and differential evo-

lution methods. In [40, 41], a model-free PID tuning based on the SPSA method

has been presented. For example, in [41], the SPSA method has been embedded

in a field-programmable gate array (FPGA) for online PID tuning. Meanwhile,

for the fuzzy logic controller case, a tabu search has been used in tuning the fuzzy

rules [42] and membership functions [43]. In [44], a simulated annealing is used for

tuning the input scaling factor of fuzzy membership functions, and has been tested

on a drilling force plant. The similar tuning strategy has been reported in [45]

by using the SPSA method for a three-dimensional fuzzy logic controller. On the

other hand, in the neural network controller case, the SPSA algorithm is used to

tune a large number of neurons of the neural network controller with applications

in a water treatment plant [46, 47] and trajectory tracking of a two-link robot [48].

However, most of the presented work only considers a small number of the control

design parameters, except for a few results in [46–48]. Therefore, it is necessary

to investigate the effectiveness of these methods in optimizing high-dimensional

control design parameters.

On the other hand, there are also another special class of data-driven tools that

are characterized by different names, such as iterative feedback tuning (IFT) [49],

correlation-based tuning (CbT) [50], virtual reference feedback tuning (VRFT)

[51], fictitious reference iterative tuning (FRIT) [52], and unfalsified control (UC)

[53]. In the IFT method, it uses iterative optimization of the controller parameters

according to an estimated gradient of a control performance criterion. In particu-

lar, two experiments are involved in IFT algorithm to estimate the gradient in each

iteration. The industrial and laboratory applications of IFT have been reported

in [54, 55]. A similar tool to IFT is CbT method. Here, the controller parameters

are tuned iteratively either to decorrelate the closed-loop output error between

designed and achieved closed-loop systems with the external reference signal or to

reduce this correlation. Unlike IFT, CbT only one experiment per iteration. In

[56, 57], the CbT method has been applied to tune the controller for the suspen-

sion system. Meanwhile, in the VRFT and FRIT methods, they employ a direct

tuning method with one-shot experimental data. In particular, it formulates the

controller tuning problem as a system identification problem by using a virtual

or fictitious reference model. So far, both VRFT and FRIT have been applied in

tuning the controller for FES-supported systems [58], vertical single-link arm [59],

active suspension system [60], time delay plant [61], and servo systems [62]. In the


UC method, it recursively falsifies control parameter sets that fail to satisfy the

performance specification. Specifically, at each iteration, it has to evaluate all the

controller candidates and select the optimal one from a finite controller set. The

UC method has been applied for designing the controller of two-link robot arm [63],

weigh belt feeder [64], and grinding process [65]. However, most of the reported

data-driven tools have been widely applied to a single-input-single-output (SISO)

plant with linear-time-invariant (LTI) system and there are few applications of

these tools to multi-input-multi-output (MIMO) plants and nonlinear systems.

1.1.4 Motivation of using SPSA as a tool for model-free

controller design

The simultaneous perturbation stochastic approximation is a highly efficient al-

gorithm that approximates the gradient based on only a small number of mea-

surements of the objective function. In particular, the flow diagram of the SPSA

algorithm is illustrated in Figure 1.2. Based on the selected SPSA coefficients,

each element of the current design parameter is perturbed in random directions

to obtain two design parameter vectors. Then, the objective function of each per-

turbed vector is measured. Furthermore, the gradient approximation is obtained

from these measured objective functions. Finally, the design parameter is updated

based on this gradient approximation until the termination criterion is satisfied.

As a result, the updated design parameter always has an opportunity to find a

good trajectory direction towards a local optimal solution.

In this study, we use the SPSA method as a tool for model-free controller design.

The motivation of utilizing the SPSA method is stated as follows. Firstly, the

searching mechanism of this algorithm is easy to implement, especially in real-

time control applications. This is because its pseudo-code only involves several

instructions, which is comfortable to execute by engineers or programmers. Also,

the algorithm requires a smaller number of coefficients that need to be specified,

and there are some guidelines providing insight into how to select these coefficients

in practical applications. Secondly, it exhibits less computation time, since only

two measurements of the objective function are required per iteration to update

the design parameter. Thirdly, the SPSA algorithm is suitable to solve a high-

dimensional optimization problem. This is because the gradient approximation

is only based on the measured objective functions, which are independent of the


Figure 1.2: General flow of SPSA algorithm

dimension of the design parameter. Note that this gradient approximation is very

efficient, since we do not need to perform a brute-force search that enumerates all

possible candidates from various combinations of elements to obtain the solution.

Moreover, based on the same reason, this algorithm is useful when it is very costly

or impossible to directly measure the gradient of the objective function. In other

words, it does not require any explicit form of the objective function. From the

above motivations, the SPSA algorithm can be a promising tool for model-free

controller design.


1.2 Goals and Contributions

The purpose of this study is to establish a framework of model-free controller

design based on SPSA. Firstly, a general step-by-step procedure of the model-free

control scheme based on the SPSA algorithm is described in Chapter 2. Then,

we apply our proposed method to three control problems:- model-free PID tuning

of MIMO systems, fuel consumption optimization of hybrid electric vehicles, and

maximizing power production of wind farms.

As an initial step, the first control problem investigates the effectiveness of various

SPSA-based algorithms for PID tuning of multi-input-multi-output (MIMO) sys-

tems. It is a challenging task because this problem involves a higher dimension of

PID parameters, unlike the existing studies which employ a small number of PID

parameters. Here, we identify the most effective SPSA-based method for high-

dimensional PID tuning based on the stability of the closed-loop system, tracking

performance, and computation time. The details of the comparative study are

presented in Chapter 3.

The second control problem, which is the maximization of the fuel efficiency of

hybrid electric vehicles, has become one of the main global issues today. In this

dissertation, a benchmark problem [66] for fuel consumption optimization using

the hybrid powertrain simulator, which consists of a highly complex hybrid elec-

tric vehicle system with various users’ driving conditions, is considered. Here, a

switching controller, which is also in a high dimension, is developed to specifically

handle various driving conditions. The effectiveness of the SPSA-based model-

free design is investigated in terms of fuel consumption reduction and the driver’s

satisfaction. This specific model-free control problem is discussed in Chapter 4.

Finally, one of the recent and attractive control problems in renewable energy

fields is to maximize the power production of large-scale wind farm systems. In this

study, we explore the applicability of the SPSA-based algorithm to this problem by

tuning the control action of each turbine. The control of multiple turbines is more

challenging than the single turbine due to the aerodynamic interactions among

the turbines, which are complex and difficult to model, besides the non-static

behavior of incoming wind. Here, by exploiting the information of the wind farm

structure, such as the turbine location and wind direction, we propose a multi-

resolution SPSA (MR-SPSA). The effectiveness of the MR-SPSA-based method


is evaluated in terms of maximum power production and convergence time. This

work is presented in Chapter 5.

In this dissertation, our SPSA-based method not only can handle a large number

of design parameters, but also can be utilized in the switching controller frame-

work and distributed controllers of a large-scale complex system in a practical

convergence time. Moreover, it is important to stress that the application of the

SPSA-based algorithm in the above cases is not directly implemented from the

standard one. For each case, some modifications to the standard SPSA algorithm

are necessary to successfully accomplish the control objective. Note that each

modification depends on each case and it would become a good guideline for other

researchers in handling similar problems.

1.3 Organization of Thesis

This thesis is organized as follows.

Chapter 2 describes a general framework of the model-free controller design by

standard simultaneous perturbation stochastic approximation. A step-by-step

procedure of the SPSA algorithm is presented. This is followed by illustrative

examples to indicate the effectiveness of the SPSA algorithm. Then, we show

how to implement the standard SPSA algorithm for a model-free design controller

based on a general control objective function.

Chapter 3 presents a performance comparison of several SPSA-based algorithms

for model-free PID tuning of MIMO systems. Specifically, the effectiveness of

four typical SPSA-based methods, which are one-measurement SPSA (1SPSA),

two-measurement SPSA (2SPSA), global SPSA (GSPSA), and adaptive SPSA

(ASPSA) are investigated through extensive simulation for several controller de-

sign examples. Their performances are evaluated in terms of the stability of the

closed-loop systems, tracking performance, and computation time. Furthermore,

the performance comparison with other stochastic optimization-based approaches

is also considered.

Chapter 4 proposes a model-free switching controller design for hybrid electric ve-

hicle systems. Here, it is expected to reduce the fuel consumption by adopting the

switching controller that depends on the driving conditions. The design method


is tested on the JSAE-SICE benchmark problem, which is developed using GT-

SUITE of Gamma Technologies, Inc. and integrated with Simulink / MATLAB.

The effectiveness of the proposed controller is compared with the sample controller

of the benchmark problem, in terms of the fuel consumption reduction and driver’s

satisfaction.

Chapter 5 provides a model-free approach based on the multi-resolution simulta-

neous perturbation stochastic approximation (MR-SPSA) for maximizing power

production of wind farms. The main advantage of the MR-SPSA-based method is

that a fast controller tuning can be achieved by exploiting the information of the

wind farm configuration, such as the turbine location and wind direction. Next, a

wind farm model with dynamic characterization of the wake interaction between

turbines is used to simulate the model-free scheme. The performance of the MR-

SPSA-based method is assessed in terms of the maximum power production and

convergence time.

Chapter 6 concludes this thesis.

Chapter 2

Model-Free Controller Design

based on Simultaneous

Perturbation Stochastic

Approximation

Simultaneous perturbation stochastic approximation (SPSA) is a method to op-

timize the design parameters such that a pre-specified objective function is mini-

mized or maximized. The fundamental theory of SPSA was introduced by James

C. Spall from John Hopkins University in 1992 [33]. This algorithm is efficient in

high-dimensional problems, providing a good solution for a relatively small num-

ber of measurements of the objective function. In the last two decades, the SPSA

has been applied to numerous applications such as queuing systems, industrial

quality improvement, aircraft design, pattern recognition, air traffic management,

sensor placement, parameter estimation, and fault detection. In this chapter, a

general framework of the SPSA algorithm is initially described. This is followed

by illustrative examples of solving artificial benchmark problems using the SPSA

algorithm. Finally, the application of the SPSA algorithm for model-free controller

design based on a general control objective function is presented.

11

Chapter 2. Model-Free Controller Design based on Simultaneous PerturbationStochastic Approximation 12

2.1 Simultaneous Perturbation Stochastic Approx-

imation

A general framework of the SPSA algorithm is presented in this section. Consider

the optimization problem given by

minθ∈Rn

f(θ) (2.1)

where f : Rn → R is an unknown objective function and θ ∈ Rn is the designvariable.

The SPSA algorithm [33] iteratively updates the design parameter to search an

optimal solution θ∗ ∈ Rn of (2.1). The update law is

θ(k + 1) = θ(k)− a(k)g(θ(k)), (2.2)

for k = 0, 1, ..., where a(k) is the gain and g(θ(k)) is the update vector given by

g(θ(k))=f(θ(k)+c(k)4 (k))−f(θ(k)−c(k)4 (k))

2c(k)

41(k)−1

42(k)−1...

4n(k)−1

. (2.3)

In (2.3), c(k) is another gain, 4(k) is the n-dimensional random perturbationvector and 4i(k) is the i-th component of the vector 4(k). For example, thegains a(k) and c(k) are given by a(k) = a/(k + 1 + A)α and c(k) = c/(k + 1)γ,

respectively, for non-negative numbers a, c, A, α and γ. Meanwhile, 4(k) is, forexample, drawn from the element-wise Bernoulli distribution{

P(4i(k) = 1) = 0.5,P(4i(k) = −1) = 0.5.

(2.4)

Note that, the selection of non-negative coefficients a, c, A, α and γ will be per-

formed by some guidance reported in [33].

Then, the SPSA algorithm is executed by the following steps:


Step I: Select the non-negative coefficients a, c, A, α and γ for the SPSA gain

sequences a(k) = a/(k + 1 +A)α and c(k) = c/(k + 1)γ. Set the initial conditions

of the design parameters θ(0) and set k = 0.

Step II: Generate n-dimensional random perturbation vector 4(k).

Step III: Obtain two values of the objective functions f(θ(k) + c(k)4 (k)) andf(θ(k)− c(k)4 (k)).

Step IV: Calculate the vector g(θ(k)) in (2.3).

Step V: Execute the update law in (2.2) and obtain θ(k + 1).

Step VI: If a pre-specified termination criterion is satisfied, the algorithm termi-

nates with the solution θ∗ := arg minθ∈{θ(0),θ(1),...,θ(k+1)}

f(θ). Otherwise, set k = k+1

and go to Step II.

On the other hand, the standard SPSA algorithm also can be applied to the

maximization problem

maxθ∈Rn

f(θ) (2.5)

where f and θ are similarly defined in (2.1). A solution to the problem in (2.5) is

obtained by the following iterative procedure

θ(k + 1) = θ(k) + a(k)g(θ(k)) (2.6)

where a(k) and g(θ(k)) are similarly defined in (2.2) and (2.3), respectively.

2.2 Convergence conditions of the SPSA algo-

rithm

This section presents the convergence conditions of the standard SPSA algorithm,

which can be described in the following theorem.


Theorem 2.1.[33] For the algorithm in (2.2), suppose that θ(0) ∈ Rn is given.Then,

limk→∞

θ(k) = θ∗ w.p.1 (2.7)

subject to the following assumptions:

(A1) a(k), c(k) > 0 ∀k, limk→∞ a(k) = 0, limk→∞ c(k) = 0,∞∑k=0

(a(k))2/(c(k))2 <

∞, and∞∑k=0

a(k) =∞.

(A2) For some α1, α2 > 0, E(f(θ(k) ± c(k) 4 (k))2) ≤ α1, and E(4i(k)−2) ≤α2 (i = 1, 2, ..., n) ∀k. Moreover, for almost all θ(k) (at each k ≥ π for someπ < ∞) and some α3 > 0, f (3)(θ) := ∂3f/∂θ>∂θ>∂θ> exists continuouslywith individual elements bounded by α3 for all θ in an open neighborhood

of θ(k).

(A3) ‖ θ(k) ‖


2.3 Illustrative Examples

In this section, we present two examples to illustrate the effectiveness of the stan-

dard SPSA algorithm in the previous section.

Example 1: Consider a smooth Rastrigin test function with n = 100 given by

f(θ) =n∑i=1

θ2i − 10cos(2πθi) + 10 (2.8)

where θ = [θ1, θ2, ..., θ100]T . Note that f(θ∗) = 0 at θ∗ = [0, 0, ..., 0]T . The

initial condition is selected to be closed to θ∗ after performing several preliminary

simulations. Let θ(0) = [0.05, 0.05, ..., 0.05]T that yields f(θ(0)) = 49.1935. Here,

we choose a = 0.002, A = 150 and c = 0.5 in the gains a(k) = a/(k + 1 + A)0.9

and c(k) = c/(k + 1)1/6. Next, 4(k) is drawn from the element-wise Bernoullidistribution in (2.4).

Figure 2.1: Responses of the objective function f(θ(k)) in Example 1

Figure 2.1 shows the responses of the objective function in 4000 iterations for

50 independent trials. In terms of the statistical analysis, the mean, best, worst

and standard deviation values of resultant objective functions are depicted as


0.0264, 0.0188, 0.0349 and 0.0043, respectively. It shows that the SPSA algorithm

converges to the optimal solution.

Example 2: Consider a non-smooth Ackley test function given by

f(θ) = 19− 20e−0.2

√1n

n∑i=1

θ2i+ e

1n

n∑i=1

sin(2π|θi|)(2.9)

where θ = [θ1, θ2, ..., θn]T , n = 20 and f(θ∗) = 0 at θ∗ = [0, 0, ..., 0]T . Let θ(0) =

[9, 9, ..., 9]T , so f(θ(0)) = 16.5848. In this example, we set the gain sequences

a(k) = 50/(k+ 11) and c(k)) = 2/(k+ 1)1/6 with 4(k) is set to be the same as inthe previous example.

Figure 2.2: Responses of the objective function f(θ(k)) in Example 2

The responses of the objective function in 2000 iterations for 50 independent trials

are shown in Figure 2.2. Meanwhile, the mean, best, worst and standard deviation

values of resultant objective functions are recorded as 2.7049×10−4, 5.4985×10−5,0.0070 and 0.0011, respectively. It indicates that the SPSA algorithm still main-

tains a good convergence in most of the trials, even for the non-smooth function.


Both examples clarify that the SPSA algorithm can solve high-dimensional prob-

lems by using only two measured objective functions per iteration. Also, the SPSA

algorithm is applicable to both smooth and non-smooth objective functions as long

as the measured objective functions are available. Therefore, these facts indicate

that this algorithm is a promising tool for a model-free control scheme.

2.4 Framework of Model-Free Controller Design

based on SPSA

Firstly, it is presented on how to apply the SPSA algorithm to the model-free

controller design. In general, let J (κ) : Rn → R be the function specifying thecontroller performance and κ ∈ Rn be the control parameter. Assume that therelation between κ and J is unknown. Then, a general model-free controller designis summarized as follows:

Step 1: Determine the initial value κ(0) and a pre-specified termination criterion.

Step 2: Perform the SPSA algorithm in Chapter 2.1 to the objective function Jand the design parameter κ := (κ1, κ2, ..., κn), i.e., by regarding J and κ as f andθ, respectively.

Step 3: After the pre-specified termination criterion is satisfied, the algorithm

terminates with the solution κ∗ = θ∗.

Next, it is worth to clarify the applicable condition of this framework. Basically,

the above framework can be applied to a given closed-loop feedback system, which

already consists of a stabilizing controller for an unknown plant model. Here, the

model-free controller design scheme is used to improve the control performance of

the given system by tuning its stabilizing controller based only on the I/O data of

the plant.

At this stage, it is important to emphasize that the above basic framework cannot

be straightforwardly implemented for real control problems. Specific to the control

problems in Chapters 3, 4 and 5, the reasons for the above fact are stated as follows:


(i) Unstable solution of the SPSA-based algorithm:

This situation corresponds to the control problems in Chapters 3 and 4. In

this case, there is a possibility that the updated control parameters grow

rapidly and yield an unstable solution. In order to demonstrate this situa-

tion, we cater a simple example. Consider

J (κ) =((κ− 1)T (κ− 1)

)3(2.10)

where κ = [κ1, κ2, ..., κn]T , n = 10 and J (κ∗) = 0 at κ∗ = [1, 1, ..., 1]T . Let

κ(0) = [0, 0, ..., 0]T , so J (κ(0)) = 1000. In this example, we set the gainsequences a(k) = 0.05/(k+ 200)0.602 and c(k)) = 0.01/(k+ 1)0.101 with 4(k)is drawn from the Bernoulli distribution. Figure 2.3 shows the response

of the objective function for 30 iterations. Notice that the J (κ) does notconverge to the local minimum value, which implies that the SPSA-based

algorithm could not solve the problem.

Figure 2.3: Responses of the objective function J (κ(k))

(ii) Impractical convergence time for a class of problems:

This condition corresponds to a class of high-dimensional control problems,


especially in Chapter 5. In this class of problems, it is required to tune a

large number of control parameters in a reasonable period of time. Here, the

SPSA-based algorithm often does not yield an acceptable convergence time,

since the number of evaluated objective functions to achieve the convergence

state is proportional to the dimension of the design parameter. This fact is

illustrated in the following example. Consider

J (κ) =((κ− 1)T (κ− 1)

)(2.11)

where κ = [κ1, κ2, ..., κn]T and J (κ∗) = 0 at κ∗ = [1, 1, ..., 1]T . In this

example, the termination criterion is given by

| J (κ(k + 1))− J (κ(k)) |< ε (2.12)

where ε is a small number. Then, the convergence time is defined by the

simulation time that is required to achieve the termination criterion in (2.12).

Here, the performance of convergence time is assessed for n = 10, n = 50,

n = 100, n = 500 and n = 1000. Let SPSA coefficients be same to the

control problem in (2.10), ε = 1 × 10−12, and κ(0) = [0, 0, ..., 0]T for eachn. Table 2.1 shows the values of convergence time for different dimensions.

It is distinctly shown that a larger value of convergence time is required for

larger n, which is impractical for a class of problems.

Table 2.1: The values of convergence time for different dimensions

Dimension n 10 50 100 500 1000Convergence time (sec) 0.1977 3.6239 18.2886 199.5673 739.2063

Based on the above evidence, the general framework of the model-free controller

design based on the SPSA algorithm cannot be directly used for real control prob-

lems. Therefore, it motivates us to consider more advanced versions of the model-

free controller design framework, which will be explained in detail in Chapters 3,

4 and 5.


2.5 Summary

A general framework of the model-free controller design based on the simultaneous

perturbation stochastic approximation has been presented in this chapter. This

framework will be used in the next three chapters with some modifications for

more specific control problems.

Chapter 3

PID Controller Tuning of MIMO

Systems

3.1 Introduction

PID (proportional-integral-derivative) control is most widely used control method

in the industry. In order to achieve better control performance, PID controller

design requires a more accurate model of the plant to be controlled. However,

it is often difficult to obtain accurate models for the real plants, and it is time

consuming to obtain such models even if it is possible. Hence, tuning strategies

for PID control parameters based on the I/O data (instead of the plant models)

have received considerable attention, which is called model-free PID tuning.

So far, model-free PID control tuning methods have been widely reported for the

case with single-input-single-output (SISO) systems. These include metaheuris-

tic and data-driven approaches, such as the particle swarm optimization [67–70],

ant colony optimization [71, 72], bacterial foraging [73, 74], spiral optimization

[75], genetic algorithm [76], simultaneous perturbation stochastic approximation

(SPSA) [40, 77], simulated annealing [78], stochastic multi-parameters divergence

optimization [79], virtual reference feedback tuning [80], iterative feedback tuning

[81], and fictitious reference iterative tuning [82, 83]. On the other hand, there are

a few results for multi-input-multi-output (MIMO) cases. For example, a genetic

algorithm-based method has been presented in [84] and the controller has been

applied to a two-input two-output binary distillation column. Similar works have

21

Chapter 3. PID Controller Tuning of MIMO Systems 22

been reported in [85–87] with the comparative assessment study of various evolu-

tionary algorithms, such as binary coded particle swarm optimization algorithm

and firefly algorithm. In [88], cooperative and co-evolving multiple swarms have

been proposed for the model-free design of PID controller of a ball mill pulverizing

system.

As shown in the above, many approaches use multi-agent-based optimization,

where the computation times per iteration are proportional to the number of

agents. As a result, these methods require heavy computation time in the design

process. Hence, it is necessary to develop a tuning strategy which requires less

computation time. Meanwhile, the SPSA is a promising tool from this viewpoint,

because it is known to be effective for a variety of optimization problems with less

computation time even for high-dimensional parameter tuning. In this sense, it is

applicable for tuning the PID controller of a more general class of systems, which

includes nonlinear and MIMO plants.

This chapter thus presents a comparative assessment of several SPSA-based meth-

ods for model-free PID tuning of MIMO systems. In particular, four types of meth-

ods, one-measurement SPSA (1SPSA) [89], two-measurement SPSA (2SPSA) [33],

global SPSA (GSPSA) [90–92], and adaptive SPSA (ASPSA) [93] are evaluated

in some MIMO controller design problems by extensive numerical simulations. In

order to clarify the benefit of the SPSA-based approaches, we consider a higher

dimension of PID parameters, unlike the existing literatures, which have consid-

ered not more than 10 PID parameters. Then, the performance of the methods is

assessed in terms of stability of the closed-loop systems, tracking performance, and

computation time. In addition, a comparative assessment between the SPSA-based

methods and others stochastic optimization-based methods, which are Simulated

Annealing (SA) [29] and Random Search (RS) [27], is also presented. So far, there

are few papers which purposely discuss on the performance comparison of various

PID controller tuning methods for MIMO systems. Therefore, it would be bene-

ficial to present this comparative study and identify the most effective model-free

approach for high-dimensional PID tuning.


3.2 Problem Formulation

Consider the MIMO PID control system depicted in Figure 3.1 where r(t) ∈ Rq,u(t) ∈ Rp, d(t) ∈ Rl, and y(t) ∈ Rq are the reference, the control input, thedisturbance, and the measurement, respectively. The plant is the MIMO system

G(s). The controller K(s) is given by

K(s) =

h11(s) · · · h1q(s)

.... . .

...

hp1(s) · · · hpq(s)

(3.1)for the PID controller

hij(s) := Pij

(1 +

1

Iijs+

Dijs

1 + (Dij/Nij)s

)(3.2)

where Pij ∈ R is the proportional gain, Iij ∈ R is the integral time, Dij ∈ R isthe derivative time, and Nij ∈ R is the filter coefficient. Next, we introduce the

Figure 3.1: PID control system

performance index for the system in Figure 3.1. Let

êi :=

∫ tft0

|ri(t)− yi(t)|2 dt, (3.3)

ûi :=

∫ tft0

|ui(t)|2 dt, (3.4)

where ri(t), yi(t), and ui(t) are the i-th elements of the vectors r(t), y(t), and

u(t), respectively, and t0 ∈ {0}⋃R+, and tf ∈ R+. The time interval [t0, tf ]

corresponds to the period for performance evaluation. Then, the performance


index is defined as follows:

J(P , I,D,N ) =

q∑i=1

w1iêi +

p∑i=1

w2iûi, (3.5)

where P := [P11 P12 ... Ppq]>, I := [I11 I12 ... Ipq]

>, D := [D11 D12 ... Dpq]>, and

N := [N11 N12 ... Npq]>, and w1i ∈ R (i = 1, 2, ..., q) and w2i ∈ R (i = 1, 2, ..., p)

are weighting coefficients, which are given by the designer. Note that p and q are

the dimensions of the control input u(t) and the measurement y(t), respectively,

which are given from the system G(s). The first term in (3.5) corresponds to the

tracking error, while the second means the control input energy. Here, the values

of w1i and w2i are selected in a similar way to the standard Linear Quadratic

Regulator (LQR) problem. Then, the model-free optimization problem can be

described as follows.

Problem 3.1. For the feedback control system in Figure 3.1, find a PID controller

K(s) which minimizes J(P , I,D,N ) with respect to P , I,D, and N based on

the measurement data (u(t),y(t)). �

3.3 Model-Free PID Controller Design using Si-

multaneous Perturbation Stochastic Approx-

imation

This section presents a model-free PID tuning method by using SPSA methods.

3.3.1 Simultaneous Perturbation Stochastic Approxima-

tion Methods

Consider the optimization problem given by (2.1) in Chapter 2.1 and we try to

obtain the solution using the iterative procedure in (2.2). Note that the solution

to the problem in this chapter will be obtained through several SPSA methods

and the differences between those methods are the choice of the function g. Here,

the function g in (2.2) for each SPSA method is summarized as follows.


(i) One-measurement simultaneous perturbation stochastic approximation

(1SPSA):

g(θ(k))=

f(θ(k)+c(k)41(k))

c(k)411(k)...

f(θ(k)+c(k)41(k))c(k)41n(k)

(3.6)where c(k) is the gain and 41i(k) is the i-th elements of a random vector41(k) ∈ Rn. Note that some guidance to choose the gain c(k) and therandom vector 41(k) is reported in [89].

(ii) Two-measurement simultaneous perturbation stochastic approximation

(2SPSA):

g(θ(k))=

f(θ(k)+c(k)41(k))−f(θ(k)−c(k)41(k))

2c(k)411(k)...

f(θ(k)+c(k)41(k))−f(θ(k)−c(k)41(k))2c(k)41n(k)

. (3.7)The detail of the algorithm is discussed in Chapter 2.1 and [33].

(iii) Global simultaneous perturbation stochastic approximation

(GSPSA):

g(θ(k))=g2(θ(k),41(k)) + b(k)42(k) (3.8)

where g2(θ(k),41(k)) is same as g(k) in (3.7), 41(k) ∈ Rn and 42(k) ∈ Rn

are random vectors that are generated independently, and b(k) is the gain.

See [90–92] for the selection of the gain b(k) and the random vector 42(k).

(iv) Adaptive simultaneous perturbation stochastic approximation

(ASPSA):

g(θ(k))=fp(H(k))−1g2(θ(k),41(k)) (3.9)

where

fp(H(k)) =

{H(k) if H(k) ∈ Sn×n,

H(k) + σI if H(k) /∈ Sn×n,(3.10)


fp : Rn×n → Sn×n be a function, σ ∈ R+ be a given large number andg2(θ(k),41(k)) is same as g(k) in (3.7). Next, H(k) is given by

H(k) =

{Ĥ(k) if k = 0,

kk+1

H(k − 1) + 1k+1

Ĥ(k) if k > 0,(3.11)

where

Ĥ(k)=1

2

[∂h(k)

2c(k)4−11 (k)+

(∂h(k)

2c(k)4−11 (k)

)T],

4−11 (k) =[4−111 (k) 4−112 (k) · · · 4−11n (k)

],

∂h(k)=h(θ(k) + c(k)41(k))− h(θ(k)− c(k)41(k)),

h(θ(k)± c(k)41(k))=

f(θ(k)±c(k)41(k)+c̃(k)42(k))−f(θ(k)±c(k)41(k))

2c(k)421(k)...

f(θ(k)±c(k)41(k)+c̃(k)42(k))−f(θ(k)±c(k)41(k))2c(k)42n(k)

,41(k) ∈ Rn and42(k) ∈ Rn are random vectors that are generated indepen-dently, 41i(k) and 42i(k) are the i-th elements of the vectors 41(k) ∈ Rn

and42(k) ∈ Rn, respectively, and c̃(k) is the gain, for example, c̃(k) := c(k).The parameters of the algorithm are chosen based on some guidance in [93].

In this thesis, the convergence conditions of the 2SPSA method are presented in

Chapter 2 while the convergence conditions of the 1SPSA, GSPSA, and ASPSA

methods are given in Appendix B.

Note that, each method uses only the evaluation (or measurement) of the objective

function (i.e., use neither the explicit form of f nor the gradient). Also, the number

of evaluations is different in each SPSA method. For example, by letting ξ be the

number for evaluating the function f in each iteration, ξ = 1 for 1SPSA, ξ = 2

for 2SPSA and GSPSA, and ξ = 4 for ASPSA. So, in order to perform a fair

comparative assessment between each method, the termination criterion M > M

is adopted in the problem of this chapter. Here, M is the counter for ξ in each

iteration and M is the chosen total number of measurements.

Remark 3.1. The idea of the termination based on the total number of measure-

ments M is reasonable, since it is closely related to the total computation time.

This allows us to fairly assess the four methods. In this study, M is sufficiently

large such that the convergence state is satisfied for all SPSA methods.


3.3.2 Model-Free Design

In this section, it is presented how to solve Problem 3.1 by using the SPSA-based

methods. Our design procedure is summarized as follows:

Step I: Let θ be (P , I,D,N ) or some of them, determine the total number of

measurements M and select the initial value θ(0).

Step II: Execute the SPSA algorithm in (2.2) by regarding the function g in (3.6)

or (3.7) or (3.8) or (3.9), and by setting f(θ) := J(P , I,D,N ).

Step III: After M > M , the output θ∗ is applied to the PID controller K(s) in

the feedback control system in Figure 3.1.

Remark 3.2. Note that, during the optimization process, there is a possibility

that the design parameters grow rapidly and suddenly are trapped in an unfeasible

region. As a result, we obtain an undesirable solution. In order to avoid this

problem, [94] has proposed a modified version of the SPSA algorithm. There, a

saturation function satδ(·) has been introduced in (2.2). That is,

θ(k + 1) = θ(k)− satδ(a(k)g(θ(k))). (3.12)

In the following part of this chapter, the improved update law in (3.12) is adopted

instead of (2.2).

3.4 Implementation and Results

In this section, the performance of the SPSA-based methods is compared using

several numerical examples. In order to evaluate the performance of the SPSA-

based methods, we perform 50 independent trials for each method in each example.

Then, after the termination criterion is satisfied, each method is evaluated based

on the following performance criteria:

(i) The statistical analysis of the objective function J(P , I,D,N ), total norm

of the error∑q

i=1 êi, and total norm of the input∑p

i=1 ûi. Specifically, the


mean, best, worst, and standard deviation values of them are observed from

50 independent trials.

(ii) The percentage of the stable closed-loop systems from 50 independent trials.

Note that the stability of the closed-loop system in this numerical test is

analyzed based on the value of Re[λmax], where λmax is the greatest real

part of the closed-loop system pole. Here, the closed-loop system is stable

if Re[λmax] < 0. Moreover, the mean, best, worst, and standard deviation

values of Re[λmax] are also observed.

(iii) The average computation time from 50 independent trials.

Note that the performance of the SA and RS-based algorithms are also evaluated

based on the same criteria.

Remark 3.3. It is important to highlight that the methods in this study are

based on the model-free approach. Namely, they use only the information of the

given input and output data to tune the PID parameters (instead of the plant

models). Thus, it is in principle impossible to show the stability of the closed-loop

system from this point of view. Nevertheless, since the plant model is given in

our numerical test, it is worth to analyze the stability of the closed-loop system

after obtaining the resulting PID parameters. So, the success rate of achieving the

closed loop stability would be a useful criterion to justify the appropriateness of

the designed PID parameters.

Remark 3.4. In this comparative study, the parameters of each SPSA-based

method, e.g., the gain sequences a(k) and c(k) are selected heuristically such that

it yields the best control performance. Therefore, these parameters may be varied

between each method.

3.4.1 Example 1 (Wind turbine)

Consider a wind energy conversion system from [95], where the plant G(s) is

given in Appendix A. The reference and the deterministic disturbance are given

as r(t) = [0 0 0]>,∀t and d(t) = 1,∀t, respectively. Here, K(s) is assumed to


Table 3.1: Design parameters of Example 1

θ PID gainθ∗

1SPSA 2SPSA GSPSA ASPSA SA RS

θ1 P11 -3.43 1.03 15.04 3.00 13.97 10.33θ2 I11 -0.45 -0.61 2.10 0.94 1.52 2.60θ3 P12 5.99 21.95 14.76 2.61 9.73 6.69θ4 I12 1.27 4.25 -7.62 2.87 1.17 1.24θ5 P21 4.81 8.10 -0.37 -0.65 1.76 3.06θ6 I21 0.66 11.37 -0.06 0.34 0.59 1.76θ7 P22 0.42 1.53 4.08 -4.02 -3.89 -1.87θ8 I22 -0.25 0.57 0.84 25.18 -6.63 2.30θ9 P31 -4.50 -1.09 -1.05 4.05 -0.80 0.28θ10 I31 1.38 0.41 1.84 -2.53 0.39 -0.43θ11 P32 0.50 0.64 0.50 4.13 0.24 0.23θ12 I32 0.35 0.09 0.06 1.35 0.06 0.10θ13 P13 -4.81 16.15 6.89 -0.53 -1.51 6.85θ14 P23 1.53 -3.73 -6.26 -0.20 -5.31 -1.99θ15 P33 -3.04 -4.18 -1.40 -1.48 -1.30 -0.11

be a PI controller in a similar way to [95] with 15 design parameters. Therefore,

we set Dij = 0, Nij = 0,∀i, j ∈ {1, 2, 3}, and Iij = ∞ for i ∈ {1, 2, 3} andj = 3. Then, the corresponding design parameters θ := (P , I) ∈ R15 are shownin Table 3.1. Our aim is to find a θ ∈ R15 which minimizes the performanceindex J in (3.5) for p = 3, q = 3, w11 = 1000, w12 = 1000, w13 = 1, w21 = 10,

w22 = 0.001, w23 = 0.004, t0 = 0, and tf = 20. The total number of measurements

is chosen as M = 500 in each trial. Each element of the initial condition θ(0)

is randomly chosen from the uniform distribution on the interval (−2, 4), whichare independent among each trial. The parameters of each SPSA-based algorithm

are summarized in Table A.1. Next, the SA-based algorithm with Boltzmanns

constant kB = 1, the initial temperature T0 = 1, the final temperature Tf = 0.01,

and the updated temperature Te(k + 1) = 0.5499Te(k) are used. However, for the

RS-based algorithm, we do not require any adjustable coefficients to be chosen.

See [29] and [27] for the detail of the SA and RS-based algorithms, respectively.

The resulting parameters θ∗s are shown in Table 3.1 while the statistical analysis

of the proposed algorithms for 50 independent trials is depicted in Table 3.2.

Note that the values of θ∗ in Table 3.1 are obtained based on the best objective

function J(P , I,D,N ) from 50 trials. It shows that the average computation

time for M = 500 in each trial is around 4 minutes. In terms of the objective


Table 3.2: Statistical result of Example 1

Algorithm 1SPSA 2SPSA GSPSA ASPSA SA RS

Objective Mean N/A1 N/A N/A N/A N/A N/Afunction Best 254.27 226.66 226.78 399.93 227.50 229.99(×10−3), Worst N/A N/A N/A N/A N/A N/AJ(P , I,D,N) Std. N/A N/A N/A N/A N/A N/A

Total norm Mean N/A N/A N/A N/A N/A N/Aof the error Best 8.99 0.89 1.19 1970 1.41 1.80(×10−5), Worst N/A N/A N/A N/A N/A N/A∑3

i=1 êi Std. N/A N/A N/A N/A N/A N/A

Total norm Mean N/A N/A N/A N/A N/A N/Aof the input Best 22.60 22.36 22.36 26.92 22.24 22.33(×10−3), Worst N/A N/A N/A N/A N/A N/A∑3

i=1 ûi Std. N/A N/A N/A N/A N/A N/A

The greatest Mean 1492.60 52.91 21.98 28270.95 106.90 64.73real part Best -0.28 -0.89 -0.75 -0.07 -0.81 -0.63of the pole, Worst 20511.55 632.75 411.38 38600.57 2522.34 513.36Re[λmax] Std. 3704.07 163.81 80.18 12922.44 396.91 137.07

Stable closed-loop26 88 92 4 58 40

system (%)Average computation

3.92 3.95 3.84 3.89 3.87 3.83time (minutes)

1 denotes non-available.

function and the total norm of the error, we see that the 2SPSA, GSPSA, SA, and

RS obtain the lower best values as compared with the 1SPSA and ASPSA-based

algorithms. This also can be seen from the output responses of the wind energy

conversion system for all the proposed methods in Figure 3.2. Note that only

the best output responses are plotted in the figure from 50 independent trials.

Here, the output responses y1, y2, and y3 are plotted in the above, middle, and

bottom sub-figures, respectively, and the responses of the 1SPSA, 2SPSA, GSPSA,

ASPSA, SA, and RS-based algorithms are represented by solid gray line, dot black

line, solid black line, thin dash black line, dot gray line, and dash-dot gray line,

respectively. It clarifies that, for all output responses, the GSPSA, 2SPSA, SA,

and RS successfully reduce the disturbance with less oscillation compared to the

ASPSA and 1SPSA. On the other hand, in terms of stability criterion, it shows

that the 2SPSA and GSPSA achieve closed-loop stability in more than 40 trials

out of 50 trials, which is relatively higher among the other algorithms. This is

also proven from the lowest mean value of Re[λmax] for both algorithms. This fact

means that the obtained controllers stabilize the plant with high probability if we

use the 2SPSA and GSPSA.


Figure 3.2: Output responses of the wind energy conversion system

3.4.2 Example 2 (Distillation column)

In order to evaluate the applicability of high-dimensional PID tuning for indus-

trial process plant, consider a binary distillation column system in [96], whose

transfer matrix G(s) is given in Appendix A. The reference and the deterministic

disturbance are given by r(t) = [0 0 0]>,∀t and

d(t) =

{1 for 0 ≤ t ≤ 10,0 for t > 10,

(3.13)

respectively. Also in this case, K(s) is assumed to be a PI controller, where

Dij = 0, Nij = 0,∀i, j ∈ {1, 2, 3}. The corresponding design parameters θ :=(P , I) ∈ R18 is tabulated in Table 3.3. Then, we solve Problem 3.1 for p = 3,q = 3, w1i = 1×106 (i = 1, 2, 3), w2i = 0.01 (i = 1, 2, 3), t0 = 0, and tf = 1000. Weset M = 600 and generate a random number drawn from the uniform distribution

on (−2, 2) for each element of θ(0), respectively, in each trial. The numericalexample is performed for each SPSA-based algorithm using the parameters in



θ PID gainθ∗

1SPSA 2SPSA GSPSA ASPSA SA RS

θ1 P11 4.38 6.47 36.31 0.22 -0.58 0.97θ2 I11 -0.71 -0.61 -3.70 -0.98 0.65 2.54θ3 P12 -1.55 6.73 -12.45 2.20 2.03 -0.17θ4 I12 0.79 0.55 -5.79 -2.47 -3.09 0.09θ5 P13 -0.96 8.38 1.68 -1.03 -0.80 -1.04θ6 I13 -0.18 0.39 0.09 -0.58 -0.28 -0.35θ7 P21 2.51 6.32 -0.21 0.42 0.14 -1.15θ8 I21 -28.83 0.31 -0.01 0.17 0.04 -0.49θ9 P22 -10.37 -19.33 -1.58 1.01 1.12 0.25θ10 I22 -10.99 5.38 -0.46 3.65 0.89 0.11θ11 P23 -2.04 11.75 -1.25 -2.89 -1.47 -1.77θ12 I23 0.42 -1.27 0.21 -1.10 2.81 2.46θ13 P31 -0.04 -4.73 -8.91 0.38 -1.17 -1.50θ14 I31 0.00 -0.40 -1.02 0.15 -0.42 -0.58θ15 P32 -6.35 12.88 6.07 0.57 0.55 -1.84θ16 I32 -24.26 1.99 1.82 0.63 0.67 -2.45θ17 P33 -3.48 -1.79 12.34 3.56 1.45 1.15θ18 I33 -0.97 -0.20 1.93 1.68 0.78 0.68

Table A.1. Here, the parameters of the SA-based algorithm is set to be similar to

Example 1.

Tables 3.3 and 3.4 show the resulting parameters and the statistical results after

50 independent trials, respectively. They show that the objective function, the

total norm of the error, and the total norm of the input are better in 2SPSA

and GSPSA-based algorithms. In particular, the GSPSA-based algorithm obtains

the lowest mean and best values of them. This fact is also supported from the

output responses y1, y2, and y3 of the binary distillation column in Figure 3.3.

In this figure, the best responses out of 50 independent trials are shown and all

the proposed algorithms are represented by the same line styles as in Figure 3.2.

Here, we can clearly see that the GSPSA and 2SPSA produce a slightly better

disturbance attenuation than the other algorithms during the given time interval.

Furthermore, the GSPSA also maintains a highest number of stable closed-loop

systems with 82 % success rate in 50 independent trials. This is followed by 2SPSA

(68 %), SA (44 %), RS (38 %), 1SPSA (26 %), and ASPSA (2 %). The average

time for 50 independent trials is about 3 minutes. Thus, we can conclude that

the GSPSA-based algorithm can find the most satisfactory solution within a few

minutes.




Objective Mean N/A 15.36 8.85 N/A N/A N/Afunction, Best 6.83 0.56 0.54 6.82 2.35 2.67J(P , I,D,N) Worst N/A 191.98 196.98 N/A N/A N/A

Std. N/A 36.47 27.18 N/A N/A N/A

Total norm Mean N/A 1.56 0.90 N/A N/A N/Aof the error Best 0.70 0.06 0.05 7.00 0.24 0.27

(×10−5),∑3

i=1 êi Worst N/A 19.52 20.09 N/A N/A N/AStd. N/A 3.70 1.24 N/A N/A N/A

Total norm Mean N/A 11.58 2.77 N/A N/A N/Aof the input, Best 0.06 0.24 0.05 0.19 0.21 0.30∑3

i=1 ûi Worst N/A 230.16 130.52 N/A N/A N/AStd. N/A 41.03 18.26 N/A N/A N/A

The greatest real Mean 9.66 0.00 -0.08 18.51 0.49 0.64part of the pole Best -0.30 -0.29 -0.44 -0.12 -0.21 -0.17(×10−2), Re[λmax] Worst 38.45 0.63 0.45 27.87 2.87 3.72

Std. 10.95 0.22 0.15 5.22 0.81 0.92



3.41 3.45 3.31 3.44 3.47 3.33time (minutes)

Figure 3.3: Output responses of the binary distillation column system


Table 3.5: Statistical result of Example 2 with time delay and stochasticdisturbance


The best objective0.31 0.03 0.03 0.18 0.25 1.25function,

J(P , I,D,N)

The best total13.39 2.77 0.48 27.98 1062.50 285.59norm of the error

(×10−8),∑3

i=1 êiThe best total

4.74 1.20 0.54 14.01 31.92 251.14norm of the input

(×10−3),∑3

i=1 ûiThe greatest real Mean 8.42 0.41 0.33 9.75 2.43 2.53part of the pole Best -0.24 -0.17 -0.24 -0.11 0.01 0.21(×10−2), Re[λmax] Worst 29.57 2.81 1.56 17.75 6.89 6.76

Std. 9.97 0.55 0.44 4.56 1.73 1.70



3.40 3.44 3.43 3.44 3.41 3.54time (minutes)

In addition, we also evaluate the performance of the SPSA-based methods for

a system with time delay and stochastic disturbance. Here, consider the binary

distillation column plant [96] with time delay inputs u1(t − 10), u2(t − 5), andu3(t − 1), and the stochastic disturbance d(t), which is a white noise sequencewith zero mean and variance σ2v = 0.3

2. Then, we solve Problem 3.1 using the

same r(t), K(s), p, q, w1i (i = 1, 2, 3), w2i (i = 1, 2, 3), t0, tf , M , and θ(0) as

in the previous deterministic disturbance case. Moreover, the parameters of the

SPSA, SA, and RS-based methods are set to be similar to the previous case.

Table 3.5 shows the statistical results after 50 independent trials in the case with

time delay and stochastic disturbance. Note that the 2SPSA and GSPSA-based

methods yield better values of the best objective function, total norm of the error,

and total norm of the input. Moreover, the number of stable closed-loop systems

for 2SPSA and GSPSA is relatively higher than the other algorithms. Thus, we

can justify a good potential of 2SPSA and GSPSA-based methods in producing a

slightly better stochastic disturbance reduction, although with a time delay input

in the plant.

Remark 3.5. In order to handle an infinite number of poles in a closed-loop

feedback system with time delay, we use 10th-order Padé approximation that


approximate time delays by rational LTI models with finite clustered poles [97].

Hence, the number of stable closed-loop systems in Table 3.5 is obtained after this

approximation is performed.

3.4.3 Example 3 (Bell 201A-1 helicopter)

Finally, we evaluate the SPSA-based methods for a system with decoupling among

output channels [98]. The plant G(s) is Bell 201A-1 helicopter system described

in Appendix A. The aim is to decouple four typical output channels yi(t) (i =

1, 2, 3, 4) such that one of the output responses follows a desired step-response while

the rest maintain its current state. For such a purpose, we design a PID controller

such that the four output responses track the four different sets of reference input,

which are given by r(t) = [r1(t) r2(t) r3(t) r4(t)]>, where

r1(t) =

{1 for 0 ≤ t ≤ 20,0 for 20 < t ≤ 80,

(3.14)

r2(t) =

0 for 0 ≤ t ≤ 20,1 for 20 < t ≤ 40,0 for 40 < t ≤ 80,

(3.15)

r3(t) =

0 for 0 ≤ t ≤ 40,1 for 40 < t ≤ 60,0 for 60 < t ≤ 80,

(3.16)

r4(t) =

{0 for 0 ≤ t ≤ 60,1 for 60 < t ≤ 80.

(3.17)

The disturbance is given by d(t) = 0,∀t. Next, K(s) is assumed to be

K(s) =

h11(s) h12(s) 0 0

0 h22(s) h23(s) 0

0 h32(s) h33(s) 0

0 0 h43(s) h44(s)

, (3.18)

and I12 = I11, D12 = D11, N12 = N11, I23 = I22, D23 = D22, N23 = N22, I32 = I33,

D32 = D33, N32 = N33, I43 = I44, D43 = D44, and N43 = N44. The detail design

parameters θ := (P , I,D,N ) ∈ R20 are listed in Table 3.6 for each corresponding



θPID

ai biθ∗

Gain 1SPSA 2SPSA GSPSA ASPSA SA RS

θ1 P11 -5 -1 -6.24 -4.20 -2.31 -8.94 -2.06 -1.99θ2 I11 0 1 1.37 1.16 1.88 -4.28 0.24 0.21θ3 D11 0 1 2.97 1.20 1.99 -4.67 0.51 0.49θ4 N11 1 30 24.06 7.11 16.15 1.26 21.30 21.26θ5 P22 -100 -10 -84.97 -19.87 -37.07 -23.66 -19.82 -19.79θ6 I22 0 1 2.68 2.74 4.44 -4.32 1.12 1.03θ7 D22 0 1 0.42 1.05 0.62 -4.66 0.96 0.88θ8 N22 1 30 17.95 4.57 3.18 -1.08 5.02 4.98θ9 P33 10 100 73.05 32.38 16.08 28.13 23.42 23.42θ10 I33 0 1 0.96 3.57 3.78 -4.89 0.40 0.38θ11 D33 0 1 0.27 0.46 0.56 -4.59 0.89 0.85θ12 N33 1 30 24.11 6.86 20.16 3.41 8.39 8.39θ13 P44 -5 -1 -2.18 -2.68 -1.66 -7.89 -1.69 -1.78θ14 I44 0 1 0.75 2.44 1.80 -4.39 0.23 0.22θ15 D44 0 1 0.51 1.91 2.30 -5.08 0.91 0.96θ16 N44 1 30 14.86 4.90 29.38 -0.89 11.16 11.17θ17 P12 0 10 8.19 1.71 6.13 -2.33 2.05 1.99θ18 P23 0 30 12.01 9.11 10.54 4.46 7.53 7.56θ19 P32 0 30 18.76 13.90 3.48 7.63 18.50 18.46θ20 P43 0 10 2.84 5.72 6.18 -0.12 4.79 4.73

PID gain. Then, we find θ∗ such that the objective function J in (3.5) is minimized

for p = 4, q = 4, w11 = 10, w12 = 10000, w13 = 10000, w14 = 10, w2i =

1 (i = 1, 2, 3, 4), t0 = 0, and tf = 80. Similarly to the previous examples, 50

independent trials are performed for each SPSA-based algorithm based on the

parameters in Table A.1. In each trial, we set M = 500 and the initial condition

θ(0) be random vector whose elements is drawn from the uniform distribution

on (ai, bi) (i = 1, 2, ..., 20), where ai and bi are given in Table 3.6. Furthermore,

we adopt the same parameters of the SA-based algorithm as in the previous two

examples.

The resulting parameters θ∗s and the statistical analysis of the proposed algo-

rithms for 50 independent trials are shown in Tables 3.6 and 3.7, respectively.

Note that only the GSPSA-based algorithm yields a better objective function,

total norm of the error, and the total norm of the input for all 50 independent

trials. On the other hand, the ASPSA-based algorithm results in an undesirable

solution in all of the trials, since the design parameters are always trapped in the




Objective Mean N/A N/A 220.60 N/A N/A N/Afunction Best 184.41 104.36 105.38 N/A 113.01 115.00(×103), Worst N/A N/A 1084.76 N/A N/A N/AJ(P, I,D,N) Std. N/A N/A 171.53 N/A N/A N/ATotal norm Mean N/A N/A 852.20 N/A N/A N/Aof the error, Best 6.28 7.99 7.53 N/A 5.75 5.37∑4

i=1 êi Worst N/A N/A 33057.85 N/A N/A N/AStd. N/A N/A 4652.32 N/A N/A N/A

Total norm Mean N/A N/A 209.49 N/A N/A N/Aof the input Best 194.20 88.65 87.88 N/A 93.64 93.11(×103), Worst N/A N/A 945.75 N/A N/A N/A∑4

i=1 ûi Std. N/A N/A 146.64 N/A N/A N/AThe greatest Mean 31.31 2.51 -0.01 6.36 45.04 0.09real part Best -0.01 -0.01 -0.01 0.21 -0.01 -0.01of the pole, Worst 691.81 50.92 0.07 183.02 2250.45 1.44Re[λmax] Std. 128.40 9.18 0.01 25.63 318.26 0.27Stable closed-loop

74 88 96 0 82 74system (%)Average computation

5.81 5.53 5.70 5.70 5.50 5.41time (minutes)

unfeasible region when approximating the Hessian matrix. Figure 3.4 shows the

best output responses y1, y2, y3, and y4 of the Bell 201 A-1 helicopter system for

all the proposed algorithms. In general, all the proposed algorithms follow the

desired input trajectory, except for the ASPSA-based algorithm. Specifically, the

1SPSA, 2SPSA, GSPSA, SA, and RS successfully tracks the given step response

and decouple the others output channels effect, especially in both y2 and y3 out-

put responses. Next, in terms of the stability analysis, the GSPSA still maintains

the highest success rate (96 %) with the lowest mean value of Re[λmax]. This is

followed by 2SPSA (88 %), SA (82 %), RS (74 %), 1SPSA (74 %), and ASPSA

(0 %). In this example, the average time for 50 independent trials is about 6

minutes. Hence, we can confirm the superiority of the GSPSA-based algorithm

in tracking performance and producing stable closed-loop system with reasonable

calculation time.


Figure 3.4: Output responses of the Bell 201A-1 helicopter system

3.5 Summary

In this chapter, the performance comparison of several SPSA-based methods for

PID tuning of MIMO systems has been conducted. The statistical analysis of

the proposed algorithms through extensive numerical examples shows that the

SPSA-based methods have a good potential in finding satisfactory results within a

few minutes. In particular, all numerical examples indicate that the GSPSA-based

algorithm outperforms the other SPSA and stochastic optimization-based methods

from the viewpoint of the control performance and the closed-loop stability.

Chapter 4

Switching Controller Design for

Hybrid Electric Vehicles

4.1 Introduction

Nowadays, hybrid electric vehicles (HEV’s) have been attracting attention as fuel

efficient vehicles, compared to conventional internal combustion engine vehicles.

As a result, some joint researches from automotive and control communities are

now focusing on the fuel consumption optimization problem. For instance, a tech-

nical committee from JSAE (Society of Automotive Engineers of Japan), SICE

(The Society of Instrument and Control Engineers) and Japanese automobile in-

dustry have developed a benchmark problem for fuel consumption optimization

using the hybrid powertrain simulator. The detail of their work has been reported

in [66].

So far, many techniques have been proposed for HEVs controller design. In [99],

the engine torque and gear number are considered as control inputs and the op-

timal control has been studied for a simplified model of HEVs. An equivalent

consumption minimization strategy (ECMS) has been proposed by [100] where an

equivalence factor between fuel energy and electrical energy has been derived. In

[101], a hybrid powertrain model has been developed to operate in multiple driv-

ing modes. They have used a dynamic programming algorithm to find an optimal

sequence of driving modes such that the overall fuel consumption is minimized.

Besides that, various fuel optimization problems based on model-free tuning meth-

ods also have been proposed in [94, 102–106] where directed rectangles, simulated

39

Chapter 4. Switching Controller Design for Hybrid Electric Vehicles 40

Table 4.1: Fuel efficiency comparison for benchmark problem

Result Approach Fuel efficiency

Optimize non-switchingBaba et. al [94] 25.41 km/L

controller using SPSAOptimize non-switching

Ahmad et. al [107] 22.55 km/Lcontroller using SPSAOptimize non-switching

Hagura et. al [105] 24.82 km/Lcontroller using SAOOptimize switching

Yamazaki et. al [108] 25.43 km/Lcontroller using PSOOptimize switching

This study 25.85 km/Lcontroller using SPSA

annealing, and genetic algorithm [102], multi-objective Genetic Algorithms [103],

composite particle swarm, genetic algorithm, and downhill-simplex [104], simul-

taneous perturbation stochastic approximation method [94], [107], and sequential

approximate optimization (SAO) [105] have been applied.

In this chapter, we address switching controller design for hybrid electric commuter

vehicles of the benchmark problem [66]. A model free tuning method [94] based on

simultaneous perturbation stochastic approximation (SPSA) is utilized to optimize

the design parameters for every switching mode simultaneously so that the fuel

consumption is minimized and the driver’s satisfaction is kept higher than 90 %.

Consequently, we obtain the fuel efficiency 25.85 km/L, which is the best result in

the benchmark problem at present, as shown in Table 4.1.

Note that this chapter is based on [106] and is a fully generalized version of it.

On the other hand, a result of switching controller design has been independently

proposed in [108] at almost the same time. There, a model-free approach based on

the particle swarm optimization has been developed. However, it is known that,

in general, the particle swarm optimization does

title model-free controller design based on simultaneous ......title model-free controller design...

Documents