ucy systems.pdf · inflnitesimal perturbation analysis and optimization for make-to-stock...
TRANSCRIPT
Infinitesimal Perturbation Analysis and Optimization for
Make-To-Stock Manufacturing Systems Based on Stochastic Fluid
Models
Christos PanayiotouDept. of Electrical and Computer Engineering,
University of Cyprus,Cyprus.
Christos G. Cassandras∗
Dept. of Manufacturing Engineering,Boston University,
Brookline, MA [email protected]
January 20, 2005
Abstract
In this paper we study Make-To-Stock manufacturing systems and seek on-line algorithms fordetermining optimal or near optimal buffer capacities (hedging points) that balance inventoryagainst stockout costs. Using a Stochastic Fluid Model (SFM), we derive sample derivatives(sensitivities) which, under very weak structural assumptions on the defining demand and ser-vice processes, are shown to be unbiased estimators of the sensitivities of a cost function withrespect to these capacities. When applied to discrete-part systems, we show that these estima-tors are greatly simplified and become nonparametric. Thus, they can be easily implementedand evaluated on line. Though the implementation on discrete-part systems does not necessar-ily preserve the unbiasedness property, simulation results show that stochastic approximationalgorithms that use such estimates do converge to optimal or near optimal hedging points.
1 Introduction
Make-To-Stock (MTS) is a common mode of operation for many industries such as retail products,appliances and cars. Typically, in MTS manufacturing systems, a positive Finished Goods Inventoryis maintained which is used to fulfill arriving orders. The fundamental problem in such systemsis to determine how much inventory to maintain so that the carrying inventory cost is balanced
∗Supported in part by the National Science Foundation under Grants EEC-0088073 and DMI-0330171, by AFOSRunder contract F49620-01-0056, and by ARO under grant DAAD19-01-0610.
1
against the cost of unfulfilled demand due to stockout. MTS systems have been studied extensivelyin the literature; see [1, 2] and references therein.
In this paper we investigate on-line optimization algorithms for determining the optimal or nearoptimal buffer capacities (hedging points) that balance inventory against stockout costs for MTSproblems (see [3], [4], [5] and references therein). Specifically, we investigate a MTS system withtwo stages, one for finished goods and one for work-in-process. First we adopt a Stochastic FluidModel (SFM) and derive cost sensitivities with respect to the buffer capacities and show that suchsensitivities are unbiased. We derive the sensitivity information through Perturbation Analysis (PA)methods [6] by appropriately adapting them to a SFM viewed as a hybrid system. Subsequently, weimplement these estimators on the sample path that is observed from a discrete-part system. Thisis possible since the estimators consist of a number of timers that measure the time between theoccurrence of certain events and possibly some instantaneous rates at specific points in time whichare either known or can be measured. Finally, these sensitivity estimators are used in stochasticapproximation algorithms to determine the optimal hedging points.
As mentioned above, to derive the sensitivity information we adopt a SFM, a framework commonlyused to deal with the analysis and control of manufacturing systems, as in [7, 8]. We emphasizethat our goal is to make use of such a model for the purpose of control and optimization ratherthan performance analysis. Though a SFM does not necessarily estimate the performance of theunderlying Discrete Event System (DES) with great accuracy, one can often identify the solution ofan optimization problem based on such a model that captures only those features of the underlying“real” system that are needed to lead to the right solution. Such observations have been made inseveral contexts (e.g., [9],[10] and references therein). Furthermore, Ordinal Optimization [11] alsosuggests that such surrogate models may perform well in identifying the set of best designs.
This paper unifies the IPA derivation approaches presented in [3] and [4] and provides the proofsnot given in these earlier papers. In [3] IPA sensitivities were obtained for a SFM with piece-wise constant input processes using finite differences. We assumed that the buffer capacity wasincreased by ∆θ and evaluated the resulting change in cost functions using geometric arguments.In [4] IPA sensitivities were obtained for the same SFM but assuming almost arbitrary (piecewisedifferentiable) input processes. In this paper we derive the general case IPA estimates and showthat if they are evaluated based on observations from a discrete-part sample path, then they are thesame as the IPA estimates of [3]. Further contributions of the paper include the derivation of IPAestimates for a tandem network with manufacturing blocking, unlike [12] where tandem networkswith communication blocking are investigated.
An important result of our analysis is that the derived IPA estimators turn out to be nonparametricwhen the evaluation is done based on a discrete-part sample path, despite the tight coupling betweenthe two machines in our model. In other words, not only are the estimators independent of thedistributions of the underlying random processes, but they also require no knowledge of any of theassociated parameters (such as demand and machine processing rates). In addition, the estimatesobtained are shown to be unbiased under very weak structural assumptions on the defining demandand service processes. However, we point out that the unbiasedness property holds only for theSFM cost functions. Once we apply the estimators using information from discrete-part samplepaths, there is no guarantee that the property still holds; however, our simulation results show thatsuch estimates are still usable for on-line control purposes to perform periodic system managementfunctions in order to trade off inventory and stockout costs.
The remaining paper is organized as follows. Next, in Section 2 we describe the model of the
2
problem and define some notation. In Section 3 we derive the Infinitesimal Perturbation Analysis(IPA) algorithm based on the Stochastic Fluid Model (SFM). In Section 4 we show how the IPAalgorithm can be implemented in the case of a discrete part manufacturing system. In Section 5we show some simulation results and we conclude with Section 6.
2 Model
In this paper we use the two machine model of [3] shown in Fig. 1. The upstream machine (M2)can process fluid at a maximum rate µ2(t) and is assumed to have an infinite supply. The outputof M2 is placed in the work-in-process buffer W which has capacity θW . Machine M1 drains W ata maximum rate µ1(t) and fills the finished part inventory buffer F . Finally, F is drained by thedemand at a rate ρ(t). In this paper, random functions µ1(t), µ2(t) and ρ(t) are assumed piecewisedifferentiable functions w.p. 1. This is in contrast to [3] where they were assumed piecewise constantprocesses.
µ2(t) W µ1(t)
Demand
ρ(t)
Finished part inventory
Work in process inventory
θF θW
F
M2 M1 )(tx)(ty
Figure 1: System Model
Let (x(t), y(t)) denote the state of the system, where x(t) ≤ θF , 0 ≤ y(t) ≤ θW correspond to theamount of fluid present in F and W respectively. It is important to note that x(t) can take negativevalues and these correspond to the amount of backlogged demand. Clearly, both x(t) and y(t) arefunctions of the thresholds θ = [θF , θW ], however, to ease the notation, we omit θ. The dynamicsof the system are given by
dx(t)dt
= γ1(t; θ)− ρ(t) ={
γ1(t;θ)− ρ(t) if x(t) < θF ,0 if x(t) = θF
(1)
dy(t)dt
= γ2(t; θ)− γ1(t; θ) ={
γ2(t;θ)− γ1(t;θ) if 0 < y(t) < θW ,0 if y(t) = 0 or y(t) = θW
(2)
where γ1(t;θ) is the inflow rate to buffer F (and the outflow of buffer W ) and γ2(t; θ) is the inflowrate to buffer W . At this point it is worthwhile to point out that, unlike communication systemmodels [12], at the boundary states, the inflow to each buffer equals to the outflow. The ratesγ1(t;θ) and γ2(t; θ) capture the coupling between the dynamics of the two buffers and are given by
γ1(t;θ) =
ρ(t) if x(t) = θF
µ1(t) if x(t) < θF and y(t) > 0,µ2(t) if x(t) < θF and y(t) = 0
(3)
γ2(t;θ) =
µ2(t) if y(t) < θW
µ1(t) if y(t) = θW and x(t) < θF
ρ(t) if y(t) = θW and x(t) = θF
(4)
3
x
θF
y
θW
1xlb − 1
xle − 1
xlb +
xlex
lb xif
xiw
1yjb − 1
yje −
yjb y
mfymw
Figure 2: Typical sample path
Fig. 2 shows a typical sample path for this system where we identify the following events. Ingeneral, the time when these events occur depends on θ, however, to ease the notation, the explicitdependence is omitted.
βqi : The trajectory of q ∈ {x, y} ceases to be empty for the ith time. This event occurs at time
instants bqi .
εqi : The trajectory of q ∈ {x, y} becomes empty for the ith time. It occurs at time instant eq
i .
φqi : The trajectory of q ∈ {x, y} becomes full for the ith time. It occurs at time instant f q
i .
ωqi : The trajectory of q ∈ {x, y} ceases being full for the ith time. It occurs at time instances wq
i .
2.1 Objective Function and Optimization Approach
In the context of manufacturing systems of the type we study in this paper, one is typically inter-ested in minimizing an expected cost function
JT (θ) = E [LT (θ)] (5)
where, θ = [θF , θW ] and LT (θ) is a sample function which in general has the form
LT (θ) =1T
∫ T
0
(cF [x(t)]+ + cW y(t) + cD[−x(t)]+
)dt
The first two terms correspond to the average inventory cost in F and W respectively and the thirdone corresponds to the average cost of backlogged demand. Furthermore, T is the length of the
4
observation interval, [x]+ = max{0, x}, and cF , cW , cD are non-negative constants. The objectiveof this paper is to obtain the hedging points θ∗ = [θ∗F , θ∗W ] that minimize the objective JT (θ).Alternatively, in several instances (e.g., kanban-based manufacturing systems) it may be desirableto keep the total inventory constant, i.e., θF + θW = θ. Thus one may formulate a non-linearoptimization problem of the form
minθF ,θW
JT (θF , θW ) s.t. θF + θW = θ (6)
Though SFMs are significantly simpler than detailed discrete event systems, it is still difficult toobtain analytical solutions, since expressions for JT (θF , θW ) are unavailable. Therefore, one needsto resort to iterative methods such as stochastic approximation algorithms (e.g., [13]) which aredriven by estimates of the gradient of a cost function with respect to the parameters of interest.In the case of the cost minimization problem above, we are interested in estimating dJT /dθ basedon directly observed (or simulated) data. We can then seek to obtain θ∗ = [θ∗F , θ∗W ] such that itminimizes JT (θ) through an iterative scheme of the form
θn+1 = θn − ηnHn(θn, ωSFMn ), n = 0, 1, . . . (7)
where Hn(θn, ωSFMn ) is an estimate of dJT /dθ evaluated at θ = θn = [θF , θW ]n and based on
information obtained from a sample path of the SFM denoted by ωSFMn . Furthermore, {ηn} is an
appropriate sequence of step sizes. As we will see in Section 4, the simple form of Hn(θn, ωSFMn )
that we derive also enables us to apply it to ωDESn , a sample path of the real discrete event system:
Kn+1 = Kn − ηnHn(Kn, ωDESn ), n = 0, 1, . . . (8)
where Kn = [KF , KW ]n is the threshold of the discrete-part system used for the nth iteration. Inother words, analyzing the SFM provides us with the structure of a gradient estimator whose actualvalue can be obtained based on data from the actual system.
3 Infinitesimal Perturbation Analysis (IPA)
In this section we derive sample derivatives for each of the components that make up the samplefunction LT (θ), θ = [θF , θW ]. So, we define
XT (θ)=∫ T
0x(t)dt, YT (θ) =
∫ T
0y(t)dt (9)
X+T (θ)=
∫ T
0[x(t)]+ dt, X−
T (θ) =∫ T
0[−x(t)]+dt. (10)
In our earlier approach [3], we derived such IPA estimates under the assumption of piecewiseconstant input processes (µ1(t), µ2(t), and ρ(t)). In that work, we assumed that θq is increased by∆θq > 0, q ∈ {F, W} and determined the change in XT and YT , ∆XT and ∆YT respectively. Inthis paper, we generalize the results for piecewise differentiable input processes. The derivation ofthe IPA derivative estimates is by differentiating the sample functions XT (·), and YT (·) directly.Before we proceed with the derivation of the sample derivatives we make the following assumptionsand derive some preliminary results.
Assumptions:
5
A1: W.p. 1, the functions µ1(t), µ2(t) and ρ(t) are piecewise analytic in the interval [0, T ].
A2: W.p. 1, except for (εy, ωx), (ωx, ωy) and (φx, βy) no two events can occur at the same time.
A3: W.p. 1, there does not exist an interval (s, s + τ), τ > 0, such that Mn,m(t) = 0, where
Mn,m(t) = µn(t)− µm(t) (11)
n 6= m, n,m ∈ {0, 1, 2} and, by definition, µ0(t) = ρ(t).
In A2, the exceptions are needed since in each pair, it is possible that the occurrence of the firstevent might also trigger the second event. For example, in Fig. 2, the event φx
i (buffer F becomesfull) at fx
i causes buffer W to cease to be empty at byj = fx
i . All three assumptions are mild andare satisfied for general systems. A3 is rather technical. If it is violated, then one can still resortto one sided derivatives and get similar results, but we keep it to simplify our analysis. The aboveassumption guarantees the existence of the sample derivatives dLT
dθFand dLT
dθW.
3.1 More Notation and Preliminary Results
In this section we are concerned with the evaluation of event time derivatives (with respect to eitherθF of θW ) for some critical events. We derive a number of linear iterative relations for evaluating therequired event time derivatives. Before we proceed with the specific results, however, we introducethe following sample path decomposition.
3.1.1 Sample Path Decomposition
In this section we decompose the sample paths of (x(t), y(t)) into Boundary Periods (BP) and Non-Boundary Periods (NBP). BPs, are intervals where the buffer content is at some boundary; full orempty. Specifically, x(t) = θF defines a Full Period (FP) while y(t) = 0 defines an Empty Period(EP) and y(t) = θW a FP. A NBP followed by a BP form a cycle. We also define the followingvariables. Let vi,0 denote the beginning point of the ith cycle in the sample path of x(t), therefore,it also denotes the beginning of the ith NBP of x(t). This point coincides with the occurrence ofthe ωx
i−1 event (buffer F ceases to be full). vi,1 denotes the end of the ith NBP (and the beginningof the ith BP) while vi,2 denotes the end point of the ith BP. vi,1 and vi,2 coincide with the φx
i andωx
i events respectively. Similarly, for the y(t) sample path, let si,0 denote the beginning of the ithNBP, si,1 the end of the ith NBP (same as the start of the ith BP) and si,2 the end of the ith BP.Points si,0 and si,2 occur due to either βy or ωy events (W ceases to be empty or full respectively).Point si,1 occurs due to either φy or εy events (W becomes full or empty respectively). Furthermore,we use the notation
Cxi = [vi,0, vi,2), i = 1, · · · , Nx
T , Cyj = [sj,0, sj,2), j = 1, · · · , Ny
T ,
to denote the cycles in the sample paths of x(t) and y(t) respectively; NxT , Ny
T denote the randomnumber of cycles in the corresponding sample path during the observation interval [0, T ]. Forexample, in Fig. 2, the intervals [by
j−1, byj ) and [by
j , wym) correspond to two consecutive cycles in y(t),
say Cyk and Cy
k+1. Thus byj−1 = sk,0, ey
j−1 = sk,1, and byj = sk,2 = sk+1,0.
6
3.1.2 Event Time Derivatives
Lemma 1 Let wxi denote the time when the ωx
i event (F ceases to be full) has occurred. Then,
∂wxi
∂θF=
∂eyj
∂θF1[wx
i = eyj
]=
{∂ey
j
∂θFif wx
i = eyj
0 otherwiseand
∂wxi
∂θW=
∂eyj
∂θW1[wx
i = eyj
]=
{∂ey
j
∂θWif wx
i = eyj
0 otherwise,
for all i = 1, 2, · · · and some j = 1, 2, · · · 1.
All proofs are included in the Appendix. Lemma 1 states that the time a FP ends at buffer F isindependent of either θF or θW unless the event is actually caused by an εy event in which case, itinherits the derivative of that event time.
Lemma 2 Let wyi denote the time when the ωy
i event (W ceases to be full) has occurred. Then2,
∂wyi
∂θF=
∂wyi
∂θW= 0, for all i = 1, 2, · · · .
Lemma 2 states that the time a FP ends at buffer W is independent of either θF or θW .
Lemma 3 Let byi denote the time when the ith βy event (W ceases to be empty) has occurred.
Then,
∂byi
∂θF=
∂fxj
∂θF1[byi = fx
j
]=
{∂fx
j
∂θFif by
i = fxj
0 otherwiseand
∂byi
∂θW=
∂fxj
∂θW1[byi = fx
j
]=
{∂fx
j
∂θWif by
i = fxj
0 otherwise,
for all i = 1, 2, · · · and some j = 1, 2, · · · 3. Also, fyj = vj,1, j = 1, 2, · · · .
Lemma 3 states that the time an EP ends at buffer W is independent of either θF or θW unlessthe event is actually caused by a φx event, in which case, it inherits the derivative of that eventtime. Lemmas 1-3 give the event time derivatives for the cycle begin and end times at the twobuffers with respect to θF and θW . Note that cycle Cx
i = [vi,0, vi,2) = [wxi−1, w
xi ), thus the derivatives
∂vi,0
∂θF= ∂vi−1,2
∂θFand ∂vi,0
∂θW= ∂vi−1,2
∂θWare evaluated using Lemma 1. Also, cycle Cy
i starts and ends either
with a βy or an ωy event, thus the derivatives ∂si,0
∂θF= ∂si−1,2
∂θFand ∂si,0
∂θW= ∂si−1,2
∂θW, are evaluated using
Lemmas 2 and 3. For notational economy, the results of Lemmas 2 and 3 are combined togetherin the following corollary.
1Using the decomposition of Sec. 3.1.1, wxi = vi,2 = vi+1,0 for all i = 1, 2, · · · and ey
j = sn,1, n ≥ j = 1, 2, · · · (nalso includes all NBPs that end with a φy event).
2wyi = sn,2 = sn+1,0 for all n ≥ i (n includes all BPs that end with a βy event).
3bi = sn,2 = sn+1,0 where again n ≥ i since it may include NBPs that start with an event ωy.
7
Corollary 1 Let si,0 denote the time when the ith cycle Cyi starts. Then,
∂si,0
∂θF=
∂fxj
∂θF1[si,0 = by
l = fxj
]=
{∂fx
j
∂θFif si,0 = by
l and byl = fx
j
0 otherwiseand
∂si,0
∂θW=
∂fxj
∂θW1[si,0 = by
l = fxj
]=
{∂fx
j
∂θWif si,0 = by
l and byl = fx
j
0 otherwise
for all i = 1, 2, · · · , and some j, l = 1, 2, · · · .
Next we derive four lemmas that show how perturbations in event times propagate from the begin-ning of a NBP (beginning of the cycle) to the end of the same NBP. The first two assume that noother events occur during the NBP while the remaining two assume an arbitrary number of events.First, for notational economy we define the following indicator functions:
1F[t] = 1[x(t) = θF ] , 1W[t] = 1[y(t) = θW ] . (12)
Lemma 4 Consider an interval [si,0, si,1) ∈ [vj,0, vj,1) or [si,0, si,1) ∈ [vj,1, vj,2), for some i, j =1, 2, · · · . Then
∂si,1
∂θF=
M2,g(si,0)M2,g(si,1)
∂si,0
∂θF
∂si,1
∂θW=
1W[si,1]− 1W[si,0]M2,g(si,1)
+M2,g(si,0)M2,g(si,1)
∂si,0
∂θW
where Mn,m(t) is defined in (11), g = 1[x(t) < θF ], t ∈ [si,0, si,1).
We point out that ∂si,0
∂θF, and ∂si,0
∂θWare given by Corollary 1. They are both 0 unless si,0 corresponds
to a βy event that coincides with a φx event.
Lemma 5 Consider an interval [vi,0, vi,1) ∈ [sj,0, sj,1) or [vi,0, vi,1) ∈ [sj,1, sj,2), for some i, j =1, 2, · · · . Then
∂v1,1
∂θF=
1Mg,0(vi,1)
∂vi,1
∂θF=
Mg,0(vi,0)Mg,0(vi,1)
∂vi,0
∂θF, i = 2, 3, · · ·
∂vi,1
∂θW=
Mg,0(vi,0)Mg,0(vi,1)
∂vi,0
∂θW
where Mn,m(t) is defined in (11), g = 1 + 1[y(t) = 0], t ∈ [vi,0, vi,1).
Note that ∂vi,0
∂θF, and ∂vi,0
∂θWare given by Lemma 1. They are both 0 unless vi,0 corresponds to an ωx
event that coincides with an εy. Lemmas 4 and 5 provide an iterative algorithm for determining theevent time derivatives (with respect to either θF or θW ) of the end point of a NBP given the eventtime derivatives of the starting point of a NBP at some buffer, when no other event occurs at theother buffer during the interval. Finally, we need to determine how a perturbation propagates from
8
the beginning of a NBP to the end of a NBP when several other events occur during the interval.However, we first need to define the following notation. We concentrate in an interval [vi,0, vi,1)and identify a sequence of points ξi,j , j = 0, · · · ,Ky
i + 1 where ξi,0 = vi,0, ξi,Kyi +1 = vi,1 and at any
other point ξi,j , y(ξi,j) transitions either from empty to non-empty or vice versa. In other words,ξi,j is a sequence with all βy or εy events in the interval [vi,0, vi,1). Next, we also define the indexfunction that takes values in {1, 2}
gi(k) ={
1 if gi(k − 1) = 22 if gi(k − 1) = 1
(13)
with gi(0) ={
1 if y(ξi,0) > 02 if y(ξi,0) = 0
Lemma 6 Consider the cycle Cxi , i = 1, 2, · · · . Then, the derivatives of vi,1, with respect to θF and
θW are given by
∂vi,1
∂θF=
(µgi(0)(vi,0)− ρ(vi,0)
)v′i,0 −
(µgi(0)(ξi,1)− µgi(1)(ξi,1)
)ξ′i,1
µgi(Kyi )(vi,1)− ρ(vi,1)
, i = 2, 3, · · ·
∂vi,1
∂θW=
(µgi(K
yi )(vi,1)− ρ(vi,1)
)−1×
(
µgi(0)(vi,0)− ρ(vi,0))v′i,0 −
Kyi∑
j=1
[(µgi(j−1)(ξi,j)− µgi(j)(ξi,j)
)ξ′i,j
]
where the prime notation indicates the event time derivatives with respect to either θF (first equa-tion) or θW (second equation).
Next, we analyze the interval [si,0, si,1) i = 1, 2, · · · , and we identify a sequence of points ζi,j ,j = 0, · · · ,Kx
i + 1 where ζi,0 = si,0, ζKxi +1 = si,1 and at any other point x(ζi,j) transitions either
from full to non-full or vice versa. In other words, ζi,j is a sequence with all φx or ωx events in theinterval [si,0, si,1). Next, we also define the index function that takes values in {0, 1}
hi(k) ={
0 if hi(k − 1) = 11 if hi(k − 1) = 0
(14)
with hi(0) ={
0 if x(ζi,0) = θF
1 if x(ζi,0) < θF
Lemma 7 The derivatives of si,1, i = 1, 2, · · · with respect to θF are given by
s′i,1 =
(µ2(si,0)− µhi(0)(si,0)
)s′i,0 +
(µhi(0)(ζi,1)− µhi(1)(ζi,1)
)ζ ′i,1
µ2(si,1)− µhi(Kxi )(si,1)
The derivatives of si,1, i = 1, 2, · · · with respect to θW are given by
s′i,1 =1W[si,1]− 1W[si,0] +
(µ2(si,0)− µhi(0)(si,0)
)s′i,0 +
(µhi(0)(ζi,1)− µhi(1)(ζi,1)
)ζ ′i,1
µ2(si,1)− µhi(Kxi )(si,1)
9
3.2 Sample Function Derivatives
During any cycle Cxi , i = 1, 2, · · · , x(t) is given by
x(t) =
{θF1F[vi,0] +
∫ tvi,0
(γ1(τ ; θ)− ρ(τ)) dτ. if t ∈ [vi,0, vi,1)θF if t ∈ [vi,1, vi,2)
(15)
where the θF1F[vi,0] term can take the value 0 only during the first cycle. For the first part ofCx
i , i.e., during the interval [vi,0, vi,1), we identify the sequence of points ξi,j , j = 0, · · · ,Kyi + 1
where ξi,0 = vi,0, ξi,Kyi +1 = vi,1 and at any other point ξi,j , y(ξi,j) switches either from empty to
non-empty or vice versa. Using this segmentation, we can rewrite x(t) as
x(t) = θF1F[vi,0] +Ky
i (t)∑
j=0
∫ ξi,j+1
ξi,j
Mgi(j),0(τ)dτ. (16)
where Kyi (t) is the number of εy and βy events in [vi,0, t) and ξKy
i (t)+1 = t (gi(j) is given by (13) andMi,j(·) by (11)). Now we are ready to differentiate the state trajectory (16) to obtain the followingresult.
Lemma 8 During any cycle Cxi , i = 1, 2, · · · , for any interval [vi,0, vi,1)
∂x(t)∂θF
= 1F[vi,0]−[µgi(0)(vi,0)− ρ(vi,0)
]v′i,0 −
[µgi(1)(ξi,1)− µgi(0)(ξi,1)
]ξ′i,11[t ≥ ξi,1] (17)
∂x(t)∂θW
=(ρ(ξi,0)− µgi(0)(ξi,0)
)v′i,0 +
Kyi (t)∑
j=1
(µgi(j−1)(ξi,j)− µgi(j)(ξi,j)
)ξ′i,j1[t ≥ ξi,j ] (18)
For any interval [vi,1, vi,2)∂x(t)∂θF
= 1∂x(t)∂θW
= 0
for all i = 1, 2, · · · . Again,{ξi,j : j = 0, · · · ,Kyi + 1} is the sequence of points with ξi,0 = vi,0,
ξi,Kyi (t)+1 = t and at any other point ξi,j, y(ξi,j) transitions either from empty to non-empty or vice
versa.
Note that the derivative of x(t) with respect to θF is a piecewise constant function that changesvalue at points t = ξi,1. The derivative of x(t) with respect to θW is again a piecewise constantfunction that changes value at points t = ξi,j . All event time derivatives with respect to θF or θW
are determined by Lemmas 1, 3, 4 and 7.
Next, for the W buffer, during any cycle Cyi , i = 1, 2, · · · , y(t) is given by
y(t) =
{θW1W[s0] +
∫ tsi,0
(µ2(τ)− γ1(τ ; θ)) dτ. if t ∈ [si,0, si,1)θW1W[s0] if t ∈ [si,1, si,2)
=
{θW1W[s0] +
∑Kxi (t)
j=0
∫ ζi,j+1
ζi,jM2,hi(j)(τ)dτ if t ∈ [si,0, si,1)
θW1W[s0] if t ∈ [si,1, si,2)(19)
where, as in previous cases, in the interval [si,0, si,1) i = 1, 2, · · · we identify a sequence of pointsζi,j , j = 0, · · · ,Kx
i (t) + 1 where ζi,0 = si,0, ζKxi (t)+1 = t and at any other point x(ζi,j) transitions
10
either from full to non-full or vice versa. Also, Kxi (t) is the number of φx and ωx events in [si,0, t).
By convention we set ρ(t) = µ0(t) and hi(j) is given by (14). Differentiating we obtain the followingresults.
Lemma 9 During any cycle Cyi , i = 1, 2, · · · , the derivative of y(t) with respect to θF is given by
dy(t)dθF
=
(µhi(0)(si,0)− µ2(si,0)
)s′i,0
+(µhi(1)(ζi,1)− µhi(0)(ζi,1)
)ζ ′i,11[t ≥ ζi,1] if t ∈ [si,0, si,1)
0 if t ∈ [si,1, si,2)(20)
The derivative of y(t) with respect to θW is given by
dy(t)dθW
=
1W[si,0] +(µhi(0)(si,0)− µ2(si,0)
)s′i,0
+(µhi(1)(ζi,1)− µhi(0)(ζi,1)
)ζ ′i,11[t ≥ ζi,1] if t ∈ [si,0, si,1)
1W[si,1] if t ∈ [si,1, si,2)(21)
The derivatives of y(t) with respect to θF and θW are both piecewise constant functions. The eventtime derivatives of ζi,1 with respect to θF and θW are given by Lemmas 1, 5 and 6.
Now we are ready to derive the derivatives of the sample functions of interest with respect to eitherθF or θW . First, using cycles, observe that XT (θ) and YT (θ) can be written as
XT (θ) =Nx
T∑
i=1
(∫ vi,1
vi,0
x(t)dt +∫ vi,2
vi,1
x(t)dt
)
=Nx
T∑
i=1
(∫ vi,1
vi,0
x(t)dt + θF (vi,2 − vi,1)
)(22)
YT (θ) =Ny
T∑
i=1
(∫ si,1
si,0
y(t)dt +∫ si,2
si,1
y(t)dt
)
=Ny
T∑
i=1
(∫ si,1
si,0
y(t)dt + θW (si,2 − si,1)1W[si,1]
)(23)
3.2.1 Evaluation of ∂XT (θ)∂θF
Differentiating (22) with respect to θF we get
∂XT (θ)∂θF
=Nx
T∑
i=1
(x(v′i,1)v
′i,1 − x(v′i,0)v
′i,0 +
∫ vi,1
vi,0
x′(t)dt + (vi,2 − vi,1) + θF
(v′i,2 − v′i,1
))
=Nx
T∑
i=1
((vi,2 − vi,1) +
∫ vi,1
vi,0
x′(t)dt
)(24)
where we use the prime notation to denote the derivative of a quantity with respect to θF . Toobtain this result we used the facts that vi,2 = vi+1,0, i = 0, · · · , Nx
T − 1, v′NxT ,2 = T ′ = 0 and
v′1,0 = 0′ = 0 and x(vi,j) = θF , i = 1, · · · , NxT − 1, j = 0, 1, 2.
11
Theorem 1 The derivative of XT (θ) with respect to θF is given by
∂XT (θ)∂θF
=Nx
T∑
i=1
[(vi,2 − vi,1) + x′(vi,0) (ξi,1 − vi,0) + x′(ξi,1) (vi,1 − ξi,1)
]
where x′(ξi,j), j = 1, · · · , Kyi +1 is the derivative of x(t) with respect to θF and is evaluated by (17)
for all i = 1, · · · , NxT .
The proof is obtained by substituting (17) in (24).
3.2.2 Evaluation of ∂XT (θ)∂θW
Differentiating (22) with respect to θW we get
∂XT (θ)∂θW
=Nx
T∑
i=1
∫ vi,1
vi,0
x′(t)dt (25)
where now the prime notation denotes the derivative of a quantity with respect to θW . Next, weevaluate x′(t) = ∂x(t)
∂θW. During the ith NBP, x(t) is given by (16) where the same segmentation as
in the previous case still holds.
Theorem 2 The derivative of XT (θ) with respect to θW is given by
∂XT (θ)∂θW
=Nx
T∑
i=1
Kyi +1∑
j=1
x′(ξi,j−1) (ξi,j − vi,0)
where x′(ξi,j), j = 1, · · · ,Kyi + 1 is the derivative of x(t) with respect to θW and is evaluated by
(18) for all i = 1, · · · , NxT .
The proof is obtained by substituting (18) in (24).
3.2.3 Evaluation of ∂YT (θ)∂θF
Differentiating (23) with respect to θF we get
∂YT (θ)∂θF
=Ny
T∑
i=1
∫ si,1
si,0
y′(t)dt (26)
where we use the prime notation to denote the derivative of a quantity with respect to θF . Also,we used the fact that si,2 = si+1,0. Next, we evaluate y′(t) = ∂y(t)
∂θF.
Theorem 3 The derivative of YT (θ) with respect to θF is given by
∂YT (θ)∂θF
=Ny
T∑
i=1
[y′(si,0) (ζi,1 − si,0) + y′(ζi,1) (si,1 − ζi,1)
]
where y′(ζi,j), j = 1, · · · ,Kxi +1 is the derivative of y(t) with respect to θF and is evaluated by (20)
for all i = 1, · · · , NyT .
12
The proof is obtained by substituting (20) in (26).
3.2.4 Evaluation of ∂YT (θ)∂θW
Differentiating (23) with respect to θW we get
∂YT (θ)∂θW
=Ny
T∑
i=1
((si,2 − si,1)1W[si,1] +
∫ si,1
si,0
y′(t)dt
)(27)
where now, the prime notation denotes the derivative of a quantity with respect to θW . Also, inthe last step we used the fact that si,2 = si+1,0. Next, we evaluate y′(t) = ∂y(t)
∂θW. During the ith
NBP, y(t) is given by (19) and we differentiate y(t) with respect to θW to obtain
Theorem 4 The derivative of YT (θ) with respect to θW is given by
∂YT (θ)∂θW
=Ny
T∑
i=1
[y′(si,0) (ζi,1 − si,0) + y′(ζi,1) (si,1 − ζi,1)
]
where y′(ζi,j), j = 1, · · · , Kxi + 1 is the derivative of y(t) with respect to θW and is evaluated by
(21) for all i = 1, · · · , NyT .
The proof is obtained by substituting (21) in (27). Theorems 1-4 constitute an algorithm forevaluating the sample derivatives of interest with respect to θF and θW . These are similar with theones derived in [3] but differ in that they include a number of “correction” terms that account forthe fact that the input processes are no longer piecewise constants.
3.3 Unbiasedness
Theorem 5 This estimators obtained in Theorems 1-4 are unbiased estimators of the objectivefunctions, i.e.,
∂E [XT ]∂θF
= E[∂XT
∂θF
],
∂E [XT ]∂θW
= E[∂XT
∂θW
],
∂E [YT ]∂θF
= E[∂YT
∂θF
],
∂E [YT ]∂θW
= E[
∂YT
∂θW
].
The proof of the theorem is rather long, tedious, and very similar to the one in [12], and does notwarrant duplication in this paper. It is based on a theorem in [14] which asserts that a sensitivityestimator is unbiased if the sample derivative exists and the sample functions are Lipschitz contin-uous. Existence is guaranteed by Assumptions A1-A3. Lipschitz continuity can be shown using thearguments from [12]. Alternatively, one can follow the approach of [3] and show that the differences|f(θ + ∆θ)− f(θ)| < K|∆θ| where f(·) is the sample function of interest, and E [K] < ∞.
13
4 IPA Algorithm Implementation for Discrete-Part Systems
The IPA algorithm described by Theorems 1- 4 is derived based on a stochastic fluid model. If themanufactured product is actually a fluid, then one can implement the algorithm directly. However,in this work, we are also interested in applying the derived algorithm to discrete-part manufacturingsystems (i.e., a system that produces discrete products which, in general, is modelled by a DES).Thus, in this section we simply adopt the results from the previous section and evaluate thembased on the sample path observed that is based on a discrete-event model. This is the approachadopted in [9, 15] where the IPA estimators are first derived based on a SFM; however, they areevaluated based on the sample path of a DES. As we will show in the sequel, while in the SFMit was possible for certain events to occur at the same time, in a DES setting this is no longerpossible. This observation results in several simplifications that make the IPA algorithm easier andnon-parametric (the algorithm no longer needs any rate information). In the DES setting, thereis no guarantee that the IPA estimators are still unbiased; however, simulation results indicatethat such an approach provides excellent results. Next, we investigate how some of the derivedsensitivities may be evaluated based on a DES sample path, but, for the sake of completeness,we first define the automaton that describes the dynamics of the system. The state automaton isdescribed by the five-tuple (E , X, Γ,F , x0) where E = {c1, c2, d1, d2, r} is the event set with ci beingthe completion of a part, di being the departure of the part from machine i, i = {1, 2} and r beinga part order arrival. X = [x, y, b1, b2]T is the state space with x ∈ Z and y ∈ Z+, and bi ∈ {0, 1}indicating that machine i is blocked; x ≥ 0 indicate the number of finished parts, x < 0 indicatethe number of unfulfilled orders and y measures the number of unfinished parts in W and M1. Fis the state transition mechanism and x0 = 0 is the initial state.
Assumption:
A4 W.p. 1, no two events can occur at exactly the same time except for (ci, di), (r, d1), and(r, d1, d2).
The exceptions are needed since a completed part will immediately depart from the machine ifthere is available space in the buffer, and since an order arrival coincides with a departure at eithermachine M1 or M1 and M2 when blocking occurs.
Next, we present how the events defined on the SFM are identified given the above automatondefinition.
βqi Trajectory of q ∈ {x, y} ceases to be empty. If x = 0 and d1 occurs, this identifies a βx
i eventand if y = 0 and d2 occurs identifies a βy
i .
εqi Trajectory of q ∈ {x, y} becomes empty. If x = 1, b1 = 0 and a d1 event occurs, this identifies
εxi . If y = 1, b2 = 0 and a d2 event occurs, this identifies an εy
i .
φqi Trajectory of q ∈ {x, y} becomes full. If x = θF , b1 = 0 and a c2 event occurs, this identifies a
φxi event. If y = θW , b2 = 0 and a c2 event occurs, this identifies a φy
i .
ωqi Trajectory of q ∈ {x, y} ceases to be full. If x = θF , and a d1 event occurs, this identifies an ωx
i
event. If y = θW , and a d2 event occurs, this identifies an ωyi .
If we evaluate Lemmas 1-2 using observations from a discrete-part sample path then, the followingsimplifying propositions/corollaries hold,
14
Proposition 1 Let wxi denote the time when the ωx
i event (F ceases to be full) has occurred in theDES sample path. Then,
∂wxi
∂θF=
∂wxi
∂θF= 0.
This proposition follows directly from Lemma 1 recognizing that wxi = ey
j implies that an orderarrival r should occur exactly the same time with a departure from M2 which is excluded byassumption A4.
Proposition 2 Let byi denote the time when the βx
i event (W ceases to be empty) has occurred inthe DES sample path. Then,
∂byi
∂θF=
∂byi
∂θF= 0.
This proposition follows directly from Lemma 3 recognizing that byi = fx
j implies that a completionof a part at the two machines occurs exactly the same time which is also excluded by assumptionA4. Note that Propositions 1 and 2 imply that that the begin and end points of a cycle in x(t) (Cx
i )and y(t) (Cy
i ) are independent of either θF or θW . In other words, the following corollaries hold.
Corollary 2 The begin and end points of a cycle at buffer x(t) are independent of either θF orθW . In other words,
∂vi,0
∂θW=
∂vi,2
∂θW=
∂vi,0
∂θF=
∂vi,2
∂θF= 0.
The proof follows from Proposition 1 by recognizing that the event ωxi indicates the end of the ith
cycle (vi,0) and the begin of cycle i + 1 (vi,2 = vi+1,0).
Corollary 3 The begin and end points of a cycle at buffer y(t) are independent of either θF orθW . In other words,
∂si,0
∂θW=
∂si,2
∂θW=
∂si,0
∂θF=
∂si,2
∂θF= 0.
The proof follows from Proposition 2 and Lemma 2 by recognizing that the events βyi and ωx
i
indicate the begin or end points of some cycle, si,0 and si,2 respectively. Using Corollaries 2 and 3we can also simplify the results from Lemmas 5 and 4 as shown in Propositions 3 and 4.
Proposition 3 Consider the interval [vi,0, vi,1) ∈ [sj,0, sj,1) or [vi,0, vi,1) ∈ [sj,1, sj,2), for somei, j = 1, 2, · · · . Then
∂vi,1
∂θF=
1− 1F[vi,0]M1,0(vi,1)
∂vi,1
∂θW= 0
where 1F[vi,0] is needed for the case where x(0) = 0.
15
The proposition follows immediately from Lemma 5 by recognizing that g = 1 + 1[y(t) = 0] = 1t ∈ [vi,0, vi,1), since at point vi,1 a part is blocked in y(t), therefore 1[y(t) = 0] = 0.
Proposition 4 Consider the interval [si,0, si,1) ∈ [vj,0, vj,1) or [si,0, si,1) ∈ [vj,1, vj,2), for somei, j = 1, 2, · · · . Then
∂si,1
∂θF= 0
∂si,1
∂θW=
1W[si,1]− 1W[si,0]M2,1(si,1)
where M2,1(t) is defined in (11).
The proof of the proposition follows from Lemma 4 by applying corollary 3 and recognizing thatg = 1[x(t) < θF ] = 1, t ∈ [si,0, si,1) since for a part to be able to leave M1, it is necessary that Fis not full. Next, as with Lemmas 6 and 7 we provide an iterative algorithm for determining theevent time derivatives of the end point of the NBP when several events occur in the other queueduring the NBP.
Proposition 5 The derivatives of vi,1, i = 1, 2, · · · with respect to θF and θW are given by
∂vi,1
∂θF= −(µ1(ξi,1)− µ2(ξi,1)) ξ′i,1
µ1(vi,1)− ρ(vi,1), i = 2, 3, · · ·
∂vi,1
∂θW= − (µ1(vi,1)− ρ(vi,1))
−1
Kyi∑
j=1
[(µ1(ξi,j)− µ2(ξi,j)) ξ′i,j
]
where the prime notation indicates the event time derivatives with respect to either θF (first equa-tion) or θW (second equation).
Recall that ξi,j corresponds to the time instance when y(t) switches between empty and non-empty.If it corresponds to a βy event, then ξ′i,j = 0 due to Corollary 3. If on the other hand, it correspondsto an εy event, then ξ′i,j is evaluated using Propositions 4 and 6 (the latter is defined next).
Proposition 6 The derivative of si,1, i = 1, 2, · · · with respect to θF , if si,1 corresponds to an εy
event (i.e., it starts an EP) is given by
∂si,1
∂θF=
(µ1(ζi,1)− µ0(ζi,1)) ζ ′i,1µ2(si,1)− µ1(si,1)
On the other hand, if it corresponds to a φy event (i.e., starts a FP),
∂si,1
∂θF=
(µ1(ζi,1)− µ0(ζi,1)) ζ ′i,1µ2(si,1)− µhi(Kx
i )(si,1)
The derivative of si,1, i = 1, 2, · · · with respect to θW , if si,1 corresponds to an εy event is given by
∂si,1
∂θW=
1W[si,1]− 1W[si,0] + (µ1(ζi,1)− µ0(ζi,1)) ζ ′i,1µ2(si,1)− µ1(si,1)
16
On the other hand, if it corresponds to a φy event
∂si,1
∂θW=
1W[si,1]− 1W[si,0] + (µ1(ζi,1)− µ0(ζi,1)) ζ ′i,1µ2(si,1)− µhi(Kx
i )(si,1).
Recall that ζi,j corresponds to the time instance when x(t) switches between full and non-full. If itcorresponds to an ωx event, then ζ ′i,j = 0 due to Corollary 2. If on the other hand, it correspondsto a φx event, then ζ ′i,j is evaluated using Propositions 3 and 5. Next, we substitute the derivedevent time derivatives into the sample function sensitivities from Lemmas 8 and 9 as shown inthe following two propositions. Here however, we need to be a little careful. For the discrete partsample path, the state can only take discrete values, therefore, strictly speaking a derivative (e.g.,∂x(t)∂θF
) is not defined. What the following propositions evaluate are “pseudo-sensitivities” of x(t)and y(t) with respect to the buffer capacities, so to avoid any confusion we use the notation x′θq
(t),y′θq
(t), q ∈ {F, W} to denote them.
Proposition 7 During any cycle Cxi , i = 1, 2, · · · , for any interval [vi,0, vi,1)
x′θF(t) = 1F[vi,0]− [µ1(τi)− µ0(τi)] τ ′i1[t ≥ ξi,1]
x′θW(t) =
∑
k
(1W[ξi,k]− 1W[sm,0] + (µ1(σi)− µ0(σi))σ′i
)
For any interval [vi,1, vi,2)x′θF
(t) = 1 x′θW(t) = 0
for all i = 1, 2, · · · .
where τi is the time when a perturbation that propagated from x(t) (buffer F) to y(t) has returnedback to x(t) through an event εy. From Propositions 3 and 5 we see that either τ ′i = 0 or, forcertain early cycles it may take a value that will cause the entire perturbation to be lost x′θF
(t) = 0.Also, sm,0 indicates the point that started the mth cycle in y(t) and which had sm,1 = ξi,k, inother words, NBP of that cycle ended with an εy event. Furthermore, σi indicates a point where aperturbation that propagated from y(t) to x(t) is returned back to y(t). Note that only one suchpoint exists in a cycle of x(t) since it contains only one point where x(t) becomes full.
Proposition 8 During any cycle Cyi , i = 1, 2, · · · , the sensitivity of y(t) with respect to θF is given
by
y′θF(t) =
{ − (µ1(ψi)− ρ(ψi)) ψ′i1[t ≥ ζi,1] if t ∈ [si,0, si,1)0 if t ∈ [si,1, si,2)
The sensitivity of y(t) with respect to θW is given by
y′θW(t) =
{1W[si,0] if t ∈ [si,0, si,1)1W[si,1] if t ∈ [si,1, si,2)
where ψi is the time when a perturbation propagates from x(t) (buffer F ) to y(t) through anevent φx. These perturbations are given by Propositions 3 (when perturbation is generated) and5 when perturbation is propagated from earlier time instants. Propositions 7 and 8 preciselydescribe the finite differences ∆x(t) and ∆y(t) in the results obtained in [3]. Therefore one canuse the algorithm from [3] to evaluate the required cost sensitivities on-line. For completeness, thealgorithm is presented next.
17
4.1 IPA Algorithm for a Discrete-Part System
First, during the entire observation period we define a sequence of points {τi}, i = 1, · · · , N∗T , such
thatτ1 < τ2 < · · · < τN∗
T≡ τ∗ (28)
where,
• τ1 = inf{t : x(t) = θF , t ≥ 0} is the time that the very first φx event occurred (F becomesfull).
• τ2n, n = 1, · · · ,N∗
T−12 is the time of the εy event (W becomes empty) that immediately follows
a φx event (i.e., the index i in the sequence {τi} is even).
• τ2n−1, n = 2, · · · ,N∗
T +12 is the time of the first φx event following an εy event (i.e., i is odd).
• N∗T = min {2n− 1 : y(t) = θW for some τ2n−1 ≤ t < τ2n, n = 1, 2, · · · }. In case the event
y(t) = θW for some τ2n−1 ≤ t < τ2n, n = 1, 2, · · · , does not occur in [0, T ], we setN∗
T = max{2n− 1 : τ2n−1 ≤ T, n = 1, 2, · · · } and also set τ∗ = T .
The first and third items above define a sequence of events{
φ̂xi : i = 1, · · · ,
N∗T +12
}while the
second item defines a sequence of events{
ε̂yi : i = 1, · · · ,
N∗T−12
}. Both are subsequences of {φx
i }and {εy
i } respectively. N∗T is the random number of events that appear in the two subsequences in
the interval [0, T ]. Furthermore, τ∗ indicates either the first occurrence of a φ̂x when y(τ∗) = θW
or the first occurrence of a φ̂x that is followed by a φy event that occurs before an εy. Finally, notethat since N∗
T stops counting after a φ̂x event, thus it must be an odd number. Furthermore, wedefine the following times
• fy∗ = inf{t : y(t) = θW , t ≥ τ∗}, is the time instant of the first φy event after τ∗.
• fxmj
= inf{t : x(t) = θF , t ≥ eyj , j = 1, · · · , Ny
T }, is the time of the first φx event thatimmediately follows after an εy event.
In order to estimate the sensitivities of XT and YT with respect to θF and θW , we simply accumulatethe following time intervals.
X ′θF
(θ) = (T − τ∗) +
N∗T−1
2∑
n=1
(τ2n − τ2n−1) (29)
Y ′θF
(θ) = −(fy∗ − τ∗)1[y(τ∗) < θW ] +
N∗T−1
2∑
n=1
(τ2n − τ2n−1) (30)
X ′θW
(θ) =Ny
T∑
j=1
(fx
mj− sj,1
)1[y(sj,0) = θW ]1[x(sj,1) < θF ] (31)
Y ′θW
(θ) =Ny
T∑
j=1
[(sj,1 − sj,0)1[y(sj,0) = θW ] + (sj,1 − sj,2)1[y(sj,1) = θW ]] (32)
18
These results are obtained from Theorems 1 - 4 when x′(t) and y′(t) are obtained from Propositions 7and 8. For example, from x′θF
(t) of Proposition 7, we see that for t < v1,1 (i.e., before the very firstbuffer F becomes full event), x′θF
(t) = 0 since by assumption x(0) = 0. For the remaining cycles,x′θF
(t) = 1 for vi,0 ≤ t < ξi,1 and for the interval ξi,1 ≤ t < vi,1, x′θF(t) takes the value 1, if τ ′i = 0
(i.e., when there is no perturbation to be propagated from y(t) back to x(t), or 0, when there is someperturbation in y(t) that returns back to x(t) and cancels the original perturbation (note that τi
corresponds to some earlier vj,1 point (j < i), thus, substituting the expression from Proposition 3makes (µ1(τi)− µ0(τi))τ ′i = 1). Summarizing, at the first ωx
1 event (at τ1), a perturbation equal to1 is generated. Then, this perturbation persists for the remaining BP of the cycle, continues intothe next cycle until it may be cancelled at point x2,1 = τ2. Next, at φx
2 (τ3) the perturbation isregenerated until it may be lost again in the following cycle at τ4 and so on. After a point (τ∗),y(t) will have no perturbation to return to x(t) and cancel its perturbation, therefore, x′θF
(t) = 1for τ∗ ≤ t < T . The remaining expressions are similarly obtained.
We emphasize that the algorithm described by (29)-(32) consists of simple timers that measurethe intervals between certain events and some indicator functions that are simple to evaluate alonga sample path. These estimators are truly non-parametric since they require no knowledge ofthe input stochastic processes that drive the system. In the next section we investigate how thisalgorithm applies in the optimization of hedging points in a discrete-part manufacturing system.
5 Simulation Results
To demonstrate our approach we consider the unconstrained optimization problem of a manufactur-ing system where discrete parts are produced at machines M1 and M2 ( constrained optimizationscenaria are considered in [5]). The part processing times at both machines are exponentiallydistributed with rates µ1 = µ2 = 1 parts per unit time. Demand arrives according to a Poissonprocess with rate ρ = 0.8 parts per unit time. Finally we assume that inventory and backlog costsare cF = 2.0, cW = 1.0 and cD = 3.0. Fig. 3 shows estimates of the cost (5) as a function of thediscrete buffer capacities KF and KW obtained through exhaustive simulation, where each pointis averaged over 10 sample paths, each 100K time units long. From the figure we see that theoptimal buffer capacities are K∗
F = 3 and K∗W = 8, while neighboring designs exhibit comparable
near-optimal performance. In addition, an interesting observation is that for KW ≤ 2, the systembecomes unstable in the sense that the backlog cost goes to infinity as T →∞.
Next, we start from a random design and use the stochastic approximation algorithm (8) to findthe optimal buffer capacities. In this algorithm, we use the derived sample derivatives, evaluatedbased on observations from the “actual” discrete-event system to obtain estimates of the gradientHn(Kn, ωSFM
n ). In other words, we use the expressions (29)-(32) and evaluate all intervals basedon the occurrence times of the corresponding events. Table 1 shows the iterations of the algorithmwhen the starting buffer capacities are set to KF = 24 and KW = 20. The gradient estimatesare evaluated from a single sample path of length T = 50K time units. Furthermore, for thisexperiment we have fixed the step size ηn = 1 for all n = 0, 1, · · · . As shown in the table,the algorithm converges very quickly to the capacities that minimize the expected cost whichdemonstrates that sample derivatives based on SFMs can be used effectively in the control andoptimization of manufacturing systems and of discrete event systems in general. At this point,it is worth pointing out that the SFM-based IPA estimates yield correct directional informationfor the optimization algorithm, even though, the equivalent SFM model of the system we study
19
1 2 3 4 5 6 7 8 9 10 11 12 13 14
35
7
9
11
10
15
20
25
30
35
40
45
50
J
K F
K W
45-50
40-45
35-40
30-35
25-30
20-25
15-20
10-15
Figure 3: Expected cost JT (KF ,KW ).
is trivial and does not provide any useful information; the corresponding SFM model is one withµ1(t) = µ2(t) = 1 and ρ(t) = 0.8 for all t ∈ [0, T ], thus the sample paths of x(t) and y(t) consist ofa single busy period that extend up to T .
6 Conclusions
In this paper we study a two-stage Make-To-Stock manufacturing systems and showed that SFMscan be very effective tools for the control and optimization of such systems. Our approach is basedon deriving IPA estimates based on SFMs (which are often unbiased), and then evaluate thembased on data from the actual discrete event system. In the future we plan to extend this approachto multi-stage systems and also include multiple part types as well. Of course, this is a gradientbased approach, thus it can only guarantee convergence to local optima but it can be used withother global optimization techniques such as Ordinal Optimization and Simulated Annealing.
20
Table 1: Optimization Algorithm Iterations
Iteration KW KF LT (KF ,KW ) ∂LT∂θW
∂LT∂θF
0 20 24 60.4 0.99 1.981 19 22 53.9 0.97 1.922 18 20 51.1 0.99 1.953 17 18 46.1 0.97 1.964 16 16 41.3 0.99 1.845 15 14 35.7 0.95 1.896 14 12 32.2 0.89 1.777 13 10 27.4 0.90 1.438 12 8 24.5 0.82 1.399 11 6 21.6 0.58 0.9910 10 5 19.8 0.35 0.6411 9 4 21.6 -0.04 0.1112 9 3 18.3 0.33 -0.0313 8 3 18.6 0.08 0.05
APPENDIX: Proofs of Lemmas, Theorems and Propositions
Proof of Lemma 1:
The event wxi occurs when the difference γ1(t;θ)−ρ(t) becomes negative, i.e., γ1(w−; θ)−ρ(w−) =
0 and γ1(w; θ) − ρ(w) < 0 where for ease of notation we let wxi = w. Also, if y(w−) > 0,
ρ(w−) < µ1(w−) or, if y(w−) = 0, ρ(w−) < µ2(w−) (otherwise x(w−) < θF (see (3))). The latter isimpossible since, ρ(w−) < µ2(w−) implies that some fluid will accumulate in W as long as θW > 0(see (2)-(4)). So we are only concerned with y(w−) > 0. In this case, the difference γ1(w; θ)−ρ(w)will become negative if µ1(t)− ρ(t) switches from positive to negative at w, or an εy event occurswith µ2(w) < ρ(w). In the first case, both processes µ1(t) and ρ(t) (the processes that determinethe time when the event ωx will occur) are independent of either θF or θW , thus the event timederivatives with respect to either θF or θW are 0. From the latter case, we get that
∂wxi
∂θF=
∂eyj
∂θFand
∂wxi
∂θW=
∂eyj
∂θW, i, j = 1, 2, · · · .
Proof of Lemma 2:
To prove the lemma, we need to show that the instant when the difference µ2(t)− γ1(t; θ) changesfrom zero to negative is independent of both θF or θW . Again to ease the notation, let wy
i = w.Next suppose that x(w−;θ) = θF , thus γ1(w−; θ) = ρ(w−) < µ1(w−) and ρ(w−) < µ2(w−).µ2(t)−γ1(t; θ) changes from non-negative to negative because either µ2(w)−ρ(w) or µ1(w)−ρ(w)switched from non-negative to negative. But, both cases are independent of θF and θW (Notethat this case also includes the case when ωx coincides with ωy but as also indicated by Lemma 1
21
both events are independent of θF and θW ). Next, assume that x(w−; θ) < θF . In this case,γ1(w−; θ) = µ1(w−) and we identify two subcases. (i) If x(w) < θF , then the change is due to aswitch of the difference µ2(w)−µ1(w) from non-negative to negative which is independent of both θF
and θW . (ii) If x(w) = θF this implies that γ1(w;θ) = ρ(w) < µ1(w) and µ2(w) < ρ(w) (otherwiseno ωy at w), so µ2(w) < ρ(w) < µ1(w). Just before the switching event, since y(w−; θ) = θW , it isimplied that µ2(w−) > µ1(w−) and since the content of buffer F is increasing it is also necessarythat µ1(w−) > ρ(w−). Summarizing, we have,
ρ(w−) < µ1(w−) < µ2(w−)µ2(w) < ρ(w) < µ1(w)
However, this suggests that more than one event should occur at time w which is excluded due toAssumption A2. Therefore, the result has been proved.
Proof of Lemma 3:
The event βy occurs when the difference γ2(t; θ) − γ1(t; θ) = µ2(t) − γ1(t; θ) becomes positive attime t = by
i = b (γ2(t; θ) = µ2(t) since y(t) = 0 and byi = b for notational simplicity). Depending
on x(b) we identify the following four cases.
i x(b−) < θF , x(b) < θF . In this case, γ1(b−; θ) = µ2(b−) and γ1(b; θ) = µ1(b). The event is causedby a sign change of the difference µ2(t)− µ1(t) from non-positive to positive. Such change isindependent of either θF or θW thus both event time derivatives are equal to zero.
ii x(b−) = θF , x(b) = θF . In this case, γ1(b−; θ) = µ2(b−) since y(b−) = 0. In addition, sincex(b−) = θF , γ1(b−) = ρ(b−). If ρ(b−) > µ2(b−), then x(b−) < θF which is a contradiction. Ifρ(b−) < µ2(b−), then y(b−) > 0 which is again a contradiction. Since, ρ(t) 6= µ2(t) for anyinterval due to Assumption A3, this case is impossible.
iii x(b−) < θF , x(b) = θF . In this case, γ1(b−; θ) = µ2(b−) < µ1(b−) and ρ(b−) < µ2(b−) otherwise,x(b−) will not be increasing. The switch is because of the φx
j event, therefore
∂byi
∂θF=
∂fxj
∂θFand
∂byi
∂θW=
∂fxj
∂θW, i, j = 1, 2, · · · .
iv x(b−) = θF , x(b) < θF . Following the arguments of case ii it can be shown that this is also animpossible case.
Proof of Lemma 4:
During the interval, [si,0, si,1) since no other event has occurred, the inflow and outflow processesare fixed. The inflow is µ2(t) since 0 < y(t) < θW and the outflow is µ1(t) if x(t) < θF , t ∈ (vj,0, vj,1)or µ0(t) = ρ(t) t ∈ (vj,1, vj,2). For the interval [si,0, si,1) we can write
θW1W[si,0] +∫ si,1
si,0
M2,g(t)dt = θW1W[si,1]
22
Differentiating both sides with respect to θF we get
M2,g(si,1)s′i,1 −M2,g(si,0)s′i,0 = 0
Rearranging terms we get the first result. Differentiating both sides of the above relation withrespect to θW we get
1W[si,0] + M2,g(si,1)s′i,1 −M2,g(si,0)s′i,0 = 1W[si,1]
Rearranging terms we get the second result.
Proof of Lemma 5:
During the interval, [vi,0, vi,1) since no other event has occurred, the inflow and outflow processesare fixed. The inflow is µ1(t) if y(t) > 0 or µ2(t) if y(t) = 0. The outflow is ρ(t). For the interval[vi,0, vi,1) we can write
θF1F[vi,0] +∫ vi,1
vi,0
Mg,0(t)dt = θF
Differentiating both sides with respect to θF we get
Mg,0(vi,1)v′i,1 −Mg,0(vi,0)v′i,0 = 1− 1F[vi,0]
Note that for the first NBP 1F[v1,0] = 0, since we assume x(s1,0) = 0, while for all other NBPs1F[vi,0] = 1, i = 2, 3, · · · . Rearranging terms we get the first two results. Differentiating both sidesof the above relation with respect to θW we get
Mg,0(vi,1)v′i,1 −Mg,0(vi,0)v′i,0 = 0
Rearranging terms we get the third result.
Proof of Lemma 6:
In the interval [vi,0, vi,1) x(t) goes from x(vi,0) = θF to x(vi,1) = θF . Using the ξi,j notation (definedearlier), for i = 2, 3, · · · , we can write (for i = 1 see Lemma 5)
θF +Ky
i∑
j=0
∫ ξi,j+1
ξi,j
(µgi(j)(τ)− ρ(τ)
)dτ = θF . (33)
Differentiating both sides with respect to θF we get,
Kyi∑
j=0
(µgi(j)(ξi,j+1)− ρ(ξi,j+1)
)ξ′i,j+1 −
(µgi(j)(ξi,j)− ρ(ξi,j)
)ξ′i,j = 0.
Note that most ρ(ξ)ξ′ terms cancel out. So, rearranging terms, we get(µgi(K
yi )(ξi,Kj
i +1)− ρ(ξ
i,Kji +1
))
ξ′i,Kyi +1 −
(µgi(0)(ξi,0)− ρ(ξi,0)
)ξ′i,0 +
Kyi∑
j=1
(µgi,j−1(ξi,j)− µgi,j (ξi,j)
)ξ′i,j = 0
23
Next, solving for ξ′i,Ky
i +1and recognizing that v′i,1 = ξ′
i,Kyi +1
and v′i,0 = ξ′i,0 we get
v′i,1 =(µgi(K
yi )(vi,1)− ρ(vi,1)
)−1
(
µgi(0)(vi,0)− ρ(vi,0))v′i,0 −
Kyi∑
j=1
[(µgi,j−1(ξi,j)− µgi,j (ξi,j)
)ξ′i,j
]
(34)Subsequently, we investigate all ξi,j points, j = 2, · · · ,Ky
i . By definition, these correspond to pointswhere either a βy or an ε event occurs. Since, these events occur in the interval (vi,0, vi,1), theycannot coincide with any φx event. Therefore, using Lemma 3 we conclude that ξ′i,j = 0 for all ξi,j
that correspond to a βy event. The remaining ξi,j points correspond to εy points that belong tocycles fully contained in the interval (vi,0, vi,1). Thus, using Lemma 4, we also find that ξ′i,j = 0
(Note that for this argument, we first use Corollary 1 to show that ∂si,0
∂θFand then Lemma 4). As a
results, we obtain the required result.
To obtain ∂vi,1
∂θWwe differentiate (33) with respect to θW and obtain (34) with the exception that
the prime notation indicates derivatives with respect to θW rather than θF which is the requiredresult.
Proof of Lemma 7:
In the interval [si,0, si,1), y(t) goes from one boundary to another boundary, therefore, using theabove notation, we can write
θW1W[si,0] +Kx
i∑
j=0
∫ ζi,j+1
ζi,j
(µ2(τ)− µhi(j)
(τ))
dτ = θW1W[si,1] . (35)
Differentiating both sides with respect to θF we get,
Kxi∑
j=0
[(µ2(ζi,j+1)− µhi(j)(ζi,j+1)
)ζ ′i,j+1 −
(µ2(ζi,j)− µhi(j)(ζi,j)
)ζ ′i,j
]= 0.
Recognizing that ζi,0 = si,0 and si,1 = ζi,Kxi +1 and rearranging terms as in the proof of the previous
lemma, we obtain
s′i,1 =(µ2(si,1)− µhi(Kx
i )(si,1))−1
×(
µ2(si,0)− µhi(0)(si,0))s′i,0 +
Kxi∑
j=1
(µhi(j−1)(ζi,j)− µhi(j)(ζi,j)
)ζ ′i,j
.
By definition, all point ζi,j correspond to points where x(t) either cease to be full or becomesfull. Using Lemma 1 we conclude that for all j = 1, · · · ,Kx
i , all ζ ′i,j = 0 for all such points thatcorrespond to ωx events (since these events occur in the interval [si,0, si,1), they cannot coincidewith an εy event). Also, for all Cx
i that are fully contained in the interval [si,0, si,1), all ∂vi,0
∂θF= 0
(due to Lemma 1) and thus, ∂vi,1
∂θF= 0 due to Lemma 4. As a result, all ζ ′i,j = 0 except possibly,
ζ ′i,1. Thus, the required result is obtained.
24
Next, we differentiate the above relation with respect to θW to obtain
1W[si,0] +Kx
i∑
j=0
[(µ2(ζi,j+1)− µhi,j
(ζi,j+1))ζ ′i,j+1 −
(µ2(ζi,j)− µhi,j
(ζi,j))ζ ′i,j
]= 1W[si,1]
Again recognizing that ζi,0 = si,0 and si,1 = ζi,Kxi +1 and rearranging terms we obtain
s′i,1 = −(µ2(si,1)− µhKx
i(si,1)
)−1×
1W[si,1]− 1W[si,0] +
(µ2(si,0)− µhi(0)(si,0)
)s′i,0 +
Kyi∑
j=1
(µhi,j−1(ζi,j)− µhi,j (ζi,j)
)ζ ′i,j
Next we concentrate on ζ ′i,j j = 2, · · · ,Kxi . Following an argument similar to the derivative with
respect to θF we get that ζ ′i,j = 0 for all j = 2, 3, · · · , therefore,the required result is obtained.
Proof of Lemma 8:
Next, we differentiate x(t) (16) with respect to θF .
x′(t) = 1F[vi,0] + ρ(ξi,0)ξ′i,0 +Ky
i (t)∑
j=0
[µgi(j)(ξi,j+1)ξ′i,j+1 − µgi(j)(ξi,j)ξ′i,j
]
= 1F[vi,0]−[µgi(0)(vi,0)− ρ(vi,0)
]v′i,0 −
[µgi(1)(ξi,1)− µgi(0)(ξi,1)
]ξ′i,11[t ≥ ξi,1]
where as earlier, ξ′i,j indicates the derivative of the time instant ξi,j with respect to θF . Also, notethat all ξ′i,j = 0, j = 2, · · · ,Ky
i (t) as indicated in the proof of Lemma 6. Next, we differentiate x(t)with respect to θW in the interval [vi,0, vi,1) to obtain
x′(t) = ρ(ξi,0)ξ′i,0 +Ky
i (t)∑
j=0
[µgi(j)(ξi,j+1)ξ′i,j+1 − µgi(j)(ξi,j)ξ′i,j
]
where now, ξ′i,j is the derivative of ξi,j with respect to θW . Differentiating x(t) in the interval[vi,1, vi,2) with respect to θF and θW we get the second result of the lemma and the proof iscomplete.
Proof of Lemma 9:
We differentiate y(t) (19) with respect to θF to obtain
y′(t) =
{ ∑Kxi (t)
j=0
[M2,hi(j)(ζi,j+1)ζ ′i,j+1 −M2,hi(j)(ζi,j)ζ ′i,j
]if t ∈ [si,0, si,1)
0 if t ∈ [si,1, si,2)
=
−µ2(ζi,0)ζ ′i,0+
∑Kxi (t)
j=0
[µhi(j)(ζi,j+1)ζ ′i,j+1 − µhi(j)(ζi,j)ζ ′i,j
]if t ∈ [si,0, si,1)
0 if t ∈ [si,1, si,2)
25
where as earlier, ζ ′i,j is the derivative of ζi,j with respect to θF . Following the argument in the proofof Lemma 7, ζ ′i,j = 0, j = 2, · · · ,Kx
i , therefore the first result is proved. Next, we differentiate y(t)(19) with respect to θW to obtain,
y′(t) =
1W[si,0]− µ2(ζi,0)ζ ′i,0+
∑Kxi (t)
j=0
[µhi(j)(ζi,j+1)ζ ′i,j+1 − µhi(j)(ζi,j)ζ ′i,j
]if t ∈ [si,0, si,1)
1W[si,0] if t ∈ [si,1, si,2)
where, ζ ′i,j is the derivative of ζi,j with respect to θW . Again ζ ′i,j = 0, j = 2, · · · ,Kxi , therefore the
proof is complete.
Proof of Proposition 5
Starting from Lemma 6 we first ignore the v′i,0 terms due to Corollary 2 to get
∂vi,1
∂θF= −
(µgi(0)(ξi,1)− µgi(1)(ξi,1)
)ξ′i,1
µgi(Kyi )(vi,1)− ρ(vi,1)
, i = 2, 3, · · ·
∂vi,1
∂θW= −
(µgi(K
yi )(vi,1)− ρ(vi,1)
)−1
Kyi∑
j=1
[(µgi(j−1)(ξi,j)− µgi(j)(ξi,j)
)ξ′i,j
]
where the prime notation indicates the event time derivatives with respect to either θF (firstequation) or θW (second equation). Next, at points vi,1, i.e., when x(t) is full and y(t) becomesblocked, y(vi,1) > 0, therefore µgi(K
yi )(vi,1) = µ1(vi,1). Next, we investigate ξ′i,1 in the first equation.
If it corresponds to an event βy (buffer W cease to be empty), it corresponds to the beginning pointof a cycle and by Corollary 3, ξ′i,1 = 0. On the other hand, if it corresponds to an event εy, thenbefore ξi,1, y(t) > 0 and thus µgi(0)(ξi,1)− µgi(1)(ξi,1) = µ1(ξi,1)− µ2(ξi,1) therefore the first resultis obtained. Similarly, ξ′i,j = 0 for all j that correspond to a βy event. Finally, recognizing thatbefore an εy event y(t) > 0 we also obtain the second result.
Proof of Proposition 6
Starting from Lemma 7, we first ignore the s′i,0 terms due to Corollary 3 to get that the derivativesof si,1, i = 1, 2, · · · with respect to θF are given by
∂si,1
∂θF=
(µhi(0)(ζi,1)− µhi(1)(ζi,1)
)ζ ′i,1
µ2(si,1)− µhi(Kxi )(si,1)
If at the beginning of the interval x(si,0) = θF , then ζ ′i,1 = 0, since it corresponds to a buffer F ceaseto be full event (see Corollary 2). As a result, when x(si,0) < θF , hi(0) = 1 and hi(1) = 0. To obtainthe final result, we also identify two cases. One when si,1 corresponds to an event εy, which impliesthat x(t) is not full (otherwise the part would be blocked in M1), therefore µhi(Kx
i )(si,1) = µ1(si,1).The other case is when si,1 corresponds to a φy event, and the denominator can be either of therate differences.
The result for the derivatives of si,1, i = 1, 2, · · · with respect to θW is similarly obtained.
26
Proof of Proposition 7
The proof follows the arguments of the previous two proofs. During any cycle Cxi , i = 1, 2, · · · , for
any interval [vi,0, vi,1)
x′θF(t) = 1F[vi,0]−
[µgi(1)(ξi,1)− µgi(0)(ξi,1)
]ξ′i,11[t ≥ ξi,1]
Now, ξ′i,1 = 0 if it corresponds to βy event (due to Corollary 3) and possibly has non-zero value forsome εy events. In this case, y(t) > 0 for t < ξi,j thus gi(0) = 1 and gi(1) = 2. So we can write
x′θF(t) = 1F[vi,0]− [µ2(ξi,1)− µ1(ξi,1)] ξ′i,11[t ≥ ξi,1]
Lets say that ξi,1 corresponds to some sk,1 (for some k) therefore, the derivatives are given by eitherProposition 4 (s′k,1 = 0) or 6 (s′k,1 may be non-zero). Substituting in the above equation we obtain
x′θF(t) = 1F[vi,0]− [µ2(sk,1)− µ1(sk,1)] s′k,11[t ≥ sk,1]
= 1F[vi,0]− [µ2(sk,1)− µ1(sk,1)](µ1(τ)− µ0(τ)) τ ′
µ2(sk,1)− µ1(sk,1)1[t ≥ sk,1]
= 1F[vi,0]− (µ1(τ)− µ0(τ)) τ ′1[t ≥ sk,1]
where τ is the time of some event on buffer F that caused a perturbation that propagated to bufferW and caused the s′k,1 to be non-zero, thus the first result is obtained.
Next, the sensitivity of x(t) with respect to θW is given by
x′θW(t) =
Kyi (t)∑
j=1
(µgi(j−1)(ξi,j)− µgi(j)(ξi,j)
)ξ′i,j1[t ≥ ξi,1]
As before, ξ′i,j = 0 for all points that correspond to βy events due to Corollary 3 while they maytake non-zero value for some points that correspond to εy events. In this case, y(t) > 0 for t < ξi,j
thus gi(j − 1) = 1 and gi(j) = 2 so we can write
x′θW(t) =
Kyi (t)∑
j=1
(µ1(ξi,j)− µ2(ξi,j)) ξ′i,j1[t ≥ ξi,1] .
Since all ξi,j points correspond to points where an εy event has occurred, their sensitivities can beevaluated by Propositions 4 or 6 and substituting we get
x′θW(t) = −
∑
k
(µ1(ξi,k)− µ2(ξi,k))(1W[ξi,k]− 1W[sm,0] + (µ1(σ)− µ0(σ))σ′)
(µ1(ξi,k)− µ2(ξi,k))1[t ≥ ξi,1] .
= −∑
k
(1W[ξi,k]− 1W[sm,0] + (µ1(σ)− µ0(σ))σ′
)1[t ≥ ξi,1] .
where sm,0 indicates the point that started the mth cycle in y(t) and which had sm,1 = ξi,k, inother words, NBP of that cycle ended with an εy event. Furthermore, σ indicates a point where aperturbation that propagated from y(t) to x(t) is returned back to y(t).
27
Proof of Proposition 8
¿From Lemma 9, ignoring the s′i,0 terms due to Corollary 3 we obtain the following sensitivity withrespect to θF
y′θF(t) =
{ (µhi(1)(ζi,1)− µhi(0)(ζi,1)
)ζ ′i,11[t ≥ ζi,1] if t ∈ [si,0, si,1)
0 if t ∈ [si,1, si,2)
Next, note that ζi,j corresponds to a point where x(t) switches between full and non-full. If this isan ωx event, then ζ ′i,j = 0 due to Corollary 2. On the other hand, it is possible to have a non-zerovalue if it corresponds to a φx event. In this case, hi(0) = 1 and hi(1) = 0, therefore
y′θF(t) =
{ − (µ1(ζi,1)− ρ(ζi,1)) ζ ′i,11[t ≥ ζi,1] if t ∈ [si,0, si,1)0 if t ∈ [si,1, si,2)
The sensitivity of ζi,1 is then obtained from either Proposition 3 or 5 and the result is obtained.Next, from Lemma 9 again, the sensitivity of y(t) with respect to θW is given by
y′θW(t) =
{1W[si,0] +
(µhi(1)(ζi,1)− µhi(0)(ζi,1)
)ζ ′i,11[t ≥ ζi,1] if t ∈ [si,0, si,1)
1W[si,1] if t ∈ [si,1, si,2)
As before ζ ′i,j = 0 if it corresponds to an ωx event. Next, to determine ζ ′i,1 when it coincides with aφx event we use Proposition 3 which implies ζ ′i,j = 0 or Proposition 5. From Proposition 5, againζ ′i,j = 0 because ξi,j indicate points where y(t) switches between empty and non-empty. Duringany NBP of y(t) (i.e., the interval we are considering here), no such events can occur. Thereforethe proof is complete.
References
[1] R. Kapuscinski and S. R. Tayur, “Optimal policies and simulation based optimization forcapacitated production inventory suystems,” in Quantitative Models for Supply Chain Man-agement (S. Tayur, R. Ganeshan, and M. Magazine, eds.), pp. 7–40, Kluwer, 1999.
[2] D. Bertsimas and I. C. Paschalidis, “Probabilistic service level guarantees in make-to-stockmanufacturing systems,” Operations Research, vol. 49, pp. 119–133, Jan-Feb 2001.
[3] C. Panayiotou, C. Cassandras, and P. Zhang, “On-line inventory cost minimization for make-to-stock manufacturing systems,” in Proceedings of the IEEE American Control Conference,pp. 4469–4474, 2002.
[4] C. Panayiotou and C. Cassandras, “Infinitesimal perturbation analysis for make-to-stock man-ufacturing systems based on stochastic fluid models,” in Proceedings of the IFAC Workshop ofDescrete Event Systems WoDES’04, 2004.
[5] I. C. Paschalidis, Y. Liu, C. Cassandras, and C. Panayiotou, “Inventory control for sup-ply chains with service level constraints: A synergy between large deviations and pertur-bation analysis,” in Annals of Operations Research, Special Volume on Stochastic Models ofProduction-Inventory Systems, vol. 126, pp. 231–258, 2004.
28
[6] C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems. Kluwer AcademicPublishers, 1999.
[7] H. Yu and C. Cassandras, “Perturbation analysis for production control and optimization ofmanufacturing systems,” Automatica, vol. 40, pp. 945–956, 2004.
[8] R. Suri and B. R. Fu, “On using continuous flow lines for performance estimation of discreteproduction lines,” in Proc. 1991 Winter Simulation Conference, pp. 968–977, 1991.
[9] C. G. Cassandras, Y. Wardi, B. Melamed, G. Sun, and C. G. Panayiotou, “Perturbationanalysis for on-line control and optimization of stochastic fluid models,” IEEE Transactionson Automatic Control, vol. 47, no. 8, pp. 1234–1248, 2002.
[10] S. Meyn, “Sequencing and routing in multiclass networks. Part I: Feedback regulation,” inProceedings of the IEEE International Symposium on Information Theory, pp. 4440–4445,2000. To appear in SIAM J. Control and Optimization.
[11] Y. C. Ho, R. S. Sreenivas, and P. Vakili, “Ordinal optimization in DEDS,” Journal of DiscreteEvent Dynamic Systems: Theory and Applications, vol. 2, pp. 61–88, 1992.
[12] G. Sun, C. Cassandras, Y. Wardi, C. Panayiotou, and G. Riley, “Perturbation analysis andoptimization of stochastic flow networks,” IEEE Transaction on Automatic Control, vol. 49,no. 12, pp. 2143 – 2159, 2004.
[13] H. J. Kushner and D. S. Clark, Stochastic Approximation for Constrained and UnconstrainedSystems. Berlin, Germany: Springer-Verlag, 1978.
[14] R. Y. Rubinstein and A. Shapiro, Discrete Event Systems: Sensitivity Analysis and StochasticOptimization by the Score Function Method. New York, New York: John Wiley and Sons,1993.
[15] C. Panayiotou, C. Cassandras, G. Sun, and Y. Wardi, “Control of communication networks us-ing infinitesimal perturbation analysis of stochastic fluid models,” in Lecture Notes in Controland Information Sciences (LCNCIS): Advances in Communication Control Networks (S. Tar-bouriech, C. Abdallah, and J. Chiasson, eds.), 2004.
29