repair and replacement decisions for warranted products under markov deterioration

368 IEEE TRANSACTIONS ON RELIABILITY, VOL. 59, NO. 2, JUNE 2010

Repair and Replacement Decisions for WarrantedProducts Under Markov Deterioration

Yue Pan and Marlin U. Thomas

Abstract—An important problem in maintenance planning is therepair versus replacement problem which consists of a sequence ofdecision points over time, whereby decisions are made to repair orreplace a machine based on its condition. This problem can be fur-ther complicated by prevailing failure characteristics, and costs.The failure modes for many machines are such that failure emergesthrough a series of deterioration levels rather than as an abruptevent. Zuo et al. [12] presented a policy for multistage deteriora-tion for machines under warranty. This paper extends that workby treating a larger, more general state space with time parametersat each state. This approach allows for a more cost effective policy.We consider a piece of equipment, such as a production machinethat is reviewed for repair verses replacement at various times. Themachine is sold with a free repair warranty (FRW) policy. So if itfails during the warranty period, it is maintained at the expenseof the manufacturer. Two kinds of repairs are adopted: minimalrepair, and replacement. We assume that there is no form of main-tenance provided between failures, and the manufacturer decideson the repair action to be taken once a failure and warranty claimis filed. The approach is to model the process as a continuous timeMarkov Chain. The state decision variables are based on the ex-pected costs to the manufacturer for failures and repairs occurringduring the warranty period. The criterion for an optimum policy isto minimize the expected total cost to the manufacturer during thewarranty period. A development of the method is provided for thecase of three functioning states. Under the policy, if a machine failsearly in the warranty period, and it’s deterioration before failureis large, it is economic to replace that machine with a new one.

Index Terms—Deterioration, free repair warranty, minimal re-pair, multi-state, replacement.

ACRONYM

FRW Free repair warranty

NOTATION

N Number of functioning states

w Length of warranty period

Rate of transition from state i,

Probability that the machinemakes transition from state i tostate j,

Manuscript received November 08, 2008; revised September 05, 2009; ac-cepted December 02, 2009. Date of current version June 03, 2010. AssociateEditor: L. Walls.

Y. Pan is with the Department of Industrial Engineering and the Schoolof Industrial Engineering, Purdue University, West Lafayette, IN 47907 USA(e-mail: [email protected]).

M. U. Thomas is with the Air Force Institute of Technology, Wright PattersonAFB, OH 45433-7765 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TR.2010.2048731

Cost of minimal repair given themachine is in state i before failureCost of replacement given themachine is in state i before failureExpected cost to the manufacturergiven the machine is currentlyin functioning state i or a higherstate at time tTransition probability of themachine currently in state i beingin state j after an additional time t

P(t) , matrix of transitionprobability functionsRate matrix for the process

, state transitionprobability density functionsDecision parameters,

Expected total cost to themanufacturer during warrantyperiod (0,w)

I. INTRODUCTION

T HE REPAIR versus replacement decision problem is avery old maintenance issue that dates back to the 1960s

[2]. The basic problem is to determine the minimum cost policyfor maintenance and replacement decisions for machines withincreasing maintenance costs due to deterioration over time.

Derman & Sacks [3] first considered a problem in whichthe amount of system deterioration during a time intervalwas described by a sequence of -independent, identicallydistributed, nonnegative random variables. The maintenanceactions, hence, were taken when accumulated deteriorationof the system reached a certain level. Derman [4] furtherinvestigated a multi-state model, where the deterioration of thesystem was described as the movement from state to state bya Markov chain. states were involved to represent theascend level of deterioration of the system, with state 0, and

denoting new, and fail states, respectively. The maintenancepolicy was to replace the system iff the observed state is ,

. Klein [7] generalized the Derman [4] model byallowing a variety of repair actions including replacement. Lam& Yeh [9] summarized the general assumptions in the study ofdeteriorating systems, and presented algorithms for differentmaintenance strategies. More studies about Markovian deterio-ration systems can be found in Kolesar [8], Anderson [1], andWood [10].

0018-9529/$26.00 © 2010 IEEE

PAN AND THOMAS: REPAIR AND REPLACEMENT DECISIONS FOR PRODUCTS UNDER MARKOV DETERIORATION 369

Fig. 1. State transaction diagram.

Maintenance planning and scheduling problems are by theirnature difficult, and can be even more challenging when war-ranties are included in the decision alternatives. Zuo et al. [11]was among the first to study maintenance actions under Markovdeterioration with warranty. That paper developed a repair/re-placement decision model for a machine which deteriorates ac-cording to a continuous parameter Markov chain gradually fromthe initial good-as-new condition at state 1, through transitionfrom state i to state , or from state i to the failed state, for

. The conditions for optimal repair or replacementdepend on the level of deterioration, and residual warranty time.

This paper extends the model in Zuo et al. [12] by consideringshocks, which extends beyond one functioning state. A policywith decision parameters for a machine with func-tioning states is proposed to choose minimal repair or replace-ment in case of failure during the warranty period. This main-tenance policy may apply to systems such as freeways, roads,airfield runways, or automobiles that are composed of subsys-tems or components that can deteriorate during warranty time.It also applies to military applications or medical devices thatincur quite different maintenance costs under different workingconditions.

We start in Section II with a description of the model. Ourapproach is to model the process as a continuous time MarkovChain with discrete states representing the level of deteriorationof the system. In Section III, an algorithm for choosing appro-priate maintenance actions is proposed, and shown by an ex-ample of a four state machine. These results are generalized for

states in Section IV. Section V provides some directions forfurther work.

II. MODEL DESCRIPTION

We consider a piece of equipment such as a large productionmachine that is reviewed for repair versus replacement at var-ious times under a prescribed maintenance plan. The machineis initially procured with a free repair warranty policy. So forfailures that occur during the warranty period (0, w], the repairor replacement expenses are incurred by the machine manufac-turer. We assume that the condition of the machine system canbe described by a Markov Chain with state space

representing the ascending level of deteriora-tion of the machine. An element fails through a deteriorationprocess that gradually builds up with operation time, and can berepresented by discrete states. State 1 corresponds to the newor good-as-new condition with subsequent higher functioningstate numbers suffering greater deterioration until failure at state

N. Under the Markov chain assumption, once the machine en-ters state , it stays in that state for a random length of time,which is exponentially distributed with parameter . We fur-ther assume that the machine cannot achieve a less deterioratedcondition unless maintenance occurs. Once the machine entersstate , it can only make a transition to a more deteriorated state

with probability . The transition diagramfor the process is shown in Fig. 1.

The maintenance policy studied in this paper is correctivemaintenance, and the maintenance actions are minimal repair,and replacement. That is, for a failure that occurs during a war-ranty period, the machine shall be either minimally repaired, orreplaced by a new machine, with actions chosen by the manu-facturer. Under such a condition, for a machine in state j beforefailure, minimal repair restores its working condition to state j,the as bad as old condition; while replacement restores it to state1, the as good as new condition.

Because it is the manufacturer who decides which mainte-nance actions to take when a warranty is claimed, the criterionof making decisions is based on minimizing the expected totalmaintenance cost to the manufacturer during the warranty pe-riod. The policy for deciding on minimal repair versus replace-ment during a fixed warranty period is as follows.

A failed machine at age t that is in functioning state ,, , is replaced by a new machine if

; otherwise, it is minimally repaired.

A sample realization of this process illustrating the decision pa-rameters is shown in Fig. 2.

We further assume the following.1) Costs of minimal repair and replacement are non-de-

creasing. , and, where , and are, respectively, the

cost of replacement, and cost of minimal repair given thatthe machine failed in state , . These costs arefixed values known to the manufacturer. Moreover, thecost of replacement is no less than the cost of minimalrepair for the same failure state, . It is thereforenot necessarily economical to replace the machine oncea failure occurs without considering alternative minimalrepair action.

2) There are no maintenance actions between failures. Inother words, neither the manufacture nor the customerwill adopt preventive maintenance actions that restore themachine’s working condition from state j to state i, where

.


Fig. 2. Minimal repair and replacement policy in warranty period �� .

3) The manufacturer has sufficient information through phys-ical measurements, e.g. vibration trends, and expert judg-ment, to determine which functioning state the machinewas in before failure.

4) Failure is detected immediately. The time of minimal re-pair or replacement is relatively short compared to themean time between failures, and hence is negligible. Thecosts associated with failure other than the cost for main-tenance are negligible.

The maintenance policy under a free repair warranty (FRW) arecharacterized by parameters , with

. Under such a maintenance policy,a machine that fails early in the warranty period with a largedeterioration will be replaced by a new one; meanwhile, it willbe minimally repaired if the remaining warranty time is less.

III. THE FOUR STATE MACHINE PROBLEM

Consider the case of a machine with states. State 1is in a good as new condition, and state 4 is the failed state. Asdescribed in Section II, the maintenance policy during warrantyis that if the machine fails during functioning state i at age t, itis replaced by a new machine at a cost to the manufacturer of

if ; otherwise, it is minimal repaired. The expectedtotal cost to the manufacturer for the machine currently in state

, or 3 at time t is given by

(1)where is the state transition probability density functionof the machine currently in state i being in state j after a periodof time x. For the case of , the first term is the expectedcost of replacement in state i expressed as a renewal equation,and the second term is the expected cost for failures that occurin a higher state.

For , a failure from state I will result in a replacementof the machine if it occurs in , or a minimal repair if itoccurs in . The manufacturer will accordingly incur cost

, or . Therefore, the first term on the right side of (1) foris the expected cost of replacement in state i, and the last

term is the expected cost for failures occurring in higher states.With state 4 being the failure state, it follows that when the

machine is working in state 3, the expected cost is given by

.(2)

For a free repair fixed warranty over the period (0, w], theproblem is

(3)

In other words, determine values of the decision parametersthat will provide a minimum cost for the manu-

facturer. As noted previously, will always be 0 because themachine is new in state 1, and hence there is no reason to replaceit. A closed form solution to (2) is difficult due to the cumber-some renewal equations in (1). However, an optimal solutioncan be derived through the following procedure.

A. Solution Procedure

An optimal solution can be derived by systematically enu-merating through choices of alternative sequences of the deci-sion parameters , computing the expected costs from (1), andselecting the choices of values that give the minimal totalcost. To illustrate the procedure, consider the case ofstates shown in Fig. 3. The expected costs and decision param-eters are derived sequentially as follows.

1) Develop the state transition probability density functionsby solving the Kolmogorov forward equations

(4)


Fig. 3. Parameters determining for � � �. (a) Decision Parameters Determining with Sequence �� . (b) Decision parameters determining with sequence�� .

to derive

2) Derive values , that result in a minimal expected totalcost .(a) Step 1. Determine . The machine is replaced by a

new one if it is of age t, , and it is in a func-tioning state 3 before failure; otherwise, it is mini-mally repaired. is the parameter determined in thisstep by minimizing the expected total cost function

from the following.

(5)

(6)

(7)

(b) Step 2. Determine . The machine is replaced by anew one if it is in functioning state 3, or it is in func-tioning state 2 with age t, , before failure; oth-erwise, it is minimally repaired. is determined in

this step by minimizing the expected total cost func-tion from in (5), and the following.

(8)

(9)

(c) Compute the expected total cost .3) Derive values , that result in a minimal expected

total cost .(a) Step 3. Determine . The machine is replaced by a

new one if it is of age t, , and it is in functioningstate 2 or 3 before failure; otherwise, it is minimallyrepaired. Derive by minimizing the expected totalcost function from in (5), in (9),and the following.

(10)(b) Step 4. Determine . The machine is replaced by a

new one if it is of age t, , and it is infunctioning state 3 before failure; otherwise, it is min-imally repaired. is derived by minimizing

in (4), from which can be easily de-termined.

(c) Compute the expected total cost, .4) Select the respective parameter values that provide the min-

imum expected total costs from Steps 1(c), and 2(c).


B. Example

To illustrate the procedure described in Section III-A, con-sider a four state machine under a 20-month FRW policy thathas state transition rates ; minimal repaircosts , , and ; replacementcosts , and ; and transition probabili-ties , , and .

We start by determining the transition probability densityfunctions by applying the Kolmogorov forward differentialequations

(11)

where is the probability that the machine deteriorationcondition makes a transition from state to state after time .From the state transition diagram in Fig. 1, the generator matrixis given by

It is easily shown from (8) that the process is described by thefollowing system of differential equations.

(12)

Taking Laplace transformations through (12), (5), (6), and (7),we get

(13)

and

(14)

Combining (13) with (14), it follows that

which on inversion leads to the expected cost to the manufac-turer given the machine is in a functioning state 1 or higher attime t given by

(15)

Substituting (15) into (7) for the case of , and furtherinto (12) and (13), leads to the equation shown at the bottom ofthe page. On setting , it follows that

.Proceeding similarly to determine the second decision pa-

rameter, , thus leading to the total expected cost. Following the same procedure

for evaluating the sequence results in ,, with the corresponding expected total cost

. Therefore, after comparingthe two costs, the solutions are , and .The warranty maintenance policy for this machine is to replacethe failed item by a new one if it is in deterioration state 2before failure with an age less than 10.26, or in state 3 before


failure with an age less than 18.77. Otherwise, the failed item isminimally repaired. The expected total warranty cost is 207.76.

IV. GENERAL CASE

We advocate the same solution procedure for the general case.For a machine with functioning states, where state is thefailure state, the decision parameters are , (is always equal to 0). These parameters are derived se-quentially. The rule to determine is that a failed machine isreplaced by a new one if it is in a state higher than beforefailure, or if it is in state with an age ; otherwise, thefailed machine is minimally repaired. Given the largest pa-rameter less than , and the smallest one greater thanamong those parameters already determined, we get byminimizing shown in

for

for

The optimum policy follows by solving this system of cost equa-tions sequentially to derive the parameter values that lead to aminimum total expected cost.

V. CONCLUDING REMARKS

A continuous parameter Markov chain model has been de-veloped for a warranty action determination policy for a ma-chine with deterioration under a fixed period free repair war-ranty. Warranty actions considered here are minimal repair, andreplacement, which are selected by the manufacturer to min-imize the expected total cost that occurs to the manufacturerduring the warranty period. The policy is characterized byparameters for a machine in which the working condition is di-vided into N states according to the deterioration level.

This paper extends the maintenance action selection policy inthe paper by Zuo et al. [2] in two ways. First, more deteriorationlevels are considered, such as shocks that produce perturbationsbut are not fatal. Second, with decision parameters for an

-state system, this policy can also be applied to machines thatincur quite varying costs under different failure conditions.

There are several directions that can be considered for furtherstudy in maintenance and warranty planning. Here, the deterio-ration consisted of a series of discrete states. It could be morepractical to represent the deterioration using continuous vari-ables. Other forms of the distribution of sojourn times in func-tioning states might also be more appropriate for representingparticular types of machine deterioration. In this paper, all in-formation pertinent to the current state of the machine was as-sumed to be known. One direction for further work would be torelax this assumption, and treat the process as a partially observ-able Markov chain. Other extensions and generalizations can bederived from different types of policies such as opportunisticreplacement and renewing warranties [5], [11], and preventivemaintenance policies [6].

REFERENCES

[1] M. Q. Anderson, “Monotone optimal preventive maintenance policiesfor stochastically failing equipment,” Naval Research Logistics Quar-terly, vol. 28, pp. 347–358, 1981.

[2] R. E. Barlow and L. Hunter, “Optimum preventive maintenance poli-cies,” Operations Research, vol. 8, no. 1, pp. 90–100, 1960.

[3] C. Derman and J. Sacks, “Replacement of periodically inspected equip-ment (an optimal optional stopping rule),” Naval Research LogisticsQuarterly, vol. 7, no. 4, pp. 597–607, 1960.

[4] C. Derman, “On optimal replacement rules when changes of state areMarkovian,” in Optimal Decision Processes, B. Richard, Ed. : theRAND Corporation, 1963, pp. 201–212, R-396-PR.

[5] B. P. Iskandar and H. Sandoh, “An opportunity based age replace-ment policy considering warranty,” International Journal of Reliability,Quality and Safety Engineering, vol. 6, no. 3, pp. 229–236, 1999.

[6] C. S. Kim, I. Djamaludin, and D. N. P. Murthy, “Warranty and discretepreventive maintenance,” Reliability Engineering and Systems Safety,vol. 84, no. 3, pp. 301–309, 2004.

[7] M. Klein, “Inspection-maintenance-replacement schedules under Mar-kovian deterioration,” Management Science, vol. 9, no. 1, pp. 25–32,1962.

[8] P. Kolesar, “Minimum cost replacement under Markovian deteriora-tion,” Operations Research, vol. 12, pp. 694–706, 1966.

[9] C. T. Lam and R. H. Yeh, “Optimal replacement policies for multistatedeteriorating systems,” Naval Research Logistics Quarterly, vol. 41,pp. 303–315, 1994.

[10] A. P. Wood, “Optimal maintenance policies for constantly monitoredsystems,” Naval Research Logistics Quarterly, vol. 35, no. 1, pp.461–471, 1988.

[11] R. H. Yeh, G. C. Chen, and M. Y. Chen, “Optimal age-replacementpolicy for nonrepairable products under renewing free replacementwarranty,” IEEE Transactions on Reliability, vol. 54, pp. 92–97, 2005.

[12] M. J. Zuo, B. Liu, and D. N. P. Murthy, “Replacement-repair Policy forMulti-state Deterioration Products Under Warranty,” European Journalof Operational Research, vol. 123, pp. 519–530, 2000.

Marlin U. Thomas is Dean, Graduate School of Engineering and Manage-ment, Air Force Institute of Technology, past Professor and past Head of theSchool of Industrial Engineering at Purdue University. He received his B.SE. atthe University of Michigan-Dearborn, and MSE and Ph.D. at the University ofMichigan. His research interests are in operations research with applications inreliability and contingency logistics. He has received numerous honors for hisresearch and leadership in industrial engineering including the 2008 IIE Frankand Lillian Gilbreth Award. He is a Fellow of the American Society for Quality,the Institute for Operations Research and Management Science, and the Insti-tute of Industrial Engineers.

Pan Yue us a graduate student at Purdue University. She received a BE degreein industrial engineering, and a BA degree in English for Science and Tech-nology in 1998 at Tianjin University. She received her MS degree in industrialengineering at Purdue University in 2002.

repair and replacement decisions for warranted products under markov deterioration

Documents