F. Zavoral et al. (Eds.): NDT 2010, Part II, CCIS 88, pp. 172–183, 2010. © Springer-Verlag Berlin Heidelberg 2010

Solving the Problem of Flow Shop Scheduling by Neural Network Approach

Saeed Rouhani1, Mohammad Fathian2, Mostafa Jafari2, and Peyman Akhavan2

1 Department of Industrial Engineering, Firoozkoh Branch, Islamic Azad University, Firoozkoh, Iran

Srouhani@iust.ac.ir 2 Department of Industrial Engineering, Iran University of Science and Technology,

Tehran, Iran

Abstract. If there is a continuous flow of production jobs for some machines, the problem of flow shop scheduling arises. As mentioned in many researches, the complexities of this problem are of exponential kind; therefore it is neces-sary to design less complex methods or algorithms for solving it. In this paper, a new solution is presented for this kind of scheduling problem by using the idea of neural networks. In fact, this research is a response to the need for solving large and complex problems of this type by non-classical methods. The purpose of the paper is to create an artificial intelligence for doing this kind of schedul-ing via the neural network training process. Here, the neural network has been trained by using training data obtained from optimal sequence of solved prob-lems of flow shop scheduling. The trained network can provide a priority which shows the sequence of the job and will be very close to the optimal sequence.

Keywords: Artificial Intelligence, Neural Networks, Scheduling, Flow shop problem.

1 Introduction

Since the publication of the first paper in the field of flow shop by Johnson in 1954 [16], more than 1200 papers have been published about different aspects of this issue in the literature of operations research. The concept of this scheduling problem be-came common since Johnson's paper. However, the topic of "flow shop" was first applied in [14]. Based on fifty years of literature in the field of scheduling for these types of problems, we first have a look at different researches which have used heuris-tic and meta-heuristic methods for solving these kinds of problems. Along with the review of the literature, for having an exact definition of flow shop problem, the prob-lem is introduced and its assumptions and types are presented. Finally the applications of neural network for solving these kinds of problems as a limited and almost new area are introduced. In the final section, the experimental research done for training the network and its results are described.

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 173

2 Defining the Problem of Flow Shop

If there is a continuous flow of production jobs for some machines, the problem of flow shop scheduling arises. Some may explain this problem similar to an assembly line. However, there are some differences like the ability of flow shop in producing different combinations and sequences, the independence of machinery, different times of jobs and the possibility of not passing a job on one machine, which caused most of the researchers to distinguish this problem from assembly line. As it is seen, this prob-lem has a one-way flow and according to Johnson, the definition of the problem can be summarized as follows [24]:

• n jobs must be done on m machines with a fixed technological sequence. • The time needed for doing job i on machine j is called pij. • The purpose is to do all the jobs with a minimum cost.

In general, we may specify n!m sequences which is impossible to calculate. Therefore, in some production environments, the sequence of jobs could be assumed the same and the possible sequences could be reduced to n!. As mentioned in many researches, this type of complexity is of exponential type and it is necessary to design less com-plex methods or algorithms for solving it. Based on this fact and for reducing the complexity of problem, the following theory has been proved [4]:

For the problem of flow shop scheduling on m machines and for minimizing the overall completion time of jobs, we just need to obtain the sequence of jobs on the first two machines.

Also some assumptions have always been stated which most of them are based on the real world facts and therefore have made different states for this kind of schedul-ing problem. In practice, applying each of these assumptions makes the problem more complete and sometimes much more difficult to solve. Some of these assumptions are [1]: unpredictable stop and failure of a machine, delay in initial setup time of a ma-chine, the variance of work time and customer pressure. Gupta (1979) have defined the traditional problem of flow shop with 21 assumptions related to the nature of jobs, machinery and production policy [13]. In recent decades, many researchers have solved this type of scheduling problem by three types of exact, heuristic and meta-heuristic methods based on these assumptions and sometimes they have modified the assumptions for getting closer to real conditions. Thus, due to existing assumptions and different kinds of them, this type of scheduling problem could be known as a production model with lots of samples.

3 Researches of Past Five Decades on Solving Flow Shop

The initial group of researchers in the field of flow shop scheduling was very small and they did research about this field only in a few universities, but today a global association of researches has been formed for this kind of scheduling problem and many related papers are published by authors from different countries. After John-son's paper, initial researches about this problem were mainly related to mathematical issues of that initial model (for example see [25]). The reason that the problem and its solution methods were not much developed could be related to the fact that it was

174 S. Rouhani et al.

NP-hard and computer powers were low at that time. Most of the papers published early in that decade contain discussions on only 2 or 3 numbers of machines.

In the second decade of flow shop problem between 1965 and 1974, on the one hand different solutions and on the other hand objective functions except total time were introduced. Branching and reduction method for solving this kind of problem was first applied by [17, 2 & 18]. The introduced Branching and reduction method, which acted based on lower bound, was finally turned into a developed Branching and reduction recitation framework for this kind of scheduling problem. Gupta (1972) introduced objective functions different from total time and compared the results with previous objective function [12]. Late in that decade, the complexity of problem solv-ing for finding the optimal solution caused a movement towards good solutions and heuristic methods [9].

The scientific emergence of NP-complete theory, which is now known as NP-hard, had a deep effect on developing flow shop scheduling problem [10]. In the third dec-ade of researches which was between 1975 and 1984, on the one hand many efforts were made to solve for complexity of these kinds of problems [3], and on the other hand a lot of heuristic methods were invented for solving this kind of problems [19].

During this decade, different types of flow shop scheduling problems were also ex-tended and sometimes in various papers and researches the mentioned assumptions were not taken into account and different states were considered. As an example, during this period, researchers got interested in the concept of the worst performance of a heuristic algorithm and highlighted this issue in their papers. Furthermore, con-sidering the time related to the objective functions in this problem became important. Finally, the third decade could be known as the time for bringing up the issue of prob-able work times. In this decade more work was done on probability modeling of this kind of problems but not on solving them. Dempster (1982) did his research on differ-ent kinds of probability models [5].

The fourth decade of researches was between 1985 and 1994. During this decade, the problem of hybrid flow shop scheduling was defined in which every stage of the problem could include parallel machines. However, what was mainly observed during this decade was the invention of meta-heuristic methods such as Tabu Search, Genetic Algorithms and Simulated Annealing. Many people with algorithm specialization were attracted to the industrial area of production planning [20].

According to assumptions related to independent and dependant setup times which was studied more in this decade and also because of using primary artificial intelli-gence in the form of expert systems and decision support systems in this field, the fourth decade could be known as the brightest decade of researches in the field of flow shop scheduling.

Years between 1995 and 2004 form the fifth decade of researches in the field of flow shop scheduling. In this decade, the variety of problems and objective functions and also the invention of many different methods are seen which almost do not follow a focused orientation. During this decade, considering the problem of batch size and job scheduling simultaneously on the one hand and taking into account batch process-ing machines on the other hand helped to link the problem of flow shop scheduling to other production planning problems [21].

Ruiz et al, (2005) did a broad research on effectiveness and efficiency of different kinds of meta-heuristic methods with separated sequence and dependent setup times

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 175

[22]. Issues, like flow shop robotics, were also brought up as the edge of the science in the end of this decade [11].

4 Recent Researches

Today, many fundamental, review and applied papers are submitted to famous monthly journals in the field of flow shop scheduling. Researches of recent year (2008) can be summarized into some categories as follows:

• Problems related to two machines • Two jobs on m machines • Flow shop with setup time • Complexity and performance • Meta-heuristic approaches for problem solving

The last case could be related directly to the course topic of meta-heuristic methods; therefore, the literature of this case will be reviewed here.

4.1 Meta-heuristic Approaches

Hybrid optimization approaches are usually used to solve flow shop scheduling prob-lems. Agarwal, Colak, and Eryarsoy described a developed heuristic (meta-heuristic) approach to solve for the traditional flow shop scheduling problem. This approach starts with an initial solution and similar to neural networks finds an appropriate solu-tion by a learning method [15].

Bagchi, Gupta, and Sriskandarajah reviewed solution techniques of flow shop problem based on TSP1 method [11, 12 &13]. They showed that TSP techniques could be appropriate for solving complex problems of this field of scheduling. In fact, they linked flow shop to the TSP family and facilitated it for presenting more meta-heuristic methods.

Researches of recent fifty years could be categorized into three types of researches: theoretical researches, mathematical researches and experimental researches. It can be predicted for sure that with enhancement of computers power in future years, aston-ishing results in theoretical and mathematical researches will emerge.

4.2 Applications of Neural Networks in Flow Shop Problem

In recent years, technological improvements in software and hardware have helped to the emergence of applied tools such as neural networks, which are used in solving complex problems. These networks are mathematical structures, which can learn different tasks by using simple biological models. Based on their extensive applica-tion, these networks have also been used in hybrid optimization and scheduling prob-lems. In scheduling literature, finding the relation between data (work times and etc.) and sequences of jobs, specifying optimal sequence of jobs and determining the strat-egy for distribution of jobs (scheduling rules) are some of the applications of neural networks in different areas of scheduling. 1 Travelling Salesman Problem.

176 S. Rouhani et al.

Due to the learning attribute between input and output variables in artificial neural networks, some researchers of this field got interested in the idea of replacing these networks with a set of heuristic and meta-heuristic models. Derya Eren Akyol (2004) simulated six heuristic methods of flow shop scheduling by applying a multi-layer forward neural network and by using a set of training data (completion times, idol times of machinery, total work time), which were obtained from one month effort and the sequences of heuristic methods [6]. It can be stated that the obtained neural net-work could be used instead of the six mentioned methods. In this research this neural network could predict the total completion time of jobs using the input data. However, generation of the optimal sequence or an initial solution was not introduced.

El-Bouri and et al, presented a paper in 2000 which included an interesting idea for application of neural networks in scheduling problems. In this research, classification application of neural network for separating different states of scheduling problem was represented in the first stage and the estimation application for presenting the sequence of jobs was showed in the second stage [7].

This approach was applied in the second paper of El-Bouri and et al (2005) in the form of scheduling phase on flow shop scheduling problem [8]. In this paper, the neural network is trained using training data obtained from optimal sequences of solved problems of flow shop scheduling. The trained network can provide a priority which shows the sequence of the job and will be very close to the optimal sequence. It is represented that local searching methods, which use the neural network solutions, have a better quality and speed in comparison with other methods [23].

One of the important points of this paper is the definition of input and output lay-ers. Since the information related to work time of machines are not enough to estimate and predict the sequence of one job and this sequence is a dependant problem, all the information related to jobs, times and machines should be given to the network via the input layer and in turn one number, which shows the priority of the job in the se-quence of operations, will be received from the network.

Fig. 1. Neural Job Classification and Sequencing System, NJCASS [8]

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 177

In general 3m inputs are given to the network: work time on m machines, average time needed for the job of each machine, and standard deviation of work time of each machine. The network has 3m-h-1 architecture and the number of neurons of hidden layers are determined by trial and error.

Although the idea of the paper is very smart, one of its main drawbacks is that it does not present a method for data generation and network training and it is not clear how the trained data and optimal sequences are obtained. The network is not flexible for the number of machines and jobs and probably it should be retrained for each number of machines and jobs. The two main papers in the field of direct application of neural networks for flow shop scheduling problem in recent years were reviewed. It can be concluded that following approaches exist in this field:

• Using neural networks as a replacement and imitator of heuristic and meta-heuristic methods for solving flow shop scheduling with more emphasis on prediction of completion time of total jobs.

• Applying neural networks in classifying different kinds of flow shop schedul-ing problems, and offering solution or providing the sequence.

• Using neural networks for learning optimal sequences and providing initial effective solutions for local searching methods and other meta-heuristic methods.

• Combining neural networks with other methods for creating hybrid meta-heuristic methods specific to flow shop scheduling problem.

5 Proposed Approach

If there is a continuous flow of production jobs for some machines, the problem of flow shop scheduling arises. Some may explain this problem similar to an assembly line. However, there are some differences like the ability of flow shop in producing different combinations and sequences, the independence of machinery, different times of jobs and the possibility of not passing a job on one machine, which caused most of the researchers to distinguish this problem from assembly line. As it is seen, this prob-lem has a one-way flow and according to Johnson, the definition of the problem can be summarized as follows:

• n jobs must be done on m machines with a fixed technological sequence. • The time needed for doing job i on machine j is called Pij. • The purpose is to do all the jobs with a minimum cost.

In this paper, a new solution is presented for this kind of scheduling problem by using the idea of neural networks. In fact, this research is a response to the need for solving large and complex problems of this type by non-classical methods. The purpose of the paper is to create an artificial intelligence for doing this kind of scheduling via the neural network training process.

5.1 Generating Training Data

It is necessary to use appropriate and optimal data for generating the data of the prob-lem, so that the network can be trained to generate the sequence of production by

178 S. Rouhani et al.

using that data. These data are the same as the data of flow shop scheduling and opti-mal sequence problem (work time of each machine for each job).

To generate these data, a sample problem was chosen in which 6 jobs must be planned on 5 machines. It should be noted that this approach can be applied for prob-lems with larger sizes. In fact, for each m in n combination, a trained optimal network is created. The following example represents a sample of data set (table 1)

Table 1. Sample of data set

machines jobs M1 M2 M3 M4 M5

J1 48 48 89 83 10

J2 54 59 7 51 34

J3 6 68 8 20 44

J4 74 54 67 64 65

J5 70 35 51 4 95

J6 24 41 12 15 57

To train the network in the above problem, the optimal sequence of jobs from the

view of Cmax or total work time should be taught to the network. For this purpose, the optimal sequence for training data should be obtained by a reliable optimal se-quence generator method. According to the sizes of this sample problem, the optimal sequence can be obtained by using WinQSB software and by full numeration tech-nique. The solution will be as follows:

Table 2. Solution for sample of data set

jobs J1 J2 J3 J4 J5 J6

Priority 6.00 4.00 1.00 3.00 5.00 2.00

Fig. 2. Optimal sequence for jobs

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 179

Fig. 3. Optimal sequence for machines

5.2 Preparing Data for Giving to the Neural Network

As the problem requires, it is not enough to have the operation times of one job on each machine for obtaining the sequence of that job, but this sequence or priority is completely dependent on other times and in fact dependant on the whole matrix. However, the neural networks also have the structure of input nodes. These nodes cannot accept the matrixes; thus, all the important information for each job should be entered in some way as an input vector and in turn, the optimal priority of that job should be trained to the network.

Using the proposed method [8], for each job beside the operation time of that job on each machine, the average operation time of jobs on each machine and standard deviation of operation time of jobs for each machine are entered as input. By this way, the whole information of the matrix is summarized in the input vector of each job. Therefore, the input structure of the network is also formed three times larger than the number of machines (m):

5.3 The Architecture of Neural Network

The applied neural network is a multi layer, forward and fully connected neural net-work. The forwardness property of this network helps it not to have loop or to return backwards. This network also composes of three layers: input, hidden and output layers; therefore, it is a multi-layer network and since each node is connected to the all nodes of the next layer, it is called a fully connected network. There are 15 input nodes, 10 neurons of hidden layer (obtained from trial and error) and one output node in this network.

180 S. Rouhani et al.

Fig. 4. The architecture of designed neural network

5.4 Introduction of Data Sets for Training, Validating and Testing

108 records of data have been generated for this research. Out of these 108 records, 72 records are considered for training set, 18 for validation set and 18 for test set. By using the validation and test sets, it can be ensured that the network is generalized and will not be over fitted. The selected activation functions are Tansig hidden layer and Purelin output layer and training algorithm is Levenberg-Marquardt (trainlm).

5.5 Results of the Research

Due to randomness, training, validating and testing of the network was done 20 times. The results are as follow.

FinalR = 0.9157 FinalRt = 0.7444

1 2 3 4 5 6 7 80.3

0.4

0.5

0.6

0.7

0.8

0.9

1train Retio

Fig. 5. Status of output convergence criteria of the network and training data

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 181

0 2 4 6 8 10 120

0.5

1

1.5

2

2.5

3

3.5

Squ

ared

Err

or

Epoch

TrainingValidationTest

Fig. 6. Comparison of error between three sets of training, validating and testing data

6 Conclusions

By reviewing researches in the field of flow shop scheduling problems, we can con-clude that these researches have been done in areas of modeling, problem extending, offering exact solution methods, designing heuristic methods, and applying meta-heuristic and hybrid strategies for reaching to intended goals and solutions.

In recent years, based on problem extensions, many heuristic and meta-heuristic methods have been developed by the help of enhanced processing technologies of computers, and they have managed to play an important role in obtaining appropriate solutions with suitable criteria for this kind of scheduling problem. Therefore, neural networks, which is one of the areas related to meta-heuristic methods, has been ap-plied in a few researches of this problem; but in recent years, this technique has been welcomed more. In the fifty-year history of researches in the field of flow shop scheduling problem, this method has hardly been applied; however, the method has been applied very effectively in these few researches and has caused a new wave to start in using this method for flow shop scheduling problems. We hope that research-ers will answer to challenges and problems of industrial world by continuing this path and by bringing optimization from theoretical problems into the complicated world of business.

This research managed to provide an artificial neural network which can solve the flow shop scheduling problem optimally by the rate of 90% for the size of 6 jobs and 5 machines. The approach of this paper can be used in designing an engine for sched-uling software in small sizes. Future studies should try to delete the constraint of problem sizes and to train the network for different problems. As additional re-searches for comparison of results it is possible to recommend the using of a recurrent neural network to solve the declared problem.

182 S. Rouhani et al.

Acknowledgements

The authors are especially grateful for the helpful comments and edition comments provided by Mr. Mahmood Rouhani and Dr. Farnaz Barzinpour.

References

1. Ashour, S.: Sequencing Theory. Springer, Berlin (1972) 2. Brown, A.P.G., Lomnicki, Z.A.: Some applications of the branch and bound algorithm to

the machine scheduling problem. Operational Research Quarterly 17, 173–186 (1966) 3. Brucker, P.: Scheduling Algorithms. Springer, Berlin (1998) 4. Conway, R.L., Maxwell, W.L., Miller, L.W.: Theory of Scheduling. Addison Wesley,

Reading (1967) 5. Dempster, M.A.H., Lenstra, J.K., Rinnooy Kan, A.H.G. (eds.): Deterministic and Stochas-

tic Scheduling. Reidel, Dordrecht (1982) 6. Akyol, D.E.: Application of neural networks to heuristic scheduling algorithms. Computers

& Industrial Engineering 46, 679–696 (2004) 7. El-Bouri, A., Balakrishnan, S., Popplewell, N.: Jobs on a single machine: A neural net-

work approach. European Journal of Operational Research 126, 474–490 (2000) 8. El-Bouri, A., Balakrishnan, S., Popplewell, N.: A neural network to enhance local search

in the permutation flowshop. Computers & Industrial Engineering 49, 182–196 (2005) 9. Framinan, J.M., Gupta, J.N.D., Leisten, R.: A review and classification of heuristics for

permutation flow-shop scheduling with makespan objective. Journal of the Operational Research Society 55, 1243–1255 (2004)

10. Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-completeness. W.H. Freeman and Company, San Francisco (1979)

11. Geismar, H.N., Dawande, M., Sriskandarajah, C.: Robotic cells with parallel machines: Throughput maximization in constant travel-time cells. Journal of Scheduling 7, 375–395 (2004)

12. Gupta, J.N.D.: Heuristic algorithms for multistage flowshop scheduling problem. AIIE Transactions 4, 11–18 (1972)

13. Gupta, J.N.D.: A review of flowshop scheduling research. In: Ritzman, L.P., Krajewski, L.J., Berry, W.L., Goodman, S.T., Hardy, S.T., Vitt, L.D. (eds.) Disaggregation Problems in Manufacturing and Service Organizations. Martinus Nijhoff, The Hague, pp. 363–388 (1979)

14. Ignall, E., Schrage, L.: Application of branch-and-bound technique to some flow shop problems. Operations Research 13, 400–412 (1965)

15. Gupta, J.N.D., Stafford Jr., E.F.: Flowshop scheduling research after five decades. Euro-pean Journal of Operational Research 169, 699–711 (2006)

16. Johnson, S.M.: Optimal two- and three-stage production schedules with setup times in-cluded. Naval Research logistics Quarterly 1, 61–68 (1954)

17. Lomnicki, Z.A.: A branch and bound algorithm for the exact solution of the three machine scheduling problem. Operational Research Quarterly 16, 89–100 (1965)

18. McMahon, G.B., Burton, P.G.: Flowshop scheduling with the branch and bound method. Operations Research 15, 473–481 (1967)

19. Morton, T.E., Pentico, D.W.: Heuristic Scheduling Systems. Wiley, New York (1993) 20. Osman, I.H., Kelly, J.P.: Meta-heuristics: Theory and Applications. Kluwer Academic

Publishers, Boston (1996)

Solving the Problem of Flow Shop Scheduling by Neural Network Approach 183

21. Potts, C.N., Kovalyov, M.Y.: Scheduling with batching: A review. European Journal of Operational Research 120, 228–249 (2000)

22. Ruiz, R., Maroto, C., Alcaraz, J.: Solving the flowshop scheduling problem with sequence dependent setup times using advanced metaheuristics. European Journal of Operational Research 165, 34–54 (2005)

23. Sabuncuoglu, I., Gurgun, B.: A neural network model for scheduling problems. European Journal of Operational Research 93, 288–299 (1996)

24. Tanaev, V.S., Sotskov, Y.N., Strusevich, V.A.: Scheduling Theory: Multi-stage Systems. Kluwer Academic Publishers, Dordrecht (1994)

25. Wagner, H.M.: An integer linear-programming model for machine scheduling. Naval Research Logistics Quarterly 6, 131–140 (1959)