A Stochastic Model for Military Air-to-Ground Munitions Demand Forecasting

Ahmed Ghanmi Defence Research and Development CanadaCentre for Operational Research and Analysis

Ottawa, Ontario, Canada, K1A 0K2 ahmed.ghanmi@forces.gc.ca

Abstract— Planning for military operations involves the consideration of a number of factors, including the required quantity of munitions. As these items typically require lengthy procurement lead times, stockpile levels must be established and maintained in anticipation of such operations. One of the key aspects of stockpile inventory management is demand forecasting. In this paper, we examine the problem of modeling and forecasting the munitions demand in military operations. We consider a continuous state space time series approach and we model the munitions demand as a non-homogeneous Poisson process with intensity function that depends on a continuous-time Markovian operational tempo process. Such a model is known in stochastic process as a Markov-Modulated Poisson Process and has been successfully applied to queuing and communication network problems. We apply the maximum likelihood estimation method to derive the MMPP model parameters. We illustrate the method with an example based upon air-to-ground munitions. This study provides military planners with a decision support method for forecasting munitions requirements in the face of abrupt changes in demand.

Keywords— military, munitions, demand forecasting, air-to-ground, stochastic process.

I. INTRODUCTION Inventory management is critical in military decision

support. Planning for ammunition stock levels for military operations in particular is a complex problem, due to the uncertainty associated with the demand of munitions and the operational tempo (i.e., intensity levels of military operations) [1]. In the military domain, low probability events (e.g., an earthquake that destroys the ammunition stock) could have serious consequences on operations.

The availability of the appropriate quantity of stock can improve the effectiveness, responsiveness, and readiness of the military. On the other hand, excessive stock (particularly ammunition) can lead to significant monetary losses due to stock damages, degradation, and obsolescence. Therefore, determining the optimum stock levels is a fundamental requirement for military inventory management [2].

One of the key aspects of stockpile inventory management is demand forecasting. Traditional approaches for determining stockpile levels for military operations include [1]: target-oriented and level of effort methods (static methods). The target oriented method determines the quantity of munitions required to defeat enemy targets planned in the scenario. It uses a probabilistic approach invoking single-shot kill probabilities for given munitions against given targets. The method considers a number of

operational planning factors (e.g., consumption rates, number of consumers, mission duration, and intensity of operations) for the calculation of the required stock level. The target oriented method is strongly dependent on the target effectiveness data and would be used for estimating stock levels of small arms ammunition.

On the other hand, the level of effort method uses historical or empirical data (from subject matter experts) to determine consumption rates for given weapon systems. It is not dependent upon specific scenarios and thus can be more generic than the target-oriented method. Simulation can be used to augment the empirical and historical data, but the ability of gamers to consider various modes of expenditure may be limited [1]. As for the target oriented method, the level of effort method would be used for small arms ammunition.

Different other quantitative approaches have been proposed in the literature to model the demand in a supply chain. In general, these approaches can be grouped into two main categories [3]: explanatory and time series methods. Explanatory methods (e.g., regression models) are statistical techniques for analyzing quantitative data to establish a functional relationship between the demand and one or more explanatory variables. Such methods assume that the explanatory variables are deterministic and non-correlated. However, when these variables are autocorrelated and/or stochastically change over time, predicting future demands based on historical data is not always suitable using an explanatory approach [4].

Time-series methods (e.g., Autoregressive Integrated Moving Average models) make forecasts based solely on historical patterns in the data and use time as an independent variable to predict demand. Although the time series approach allows for time-dependent stochastic explanatory variables, it could be overly complicated and even problematic when applied to time series data that exhibits multiple seasonal patterns [3]. Seasonal time series problems can be modeled using state space methods [5]. State space models refer to a class of probabilistic graphical models that describe the probabilistic dependence between the state variable and the observed measurement. The state or the measurement can be either continuous or discrete [6].

One of the well-known state space models in stochastic processes is the Markov-Modulated Poisson Process (MMPP). MMPP is a doubly stochastic process where the intensity of a Poisson process is defined by the state of a Markov chain [7]. The Markov chain can therefore be said to modulate the Poisson process and this modulation introduces

DRDC-RDDC-2016-P046© Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2016

© Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2016

correlations between successive inter-arrival times. Generally, MMPP models are useful when the system and its components evolve in a randomly changing environment. Compared with other standard forecasting methods (i.e., explanatory and time series methods), this model can handle autocorrelated, time-dependent stochastic covariates in continuous-time.

MMPP models have successfully been used in many applications, including queuing and communication network problems, traffic modeling, weather forecasting, seismology, etc. [8]. Muscariello et al. [9] proposed an MMPP-based hierarchical model to analyze the Internet traffic problem. They used the notion of sessions and flows to mimic the real hierarchical behavior of the packet generation process by Internet users. To illustrate their approach, they analyzed different Internet traffic data collected at an edge router under various networking scenarios and loads. Results indicated that the MMPP traffic model accurately predicts real Internet traffic. The authors suggested using the model as a management tool for network performance analysis, planning and dimensioning. Lu [10] applied MMPP to model and characterize large occurrence patterns of New Zealand deep earthquakes. He used two seismicity level states to capture the hidden Markov chain process associated with the occurrence times and magnitudes of deep earthquakes in the region. He developed a procedure to estimate the MMPP parameters through application of the Expectation–Maximization (EM) algorithm and statistical inference. The study indicated that the MMPP model provides reasonable predictions of the magnitude distribution of deep earthquakes that tends to vary correspondingly with the occurrence rates of events.

Lin et al. [11] used MMPP to model and forecast motor vehicle accidents on highway 401 (Ontario, Canada), one of the busiest highways in North America. They proposed a continuous-time stochastic approach to study the highway accidents problem and used a non-homogeneous Poisson model to analyze the accident counting and winter weather processes. To illustrate the effectiveness of their MMPP model, the authors studied a large winter data set provided by the Ministry of Transportation of Ontario. Using three aggregated states of weather conditions, they found that the MMPP model performs well for predicting accidents on Highway 401, and provides much more accurate predictions for collision frequencies compared with the standard and static Poisson model. Thayakaran and Ramesh [12] applied MMPP to model and analyze rainfall time series data at the Beaufort Park station, Bracknell, in the United Kingdom. They examined three types of time-varying covariates (namely temperature, sea level pressure, and relative humidity) that could affect the rainfall arrival process. They used maximum likelihood estimation to infer the MMPP model parameters and conducted likelihood ratio tests to compare different models. They applied the model to make statistical inferences about the accumulated rainfall in the station.

In this paper, we apply MMPP to a demand forecasting problem (air-to ground munitions demand) in a military context. It has been observed from historical data that the number of air-to-ground munitions dropped in military operations would vary randomly in time and with the operational tempo (state variable) and may exhibit various seasonal patterns (i.e., different cycles of demand). For this

problem, we consider a continuous state space time series approach and model the munitions demand as a non-homogeneous Poisson process with intensity function that depends on a continuous-time Markovian operational tempo process.

The paper is divided into five sections and is structured as follows. Section 2 presents an overview of the air-to-ground munitions used in modern military operations. Section 3 discusses the munitions demand forecasting model and assumptions, including details of the MMPP model formulation and parameters estimation. Section 4 provides an example application using different air-to-ground time series data sets to illustrate the method. Concluding remarks and future work are indicated in section 5.

II. AIR-TO-GROUND MUNITIONS

Air-to-ground munitions are aircraft ordnances used by military combat and/or unmanned aircraft to attack ground targets. They include bombs, machine guns, autocannons, air-to-ground missiles, rockets, air-launched cruise missiles and grenade launchers. From an inventory management perspective, these munitions are expensive, have potential obsolescence risks, and typically require lengthy procurement lead times. From a transportation perspective, air-to-ground munitions can be carried in a bomb bay of an aircraft or hung from a hard point. For many weapons, there is a limit to the length of time they can be flown (e.g., because of vibration damage); after this time their safety or effectiveness is not guaranteed [13].

In general, air-to-ground munitions can be grouped into two main categories [14]: unguided and precision-guided bombs. An unguided munition, also known as a free-fall bomb, gravity bomb, dumb bomb, or iron bomb, is a conventional aircraft-delivered munition that does not contain a guidance system and hence, simply follows a ballistic trajectory. A precision-guided munition, also known as a smart bomb, is a guided munition intended to precisely hit a specific target, and to minimize collateral damage. For illustration purposes, Fig. 1 depicts an example of an air-to-ground munition known as the GBU (Guided Bomb Unit) -12 Paveway II laser-guided bomb. GBU-12 is based on the general-purpose bomb, but with the addition of a nose-mounted laser seeker and fins for guidance. With a nominal weight of 227 kg, GBU-12 is one of the smallest guided munitions in current service, and one of the most common air-dropped weapons in the world [14].

Fig. 1. Picture of the GBU-12 laser-guided bomb.

III. DEMAND FORECASTING MODEL

In this paper, the munitions demand problem is modeled using MMPP. MMPP can be viewed as a stochastic process, where the rate of event arrivals is modulated by an m-state continuous-time Markov chain (i.e., state transitions occur in a continuous time domain). Let the m × m matrix Q = { }, 1 ≤ i, j ≤ m, denote the transition rate matrix containing the switch intensities on the continuous-time Markov chain. Let Ʌ = { }, 1 ≤ i ≤ m, denote the m × m diagonal matrix containing the Poisson intensity rates associated with states.

It is assumed that the elements of Q and Ʌ do no vary with time and thus the MMPP is a non-homogeneous process with stationary parameters. The parameters are considered to be identifiable when all the Poisson intensities are distinct.

The element (i ≠ j) represents the rate at which the discrete rate process transitions from state i to state j, and the element characterizes the rate at which the process transitions from state i. For a continuous-time Markov chain problem, because the rate of leaving state i is the sum of the rates of transition from state i to all states j (j ≠ i), the elements in each row of Q are dependent and sum to zero.

A challenge in describing a time series by an MMPP model is to accurately estimate the parameters of the model while keeping the number of states small enough to make the model’s performance tractable. Parameter estimation approaches are roughly classified into two categories: likelihood-based and Bayesian approaches. The likelihood-based approach is naturally viewed as a function of the parameters to be estimated. The parameters that maximize the likelihood of the observed data are called the Maximum Likelihood Estimate (MLE). Methods to infer the MMPP parameters using MLE include standard optimization and iterative algorithms such as the EM algorithm [15] for hidden Markov models. The Bayesian approach involves placing a probability distribution over model parameters.

In this paper, we assumed (for simplicity) that the Markov states are observed (not hidden) and applied the MLE approach to estimate the MMPP parameters using the standard univariate optimization method. However, this approach requires that the number of states be specified as an input in advance in order to compute the parameters. For the purpose of this analysis, we considered the following operational tempo states (corresponding to the intensity levels of military operations) to characterize the munitions demand stochastic process:

Zero-intensity: The zero-intensity state correspondsto situations where no air-to-ground munitions areemployed in a given time period.

Low-intensity: The low-intensity state corresponds toscenarios where only a few munitions are employedin a time period (e.g., 2-4 munitions per day).

Medium-intensity: The difference between the lowand medium intensity states is determined by thetotal number of munitions employed in a timeperiod. In the medium-intensity state, the number ofemployed munitions would be higher than the low-intensity state (e.g., 5-8 munitions per day).

High-intensity: This state corresponds to the highintensity combat operations and the number ofemployed munitions would be high (e.g., more than8 munitions per day) compared to the low-intensityand the medium-intensity states.

The boundaries of the different states can be determined using clustering algorithms such as the k-means algorithm [16]. The k-means clustering algorithm is a method of vector quantization that partitions a number of observations into a group of clusters fixed a priori. The procedure determines a set of cluster centroids and minimizes the within-cluster sum of distance functions of each point in the cluster to the centroids.

Consider a set of munition demands and operational tempo states observed at different time points during a time period T. Let be the total number of munitions employed during the operational tempo state i, , the total observed time for which the operational tempo process is at state i, and

, the number of transitions from state i to state j. The transition rates and the demand intensities of the continuous-time Markov chain can be estimated using MLE (by maximizing the log-likelihood function of the parameters) as follows [11]:

The steady state probabilities of the operational tempo states are defined as = ( ) and satisfy the matrix equation Q = 0, where is the transpose of . The model is implemented in Matlab.

The dynamics of the MMPP method, as applied to the munitions demand forecasting problem (with three states, for example), are graphically illustrated in Fig. 2. The bottom row is the (Markov) state diagram for the operational tempo process. The top row shows the different possible munitions demand intensities that depend on the current state of the operational tempo process.

Fig. 2. Pictorial representation of the MMPP method.

IV. ILLUSTRATIVE EXAMPLE

This section presents and discusses an example of air-to-ground munitions demand forecasting to illustrate the method. Two time series data sets of different trends, corresponding to two munition types, are constructed and examined. The data sets are used for illustration purposes only and do not represent the requirements for any military operations.

A. Munitions Demand Time Series Consider an operational scenario (illustrative) where the

number of dropped munitions per day for two ordnance types (ordnance A and B), observed during six consecutive months (180 days), is given in Fig. 3. The maximum number of employed munitions per day is 12 for ordnance A and 6 for ordnance B. Both munitions are used simultaneously in the

1 2 3

Fig. 4. ACF plots of time series for ordnance A (left) and ordnance B (right).

C. Parameters Estimation We applied MLE to estimate the MMPP parameters (Q,

Ʌ) and calculated the steady state probabilities of states ( ) for both data sets (ordnances A and B) as well as for a combined data set (Table 1). The combined data set represents the total demand of both ordnances A and B. We also verified the stationarity of the combined process. We next validated the model assumption made by the MMPP method that the demand intensity is constant in a given operational tempo state. There are different statistical methods for testing the homogeneity of Poisson processes, namely: Laplace, Anderson-Darling, Lewis-Robinson, Mann, etc. [18, 19]. We performed the Anderson-Darling test (on both data sets as well as on the combined set) which tests the following null hypothesis:

Null hypothesis (Ho): the underlying point process ishomogeneous Poisson process with constantintensity.

Alternative hypothesis (H1): the underlying pointprocess is non-homogeneous Poisson process withvarying (increasing or decreasing) intensity.

The corresponding p-value for each of the intensities is greater than a significance level of 5% (for example), indicating that the test statistic is not in the rejection region and that the null hypothesis of a homogeneous Poisson process with constant intensity cannot be rejected..

TABLE I. MMPP MODEL PARAMETERS FOR TIME SERIES OF ORDNANCES A AND B.

Munitions Demand Intensity (munitions/day) State Transition Matrix Steady Sate

Probabilities

Ordnance A

Ordnance B

Ordnances A and B

In Table 1, the indexes (0, L, M, and H) of the demand intensities and the steady state probabilities refer to the different operational tempo states. The analysis indicates that for ordnance A (four states), the zero-intensity state represents the most probable state in the longer term (47%), followed by the low-intensity state (33%), medium-intensity state (12%), and high-intensity state (8%). However, the highest intensity rate is in the high-intensity state (10.47 munitions per day), followed by the medium-intensity state (6.62 munitions per day), and the low-intensity state (3.17 munitions per day). Obviously, the rate for the zero-intensity state is null. From a state transition rate perspective, given that the steady state probability of the zero-intensity is 47%, the transition from the zero-intensity state to the low-intensity state would be the most common transition in the process. The overall average demand (calculated using the demand intensities and the steady state probabilities) would be 2.7 munitions per day.

Similar trends are observed for the ordnance B data set. However, it is important to note that the number of states for ordnance B is only three (zero, low, and medium-intensity). The total number of states is mainly determined by the maximum number of munitions per day in the data set (i.e., 12 munitions per day for ordnance A and 6 munitions per day for ordnance B). The analysis indicates that the zero-intensity state represents the most probable state in the longer term (64%), followed by the low-intensity state (31%), and the medium-intensity state (5%). The transition from the zero-intensity state to the low-intensity state would be the most common transition in the process. The overall average demand for ordnance B would be 1.1 munitions per day.

For the combined time series data (four states), the analysis indicates that the low-intensity state represents the most probable state in the longer term (35%), followed by the zero-intensity state with a comparable probability (33%), medium-intensity state (18%), and high-intensity state (14%). The overall average demand for both ordnances would be 3.8 munitions per day.

D. Demand Probability Distribution To determine the steady state probability distribution of

the demand, we considered the method developed by Fearnhead and Sherlock [20]. The method constructs a meta-Markov process on an extended state space and creates a generator matrix (G) for the metaprocess from the transition matrix (Q) and the diagonal intensity matrix (Ʌ). Let p be the maximum number of munitions in the set of demand observations (e.g., p = 12 for ordnance A and p = 6 for ordnance B), the generator matrix is defined as follows [20]:

Where 0 is a matrix or vector where all of whose elements are zeros, is the intensity rates vector and G is a square matrix of size (m(p+1) + 1) × (m(p+1) + 1). The block of square matrices comprising of the top m rows of

give the m × m conditional transition matrices,

where is the time interval between the different observations ( day in this problem). These conditional transition matrices are used, along with the vector , to determine the steady state probability distribution of the munitions demand.

Fig. 5 depicts the steady state probability distribution for the demand of ordnance A and shows the stationary probabilities for a demand of 0, 1, 2, …, 12 munitions per day. Results indicate that the probability for a demand of zero munitions per day would be about 38% (highest) and the probability for 12 munitions per day would be less than 2%, consistent with the initial data set. The demand distribution has a long tail on right with a mean of about 2.7 munitions per day and a standard deviation of 3.3 munitions per day.

Fig. 5. Steady state demand distribution (ordnance A)

Fig. 6 depicts the steady state probability distribution for the demand of ordnance B and shows the stationary probabilities for a demand of 0, 1, 2, …, 8 munitions per day. As for the demand of ordnance A, the highest probability in the ordnance B distribution is for zero munitions (58%). The distribution has a mean of about 1.1 munitions per day and a standard deviation of 1.7 munitions per day.

Fig. 6. Steady state demand distribution (ordnance B)

Fig. 7 depicts the steady state demand probability distribution for the combined time series (ordnances A and B). It indicates that the probability for a demand of zero munitions per day would be about 24% (highest) and the probability for 16 munitions per day (for example) would be less than 1%. The distribution has a mean of about 3.8

munitions per day (sum of the means of ordnances A and B) and a standard deviation of 3.7 munitions per day.

Fig. 7. Steady state demand distribution (ordnances A and B)

E. Future Work In this paper, we applied MMPP to model the stochastic

process of the munitions demand in military operations and used MLE to estimate the model parameters, assuming that the states of the Markov states are observed (not hidden). An extension of the model would be to consider hidden Markov states and to use the EM algorithm to capture these states through an iterative procedure.

The MLE approach provides single estimates of the MMPP parameters. Another extension of the model would be to use a Bayesian approach to infer the parameters through application of a Forward-Backward type algorithm as part of a blocked Gibbs sampler within a Markov Chain Monte Carlo (MCMC) framework. MCMC is simulation method for sampling from a probability distribution based on constructing a Markov chain that has the desired distribution as its equilibrium distribution [21]. Forward-Backward algorithm is an inference algorithm for hidden Markov models which computes the posterior marginal distributions of all hidden state variables given a sequence of observations. The algorithm uses the principle of dynamic programming and combines two steps (forward and backward) to get the probability of being in each state at a specific time. In the forward step, the algorithm determines a set of probabilities (forward probabilities) which provide the probability of ending up in any particular state given the first observations in the sequence. In the backward step, the algorithm calculates another set of probabilities (backward probabilities) which provide the probability of observing the remaining observations given any starting point. The forward and backward probabilities are then combined to obtain the distribution over states at any specific point in time given the entire observation sequence [22].

The demand forecasting model discussed thus far is an important module of a larger inventory framework being developed for managing air-to-ground munitions of military operations. For the inventory framework development, we apply exact and approximate dynamic programming techniques to determine optimal stock levels and ordering policies for air-to-ground munitions. For this problem, we consider a single-item, infinite horizon, periodic review, stochastic inventory problem in which demands are described by continuous-time stochastic process. We use a traditional inventory cost structure with per unit ordering,

holding and backlogging penalty costs that are incurred at the end of each period. A comprehensive literature review of models for inventory management under uncertainty can be found in [23]. This research is ongoing and details about the method implementation and the complete results will be documented in a forthcoming paper

V. CONCLUSION The paper presents a method for analyzing the stochastic

process of the munitions demand for military operations using MMPP - a continuous state space time series approach. In this approach, the demand for munitions is described by a non-homogeneous Poisson process with intensity function that depends on a continuous-time Markovian operational tempo process. Four potential operational tempo states, representing different intensity levels of military operations, are considered in the demand forecasting model: zero, low, medium and high intensity states. A maximum likelihood approach is used to estimate the MMPP model parameters. The advantage of the MMPP method over standard regression models is its ability to handle autocorrelated, time-dependent stochastic covariates in continuous-time. An example based on air-to-ground munitions used in military operations is presented to illustrate the method. Two time series data sets of different trends are also analyzed and discussed. This study provides military planners with a decision support method for forecasting munitions requirements in the face of abrupt changes in demand. An extension of the model would be to use Bayesian approach for inferring the MMPP parameters through application of a Forward-Backward type algorithm to capture potential hidden states. The demand forecasting model will be integrated in an inventory framework being developed for managing air-to-ground munitions. Future work would also include the application of the model to a real case problem using a large time series data set from historical military operations.

ACKNOWLEDGMENT The author would like to express his sincere appreciation

to Mr. David Shaw for his helpful discussions on MMPP and inventory management modeling.

REFERENCES [1] W. S. Andrews and W. J. Hurley, “Approaches to determining army

operational stockpile Levels”, Canadian Military Journal, vol. 5(2), pp. 34-46, 2004.

[2] W. L. Bean, Inventory management under uncertainty: a military application, University of Pretoria, 2011.

[3] G. E. P. Box, G.M. Jenkins, and G. C. Reinsel, Time series analysis: forecasting and control, John Wiley & Sons, Inc., New Jersey, 2008.

[4] M. E. Nenni, L. Giustiniano, and L. Pirolo, “Demand forecasting in the fashion industry: a review”, International Journal of Engineering Business Management, Special Issue on Innovations in Fashion Industry, vol. 5(37), pp. 1-6, 2013.

[5] J. Durban and S. J. Koopman, Time series analysis by state space methods, Oxford University Press, 2012.

[6] D. Koller and N. Friedman, Probabilistic graphical models, Cambridge, MA: MIT Press, 2009.

[7] D. R. Cox, “Some statistical methods connected with series of events”, Journal of the Royal Statistical Society: Series B, vol. 17, pp. 129–164, 1955.

[8] W. Fischer and K. Meier-Hellstern, “The Markov-modulated Poisson process (MMPP) cookbook: performance evaluation”, vol. 18 (2): pp. 149, 1993.

[9] L. Muscariello, M. Mellia, M. Meo, M. Ajmone, and R. L. Cigno, “An MMPP-based hierarchical model of internet traffic”, IEEE Communications Society, pp. 2143-2147, 2004.

[10] S. Lu, “Markov modulated Poisson process associated with state-dependent marks and its applications to the deep earthquakes”, Annals of the Institute of Statistical Mathematics, vol. 64, pp. 87-106, 2012.

[11] C. Lin, M. J. Kim, and V. Makis, “A stochastic model for highway accident predictions with winter data”, The Open Statistics and Probability Journal, vol. 5, pp. 6-13, 2013.

[12] R. Thayakaran and N. I. Ramesh, “Markov modulated Poisson process models incorporating covariates for rainfall intensity”, Water science and technology: a Journal of the International Association on Water Pollution Research, vol. 67(8), pp. 1786-1792, 2013.

[13] B. Fox, M. Boito, G. C. Graser, and O. Younossi, Test and evaluation trends and cost for aircraft and guided weapons. RAND Corporation. 1-223, 2004.

[14] Royal Air Force, Aircraft and weapons, Jay Myers, Royal Air Force Media and Communications, Headquarters Air Command. pp. 1-61, 2013.

[15] T. Rydén, “An EM algorithm for estimation in markov-modulated Poisson processes”, Computational Statistics & Data Analysis, vol., 21, pp. 431-447, 1996.

[16] C. C. Aggarwal and C. K. Reddy, Data clustering: algorithms and applications, Chapman and Hall/CRC, 2013.

[17] B. L., Bowerman, R. T., O’Connell, and A. B. Koehler, Forecasting, time series and regression, 4th edition Belmont, CA: Thomson Brooks/Cole, 2005.

[18] I. V. Basawa and B.L.S. Prakasa Rao, Statistical inference for stochastic processes, Academic Press, New York, 1980.

[19] R. Jiang, Introduction to quality and reliability engineering, Springer Series in Reliability Engineering, Science Press Beijing, 2015.

[20] P. Fearnhead and C. Sherlock, “An exact gibbs sampler for the Markov-modulated Poisson process”, Journal of the Royal Statistical Society: Series B, vol. 68 (5), pp. 767-784, 2006.

[21] C. Y. Geyer, “Practical Markov chain Monte Carlo”, Statistical Science, vol. 7, pp. 473-483, 1992.

[22] E. Charniak, Statistical language learning, Cambridge, Massachusetts: MIT Press, 1993.

[23] S. Ziukov, “A literature review of models of inventory management under uncertainty”, Business Systems and Economics, vol. 5(1), pp. 26-34, 2015.