Contents Order-1 process Order-q process Extensions References
Order-q dependent stochastic processes inBayesian applications
Luis E. Nieto-Barajas
Department of Statistics, ITAM, Mexico
Statistics Colloquium, UNM, USA
October 2, 2015
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Contents
Order-1 process
Application in survival analysis
Order-q process
Application in time series modelingApplication in disease mapping
Extensions
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−1 process
η1 η2 η3 η4 η5
θ1 θ2 θ3 θ4 θ5
6@@@@@@R
6@@@@@@R
6@@@@@@R
6@@@@@@R
6
Dependence among {θk} is induced through latents {ηk}Close form expressions when use conjugate distributions
Want to ensure a given marginal distribution
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−1 process
Nieto-Barajas & Walker (2001):
Beta process: {θk} ∼ BeP1(a, b, c)
θ1 ∼ Be(a, b), ηk | θk ∼ Bin(ck , θk),
θk+1 | ηk ∼ Be(a + ηk , b + ck − ηk)
⇒ θk ∼ Be(a, b) marginally
Gamma process: {θk} ∼ GaP1(a, b, c)
θ1 ∼ Ga(a, b), ηk | θk ∼ Po(ckθk),
θk+1 | ηk ∼ Ga(a + ck , b + ηk)
⇒ θk ∼ Ga(a, b) marginally
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−1 process
Nieto-Barajas & Walker (2001):
Beta process: {θk} ∼ BeP1(a, b, c)
θ1 ∼ Be(a, b), ηk | θk ∼ Bin(ck , θk),
θk+1 | ηk ∼ Be(a + ηk , b + ck − ηk)
⇒ θk ∼ Be(a, b) marginally
Gamma process: {θk} ∼ GaP1(a, b, c)
θ1 ∼ Ga(a, b), ηk | θk ∼ Po(ckθk),
θk+1 | ηk ∼ Ga(a + ck , b + ηk)
⇒ θk ∼ Ga(a, b) marginally
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−1 process
Nieto-Barajas & Walker (2001):
Beta process: {θk} ∼ BeP1(a, b, c)
θ1 ∼ Be(a, b), ηk | θk ∼ Bin(ck , θk),
θk+1 | ηk ∼ Be(a + ηk , b + ck − ηk)
⇒ θk ∼ Be(a, b) marginally
Gamma process: {θk} ∼ GaP1(a, b, c)
θ1 ∼ Ga(a, b), ηk | θk ∼ Po(ckθk),
θk+1 | ηk ∼ Ga(a + ck , b + ηk)
⇒ θk ∼ Ga(a, b) marginally
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis
Hazard rate modelling
If T is a discrete r.v. with support on τk then
h(t) = θk I (t = τk)
with {θk} ∼ BeP1(a, b, c)
If T is a continuous r.v. and {τk} are a partition of IR+ then
h(t) = θk I (τk−1 < t ≤ τk)
with {θk} ∼ GaP1(a, b, c)
This is old stuff!, but what it is new is that there is anR-package called BGPhazard that implements these models
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis
Hazard rate modelling
If T is a discrete r.v. with support on τk then
h(t) = θk I (t = τk)
with {θk} ∼ BeP1(a, b, c)
If T is a continuous r.v. and {τk} are a partition of IR+ then
h(t) = θk I (τk−1 < t ≤ τk)
with {θk} ∼ GaP1(a, b, c)
This is old stuff!, but what it is new is that there is anR-package called BGPhazard that implements these models
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis
Example: Discrete survival model
We analyse the 6-MP clinical trial data which consists ofremission duration times (in months) for children with acuteleukemia.
The study consisted in comparing drug 6-MP versus placebo.We concentrate on the 21 patients who received placebo.
Observed time values range from 1 to 23 and there are nocensored observations.
To define the prior we took a = b = 0.0001 and ct = 50 forall t. We use command BeMRes to fit the model and thecommand BePloth to produce graphs.
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis: Order-1 Beta process
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
Estimate of hazard rates
time
Haz
ard
rate
+ ++
+ +
+
+
+
+
+
+
+
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
−
+
Hazard functionConfidence band (95%)Nelson−Aalen based estimate
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis: Order-1 Beta process
0 5 10 15 20
0.0
0.2
0.4
0.6
0.8
1.0
Estimate of Survival Function
times
Model estimateConfidence bound (95%)Kaplan−MeierKM Confidence bound (95%)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis
Example: Continuous survival model
We define a piecewise hazard function
The data are survival times of 33 leukemia patients. Timesare measured in weeks from diagnosis. Three of theobservations were censored.
The prior was defined by taking a = b = 0.0001 and
ck |ξiid∼ Ga(1, ξ) for k = 1, . . . ,K and ξ ∼ Ga(0.01, 0.01). We
took K = 10 intervals and chose the partition τk such thateach interval contains approximately the same number ofexact (not censored) observations.
We used the command GaMRes to fit the model andcommand GaPloth to produce graphs.
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Survival Analysis: Order-1 Gamma process
0 20 40 60 80 100 120 140
0.0
0.2
0.4
0.6
0.8
1.0
Estimate of Survival Function
times
Model estimateConfidence bound (95%)Kaplan−MeierKM Confidence bound (95%)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−2 process
η1 η2 η3 η4 η5
θ1 θ2 θ3 θ4 θ5
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R ?
Throw more arrows to induce higher order dependence
There is no way to obtain a given marginal distribution:say beta or gamma
Unless we include an extra latent (layer)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−2 process
η1 η2 η3 η4 η5
θ1 θ2 θ3 θ4 θ5
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R ?
Throw more arrows to induce higher order dependence
There is no way to obtain a given marginal distribution:say beta or gamma
Unless we include an extra latent (layer)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−2 process
η1 η2 η3 η4 η5
θ1 θ2 θ3 θ4 θ5
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R
HHHHH
HHHHHHHj
?
@@@@@@R ?
Throw more arrows to induce higher order dependence
There is no way to obtain a given marginal distribution:say beta or gamma
Unless we include an extra latent (layer)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order−2 process
ω
η1 η2 η3 η4 η5
θ1 θ2 θ3 θ4 θ5
?
@@@@@@R
HHHH
HHHHHH
HHj?
@@@@@@R
HHHH
HHHHHH
HHj?
@@@@@@R
HHHH
HHHHHH
HHj?
@@@@@@R ?
?
���
���
@@@@@@R
HHHHHH
HHHHHj
������
������
With this common ancestor ω we can through more arrowsand still ensure a given marginal
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Space and time process
This idea can be use to induce time and/or spatial dependence
'
&
$
%
'
&
$
%
'
&
$
%
t = 1 t = 2 t = 3
θ1,1(η1,1)
θ2,1(η2,1)
θ3,1(η3,1)
θ1,2(η1,2)
θ2,2(η2,2)
θ3,2(η3,2)
θ1,3(η1,3)
θ2,3(η2,3)
θ3,3(η3,3)
θ1,4(η1,4)
θ2,4(η2,4)
θ3,4(η3,4)
��>
���������
CCCCCCCCO ZZ}
ω
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order-q beta process
Jara & al. (2013):
Order−q (AR) beta process: {θt} ∼ BePq(a, b, c)
ω ∼ Be(a, b) ηt | ωind∼ Bin(ct , ω)
θt | η ∼ Be
a +
q∑j=0
ηt−j , b +
q∑j=0
(ct−j − ηt−j)
θt ∼ Be(a, b) marginally
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order-q beta process
Properties:
Corr(θt , θt+s) =(a + b)
(∑q−sj=0 ct−j
)+(∑q
j=0 ct−j
)(∑qj=0 ct+s−j
)(a + b +
∑qj=0 ct−j
)(a + b +
∑qj=0 ct+s−j
) ,
for s ≥ 1.
If ct = c for all t then {θt} becomes strictly stationary with
Corr(θt , θt+s) =(a + b) max{q − s + 1, 0}c + (q + 1)2c2
{a + b + (q + 1)c}2.
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Autocorrelation in {θt}
5 10 15
0.0
0.2
0.4
0.6
0.8
1.0
lag
5 10 150.
00.
20.
40.
60.
81.
0lag
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Order-q beta process
Example: Unemployment rate in Chile
Bimonthly data from 1980 to 2010
Use our BePq as likelihood for the data {Yt}Took priors for (a, b, c): a ∼ Un(0, 1000), b ∼ Un(0, 1000)
and ct | λiid∼ Po(λ) and λ ∼ Un(0, 1000)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Time series: Yt = Unemployement in Chile
1980 1985 1990 1995 2000 2005 2010
0.05
0.10
0.15
0.20
0.25
Year
Unem
ploym
ent Ra
te in C
hile
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
●● ●
●● ● ●
●●
●
●
●●
●
●
q=3q=4q=5q=6q=7q=8
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Time series: Yt = Unemployement in Chile
1980 1990 2000 2010 2020
0.00.1
0.20.3
0.4
Year
●
●
●
●
●
●
●
●
●
● ●●
● ●
●●
●● ●
●● ● ● ●
●
●
●
●●
●●
BePBDM
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Spatial process
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Spatial process
Nieto-Barajas & Bandyopadhyay (2013):
Spatial gamma process: {θt} ∼ SGaP(a, b, c)
ω ∼ Ga(a, b) ηij | ωind∼ Ga(cij , ω)
θi | η ∼ Ga
a +∑j∈∂i
cij , b +∑j∈∂i
ηij
∂i is the set of neighbours of region i
θt ∼ Ga(a, b) marginally
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Disease mapping
Study: Mortality in pregnant women due to hypertensive disorderin Mexico in 2009. Areas are the States
Yi = Number of deaths in region iEi = At risk: Number of births (in thousands)λi = Maternity mortality rate
Zero-inflated model
f (yi ) = πi I (yi = 0) + (1− πi )Po(yi | λiEi )
λi = θi exp(β′xi ) πi =ξie
δ′zi
1 + ξieδ′zi
β is a vector of reg. coeff. s.t. βk ∼ N(0, σ20)
θi ∼ SGaP(a, a, c)ξi ∼ Ga(b, b)
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Disease mapping
Six explanatory variables:
X1 number of medical units (hospitals + clinics)
X2 proportion of pregnant women with soc. sec.
X3 prop. of pregnant women who were seen by a physician inthe first trimester of pregnancy
X4 public expenditure in health per capita in thousands of MX
Z1 poverty index
Z2 proportion of births in clinics and hospitals
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Estimated mortality rate λi
[3.05,6.33)[6.33,6.67)[6.67,7.38)[7.38,8.73)[8.73,21.07]
1
2
3
4
7
8
5
6 9
10
11
12
1314
151617
18
19
20
21
22
23
24
25
26
27
28
29 30
31
32
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Estimated zero inflated prob. πi
[0,0.01)[0.01,0.04)[0.04,0.06)[0.06,0.5)[0.5,0.6]
1
2
3
4
7
8
5
6 9
10
11
12
1314
151617
18
19
20
21
22
23
24
25
26
27
28
29 30
31
32
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
Extensions
Construct dependent Bayesian nonparametric priors (DDP,dPT)
Use same ideas with stochastic processes instead of randomvariables
Dependent Dirichlet processes using multinomial processes aslatents
Dependent gamma processes using Poisson processes aslatents
These constructions are currently under study
Luis E. Nieto-Barajas Order-q stochastic processes
Contents Order-1 process Order-q process Extensions References
References
Jara, A., Nieto-Barajas, L. E. & Quintana, F. (2013). A time series model forresponses on the unit interval. Bayesian Analysis 8, 723–740.
Nieto-Barajas, L. E. & Bandyopadhyay, D. (2013). A zero-inflated spatialgamma process model with applications to disease mapping. Journal ofAgricultural, Biological and Environmental Statistics 18, 137–158.
Nieto-Barajas, L. E. & Walker, S. G. (2002). Markov beta and gammaprocesses for modelling hazard rates. Scandinavian Journal of Statistics 29,413–424.
Luis E. Nieto-Barajas Order-q stochastic processes