xpose 4 bestiary - sourceforgexpose.sourceforge.net/bestiarium_v1.0.pdf · xpose 4 bestiary version...
TRANSCRIPT
Xpose 4 Bestiary
Version 1.0
E Niclas Jonsson and Andrew Hooker
Pharmacometrics Research GroupUppsala University
A bestiary, or Bestiarum vocabulum, is a compendium of beasts.
ContentsIntroduction 6
Data file checkout 8
data.checkout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Data visualization 10
dv.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
pred.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
ipred.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
cov.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
cov.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
cov.splom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
cov.summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Overall goodness of fit assessment 18
dv.preds.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
ind.plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
basic.gof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
addit.gof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
runsum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Structural model diagnostics 24
dv.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
dv.vs.ipred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
dv.vs.pred.ipred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
dv.vs.pred.by.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
dv.vs.ipred.by.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
dv.vs.pred.by.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
dv.vs.ipred.by.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
cwres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
wres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2
iwres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
cwres.wres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
cwres.vs.idv.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
wres.vs.idv.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
cwres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
wres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
cwres.wres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
cwres.vs.pred.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
wres.vs.pred.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
cwres.vs.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
wres.vs.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Residual model diagnostics 45
absval.cwres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
absval.wres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
absval.iwres.vs.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
absval.iwres.vs.ipred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
absval.wres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
absval.iwres.vs.idv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
absval.iwres.cwres.vs.ipred.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
absval.iwres.wres.vs.ipred.pred . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
cwres.dist.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
wres.dist.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
iwres.dist.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
cwres.dist.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
wres.dist.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
iwres.dist.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
ind.plots.cwres.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
ind.plots.wres.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
ind.plots.cwres.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
ind.plots.wres.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3
absval.cwres.vs.pred.by.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
absval.wres.vs.pred.by.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
absval.iwres.vs.ipred.by.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
absval.cwres.vs.cov.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
absval.wres.vs.cov.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
absval.iwres.vs.cov.bw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
autocorr.cwres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
autocorr.wres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Parameter distribution diagnostics 72
parm.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
ranpar.hist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
parm.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
ranpar.qq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
parm.splom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
ranpar.splom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
parm.vs.parm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
tabulate.parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
parm.summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Model comparison 82
basic.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
add.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
absval.dpred.vs.cov.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
absval.dipred.vs.cov.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
absval.dcwres.vs.cov.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
absval.dwres.vs.cov.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
absval.diwres.vs.cov.model.comp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
dOFV.vs.id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
dOFV.vs.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
dOFV1.vs.dOFV2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4
Model development 94
parm.vs.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
ranpar.vs.cov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
xpose.gam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
xp.plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
xp.ind.inf.fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
xp.ind.inf.terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
xp.ind.stud.res . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
xp.akaike.plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Getting more help 104
What is not included? 105
How to make NONMEM generate input to Xpose 106
How Xpose identifies NONMEM runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Xpose data variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Default variable definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Changing the default variable definitions . . . . . . . . . . . . . . . . . . . . . . . 106
How Xpose identifies data items 109
How to extend and modify the behavior of Xpose specific functions 110
5
IntroductionXpose is an R library for post-processing of NONMEM output. It takes one or more standardNONMEM table files as input and generates graphs or other analyses. It is assumed that eachNONMEM run can be uniquely identified by a run number (see Section “How to make NONMEM
generate input to Xpose” on page 106) for how to generate the appropriate input to Xpose).Xpose is implemented using the lattice graphics library in R.
The Xpose library has five sub-libraries:
xposedata Contains functions for managing the input data and manipulating the Xpose database.
xposegeneric Contains generic wrapper functions around the lattice functions. These func-tions can be invoked by the user but require quite detailed instructions to generate thedesired output.
xposespecific These functions are single purpose functions that generate specific outputgiven only the Xpose database as input. The behavior can to some extent be influencedby the user.
xposeclassic Historically, Xpose has had a text based menu interface to make it simple for theuser to invoke the Xpose specific functions. This interface is called Xpose Classic. Giventhe limitations a text based interface imposes, Xpose Classic is not very flexible but maystill be useful for quick assessment of a model.
xposesspecial These functions are the interface between Xpose and PsN, i.e. they do notpost-process NONMEM output but rather PsN output.
This document is intended to be a showcase for the graphical displays that can be producedusing the functions in the Xpose specific library. Many of the graphs can be generated usingthe classical Xpose text based interface (albeit in a less flexible way) but not all. On the otherhand, to use the Xpose specific functions directly, it is necessary to type the functions on thecommand line and, in some cases, do some simple data management before the functions areused.
In the following, each of the Xpose specific functions will be exemplified. Each display will havea legend that hopefully will be a good starting point for reports and papers. Together with eachexample will be the code used to generate the graph starting from NONMEM table files.
Each example assumes that the Xpose library has been attached:
> library(xpose4)
and that appropriate table files for Xpose have been produced. When preparing this docu-ment, the table files were located in the directory above from where R was running, hence thedirectory="../" argument to xpose.data in all of the examples.
There are quite a few specific functions in the Xpose library and for the sake of presentation,they have been categorized into sections:
6
Data visualization Graphs intended to plot the raw data, for example concentrations versustime or histograms of the covariates.
Overall goodness of fit assessment Xpose has a number of composite displays (multiplegraphs of different kinds on the same page) that provide diagnostics for multiple aspectsof the fit. These plots go into this section.
Structural model diagnostics These are graphs that are more geared towards diagnosingthe appropriateness of the structural model.
Residual model diagnostics Plots for diagnosing the statistical portions of the model (for ex-ample, residual error and between subject variability).
Parameter distribution diagnostics Plots in this section focus on the inter-individual variabil-ity model.
Model comparison Graphs for comparing various aspects of two models fit to the same data.
Model development Graphs and functions that are supposed to help the modeler decide whatto do next with the model. Most of these functions focus on aspects of covariate inclusion.
Please note that these sections are “fuzzy”, in the sense that many plots could be categorizedinto several different categories.
7
Data file checkoutExploratory graphical analysis of the raw data is important and Xpose has numerous functionsto support this activity. However, most of this functionality assumes that the data is available inNONMEM table file format. This is usually fine but one needs to be aware that Xpose excludesall lines with WRES=0 and only the first line for each unique ID number when plotting covariatesand/or parameters (this is the default behavior and can be changed by invoking appropriatearguments when executing the specific functions).
One challenge that is related to the exploratory analysis but usually overlooked is checkingif the data file is correctly coded, for example making sure that all individuals have the doserecords they are supposed to have. Typically such checking is done by “reading” the data file, atedious and error prone activity. Xpose offers some support with the data.checkout function.
8
data.checkout
AGE
ID
123456789
10111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364
30 40 50 60 70
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
Figure 1: The ID-number versus the values in the AGE column.
This graph can be used to check the con-tents of the data file columns. The ID num-ber is plotted versus the values of each row inthe column. In this particular case only onepanel=column is shown but the function gen-erates one panel per column (excluding the IDcolumn) in the data set. hlin specifies the rownumber in the data file that contains the col-umn headers. Check the help page for further
details.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- data.checkout(obj = xpdb,
+ datafile = "../newdata2.dta",
+ csv = F, hlin = 1, max.plots.per.page = 1)
> print(xplot@plotList[[2]])
9
Data visualizationSome people claim that exploratory analysis of the observed data is a sufficient data analysismethodology and that models offer only marginal additional benefit. There are of course peoplethat argue for the opposite. Regardless, exploratory graphical analysis is a powerful tool whichmay reveal a huge amount of information. It is sometimes also necessary to demonstrate thatconclusions drawn from a model based analysis is supported by the observed data itself. Onthe other hand, one may argue that the purpose of a model based analysis is to reveal thingsthat are not obvious from the raw data alone. With complicated underlying data structures anddependencies, as in PKPD data, this is even more true.
The graphical displays in this section depend only on the raw data and aim at supporting ex-ploratory graphical analysis to establish candidate models and to establish distributional repre-sentations of the underlying study population.
10
dv.vs.idv
Observations vs. Time (Run 1)
Time
Obs
erva
tions
0
50
100
150
2 4 6 8 10 12
● ●
●
●●
● ●
● ●
●
●
●●
●
●
●
●
● ●
● ● ● ● ●● ● ● ● ●
● ●
●
●
●●
●●
●
●
● ● ● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●● ●
●
●
●
●
●
●
●●
●
●
●● ●
● ●
● ●● ●
● ● ●
● ●● ●
●● ● ● ● ●
●
●
●●
●● ● ●
● ●
●●
●
● ● ● ● ● ● ●
●
● ● ● ●
● ●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●● ●
●●
● ●● ●
●● ● ● ● ● ● ● ● ●
●●
●●
● ●●
●● ● ●
●● ● ●
● ●● ●
● ●
●●
●
● ● ●
●● ●
●●
●● ● ● ●
●
●
●
● ● ●●
● ●●
●
●
●
●●
●●
●● ●
●●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●
● ●
●
●
●●
●●
●
● ●
●
●● ●
● ● ● ●●
● ●
● ●● ● ●
● ● ●● ●
●
●
●
●
●
●●
●●
●
●●
● ● ● ● ● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●● ●
●●
●
●
●
●
●
●
●
●
● ● ● ● ● ● ● ●
●
● ●●
●● ● ● ●
●
● ●
●●
●
● ●
● ●
●
●
●
●
●
●●
● ● ● ● ● ● ● ● ● ●
●
●
●
●
● ● ● ● ● ●
●
●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
● ●
●
● ●●
●
●●
●● ●
●
●
●
●
●● ● ● ● ●
●
● ●●
●●
●● ●
●
●
●
●●
●
● ●
●●● ●
● ● ● ● ● ● ● ●
●
●●
● ●
● ●● ● ●
●
●
●
●
● ●
●● ●
●
● ●● ● ●
●
● ● ●● ● ●
● ●●
●
●●
● ● ● ●
●●
●
●
● ● ●
●
●
●
●
● ●
●●
●
● ●
●●
● ● ● ● ● ● ● ●
● ●
●●
●
● ●●
●
●
●
●●
●
●●
● ●
● ● ●●
●●
●●
●●
●
●●
●● ●
● ●●
●
214357392615316
46
36177
4646
17
7
346121433931353
17
2
6 3333
1717
46
50344461134321 10
1746 17
1 23330
2
50443461131453 61
13317 47
3022
33
Figure 2: Observed data versus time. The data points from each individual areconnected with a line and each data point is indicated by a circle. Extreme datapoints (outermost 5%) are labeled with the ID-number of the corresponding indi-vidual. The red line is a smooth.
This graph is used for an overall visualizationof the data over the main predictor (idv oridependent variable). To use this graph for ini-tial PK model selection it is useful to use timeafter last dose on the x-axis. The Xpose func-tion add.tad can be used to compute this. Auseful variation of this graph is to use a semi-
logarithmic scale (logy=TRUE), no id-numbers(ids=FALSE) and no smooth (smooth=NULL):
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.idv(xpdb, ids = TRUE)
> print(xplot)
11
pred.vs.idv
Population predictions vs. Time (Run 1)
Time
Pop
ulat
ion
pred
ictio
ns
0
20
40
60
80
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●● ● ● ● ●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●●
●
● ●
●
●
●●
●●
●●
● ●
●
●
●●
●●
●●
● ●
●
●
●●
●●
●●
● ●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●●
●●
● ● ● ● ●
● ●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
● ●
●
●
●
●
●●
●●
● ●
●
●
●
●
●●
●●
● ●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●●
● ●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●
●●
● ● ●
● ●
●
●
●
●
●●
●●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●●
●●
● ● ● ● ●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●●
● ● ● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●●
● ● ● ● ● ●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
Figure 3: Population predictions versus time. The predictions from each individ-ual are connected with a line and each data point is indicated by a circle. The redline is a smooth.
This graph is used for visualization of thepopulation predictions over the main predictor(idv). Its is more useful as method of com-municating model properties than goodness-of-fit or diagnostics since there is no referenceto which it should be compared. See also
“dv.preds.vs.idv” on page 19.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- pred.vs.idv(xpdb)
> print(xplot)
12
ipred.vs.idv
Individual predictions vs. Time (Run 1)
Time
Indi
vidu
al p
redi
ctio
ns
0
50
100
150
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●●
●●
● ●●
●● ● ●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●● ●
●
●
●
●
●
●
●●
●
●
●
● ●
● ●●
●● ● ●
● ●
● ●●
●● ● ● ● ● ●
●
●
●●
●● ● ● ● ●
●● ● ● ● ● ● ● ● ●
●
● ● ● ● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●
● ●
●
●●
●●
●● ● ● ● ● ● ●
● ●
●
●● ● ●
●●
●●
●
● ●
●
●●
●● ● ● ●
●
●● ● ● ● ● ●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●●
● ● ●● ●
● ●●
● ● ● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●●
● ● ● ● ● ●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
● ● ● ● ● ● ● ●
●
●
●●
●●
● ●● ●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
● ● ● ● ● ● ● ● ● ●
●
●
●
●●
● ● ● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
● ●●
●●
●●
●●
●
●
●
●
●● ● ● ● ●
●
●●
●
●●
●
●●
●
● ●●
●●
●●
●●● ●
●●
● ● ● ● ● ●
●● ●
●●
●●
●● ●
●
●
●
●
●
●●
● ●
●
● ●● ● ●
●●
● ● ● ● ● ● ● ●
●
●●
● ● ● ●
●
● ●●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
● ● ● ● ● ● ● ●
● ●
●
●
●
●●
●
●
● ●
●
●
●
●●
● ●
●
● ●●
●●
●●
● ●
●
● ● ● ● ● ●● ●
●
214339573152061
46
36 7
1746
46177
612134433913344
2
33463317
334617
504461341321 1053
1730
3017 332233
50443461131453 61
3017 473330
2332
Figure 4: Individual (posterior Bayes) predictions versus time. The predictionsfrom each individual are connected with a line and each data point is indicated bya circle. Extreme data points (outermost 5%) are labeled with the ID-number ofthe corresponding individual. The red line is a smooth.
This graph is used for visualization of theindividual predictions over the main predic-tor (idv). Its usefulness lies more in modelcommunication rather than in goodness-of-fit or diagnostics since there is no referenceto which it should be compared. See also
“dv.preds.vs.idv” on page 19.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ipred.vs.idv(xpdb, ids = TRUE)
> print(xplot)
13
cov.hist
WT
Den
sity
0.000
0.005
0.010
0.015
0.020
0.025
60 80 100 120 140
SEXC
ount
0
10
20
30
40
1 2
RACE
Cou
nt
0
10
20
30
40
1 2 3
AGE
Den
sity
0.00
0.01
0.02
0.03
0.04
30 40 50 60 70
Distribution of covariates (Run 1)
Figure 5: Histograms of WT, SEX, RACE and AGE. For the continuous covariatesthe y-axes indicates the probability density and the line is a smoothed represen-tation of this density. For the categorical covariates, the y-axes shows the countsof individuals in the respective covariate category.
This graph is used for visualization of the dis-tribution of the covariates. By default only thevalues on the first row of each individual is plot-ted, i.e. the display shows the distribution ofthe covariates at baseline (assuming that thefirst row is at time 0).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("WT",
+ "SEX", "RACE", "AGE")
> xplot <- cov.hist(xpdb)
> print(xplot)
14
cov.qq
Quantiles of Normal
Qua
ntile
s of
WT
60
80
100
120
140
−2 −1 0 1 2
●●
●●●●●●●●
●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●
●
●
●
Quantiles of Normal
Qua
ntile
s of
AG
E
30
40
50
60
70
−2 −1 0 1 2
●
●
●●
●●●●●●
●●●●●●●●●●●●●
●●●●●
●●●●●●●●
●●●●●●●
●●●●
●●●●●●●
●●●●●
●●● ●●
Distribution of covariates (Run 1)
Figure 6: QQ-plots of WT, SEX, RACE and AGE. The quantiles of the distributionof body weight and age are plotted versus quantiles of the Normal distribution.The black diagonal lines is the line of identity.
This graph is used for visualization of the dis-tribution of the covariates. By default only thevalues on the first row of each individual is plot-ted, i.e. the display shows the distribution ofthe covariates at baseline (assuming that thefirst row is at time 0).
This is a very powerful way of comparing thedistribution of a variable to the Normal distribu-tion. For covariates we typically do not makeany assumptions regarding the distribution of
covariates so deviations from the line of iden-tity is usually no cause to worry.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("WT",
+ "AGE")
> xplot <- cov.qq(xpdb)
> print(xplot)
15
cov.splom
Scatterplot matrix of covariates (Run 1)
WT100
120
140100 120 140
60
80
100
60 80 100
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●● ●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●● ●
●
●●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
AGE50
60
7050 60 70
30
40
50
30 40 50
Figure 7: Scatterplot matrix of WT, AGE and SEX. Each panel includes the datafrom each individual and each pair of covariates are plotted twice, once in theupper triangle and once in the lower. The red lines are smooths.
This graph is useful for investigating the corre-lation between covariates. As with all covariateplots, the default is to only use the data fromthe first row of each individual.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("WT",
+ "AGE", "SEX")
> xplot <- cov.splom(xpdb)
> print(xplot)
16
cov.summary+----+--------+--+----+
| |Category|N |% |
+----+--------+--+----+
|RACE|1 |43|67.2|
| |2 |20|31.2|
| |3 |1 |1.6 |
+----+--------+--+----+
|SEX |1 |42|65.6|
| |2 |22|34.4|
+----+--------+--+----+
+---+-----+-----+-----+------+-----+-----------+--+
| | Mean| SD| Q1|Median| Q3| Range| N|
+---+-----+-----+-----+------+-----+-----------+--+
| HT|171.2|10.38| 165| 173| 179| 140-188|64|
+---+-----+-----+-----+------+-----+-----------+--+
| WT|86.07|15.54|75.97| 85.35|96.15|51.03-136.8|64|
+---+-----+-----+-----+------+-----+-----------+--+
|AGE|53.88|10.34| 49.5| 56| 62| 24-69|64|
+---+-----+-----+-----+------+-----+-----------+--+
This command can be used to summarize thecovariate information in the data set. By de-fault, only the covariate values present in thefirst data record of each individual are used,i.e. the baseline values of the covariates. Thisbehavior can be changed by setting the argu-ment onlyfirst to FALSE. It is also possible to
re-direct the output to a csv file which can beimported into spreadsheet or word processingprograms.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> cov.summary(xpdb)
17
Overall goodness of fit assessmentOnce a model candidate has been fit to the data we need to check if it is any good. The firstlevel of such checks can be quite high level; for example, does a plot of the individual predictionsversus time look anything like the real observations versus time. If the model fails at this level itprobably needs some substantial changes and we don’t need to worry about minor deficienciesin the residual error model.
Sometimes one graph is not enough for this high level check of the model and we may needcomposite displays of two or more graphs, for example the output from the basic.gof function.
Overall goodness of fitness plots are also well suited for communicating the appropriateness ofthe model to a wider audience in a succinct manner.
18
dv.preds.vs.idv
Observations / Individual predictions / Population predictions vs. Time (Run 1)
Time
Obs
erva
tions
/Pre
dict
ions
0
50
100
150
2 4 6 8 10 12
● ●
●
● ●● ●
● ●
●
●
● ●
●
●
●
●● ●
● ● ● ● ● ● ● ● ● ●
● ●
●
●●
● ● ● ●
●● ● ● ●
● ● ● ●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
● ● ●● ● ● ● ●
●●
●●
●●
● ●
●
●
● ● ●
● ●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
●●
● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ●●
● ● ● ●● ● ● ● ●
●
●
●
●●
●
● ●
●
●
●● ● ●
●● ● ● ● ●
●●
●
● ●● ●
●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ●● ●
● ●● ●
●
● ● ●
● ● ●● ● ● ● ● ● ●
●●
●● ● ● ● ● ● ●
●
●
●● ●
● ●● ● ●
● ●
●
●●
●● ●
●●
●
●●
● ● ●●
● ●
●
●
● ●
● ● ●
● ●
●● ● ●
● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
●
●
●
●●
● ●● ●
●
● ● ● ● ● ● ● ●
●
●
● ●● ● ●
●
●
●
●
●
●
● ●● ● ●
● ●●
●
●
●
●●
●●
● ● ● ● ● ● ● ●
●
● ● ●● ● ● ● ● ●
● ●● ● ●
● ●● ●
●
●
●
●
●● ●● ● ● ● ● ● ● ● ● ●
●●
●●
● ● ● ● ● ●
●
●
● ●
● ●
●
●
●
● ●●
●●
●●
●●
●
●
● ● ●●
●●
● ● ●● ●
●● ● ● ●
● ● ● ● ●
●
●
●●
● ● ● ● ● ●
●● ● ●
● ●
● ● ●●
●
●● ●
●● ●
● ●● ●● ● ● ● ● ● ● ●
●● ● ● ●
● ● ● ● ●
●
●
●
●
● ●● ● ●
●● ● ● ● ●●
● ● ● ● ● ● ● ● ●
●
● ● ● ● ● ●
● ● ●
●
● ● ●
●
●
●●
● ●● ● ●
● ●● ● ● ● ● ● ● ● ● ●
● ●● ● ●
● ● ●●
●●
● ●●
● ● ● ●
● ● ● ●● ●
● ●● ●
●
● ●● ● ● ● ● ● ●
214357392615316
46
36 177
46 46
177
346121433931353
1726 3333 1717
46
50344461134321 10
1746 171 233302
50443461131453 61
1 3317 47302 233
Observations
2 4 6 8 10 12
●●
●●
●●
●●
●
● ●●
●●
●●
●●
●
● ● ● ● ● ● ● ● ● ●
●●
●
●●
● ● ● ●
● ● ● ● ● ● ● ● ●
●
● ●●
●●
●●
●
●
●
●
●
●
●
●
●●
●● ● ● ● ● ●
●●
●●
●●
● ●
●
●
●● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
●●
● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●●
●
●●
● ●●
●● ● ● ●
● ●
● ●●
●●
● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●
● ● ● ● ● ● ● ● ●
● ●●
● ● ● ● ● ● ●●
● ● ● ● ● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ●●
●●
●●
●●
●●
● ● ● ● ● ●●
●
●
● ●●
●●
●●
●●
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
●●
●
●●
●●
●
●●
● ● ● ● ● ● ● ●
●● ●
●●
● ●●
●
●
●
●
●●
● ● ● ●
●● ●
●●
●
●
●●
●● ● ● ● ● ● ● ●
●●
● ● ● ● ● ● ● ●
● ● ●●
● ● ● ●● ●
●
●
●
●● ●● ● ● ● ● ● ● ● ● ●
●●
●● ● ● ● ● ● ●
●
●
● ●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
● ● ● ● ●
●● ● ● ● ● ● ● ● ●
●
●
●●
● ● ● ● ● ●
●● ● ● ● ●
●● ●
●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
●
●
●
●
●● ● ● ●
● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●
●
● ● ● ● ● ●
●● ● ●
● ● ●●
●
●
●●
●● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ●●
●● ● ● ●
●
● ●●
●●
● ● ● ●
●● ● ● ● ●
● ● ● ●
●● ● ● ● ● ● ● ● ●
214339573152061
4636 7
174646177
612134433913344
233463317
334617
504461341321 1053
17303017 332233
50443461131453 61
3017 473330 2332
Individual predictions
2 4 6 8 10 12
● ●
●
●
●●
●●
●
● ●
●
●
●●
●●
●●● ● ● ● ● ● ● ● ● ●
● ●●
●●
● ●● ●
● ● ● ● ● ● ● ● ●
● ●
●
●
●●
●●
●
● ●
●
●
●●
●
● ●●
●●
● ●● ●
● ●●
●●
●● ●
● ●●
●●
● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ●
●
●
●●
●●
●●
● ●●
●●
● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●
● ● ●● ● ● ●
● ● ●● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ●
●
●
●●
●●
●●
●●
●
●●
●●
●●
● ●
●
●
●●
●●
●●
● ● ● ● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ●
●
●
●●
●●
●●● ● ● ● ● ● ● ●
● ●●
●●
● ●●
● ●
●
●
●●
●●
●●
● ●●
●●
● ●●
●●
● ● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
● ●
●
●
●
●
● ● ● ● ● ● ● ● ● ●
● ● ●● ● ● ● ● ● ●
●
●
●●
● ●
●
●
●●
●●
●●
●
●
●
●
●
● ●●
●●
● ● ● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ●●
●●
●● ● ●
● ●●
●●
● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ●●
●●
● ● ● ● ●
● ●
●
●
●●
●●
●● ● ● ● ● ●
● ●●
●●
● ● ● ● ●
●● ● ● ● ● ●
● ●●
●● ● ●
●
● ●●
●●
● ● ● ● ●● ● ● ● ● ● ● ● ● ●
● ●●
●●
● ● ● ●
● ● ●● ●
● ● ● ●
● ●●
●●
●● ● ● ●
● ●●
●●
● ●● ● ●3213943535661 3
1728303336424655
321343943535661
2930333642464755
3213439435361 3
1728293033364655
32134394353566126172829303336
Population predictions
Figure 8: Observed data, individual and population predictions versus time. Thedata points from each individual are connected with a line and each data pointis indicated by a circle. Extreme data points (outermost 5%) are labeled with theID-number of the corresponding individual. The red line is a smooth.
This graph is used for an overall visualiza-tion of the performance of the model. If thetwo right panels (the individual and popula-tion predictions respectively) closely resem-bles the left panel (the observed data), thereis not much more the model can be improved.The difference between the middle and righthand panel gives an indication of the maxi-
mum improvement the inclusion of covariatesmay have.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.preds.vs.idv(xpdb,
+ grid = TRUE, ids = TRUE)
> print(xplot)
19
ind.plots
Individual plots (Run 1)[ID<=17]
Time
Obs
erva
tions
/ P
redi
ctio
ns
0
50
100
150
●●●
●●●●
● ●
ID:1
2 4 6 8 10
●●
●●●
●●
● ● ●
ID:2
●●●●●●●● ● ●
ID:3
2 4 6 8 10
●●●
●●●● ● ●
ID:4
●●●●●●● ● ●
ID:5
●●
●●
●●●● ●
ID:6●
●
●●
●●
●
ID:7
0
50
100
150
●●●●●●● ● ●
ID:8
0
50
100
150
●●●
●●● ● ●
ID:9
●
●● ● ●
ID:10
●●●●●●● ● ●
ID:12
●●●●●●●● ● ●
ID:13
2 4 6 8 10
●●●●●●●● ● ●
ID:14
●●●●●●●● ● ●
ID:15
2 4 6 8 10
●●●●●●●● ● ●
ID:16
0
50
100
150●
●
●●●
●●●
●●
ID:17
DV IPRED PRED● ● ●
Figure 9: The observed data, individual and population predictions versus timefor each individual are plotted in separate panels. The ID-number of each individ-ual is given in the strip above each panel. The predictions are connected with aline and the observed data points are indicated by symbols.
This graph is used for an overall visualizationof the performance of the model. As an overallvisualization of a fit it is useful, especially forsingle dose PK profiles, or if time after dose isused as the independent variable. The plotscan be misleading, especially if there is highshrikage, leading to the appearance of a “per-fect fit” between the individual preditions andthe observed data. Additionally, one “diag-nostic signal” in the plot is the differences be-tween observed and predicted values, some-
thing that is not the focus of this display. Resid-ual and/or observed vs predicted plots are bet-ter diagnostic tools to highlight these differ-ences.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ind.plots(xpdb, grid = TRUE,
+ subset = "ID<=17")
> print(xplot)
20
basic.gof
Population predictions
Obs
erva
tions
0
50
100
150
0 20 40 60 80
● ●
● ● ● ●
●
●●
● ●●
●
●
●● ●
●●
●●●●●●●●●●●● ●● ●
●
●
●●
●●●●●●●
●●
●●
●● ●
●
●
●●
●
●
●●
●
●
●
●● ●● ●● ●
●
●
●● ●●
●●
●●
● ● ●
●
●
●●●●●●●
●●
●●●●●●●●●●
●●●●●●●●
●●
●●●●●●●●●●●●●
●●●●●
●●●
●
● ●
●
●●
●
●
●
●●●●●●
● ●●●
●●●
●●● ●
● ●●
●●●●●●●●●
●●●●●●●●●●●●● ●●●
●●●●●●● ●●●●●●●●●● ●
●●
●●●●●●●
●●●
●●●●●●● ●
●●
●●●●●● ●
● ●
●
●●
● ●●
●●
●●●
● ●●
● ● ●●
●
●●
● ●
● ● ●
● ●
●
●
●●●●●●●
●●●●●●●●●● ●●●
●
● ●● ●
●●
● ●
●
●●●●●●●●●
●● ●● ● ●
●
● ● ●● ●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●●●●●●●●●●●●●● ●
●
●●●●
●●●● ●
●●
● ●●
●
●
●
●●●●●●●●●●●●●●●●●●
●●
● ●
●
●
●●
●● ●
●
●
●
●●
●●
●●
●
●●●●●
●●
●● ●
●
●
●●●●●● ● ● ●
●
●●●●● ●●
●
●
●
●●●● ●
● ● ●●
●●●●
●● ●
● ●
●
●●●●●●●●
●●
●●●●●● ● ● ●
●
● ● ●● ●
●
●
●
●
●●●●●
●●●●●● ● ● ●●●
●●●●●●●
●
●●●
●● ●●
●●●●●● ●
●
●
●
●●●●●●●●●●●●●●
● ● ●●●
●●●●●
● ●
●●
●●●●
● ●● ● ●●
●●● ●● ● ●●
●
●
Individual predictionsO
bser
vatio
ns
0
50
100
150
0 50 100 150
●●
●●●●
●
●
●
●●●
●
●
●●●
●●
●●●●●●●●●●
●●●●
●●
●
●●
●●●●●●●●
●●●
●● ●
●●
●
●
●
●
●●
●
●
●
●●●●●●●●
●
●●●●
●●
●●
●●●
●
●
●●●●●●●●●
●●●●●●●●●
●●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●
●
● ●
●
●●
●
●
●
●●●●●
●●●●
●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●
●●●●●●●●●●
●●●●
●●●
●●
●
●●
●●●
●●
●●●
●●●●●●●
●●
●
● ●
●●●●
●●
●
●●●●●●●●
●●●●●●●
●●●●●●
● ●●●
●●
●●
●
●●●●●●●●
●●●
●●●●
●
●●●●●
●●
●
●
●
●
●● ●●
●●
●●
●
●●●●●●●●●●●●●●
●●●
●
●●●
●●●●●
●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●
●●
●
●●
●
●
●●
●● ●
●
●
●
●●
●●
●●
●
●●●●●
●●
●● ● ●
●
●●●●●●●●●●
●●●●●●●
●
●
●
●●●●●
●●●●
●●●●●
●●●●
●
●●●●●●●●
●●●●●
●●●●●●
●
●●●●●
●
●
●
●
●●●●●
●●●●●●●●●●●
●●●●●●
●●
●●●●
●●●
●●●●●
●●
●●
●
●●●●●●●●●●
●●●●●●●
●●
●●●●
●●●
●●
●●●●
●●●●●●
●●●●●●●●●●
Individual predictions
|iWR
ES
|
0.0
0.1
0.2
0.3
0 50 100 150
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●●●
●
●
●● ● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●
●
●● ●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●●● ● ●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
Time
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
● ●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●●
●●
●● ● ●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
● ●
●●
● ●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●
●
● ●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
● ●
●
●●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
● ●
●● ●
●
●
●
●●
●
●
●
●●
●
●●
●
Basic goodness−of−fit plots (Run 1)
Figure 10: Basic goodness of fit plots. The top panels show the observed data versus the population andindividual predictions, respectively. The bottom left panel shows the absolute values of the individuallyweighted residuals and the bottom right panel shows the population conditional weighted residuals.Individual data points are indicated by blue circles and are connected by blue lines (except bottom leftpanel). Extreme values are labeled by the corresponding ID-number. The red lines are smooths, theblack diagonal lines (top panels) are the lines of identity and the black horizontal line (bottom right) isthe zero line.
Overall goodness of fit assessment usuallymeans considering multiple aspects of the per-formance of the model. Typically this involvesinspecting multiple graphs. In this display fourmainstay goodness of fit plots are collected onthe same page (see below).
In this version of this display, the conditionalweighted residuals are used in the bottom right
panel. The argument force.wres=TRUE to thebasic.gof function determines which versionof the weighted residuals are used.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- basic.gof(xpdb)
> print(xplot)
21
addit.gof
Population predictions
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●● ●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
● ●●
●
●
● ●
●●
● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
● ●●
●
●●
●
●
●
●
●●
●●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●
●
TimeA
bsol
ute
valu
e of
iWR
ES
0.0
0.1
0.2
0.3
2 4 6 8 10 12
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
● ● ●
● ●
●●● ●
● ●
●
●
●
● ●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
● ●
●
●●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●
● ● ●
●
●
●
●
● ●
● ●
●
●
● ● ●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
● ●
● ● ● ●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●
●●
●●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
● ●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●● ● ● ●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●● ●
●
● ●●
●
●
●
●
●
● ●
●
● ●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
● ●
●●
●
● ●
●
●
●● ●
●●
● ●●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
Time
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
2 4 6 8 10 12
Time
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
1 10
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
● ●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
● ●
●
●●
●●
●● ● ●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
● ●
●●
● ●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●
●
● ●
●
●●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
● ●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
● ●
●● ●
●
●
●
●●
●
●
●
●●
●
●●
●
Additional goodness of fit plots (Run 1)
Figure 11: Additional goodness of fit plots. The top left panel show the conditional weighted residu-als versus the population predictions, the top right panel shows the absolute values of the individualweighted residuals versus time, the bottom left panel shows the smooth of the conditional weightedresiduals versus time (data points are suppressed) and the bottom right panel shows the conditionalweighted residuals versus the logarithm of time. Individual data points are connected by blue circles andextreme values are labeled by the corresponding ID-number.
This display shows another set of goodness offit plots. The two top panels are elaboratedon elsewhere. The bottom panels are a com-plement to the bottom right panel of the basicgoodness of fit plots. In this version the datapoints are omitted to facilitate the comparisonof the smooth and the zero line (which ideallyshould overlap). The bottom right panel has
the idv(=x)-axis on the log-scale. The latteris useful if there are large x-values that domi-nate the display.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- addit.gof(xpdb)
> print(xplot)
22
runsum
Summary of run 1
050
100
150
0 20 40 60 80
● ●
● ● ● ●
●●●
● ●●
●●
●● ●
●●
●●●●●●●●●●●● ●●● ●
●
●●
●●●●●●●●●
●●
●● ●
●●
●●
●●
●●
●
●
●
●● ●●● ● ●
●
●●● ●
●●
●●●
● ● ●
●
●●●●●●
●●●●●●●●●●
●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●
●● ●
●
● ●
●
●
●
●●●●●●
● ●●●
●●●
●●● ●
● ●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●
●●●●●●●●●●
●●●●●● ●
● ●●
●●
● ●● ●
●●
●●
● ●●
● ● ●● ●
●●● ●
● ● ●● ●
●
●
●●●●●●●●●●
●●●●●●●●●●●
● ●● ●
● ●● ●
●
●●●●●●●●●
●●● ● ● ●●
● ● ● ● ●●
●
●
●
●
●●
●●●
●●
●
●
●
●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●
● ●●
●
●
●
●●●●●●●●●●●●●●●●●●●●
● ●●
●
● ●
●● ●
●
●●
●●
●●
●●
●
●●●●●
●●●
● ●
●
●
●●●●●● ● ● ●
●
●●●●●●●
●
●
●
●●●●●
● ● ●●
●●●●●● ●
● ●
●
●●●●●●●●●●
●●●●●● ● ● ●●
● ● ●● ●
●
●
●
●
●●●●●●●●●●●● ● ●●●
●●●●●●●
●●●●
●● ●●
●●●●●● ●
●
●
●
●●●●●●●●●●●●●●
● ● ●●●
●●●●●
●●●●
●●●●
●● ● ● ●●●●● ●●● ● ●●●
DV vs PRED
050
100
150
0 50 100 150
●●
●●●●
●●●
●●●●●
●●●
●●
●●●●●●●●●●●●●●
●●
●
●●
●●●●●●●●
●●●
●● ●
●●
●
●
●●
●●
●
●
●
●●●●●●●●
●
●●●●
●●
●●
●●●●
●
●●●●●●●●●
●●●●●●●●●
●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●
●
● ●
●
●
●
●●●●●
●●●●
●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●
●●●●●●●●●●
●●●●●
●●●●
●
●●
●●●●
●●
●●
●●●●●●●
●●
●● ●
●●●●●●
●
●●●●●●●●
●●●●●●●
●●●●●●
● ●●●
● ●●●
●
●●●●●●●●●●●
●●●●
●
●●●●●
●●
●
●
●
●●● ●
●
●●
●●
●
●●●●●●●●●●●●●●●
●●●
●●●●●
●●●●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●●●
●
●●●
●
● ●
●● ●
●
●●
●●
●●
●●
●
●●●●●●
●●● ● ●
●
●●●●●●●●●●
●●●●●●●
●
●
●
●●●●●
●●●●
●●●●●
●●●●
●
●●●●●●●●●●
●●●●●
●●●●●
●●●●●
●
●
●
●
●●●●●●
●●●●●●●●●●
●●●●●●
●●
●●●●●●●
●●●●●
●●●
●
●
●●●●●●●●●●
●●●●●●●
●●
●●●●
●●●
●●
●●●●
●●●●●●
●●●●●●●●●●
DV vs IPRED
0.0
0.1
0.2
0.3
0 50 100 150
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●●●
●
●●
●●●●●
●
●
●
●●
●●●●
●
●●
●
●●
●●
●
●
●● ●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●●●
●●●●
●●
●● ● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●●●●
●
●● ●
●
●
●
●●
● ●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●●
●●● ● ●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
|IWRES| vs IPRED
−2−
10
12
3
2 4 6 8 10 12
●
●
●
●
●●
●
●
●●
●●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
● ●
● ●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
● ●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●●
●
● ● ●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
● ●
●
●●
●●
●● ● ●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
● ●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ● ●
●
●
● ●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●
●
●
●
●●●
● ●
●●
● ●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
● ●
●
●
●
●
●●
●
●● ●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●● ●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●
●
● ●
●
●●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
● ●
●
●●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
● ●
●● ●
●
●
●
●●
●
●
●
●●
●
●●
●
CWRES vs TIME
MINIMIZATION SUCCESSFULNO. OF FUNCTION EVALUATIONS USED: 125NO. OF SIG. DIGITS IN FINAL EST.: 3.4ETABAR IS THE ARITHMETIC MEAN OF THE ETA−ESTIAND THE P−VALUE IS GIVEN FOR THE NULL HYPOTHETABAR: 7.5792E−03 −7.8643E−03 −7.3866E−02SE: 5.5890E−02 4.3281E−02 7.8359E−02P VAL.: 8.9213E−01 8.5582E−01 3.4585E−01
ETAshrink(%): 7.6098E−01 6.6939E+00 1.7430E+01EPSshrink(%): 1.4530E+01
Objective: 1882.75345663931 ParTH1TH2TH3OM1:1OM2:2
Val17.748176.79731.442560.45 0.37
RSE0.0590.0520.1080.130.19
ParOM3:3SI1:1
Val0.77 0.13
RSE0.290.065
$PROB Simulated prazosin data;; 1. Based on: First run [nodOFV];; Initial run that is to cover as many options as possible.;; 2. Structural model:;; One compartment model with first order absorption;; 3. Covariate model:;; No covariates;; 4. Inter−individual variability:;; ETAs on CL, V and KA;; 5. Residual variability:;; Proportional;; 6. Estimation:
;; NM7, FOCE INTER$INPUT ID SEX AGE RACE HT SMOK HCTZ PROP CON DV RATE EVID OCC$DATA newdata2.dta IGNORE=@$SUBROUTINE ADVAN2 TRANS2$PK TVCL = THETA(1) TVV = THETA(2) TVKA = THETA(3) CL = TVCL*EXP(ETA(1)) V = TVV *EXP(ETA(2)) KA = TVKA*EXP(ETA(3)) S2=V$ERROR IPRED = F IRES = DV − F W = F IF(W.EQ.0) W = 1 IWRES = IRES/W Y = IPRED + W*EPS(1)$THETA (0,18.7) ; CL (L/h) (0,87.3) ; V (L) (0,2.13) ; KA (1/h)$OMEGA 0.128 ; omCL 0.142 ; omV 1.82 ; omKA$SIGMA 0.0231 ; Prop error$EST NOABORT METHOD=1 INTER PRINT=1 MSFO=m$COV PRINT=E$TABLE ID TIME IPRED IWRES CWRES CL V KA ETA1 ETA2 ETA3
Figure 12: Summary of run 1.
This display is intended as a (printed) one sidesummary of a run. It combines the basic good-ness of fit graphs (see section “basic.gof” onpage 21), some NONMEM output and the modelfile. (The model file is allotted the remain-ing space after the graphs and NONMEM outputhave been included. If there is not enoughspace the model file will be truncated.) Theparameter estimates and the relative standard
errors (RSE) are given, were the latter is theSE/estimate. The variability parameter esti-mates are converted to standard deviationsand correlations (but the RSE will still refer tothe variance and covariance estimates).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> runsum(xpdb, dir = "../")
23
Structural model diagnosticsThe first step in a typical model development process is to get the structural (fixed effects)model describing the data well. If the overall diagnostic plots indicate that there are someissues with the model, a reasonable first step is typically to investigate if the structural model isappropriate.
24
dv.vs.pred
Observations vs. Population predictions (Run 1)
Population predictions
Obs
erva
tions
0
50
100
150
0 20 40 60 80
● ●
● ●●
●
●
● ●
● ●
●
●
●
●
●●
●
●
●●●●●●●●●●
●●
●●
●
●
●
●●
●● ●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●● ● ●
●●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●● ●● ●
● ●
●●
●●●●●●
● ●●●
●●●●●
●●
●
●
●
●●●●● ● ●
●
●●
●●●
●●
● ● ●
●
●●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●●
●●
● ●
●
●● ●
● ●●
● ●●
●●●●●●●●● ●● ●
●●
●●
● ●● ● ●
●● ●
●●
● ● ●
●
●●
● ●
●●
●●● ●●
●●
●
●●
●● ●
●● ● ●
●
●
●
● ●●
●●
●●
●●
●
●
●
● ●
●
●
●
●
●●
● ●
●
● ●●
●
●
●●
● ●
●●
●
●●
●
●
●●●
●● ● ●
●●●
● ●● ● ●
● ● ●●●
●
●●
●●
●
●
● ●
●
●●●●●●
●●
●
●●
●
● ●●
●
● ●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●●●●
●● ● ● ●
●
●●
●
●
●
● ●
● ●
●●
●
●●
●●
●
●
●
●
●●●●●●●●●●
● ● ● ● ● ●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●●
● ●●
●
●
●
●●
●
●
● ●●
●●
●●
●
●
●
● ● ● ● ●●
●
●
●
●
● ●●
●●
●●
●
●
●●
● ●
●
●●
● ●
●
●●●●●●●●
●●
● ● ●● ●
● ●●
●
●
● ●●
● ●
●
●
●
●
●●●●●
●●
● ●● ● ●
● ●
●
●
● ● ● ●●
●
●
●
● ● ●
●
● ●●
● ●
●●
●
● ●
●
●
●
●●●●●●●●
●●
●●
● ●
●●
●
●●
●●●
●
●
●●
●
●
●●
●●
●●
●● ●●
●●
● ●● ●
●
●
●
●
105055508443610
33
4147
4130233
2
574210
575410
55
55
332
30
3817
60
2
48
42
55
55
2936
2936
1
246
63317
3317
46
29
42
29
306
421
55
46
1736
7
4646
17
7
Figure 13: The observed data versus the population predictions. The data pointsfrom each individual are connected with a line and each data point is indicatedby a circle. Extreme data points (outermost 5% of the data) are labeled with theID-number of the corresponding individual. The black line is the line of identityand the red line is a smooth.
This graph is a mainstay goodness of fit plot.The aim is for the line of identity to go throughthe middle of the data density. A good signis if the smooth and the line of identity over-lap, although this may occur even if the modelcorrectly describes the data. If the struc-tural model does not include important covari-
ates, the graph can have a marked vertical"chopped" look to the far right.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.pred(xpdb, ids = TRUE)
> print(xplot)
25
dv.vs.ipred
Observations vs. Individual predictions (Run 1)
Individual predictions
Obs
erva
tions
0
50
100
150
0 50 100 150
● ●
● ●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●●●●●●●
●●
●●
●●
●
●
●
● ●
●●●●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●●●●
●●
●●
●●●●●●
●●●●
●●●●●
●●
●
●
●
●●●●●●●●
●●
●●●●
●●
●●●●●
●
● ●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●●
●●
●●
●
●●●
●●●●●●
●●●●●●●●●●
●●●
●
●●
●●●●●
●●●●●
●●●●
●
●●●
●●
●●●●
●●
●
● ●●
●●
●●●●●●●●
●●●
●●
●●
●●
●
●
●
● ●
●
●
●
●
●●
● ●
●●
● ●●
●
●
●
● ●
●●
●●
●●
●
●●●●●●●●
●●
●●●●●
●●●●●
●
●●
●●
●
●
● ●
●
●●●●●●
●●
●
●●
●
●●●
●
●●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●●●●●
●●●●●
●
●● ●
●
●
● ●
● ●
●●
●
● ●
●●
●
●
●
●
●●●●●●●●●●
●●●●●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●●●●●
●●
●●●
●●●●●●
●
●
●
●
● ●●
●●
●●
●●
●●
●●
●
●●
●
●
●
●●●●●
●●●
●●
●●●●●
● ●●●●
●●●
● ●
●
●
●
●
●●●●●
●
●●●●●●●●●
●
●●●●●
●
●
●
●● ●
●
●
●●
●●
●●
●
● ●
●
●
●
●●●●●●●●
●●
●●
● ●
●●
●
●●
●●●
●
●
●●
●
●
●●
●●
●
●●
●●●
●●●●●
●●●
●● 52
18522935137
24
19
1846
241
355233
48123030
19
33
59
4
59
199
17
1
61
6
7
1746
30
42
2
172
36
33
462
630
48
17
46
36
46
17
7
7
17
46
36
46
17
7
7
Figure 14: The observed data versus the individual predictions. The data pointsfrom each individual are connected with a line and each data point is indicated bya circle. Extreme data points are labeled with the ID-number of the correspondingindividual (outermost 5% of the data). The black line is the line of identity and thered line is a smooth.
This graph can be used for overall goodness offit assessment if the epsilon shrinkage is low.The aim is for the line of identity to go throughthe middle of the data density. A good sign isif the smooth and the line of identity overlap.If the degree of shrinkage is low this can be avery informative graph. If there is high shrink-
age (over 30%) then this plot may show themisleading “perfect fit” phenomena.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.ipred(xpdb, ids = TRUE)
> print(xplot)
26
dv.vs.pred.ipred
Observations vs. Population predictions / Individual predictions (Run 1)
Predictions
Obs
erva
tions
0
50
100
150
0 50 100 150
● ●
● ●●
●
●
●●
● ●
●
●
●
●
●●
●
●
●●●●●●●●●●●●
●●
●●
●
●●
●●●●
●●●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●● ●
●
●
●●●
●
●
●
●●
● ●●
●
●
●●●●●
●●●●
●●●●●●●●
●●
●●●●●●●
●
●●
●●●●●●●
●●●
●●●●●
●●●●●●
●
● ●
●
●●
●
●
●
●●
●●
●
●
● ●
●
●
●●
●
●●
● ●
●●
●
●●●●
●●●●●
●●●●●●●●●●
●●●
●●●
●●●●●●●●●●●●●
●●
●●●
●●
●●●●●
●●●
●●
●●●
●●●●●
●●
●●●●
●
●●
●●
●
●
●
● ●
●●
●
●
●●
● ●
●
● ● ●
●●
●●
● ●
● ●●
● ●
●
●
●●●●●●●
●●●
●●●●●
●●●●●
●
● ●
●●
●●
● ●
●
●●●●●●●●●
●●●
● ●●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●●●●
●●●●●●
●●
●
●
●●●
●●
●●
●
●●
● ●
●
●
●
●
●●●●●●●●●●
●●●●●●●
●
●●
● ●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●●
● ●
●
●
●●●●
●
●●
●●●
●●●●●●●
●
●
●
●●●
●●
● ●●●
●●
●●●
● ●●●
●
●●●●●●●
●●●
●●●●●
● ● ●●●
● ● ●
● ●
●
●
●
●
●●●●●
●●●
●●●● ● ●●●
●●●●●
●
●
●
●●●
●
●●●
●●●●●
● ●
●
●
●
●●●●●●●●●●●
●●●
● ● ●
●●
●●●●
●
●●
●●
●●●●
●●
●●●●
●●● ●●●
●●
●
●
105055508443610
3341
4741302
332
574210575410
5555
33230
381760
2
48
42
55
55
2936
2936
1246
63317
3317
46
294229
306421
55
46
17367
4646
177
Population predictions
0 50 100 150
● ●
● ●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●●●●●●●●●
●●●
●●
●
●
● ●
●●●●
●●●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●●
●
●●●
●
●
●
●●
●●●
●
●
●●●●●
●●●●
●●●●●●●
●●●
●●●●●●●
●●
●
●●●●●●●●
●●
●●●●●●
●●●●●
●
● ●
●
●●
●
●
●
●●
●●
●
●
● ● ●
●
●●
●
●●
● ●
●●
●
●●●●●●●●●
●●●●●●●●●●●
●●
●
●●
●●●●●●●●●
●●●●●
●
●●●●●
●●●●●
●●● ●
●
●●●●●
●●●●●
●●●
●●
●●
●●
●
●
●
● ●
●●
●
●
●●
●●
●●●●●
●●
●
● ●
● ●●●
● ●
●
●●●●●●●
●●●
●●●●●
●●●●●
●
● ●
●●
●●
●●
●
●●●●●●●●●
●●●
●●●
●
●●●
● ●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●●●●●●●●
●●●●●
●●
●●
●
●●●
●●
●●
●
●●
●●
●
●
●
●
●●●●●●●●●●
●●●●●●
●●
●●
●●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●●
● ●●
●
●●●●●●●●●●
●●●●●●
●
●
●
●
●● ●
●●
●●●●
●●
●●●●●●
●
●
●●●●●●●
●●●
●●●●●
●●●●●
●●●
● ●
●
●
●
●
●●●●●
●
●●●●●●●●●●
●●●●●●
●
●
●●●●
●
●●
●●●●●
● ●
●
●
●
●●●●●●●●●●
●●
●●
●● ●
●●
●●●●
●
●●
●●
●●●●
●●●
●●●●●●●●
●●●
●● 52
18522935137
24
191846241355233
48123030
1933
59
4
59199
17
1
61
6
7
1746
30
42
2
17236
33
462
630
48
17
46
36
46
17
7
7
17
46
36
46
17
7
7
Individual predictions
Figure 15: The observed data versus the population and individual individualpredictions. The data points from each individual are connected with a line andeach data point is indicated by a circle. Extreme data points are labeled with theID-number of the corresponding individual. The black line is the line of identityand the red line is a smooth.
This is the combination of the previous twographs and is also used for overall goodnessof fit assessment. Apart from the obvious com-pactness of the display, this graph has theadded benefit of providing a direct compari-son of the populations and individual predic-tion. For example, it may give an indication asto how much (or little) of the variability the fixed
effect part of the model explains.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.pred.ipred(xpdb,
+ grid = TRUE, ids = TRUE)
> print(xplot)
27
dv.vs.pred.by.cov
Observations vs Population predictions (Run 1)
Population predictions
Obs
erva
tions
0
50
100
150
0 20 40 60 80
● ●
●
●
●
●
●●
●
●
●●●●●●●●●●
●●●●●●●
●●
●●●●●●●●●●
●●●●●●●
●
●●
●●●●●●●●
●●●●
●●●
●●●●●●
●
● ●
●
●●
●
●
●
●●
●●
●
●
● ●
●
●
●●
●
●●
● ●
● ●
●
●●●●●●●●●
●●●●●●●●●●● ●
●●
●●
●●●●●● ●
●●●
●●●
●●
●●
●●
●
●
●
● ●
●●
●
●
●●
● ●
●
● ● ●
●●
●●
● ●
● ●●
● ●
●
●
●
● ●
●●
●●
● ●
●
●●●●●●●●●
● ●●
● ● ●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●●●●●● ●●● ●
●●
●
●
●● ●
●●
●●
●
●●
● ●
●
●
●
●
●●●●●●●●●●
●● ●●● ●●
●
●●
● ●
●
●
●
●
●
●
●
●●●●●
●
●●
● ●
●
●
●● ●
●●
● ●●●
●●● ● ● ● ● ●●●
● ●●●●
●
●
●●● ● ●
● ●
●
●
●
●●●●●●●●●●
●●
● ●
● ● ●
●●
●●●
●
●
●●
●●
●●●
●
●●
●● ●●
●●● ●● ●
●●
●
●
443644
4422
47302332
5742573657
381760
2
48
42
2936
2936
2
33173317
294229
30
42
3317
1736
17
SEX:1
0 20 40 60 80
● ●
● ●●
●
●
●●
● ●●
●●
●
●
●●
●●●●
●●●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ● ●● ●
●
●
● ●●
●
●
●
●●
● ●●
●
●●● ●●● ● ● ● ●●●● ●
●●
●● ●
●●●
● ●●● ● ●
●
●●
●●●●●●●
●●●
●●●●●
● ● ●●●●
●
●
● ●
●
●
●
●●
●● ●●
●
●●
●●●
●● ● ● ● ●●
●
●
●
●●
● ●●
● ●● ●
●
●●●●●●●
●●●
●● ●● ●
● ● ●●●
● ● ●
● ●
●
●
●
●
●●●●●
●●
● ● ●
●
● ●●
105055
595959
105410
59
50
1
55
55
1
6
46
46
611
4646
7
SEX:2
Figure 16: The observed data versus the population predictions given SEX. Thedata points from each individual are connected with a line and each data point isindicated by a circle. Extreme data points are labeled with the ID-number of thecorresponding individual. The black line is the line of identity and the red line is asmooth.
This graph is an extension to the DV ver-sus PRED graph (Section “dv.vs.pred” onpage 25). The purpose is to check the good-ness of fit in different subsets of the data asdefined by one or more covariates.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("SEX")
> xplot <- dv.vs.pred.by.cov(xpdb,
+ grid = TRUE, ids = TRUE)
> print(xplot)
28
dv.vs.ipred.by.cov
Observations vs Individual predictions (Run 1)
Individual predictions
Obs
erva
tions
0
50
100
150
0 50 100 150
●●●●
●●
●
●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●
●
● ●●●
●●
●●●
●●●●●●●●●
●
●●
●
●●●●●●●●●●●
●
●● ●
●
●●
●●
●●●●●●●
●
●
●
●●●●●
●●●●
●
●●●●●
●
●
●
●
●●●●●●●●●●
●●●●●●
●●
●●●●
●●●
WT
●●
●●●●
●
●
●
●●●●
●●
●
●●
●●●●
●●
●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●
●●●
●●
●●●
●
● ●●●
●●
●●●
●●●●●●●●
●●●●●●●
●●
●
●●●●●
●●●●●
●●
●● ●
●
●●
●●
●●●●●
●●●●
●
●●●●●●●●●●
●●●●●
●
●
●
●
●●●●●●●●●●
●●●●●
●●
●●
●
WT
0 50 100 150
●●
●●●●
●
●
●
●●●●
●●
●●
●●●●
●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●
●●
●●●
●●
●●●
●●●●●●●
●●
●
● ●
●●●●
●●
●
●●●●●●●●
●●●●●●●
●●
●
●●●●●
●●●●●
●●●
●
●
●
●●
●
●
●●●●●●●●●●
●●●●●●●●●●
●●●●●
●●●●●●
●●
●●
●
WT
●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●
●●●
●●
●
●
●
●●●●
●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●
●●●●●●●
●●
●
● ●
●●●●
●●
●
●●●
●●●●
●
●●●●●
●●
●
●
●
●●●●
●
●●●
●
●
●
●●
●
●
●●●●●●
●●
● ● ●
●
●●●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●●
●●●
●●
WT
0 50 100 150
●●●●●
●●●
●●
●●●●●●●●●●●●●●●●●●
●●●
●●
●
●
●
●●●●●●●●
●
●●●●●●●●●●●
●●●
●●
●●●
●●●●●●●●●
●●●●●●
●●●●
●●●●●
●●●●
●
●●●
●●●●
●
●●●●●
●●
●
●
●
●●●●
●
●●●●●●●●●●●●
●●
●●
● ● ●
●
●●●●●
●●●●
●●●●●●●●●●●●●●
●●●
●●●●●●●
●●●●●
WT
0
50
100
150
●●
●● ●
●●
●
●
●●●●●●●●
●
●●●●●●●●●●
●●●
●
●●
●
●
●
●●●●●
●●●●
●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●
●●●●
●●●●●
●●●●
●
●●●●●●●●●●●●●●●
●●
●●●
●●
●
●●●●●●●●●●
●●●●●
●●●●
●●●●●●●
●●
●●●●
●●●
●●
●●●●
●●●●●●
●●●●●●●●●●
WT
Figure 17: The observed data versus the population predictions given intervalsof WT. The data points from each individual are connected with a line and eachdata point is indicated by a circle. The black line is the line of identity and the redline is a smooth.
This graph is an extension to the DV ver-sus IPRED graph (Section “dv.vs.ipred” onpage 26). The purpose is to check the good-ness of fit in different subsets of the data asdefined by one or more covariates. In this casethe covariate is continuous so the subsets aredefined by intervals of the covariate. The inter-vals are defined using the equal.count func-tion in R and have by default approximately the
same number of data points in each interval(intervals overlap by 5̃0%).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("WT")
> xplot <- dv.vs.ipred.by.cov(xpdb,
+ grid = TRUE)
> print(xplot)
29
dv.vs.pred.by.idv
Observations vs. Population predictions (Run 1)
Population predictions
Obs
erva
tions
0
50
100
150
0 20 40 60 80
●
●●●
●●
●●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●●●●●●●●●●
●●●
●
●
●
●●●
● ●●
●●●●●●●●●●●●
●●●●●●
●●
● ●●
●●●
●
●
●
●
●
●●● ●●●
● ●●
●●
● ●
●
●
●
●
●●●
●
●
●
●●●●
●
●
●●●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
● ●●
●
●
●● ●●
● ●
●
●●●● ●●
●
●
●
● ●●●
●● ●●●
●
●
●●
●●●
●●●
● ●●●●●
Time
● ●
●
●●
● ●
●
●●●●●
●
●
●
●●●●
●
●
●
●●
●
●
●
●● ● ●●●
●
●
●●
●●●●●●●●●●●●●●●●●●●●
●●
●
●
●● ● ●
● ●
●●
●●●●●●● ●●●
●●●●●●●●
●●●●●●●●
●● ●
●
●●
●
●●
●● ●
●
● ●
●
●●●●●●●●
●●
●
●
●●●●●
● ●
●
●●
●
●
●●●
●●
●●●●●●●●
●●●●
●
●
●
●●●● ●●●
●●
●
●●
●
●●
●● ● ●
● ● ● ●
●●
●
●●● ● ●
● ●●
●
●●●● ● ● ● ●
●●
●
●
●●● ● ● ●●●
●● ●
● ●
●●
●●●
● ● ●●
●●●
●● ● ● ●● ●
● ●
Time
0 20 40 60 80
● ● ●
●
●●
● ●
●●●●●●
●
●
●●●●
● ●
●
●
●
●●
●
●●● ●●
●●
●
● ●●●●●●●●●●●
●●●●●●●●●●
●
●●
●
●●
● ●
●● ●
●
●●●●●●●● ●●●●●●●●●●●●
●●●●●●●●
●● ●
●●●
●
●
● ●● ●
● ●
● ●
●●●●●●●●
●
●●
●
●●●●●●
● ●●
●●
●
●
●●
●
●●● ●●●●●●●●
●●
●
●●●● ●●●●●
●
●
●●
●●
●●
● ●
●● ● ●
●●●
●●●
● ●
●● ●
●
●●●● ●● ● ●● ●●
●
●● ●● ● ●●●●
●
●●
●● ●
●
●●●
●● ● ●
●●●●● ● ●●● ●
●
Time
● ● ●●
●●
●●● ●●●●●
●
●● ●
●
●
●●●●●
●●●●●●●●●●●●●●●●
●
●
●
●●●●●●
●●●●●● ●●●●●●●●●
●●●●●●
●●● ●
●●
● ● ●
● ● ●
●●●●●●
● ●
●
●●●●●●
● ●●
●●● ●●●
●●●
●●●● ●●●
● ●
●
●
●●
●
●●●
●●●
●●●
●●
●●● ●●●●
● ●● ●●●●●●
●●
●●●
●●●
●●●
●●●
●●●●
Time
0 20 40 60 80
●
● ●●
●●
●
●●●● ● ●●●●●
●●
●●
●●
● ●●●● ●● ●●●●●●●●●●●●●●●●●●●●
●● ●
●
●●●●●●●●
●●●●●●● ●●●●●●●●
●●
●●●●●●●●●
●●●●●● ●
●
●●
● ●● ●
● ●
●●●●●●●●
● ●● ●
●●●●●●●● ● ● ●
●●
●●●●●●●●●
●●
● ●●●●●●●●●●
●
●● ●
●
●●●●
●
●●●●●●●●
●●●
●●●
●
●●●●●●●●● ●
●●●
●●●●●●●●●
●●●
●●●●●●●●●
●●●●
●●●●●
●●● ●●
Time
0
50
100
150
● ●
● ●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●
●●
●●●●●
●
●●●●●●●●●●●●●●
●●●●●●●
●●
● ●●●
● ●
●●●●●●
●
● ●
●● ●● ● ●●●
●●●●●●●●
●●●●●●●● ●●
●
●●
●
●●●●●●●●
●●●
●●●●●
●
●●●●●●● ●●●●●●●
●●●
●
●●●●●●●
●●
●●●●●●●
●●●
Time
Figure 18: The observed data versus the population predictions given intervalsof TIME. The data points from each individual are connected with a line and eachdata point is indicated by a circle. The black line is the line of identity and the redline is a smooth.
This graph is an extension to the DV ver-sus PRED by covariates graph (Section“dv.vs.pred.by.cov” on page 28). The purposeis to check the goodness of fit in different sub-sets of the data as defined by intervals if the in-dependent variable. The intervals are definedas described on page 28.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.pred.by.idv(xpdb,
+ grid = TRUE)
> print(xplot)
30
dv.vs.ipred.by.idv
Observations vs. Individual predictions (Run 1)
Individual predictions
Obs
erva
tions
0
50
100
150
0 50 100 150
●
●
●●
●●
●●●
●
●●
●●●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●●●●●●●●
●●●●●●
●
●
●
●●●●●●
●●●●●
●●●●●●●●●
●●●
●●●
●●●
●●●
●
●
●
●
●
●●●●●●
●●●
●●
●●
●
●
●
●
●●●●
●
●
●●●●●
● ●●●
●
●
●
●●●●
●●
●
●
●●
●
● ●
●
●●●
●
●
●
●●●
●●
●
●●●●●●
●
●
●
●●●●
●●●●●
●
●
●●
●●●
●●●
●●●●
●●
Time
●●
●
●●
●●
●
●●●●●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●●●●
●●
●
●●
●●●●
●●●●●●●●
●●●●●●●●
●●
●
●
●●●●●●
●●
●●●●●●●●●●
●●●●●●
●●●
●●●●
●●●●●●
●
●●
●
●●
●●●
●
●●
●
●●●●●●●●
●●
●
●
●●●●●
●●
●
●●
●
●
●●
●
●●
●●●●●●
●●
●●●●
●
●
●
●●●●●●●
●●
●
●●
●
●●
●● ● ●
●●●●
●●
●
●●●●●
●●●
●
●●●●●●●●●
●
●
●
●●●●●●
●●
●●●
●●
●●
●●●
●●●●
●●●
●●●●●●●●●
Time
0 50 100 150
●●●
●
●●
●●
●●●●●●
●
●
●●●●
● ●
●
●
●
●●
●
●●●●●
●●
●
●●●●●●
●●●●●●●●●●●●●●●●
●
●●
●
●●●●
●●●
●
●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●
●●●
●
●
●●●●
●●
●●
●●●●
●●●●
●
●●
●
●●●●●●
●●●
●●
●
●
●●
●
●●●●●●
●●●●●
●●
●
●●●●●●●
●●
●
●
●●
●●
●●
● ●
●●●●
●●●
●●●
●●
●●●●
●●●●●●●●●●
●
●
●●●●●●●●●
●
●●
●●●
●
●●●
●●●●
●●●●●●●●●●●
Time
●●●●●
●
●●●●●●
●●●
●● ●
●
●
●●●●●
●●●●●●●●●●●●●●●
●
●
●
●
●●●●●
●
●●●●●●●●●●●●●●●
●●●
●●●●●
●●●
●
●●●
●●●
●●●●●●
●●
●
●●●●●
●●●
●●
●●●●●
●●●
●●●●●●●
● ●
●
●
●●
●
●●●
●●●
●●●●●
●●●●●●●●●●●●●
●●●
●●
●●●
●●●
●●●
●●●
●●●●
Time
0 50 100 150
●
●●●
●●●
●●●●●●●●●●
●●
●●
●●
●●●●●●
●●●●●
●●●●●●●●●●●●●●●●
●●●
●
●●●●●●●●
●●●●●●●●●●●●●●●
●●
●●●●●●●●
●
●●●●●
●●●
●●●●
● ●
●●
●●●●●●●●
● ●●●
●●●●●●●
●●●●
●●
●●●●●●●●●
●●
●●●●●●●●●●●
●
●● ●●
●●●●
●
●●●●
●●●●
●●●
●●●●
●●●●●●●●●●●
●●●●●●●
●●●●
●●●
●●●●
●●●●●●●●●
●●●●●
●●●●●
Time
0
50
100
150
●●
●●●
●●●●●●●
●●
●
●●●●●●●●●●●●●●●●●●●
●
●●
●●●
●●●
●●●●●●●●●●●
●●●●●●
●●●●●
●
●●●●
● ●
●●●●●●
●
● ●
●●●●●●●
●
●●●●●●●●
●●●●●●●●●●
●
●●
●
●●●●●●●●
●●●
●●●●●
●
●●●●●●●●●●●●●●
●●●
●
●●●
●●●●
●●
●●●●●●●●●●
Time
Figure 19: The observed data versus the population predictions given intervalsof TIME. The data points from each individual are connected with a line and eachdata point is indicated by a circle. The black line is the line of identity and the redline is a smooth.
This graph is an extension to the DV ver-sus IPRED by covariates graph (Section“dv.vs.ipred.by.cov” on page 29). The purposeis to check the goodness of fit in different sub-sets of the data as defined by intervals of theindependent variable. The intervals are de-fined as described on page 28.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- dv.vs.ipred.by.idv(xpdb,
+ grid = TRUE)
> print(xplot)
31
cwres.vs.idv
Conditional weighted residuals vs. Time (Run 1)
Time
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●● ● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
Figure 20: The conditional weighted residuals versus time (the independent vari-able). Each residual is indicated by a blue circle. The red line is a smooth.
This graph is intended to check if the CWRESare independent of the independent variable(the independence of the residuals to the mainpredictor is a fundamental assumption in all re-gression analyses). A horizontal smooth cen-tered on the horizontal zero line is an indica-
tion that this is the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.vs.idv(xpdb)
> print(xplot)
32
wres.vs.idv
Weighted residuals vs. Time (Run 1)
Time
Wei
ghte
d re
sidu
als
−2
0
2
4
6
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●
●
● ●●
●
●
●
●
●●
●
●● ●
●
●
●
●
● ●
●
●
● ●
●●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●● ● ●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
● ●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●● ● ●
● ●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
● ● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
4810
1062440 208
4610
653
71719
48
1852
2
493421 23
4848563
594841
41
24
4151
4816448 46
30 37 642 591
19
24
642148
1737
28
2920
3
47
30
24
51
33
2
65
Figure 21: The weighted residuals versus time (the independent variable). Eachresidual is indicated by a blue circle. Extreme data points are labeled with theID-number of the corresponding individual. The red line is a smooth.
This graph is intended to check if the WRESare independent of the independent variable(the independence of the residuals to the mainpredictor is a fundamental assumption in all re-gression analyses). A horizontal smooth cen-tered on the horizontal zero line is an indica-tion that this is the case.
It is more correct to use CWRES instead ofWRES (Section “cwres.vs.cov” on page 43).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- wres.vs.idv(xpdb, ids = TRUE)
> print(xplot)
33
iwres.vs.idv
Individual weighted residuals vs. Time (Run 1)
Time
Indi
vidu
al w
eigh
ted
resi
dual
s
−0.2
0.0
0.2
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
6 204824
5240
26 8
6653552
1530 26
48
18
1052584981434
4959 5463 415
5534
24
55 6443
484851 1
1193629824
14
64
29
1321546448
1440
30 25433 65
481340
Figure 22: The individual weighted residuals versus time (the independent vari-able). Each residual is indicated by a blue circle. Extreme data points are labeledwith the ID-number of the corresponding individual. The red line is a smooth.
This graph is intended to check if the IWRESare independent of the independent variable(the independence of the residuals to the mainpredictor is a fundamental assumption in all re-gression analyses). A horizontal smooth cen-tered on the horizontal zero line is an indica-
tion that this is the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- iwres.vs.idv(xpdb, ids = TRUE)
> print(xplot)
34
cwres.wres.vs.idv
Conditional weighted residuals / Weighted residuals vs. Time (Run 1)
Time
Res
idua
ls
−2
0
2
4
6
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
● ●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●●
● ●●
●
● ●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●
●● ● ●● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
● ●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
● ●●●
●
●
●
●●
●
●
●
●
● ●
●● ●
●
●
●
●●
●
●
●
●●
●
●●
●
2481026 40 20326
95317 630 197
48
581829 498 1444 52
55184159 34563
41
55246443 5457 4650
2981 1419
6424 10
1321542935 646314
153048514033 652
Conditional weighted residuals
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●
●●●
●
●
●
●
●● ● ●●
●
● ● ●
●
●
●
●
●●
●
●● ●
●
●
●
●
● ●●
●
● ●
●●
●●
●
● ●
●
● ●●
●
●
●
●
●
●
●●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●● ● ● ●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
● ●
● ●
●
●
●
● ●
●
●●
●
●
● ●
●
●●
●●
●● ● ●● ●
●●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
● ● ●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
● ● ●●
●
●●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●●
●●
●
●●
●
● ●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●● ●
●●
●
●
●
●
● ●
●●
●
●
●
●●
●
●
●
●
●●
●● ●
●
●
●
●●
●
●
●
● ●
●
● ●
●
4810106
2440 208
4610
653
71719
48
18522493421 23
484856359 4841
41
24
4151
4816448 46
30 37 642 59119
24
642148
173728
2920
3
4730
24
5133
2
65
Weighted residuals
Figure 23: The conditional weighted residuals and weighted residuals versustime (the independent variable). Each residual is indicated by a blue circle. Ex-treme data points are labeled with the ID-number of the corresponding individual.The red line is a smooth.
This graph is intended to check if the weightedresiduals (conditional or not) are indepen-dent of the independent variable . Horizontalsmooths centered on the horizontal zero linesare an indication that this is the case.
Apart from the model development usage ofthis graph, it also provides an evaluation of thenecessity of using CWRES instead of WRES.This can be useful if the computation of theCWRES is cumbersome. With modern ver-
sions of NONMEM (7 and beyond), CWREScan be easily obtained and there is really noreason not to use CWRES.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.wres.vs.idv(xpdb,
+ ids = TRUE)
> print(xplot)
35
cwres.vs.idv.bw
Time vs. Conditional weighted residuals (Run 1)
Conditional weighted residuals
Tim
e
[1,2.83]
(2.83,4.67]
(4.67,6.5]
(6.5,8.33]
(8.33,10.2]
(10.2,12]
−2 −1 0 1 2 3
●
●
●
●
●
●
●
Figure 24: Intervals of time (the independent variable) versus the conditionalweighted residuals. The distribution of the residuals for each time interval is visu-alized by a box and whisker plot.
This graph is intended to check if the CWRESare independent of the independent variable(the independence of the residuals to the mainpredictor is a fundamental assumption in allregression analyses). It is similar to the xy-plot with CWRES versus time (see Section“cwres.vs.idv” on page 32) but may be moreuseful when there is a lot of data points. Thesolid dots in the middle of the boxes indicatethe medians of the residual distributions andif the residuals are independent of time, theyshould fall close and randomly around the ver-
tical zero line.
The number of time intervals may have to beadjusted (with the bins argument) to some-thing that is appropriate in relation to the num-ber of observations.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.vs.idv.bw(xpdb,
+ bins = 6)
> print(xplot)
36
wres.vs.idv.bw
Time vs. Weighted residuals (Run 1)
Weighted residuals
Tim
e
[1,2.1]
(2.1,3.2]
(3.2,4.3]
(4.3,5.4]
(5.4,6.5]
(6.5,7.6]
(7.6,8.7]
(8.7,9.8]
(9.8,10.9]
(10.9,12]
−2 0 2 4 6
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
● ●
●
● ●●● ● ●
● ●
Figure 25: Intervals of time (the independent variable) versus the weighted resid-uals. The distribution of the residuals for each time interval is visualized by a boxand whisker plot.
This graph is intended to check if the WRESare independent of the independent variable(the independence of the residuals to the mainpredictor is a fundamental assumption in allregression analyses). It is similar to the xy-plot with WRES versus time (see Section“wres.vs.idv” on page 33) but may be moreuseful when there is a lot of data points. Thesolid dots in the middle of the boxes indicatethe medians of the residual distributions andif the residuals are independent of time, theyshould fall close and randomly around the ver-tical zero line.
The time intervals are constructed such thatthere is an approximately equal number ofdata points in each boxplot.
It is more correct to use CWRES instead ofWRES (Section “cwres.vs.idv.bw” on the pre-vious page).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- wres.vs.idv.bw(xpdb)
> print(xplot)
37
cwres.vs.pred
Conditional weighted residuals vs. Population predictions (Run 1)
Population predictions
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Figure 26: The conditional weighted residuals versus the population predictions.Each residual is indicated by a blue circle. The red line is a smooth.
This graph is intended to check if there areany trends in the residuals with the popeula-tion predictions. The aim is to have no trends,i.e. that the smooth is horizontal.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.vs.pred(xpdb)
> print(xplot)
38
wres.vs.pred
Weighted residuals vs. Population predictions (Run 1)
Population predictions
Wei
ghte
d re
sidu
als
−2
0
2
4
6
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●● ●
●
●
● ●
●●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
●
● ●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●
●
● ● ●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●●
●
●● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
● ●
●
●
●
●
Figure 27: The weighted residuals versus the population predictions. Each resid-ual is indicated by a blue circle. The red line is a smooth.
This graph is intended to check if there areany trends in the residuals with the predictions.The aim is to have no trends, i.e. that thesmooth is horizontal.
It is more correct to use the CWRES (see Sec-tion “cwres.vs.pred” on the preceding page).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- wres.vs.pred(xpdb)
> print(xplot)
39
cwres.wres.vs.pred
Conditional weighted residuals / Weighted residuals vs. Population predictions (Run 1)
Population predictions
Res
idua
ls
−2
0
2
4
6
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
● ●
●●
● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●●
●●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●
●
Conditional weighted residuals
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●
●
●
●●
●
●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●
●
●
● ●
●●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
● ●
●
●●
●
●●
●
●
●
●●
●
●●
●●
●
● ●●
●
●
● ●
●●
● ●● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●
●● ● ●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●●
●●
●
●● ●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
● ●
●
●
●
●
Weighted residuals
Figure 28: The conditional weighted residuals and the weighted residuals versusthe population predictions. Each residual is indicated by a blue circle. The redline is a smooth.
This graph is intended to check for trends inthe residuals over the predictions. Horizontalsmooths centered on the horizontal zero linesare an indication that there are no unexplainedtrends.
Apart from the model development usage ofthis graph, it also provides an evaluation of thenecessity of using CWRES instead of WRES.This can be useful if the computation of the
CWRES is cumbersome. With modern ver-sions of NONMEM (7 and beyond), CWREScan be easily obtained and there is really noreason not to use CWRES.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.wres.vs.pred(xpdb)
> print(xplot)
40
cwres.vs.pred.bw
Population predictions vs. Conditional weighted residuals (Run 1)
Conditional weighted residuals
Pop
ulat
ion
pred
ictio
ns
[0.968,15.6]
(15.6,30.3]
(30.3,45]
(45,59.7]
(59.7,74.3]
(74.3,89]
−2 −1 0 1 2 3
●
●
●
●
●
●
●
●
●● ●
Figure 29: Intervals of the population predictions versus the conditional weightedresiduals. The distribution of the residuals for each time interval is visualized bya box and whisker plot.
This graph is intended to check if there are anytrends in the CWRES over the population pre-dictions. It is similar to the xy-plot with CWRESversus PRED (“cwres.vs.pred” on page 38) butmay be more useful when there is a lot of datapoints. The solid dots in the middle of theboxes indicate the medians of the residual dis-tributions and if they fall close and randomlyaround the vertical zero line, it is an indicationthat the model is performing adequately. The
number of time intervals may have to be ad-justed (with the bins argument) to somethingthat is appropriate in relation to the number ofobservations.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.vs.pred.bw(xpdb,
+ bins = 6)
> print(xplot)
41
wres.vs.pred.bw
Population predictions vs. Conditional weighted residuals (Run 1)
Conditional weighted residuals
Pop
ulat
ion
pred
ictio
ns
[0.968,9.77]
(9.77,18.6]
(18.6,27.4]
(27.4,36.2]
(36.2,45]
(45,53.8]
(53.8,62.6]
(71.4,80.2]
(80.2,89]
−2 −1 0 1 2 3
●
●
●
●
●
●
●
●
●
●
●● ●
● ●
Figure 30: Intervals of the population predictions versus the weighted residuals.The distribution of the residuals for each time interval is visualized by a box andwhisker plot.
This graph is intended to check if there are anytrends in the WRES over the population pre-dictions. It is similar to the xy-plot with WRESversus PRED (“wres.vs.pred” on page 39) butmay be more useful when there is a lot of datapoints. The solid dots in the middle of theboxes indicate the medians of the residual dis-tributions and if they fall close and randomly
around the vertical zero line, it is an indicationthat the model is performing adequately.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.vs.idv.bw(xpdb,
+ bins = 6)
> print(xplot)
42
cwres.vs.cov
AGE
CW
RE
S
−2
−1
0
1
2
3
30 40 50 60 70
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
● ●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
● ●
●
●
●
●●
●
●●
●
HTC
WR
ES
−2
−1
0
1
2
3
140 150 160 170 180 190
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
WT
CW
RE
S
−2
−1
0
1
2
3
60 80 100 120 140
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
SEX
CW
RE
S
−2
−1
0
1
2
3
1 2
● ●
●
Conditional weighted residuals vs Covariates (Run 1)
Figure 31: The conditional weighted residuals versus covariates. In the panelswith continuous covariates, each residual is indicated by a blue circle. The redlines are smooths. In panels with categorical covariates, the distribution of theresiduals displayed using box and whiskers plots.
This graph is intended to check if the CWRESare independent of the covariates. Horizontalsmooths (categorical covariates) or medians(categorical covariates) centered on the hori-zontal zero line suggest that this is the case.
If the shrinkage is low it may be more informa-tive to use eta versus covariate plots instead.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("AGE",
+ "HT", "WT", "SEX")
> xplot <- cwres.vs.cov(xpdb)
> print(xplot)
43
wres.vs.cov
AGE
WR
ES
−2
0
2
4
6
30 40 50 60 70
●●
●
●
●●
●
●
●●●●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●● ●
●●
●●●●●● ●
●
●●●●●
●
●
●●
●
● ●●
●
●
●
●
●●●
●●●●●●●●●●
●
●●●●
●
●
●
●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●●●
●●●●
●
●
●●
●
●●●●●●●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●●●●
●●
●
●●
●●
●
●●
●
●
●●
●
●●
●●●●●●●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●●
●
●●
●●●●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●●
●
●
●
●
●
●●●
●●●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●●●
●●
●●●●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●
●●●
●●●●
●
●●●●●
●●●
●●
●●
●●
●
●
●
●●
●
●●●●
●●●
●●
●●●●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●●●●●●
●●●●
●
●
●●●
●
●●
●
●●
●●
●●
●●
●●●
●
●
●
●●
●●
●
●●
●
●●
●
HTW
RE
S
−2
0
2
4
6
140 150 160 170 180 190
●●
●
●
●●
●
●
●●●●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●● ●
●●
●●●●●●●
●
●●●●●
●
●
●●
●
● ●●
●
●
●
●
●●●
●●●●●●●●●●
●
●●●●
●
●
●
●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●● ●●
●
●●●
●
●
●●●
●●●●
●
●
●●
●
●●●●●●●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●●●●
●●
●
●●
●●
●
●●
●
●
●●
●
●●
●●●●●●●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●●
●
●●
●●●●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●●
●
●
●
●
●
●●●
●●●●
●●
●
●●
●
●●
●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●●●
●●
●●●●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●
●●●
●●●●
●
●●●●●
●●●
●●
●●
●●
●
●
●
●●
●
●●●●
●●●
●●
●●●●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●●●●●●●●●●
●
●
●●●
●
●●
●
●●
●●
●●
●●
●●●
●
●
●
●●
●●
●
●●
●
●●
●
WT
WR
ES
−2
0
2
4
6
60 80 100 120 140
●●
●
●
●●
●
●
●●●●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●● ●
●●
●●●●●●●
●
●●●●●
●
●
●●
●
● ●●
●
●
●
●
●●●
●●●●●●●●●●
●
●●●●
●
●
●
●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●●●
●●●●
●
●
●●
●
●●●●●●●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●●●●
●●
●
●●
●●
●
●●
●
●
●●
●
●●
●●●●●●●●
●●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●●●●
●
●●
●●●●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●●●●
●
●
●
●
●
●●●
●●●●
●●
●
●●
●
●●
●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●●●
●●
●●●●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●
●●●
●●●●
●
●●●●●
●●●
●●
●●
●●
●
●
●
●●
●
●●●●
●●●
●●
●●●●●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●●●●●●
●●●●
●
●
●●●
●
●●
●
●●
●●
●●
●●
●●●
●
●
●
●●
●●
●
●●
●
●●
●
SEX
WR
ES
−2
0
2
4
6
1 2
● ●
●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
Weighted residuals vs Covariates (Run 1)
Figure 32: The weighted residuals versus covariates. In the panels with con-tinuous covariates, each residual is indicated by a blue circle. The red lines aresmooths. In panels with categorical covariates, the distribution of the residualsdisplayed using box and whiskers plots.
This graph is intended to check if the WRESare independent of the covariates. Horizontalsmooths (categorical covariates) or medians(categorical covariates) centered on the hori-zontal zero line suggest that this is the case.
If the shrinkage is low it may be more informa-tive to use eta versus covariate plots instead.It is more correct to use CWRES instead ofWRES (Section “cwres.vs.cov” on the previous
page).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("AGE",
+ "HT", "WT", "SEX")
> xplot <- wres.vs.cov(xpdb)
> print(xplot)
44
Residual model diagnosticsIt is necessary to correctly characterise the statistical model (residual error, betwen subjectvariability, etc.) before any firm conclusions can be drawn from the model. Sometimes it mayalso be hard to judge the appropriateness of the structural model before the statistical modelis in reasonably good shape (and vice versa). Additionally, to use statistical criteria (like p-values) to discriminate between contending models, it is necessary to have a good statisticalmodel. One primary way of diagnosing the statistical model is by using residuals, the differencebetween observations and model predictions (both population and individual predictions). Thissection utilizes different types of residuals to investigate, primarily, the statistical portion of apopulation model.
45
absval.cwres.vs.pred
|Conditional weighted residuals| vs. Population predictions (Run 1)
Population predictions
|Con
ditio
nal w
eigh
ted
resi
dual
s|
0
1
2
3
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Figure 33: The absolute values of the conditional weighted residuals versus thepopulation predictions. Each residual is indicated by a blue circle. The red line isa smooth.
This graph is intended to check if the CWRESare independent of the population predictions.Horizontal smooths suggest that this is thecase. This graph is useful for diagnosingboth the total variability explained in the model(residual unexplained variability, between sub-
ject variability, etc.).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.cwres.vs.pred(xpdb)
> print(xplot)
46
absval.wres.vs.pred
|Weighted residuals| vs. Population predictions (Run 1)
Population predictions
|WR
ES
|
0
1
2
3
4
5
6
0 20 40 60 80
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
● ●
●
●
●●
●●
●
● ●●●
●
●
●●
●
●
●
●
●
●
●●
● ●
●●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
● ●●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●●
●
●
●
●
Figure 34: The absolute values of the weighted residuals versus the populationpredictions. Each residual is indicated by a blue circle. The red line is a smooth.
This graph is intended to check if the WRESare independent of the population predictions.Horizontal smooths suggest that this is thecase. This graph is useful for diagnosingboth the total variability explained in the model(residual unexplained variability, between sub-ject variability, etc.).
In general it is better to use the CWRES in-
stead (see “absval.cwres.vs.pred” on the pre-ceding page
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.wres.vs.pred(xpdb)
> print(xplot)
47
absval.iwres.vs.pred
|Individual weighted residuals| vs. Population predictions (Run 1)
Population predictions
|IWR
ES
|
0.0
0.1
0.2
0.3
0 20 40 60 80
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
Figure 35: The absolute values of the individual weighted residuals versus thepopulation predictions. Each residual is indicated by a blue circle. The red line isa smooth.
This graph is intended to check if the IWRESare independent of the population predictions.Horizontal smooths suggest that this is thecase. This graph is useful for diagnosing theresidual unexplained variability (RUV) model.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.iwres.vs.pred(xpdb)
> print(xplot)
48
absval.iwres.vs.ipred.
|Individual weighted residuals| vs. Individual predictions (Run 1)
Individual predictions
|iWR
ES
|
0.0
0.1
0.2
0.3
0 50 100 150
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●
●
●
●● ●●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
Figure 36: The absolute values of the individual weighted residuals versus theindividual predictions. Each residual is indicated by a blue circle. The red line isa smooth.
This graph is intended to check if the IWRESare independent of the individual predictions.Horizontal smooths suggest that this is thecase. This graph is useful for diagnosing theresidual unexplained variability (RUV) model.
If the ε-shrinkage is high, this graph may not
be very informative.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.iwres.vs.ipred(xpdb)
> print(xplot)
49
absval.wres.vs.idv
|Conditional weighted residuals| vs. Time (Run 1)
Time
|CW
RE
S|
0
1
2
3
2 4 6 8 10 12
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●● ● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
Figure 37: The absolute values of the conditional weighted residuals versus theindependent variable. Each residual is indicated by a blue circle. The red line isa smooth.
This graph is intended to check that the vari-ability of the CWRES are independent of theindependent variable. Horizontal smoothssuggest that this is the case. This graph is use-ful for diagnosing the total variability explainedin the model (residual unexplained variability,
between subject variability, etc.).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.wres.vs.idv(xpdb)
> print(xplot)
50
absval.iwres.vs.idv
|Individual weighted residuals| vs. Time (Run 1)
Time
|iWR
ES
|
0.0
0.1
0.2
0.3
2 4 6 8 10 12
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ● ●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
Figure 38: The absolute values of the individual weighted residuals versus theindependent variable. Each residual is indicated by a blue circle. The red line isa smooth.
This graph is intended to check if the variabilityof the IWRES are independent of the indepen-dent variable. Horizontal smooths suggest thatthis is the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.iwres.vs.idv(xpdb)
> print(xplot)
51
absval.iwres.cwres.vs.ipred.pred
Population predictions
|Con
ditio
nal w
eigh
ted
resi
dual
s|
0
1
2
3
0 20 40 60 80
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●●
●
●●● ●
●
●
● ●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●●
●●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●●
●●
●
●
●
●
●●
●
●● ●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Individual predictions
|iWR
ES
|
0.0
0.1
0.2
0.3
0 50 100 150
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●● ●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●● ●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●● ● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
(Condotional) Weighted residuals vs. Predictions (Run 1)
Figure 39: The absolute values of the population and individual weighted resid-uals versus the population and individual predictions. Each residual is indicatedby a blue circle. The red line is a smooth.
This graph is a composite of “ab-sval.cwres.vs.pred” on page 46 and “ab-sval.iwres.vs.ipred” on page 49.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.iwres.cwres.vs.ipred.pred(xpdb)
> print(xplot)
52
absval.iwres.wres.vs.ipred.pred
Population predictions
|WR
ES
|
0
1
2
3
4
5
6
0 20 40 60 80
● ●
●
●●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●●●
●●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
● ●●
●●
●
●
●●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
●
● ● ●●
●
●
●
●
●●
●●●
●●●
●
●●●●
●
●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●
●●●
●●
●
●●●
●●●
●
●
●
●●
●
●
● ●● ●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●●●
●●
●●
●●
● ●
●
●
●●
●●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●● ●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
● ●●
●
● ● ●
●●
● ●● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●●●●
● ●
●
●●
●
●
●
●
●●
● ●●
●
●●
●●●
●
●●
●
●● ● ●
●●●●
●●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
● ●
●
●
●
●●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●●●
●●●
●●●
●●
●●
●
●
●
●●
●●
●●
● ●
●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●●
●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
●
●● ●
●
●●
●
●
●
● ●
●
●
●●
●
●
●
●
●● ●
● ●●
●
●
●
●●●●●●
●
●
●
●
●
●
●●●
●
● ● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
● ●
●
●
●
●
Individual predictions
|iWR
ES
|
0.0
0.1
0.2
0.3
0 50 100 150
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●● ●
●
●
●●
● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●● ●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●● ● ●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
Weighted residuals vs. Predictions (Run 1)
Figure 40: The absolute values of the population and individual weighted resid-uals versus the population and individual predictions. Each residual is indicatedby a blue circle. The red line is a smooth.
This graph is a composite of “ab-sval.wres.vs.pred” on page 47 and “ab-sval.iwres.vs.ipred” on page 49.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- absval.iwres.wres.vs.ipred.pred(xpdb)
> print(xplot)
53
cwres.dist.hist
Distribution of Conditional weighted residuals (Run 1)
Conditional weighted residuals
Den
sity
0.0
0.1
0.2
0.3
0.4
−2 −1 0 1 2 3
Figure 41: The distribution of the weighted residuals. The height of the barsrepresent the density of the distribution and the line is a smoothed representationof the density.
The purpose is to visualize the distribution ofthe CWRES. Nominally the CWRES shouldbe normally distributed (CWRESi = N(0, 1)),hence 95% of the CWRES points should bebetween -2 and 2. The center of the distribu-tion should be 0.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.dist.hist(xpdb)
> print(xplot)
54
wres.dist.hist
Distribution of Weighted residuals (Run 1)
Weighted residuals
Den
sity
0.0
0.1
0.2
0.3
0.4
−2 0 2 4 6
Figure 42: The distribution of the weighted residuals.The height of the bars rep-resent the density of the distribution and the line is a smoothed representation ofthe density.
The purpose is to visualize the distributionof the WRES. Nominally the WRES shouldbe normally distributed (WRESi = N(0, 1)),hence 95% of the CWRES points should bebetween -2 and 2. The center of the distribu-tion should be 0. Typically the CWRES shouldbe used instead of the WRES as the WRES
may not be normally distributed even if themodel is describing the data well.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- wres.dist.hist(xpdb)
> print(xplot)
55
iwres.dist.hist
Distribution of Individual weighted residuals (Run 1)
Individual weighted residuals
Per
cent
0
5
10
15
20
−0.2 0.0 0.2
Figure 43: The distribution of the individual weighted residuals.The height of thebars represent the percentage of points in the interval.
The purpose is to visualize the distribution ofthe IWRES. The IWRES should be normallydistributed. In this particular graph, the y-axisis set to “percent” and the density line is madeinvisible by using a transparent color.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- iwres.dist.hist(xpdb,
+ type = "percent", hidcol = "transparent",
+ ylb = "Percent")
> print(xplot)
56
cwres.dist.qq
Distribution of Conditional weighted residuals (Run 1)
Quantiles of Normal
Qua
ntile
s of
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
3
−2 0 2
● ● ●
●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●
●●●●
●●●●●
●
Figure 44: The quantiles of the conditional weighted residuals versus the quan-tiles of the standard normal distribution. Each data point is represented by a bluecircle. The black solid line is the line of identity.
The purpose is to directly compare the shapeof the distribution of the CWRES to a normaldistribution. If all points fall on the line of iden-tity, the shape of the CWRES distribution is ex-actly normal.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- cwres.dist.qq(xpdb)
> print(xplot)
57
wres.dist.qq
Distribution of Weighted residuals (Run 1)
Quantiles of Normal
Qua
ntile
s of
Wei
ghte
d re
sidu
als
−2
0
2
4
6
−2 0 2
●
●
●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●●●
●●●●●●●●
●●●●●●
●●●●
●
●
●
●
●
Figure 45: The quantiles of the weighted residuals versus the quantiles of anormal distribution. Each data point is represented by a blue circle. The blacksolid line is the line of identity.
The purpose is to directly compare the shapeof the distribution of the WRES to a normal dis-tribution. If all points fall on the line of identity,the shape of the WRES distribution is exactlynormal.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- wres.dist.qq(xpdb)
> print(xplot)
58
iwres.dist.qq
Distribution of Individual weighted residuals (Run 1)
Quantiles of Normal
Qua
ntile
s of
Indi
vidu
al w
eigh
ted
resi
dual
s
−0.2
0.0
0.2
−2 0 2
●●
●●●
●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●
●●●●
●●
● ●
Figure 46: The quantiles of the individual weighted residuals versus the quantilesof a standard normal distribution. Each data point is represented by a blue circle.The black solid line is the line of identity.
The purpose is to directly compare the shapeof the distribution of the IWRES to a normaldistribution. If all points fall on the line of iden-tity, the shape of the IWRES distribution is ex-actly normal.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- iwres.dist.qq(xpdb)
> print(xplot)
59
ind.plots.cwres.hist
Individual plots of CWRES (Run 1)
Conditional weighted residuals
Den
sity
0.0
0.2
0.4
0.6
ID:1
−2 −1 0 1 2
ID:2 ID:3
−2 −1 0 1 2
ID:4
ID:5 ID:6 ID:7
0.0
0.2
0.4
0.6
ID:8
0.0
0.2
0.4
0.6
ID:9 ID:10 ID:12 ID:13
−2 −1 0 1 2
ID:14 ID:15
−2 −1 0 1 2
ID:16
0.0
0.2
0.4
0.6
ID:17
Figure 47: The individual distributions of the CWRES. The height of the barsrepresent the density of the distribution and line is a smoothed representation ofthe density.
This graph can be used to visualize the individ-ual distributions of the CWRES. The underly-ing assumption is that the CWRES should benormally distributed. If all individual CWRESdistributions are symmetrically/normally dis-tributed then this will be the case. On the otherhand, the overall distribution can still be nor-mally distributed even if not all individual dis-tributions are. It may then be useful to iden-
tify the individuals that appears to have non-symmetric/non-normal distributions, as thesemay be outliers or carry unique information.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ind.plots.cwres.hist(xpdb,
+ grid = TRUE, subset = "ID<=17")
> print(xplot)
60
ind.plots.wres.hist
Individual plots of WRES (Run 1)
Weighted residuals
Den
sity
0.0
0.2
0.4
0.6
ID:1
−2 0 2 4
ID:2 ID:3
−2 0 2 4
ID:4
ID:5 ID:6 ID:7
0.0
0.2
0.4
0.6
ID:8
0.0
0.2
0.4
0.6
ID:9 ID:10 ID:12 ID:13
−2 0 2 4
ID:14 ID:15
−2 0 2 4
ID:16
0.0
0.2
0.4
0.6
ID:17
Figure 48: The individual distribution of the WRES. The height of the bars rep-resent the density of the distribution and the line is a smoothed representation ofthe density.
This graph can be used to visualize the individ-ual distributions of the WRES. The underlyingassumption is that the WRES should be nor-mally distributed. If all individual WRES distri-butions are symmetrically/normally distributedthen this will be the case. On the other hand,the overall distribution can still be symmetri-cally/normally distributed even if not all indi-vidual distributions are. It may then be use-ful to identify the individuals that appears tohave non-symmetric/non-normal distributions,as these may be outliers or carry unique infor-
mation.
It is probably better to use theind.plots.cwres function ( on the previouspage).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ind.plots.wres.hist(xpdb,
+ grid = TRUE, subset = "ID<=17")
> print(xplot)
61
ind.plots.cwres.qq
Individual Q−Q plots of CWRES (Run 1)
Quantiles of Normal
Qua
ntile
s of
Con
ditio
nal w
eigh
ted
resi
dual
s
−2
−1
0
1
2
●● ●
● ●●
●●
●
ID:1
−1 0 1
●
● ●● ●
● ●●
●
●
ID:2
● ●●
● ●● ●
● ●●
ID:3
−1 0 1
●● ● ● ● ● ● ●
●
ID:4
●
●● ● ● ●
●●
●
ID:5
●
● ●● ● ● ●
●●
ID:6
● ●●
●● ●
●
ID:7
−2
−1
0
1
2
●●
●
●
● ● ● ●
●
ID:8
−2
−1
0
1
2
●
● ● ● ● ●
●●
ID:9
●
●
● ●
●
ID:10
●●
● ● ● ● ●● ●
ID:12
●●
● ● ●●
● ● ●
●
ID:13
−1 0 1
●●
● ● ● ● ●●
●
●
ID:14
●● ● ●
● ●●
●
●●
ID:15
−1 0 1
●
● ●● ●
● ● ●● ●
ID:16
−2
−1
0
1
2
●●
● ● ●● ●
● ●●
ID:17
Figure 49: QQ plots of the individual CWRES distributions. The quantiles ofthe CWRES for each individual are plotted against the quantiles of the normaldistribution.The diagonal black line is the line of identity.
This graph can be used to compare theCWRES distribution of each individual againstthe nomral distribution. If the individual distri-bution is normally distributed the points shouldfall on the line of identity.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ind.plots.cwres.qq(xpdb,
+ grid = TRUE, subset = "ID<=17")
> print(xplot)
62
ind.plots.wres.qq
Individual Q−Q plots of WRES (Run 1)
Quantiles of Normal
Qua
ntile
s of
Wei
ghte
d re
sidu
als
−2
0
2
4
● ●●
● ●●
● ●●
ID:1
−1 0 1
●
● ● ● ● ● ● ●●
●
ID:2
●●
● ●● ●
● ● ●●
ID:3
−1 0 1
●● ● ● ● ● ● ●
●
ID:4
●● ● ● ● ●
●● ●
ID:5
●
● ●● ● ● ●
● ●
ID:6
● ● ● ● ●●
●
ID:7
−2
0
2
4
●● ●
● ● ● ● ● ●
ID:8
−2
0
2
4
●● ● ● ● ●
●●
ID:9
● ●
●●
●
ID:10
● ●● ● ● ● ● ● ●
ID:12
● ● ● ● ● ● ● ● ● ●
ID:13
−1 0 1
●● ● ● ● ● ● ●
●●
ID:14
● ● ● ● ● ●● ● ●
●
ID:15
−1 0 1
● ● ● ● ● ● ● ●● ●
ID:16
−2
0
2
4
●●
● ● ●● ● ●
●●
ID:17
Figure 50: QQ plots of the individual WRES distributions. The quantiles of theWRES for each individual are plotted against the quantiles of the normal distribu-tion.The diagonal black line is the line of identity.
This graph can be used to compare the WRESdistribution of each individual against the nom-ral distribution. If the individual distribution isnormally distributed the points should fall onthe line of identity.
If the FOCE estimation method is used, it isprobably better to use the ind.plots.qq func-
tion ( on the preceding page).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- ind.plots.wres.qq(xpdb,
+ grid = TRUE, subset = "ID<=17")
> print(xplot)
63
absval.cwres.vs.pred.by.cov
|Conditional weighted residuals| vs Population predictions (Run 1)
Population predictions
|CW
RE
S|
0
1
2
3
0 20 40 60 80
●
●●
●●
●
●
●●
●
●
●
●
●●●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
●
●●
●
●●
●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
AGE
●●
●
●
●
●
●
●
●
● ●● ● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●● ●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
● ●
●
● ●
●●
●●●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
AGE
0 20 40 60 80
●●
●
●
●
●
●
●
●
● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
● ●●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
● ●●●
AGE
●●
●
●
●
●
●
●
●●
●
●
● ● ●
●
●
●●●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●
●●● ●
●●
● ●● ●●
●
●●
●●
●
●
●
●●
●
●●
● ●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●
●●
●
●●●
●
● ●●●
AGE
0 20 40 60 80
●
●
●
● ● ●
●
●
●●●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●●●
●●●
●
●
●
●●
●
●
●●
●● ●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●●
●●
●●
●●
●
●●● ●
●●
● ●● ●●
●
●●
● ●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●●
●●●
●
●
●
● ●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
AGE
0
1
2
3
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●●●
●●●
●
●
●
●●
●
●
●●
●● ●
●
● ●
●
●●●
●
●
●
● ●
●
●●
●●●●●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
AGE
Figure 51: The absolute values of the conditional weighted residuals versus thepopulation predictions by age. Each residual is indicated by a blue circle. The redline is a smooth.
This graph is intended to check if the CWRESare independent of the population predictionsin each interval of the covariate. Horizontalsmooths in each panel would suggest that thisis the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("AGE")
> xplot <- absval.cwres.vs.pred.by.cov(xpdb,
+ grid = TRUE)
> print(xplot)
64
absval.wres.vs.pred.by.cov
|Weighted residuals| vs. Population predictions (Run 1)
Population predictions
|WR
ES
|
0
1
2
3
4
5
6
0 20 40 60 80
●
●
●
●●
● ●●
●
●
●
●
●
●
●●
●
●●●●
●●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●●
●
●
●●●
●
●●
●
●
●
●●
●
●
● ●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●● ●
●●●
●●
●●
●● ●
●
●
●
●
●●
●
●●
●
●
● ● ●
●●
● ●
● ●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●●
●●●
●
●
●
●
●● ● ●
●●●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●
● ●
●●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●●
●●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●●
●
●● ●
●
●●
●
●
●
● ●●
● ●● ●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●●
●●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
● ●
●
●
●
●
SEX:1
0 20 40 60 80
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
● ●●
●●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●●
● ●
●●
●
● ● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●●
●
●●
●
●
●
●●●
● ●
●
●●
●
●●
● ●
●
●
●
●
●
●
● ● ●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●●
●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
SEX:2
Figure 52: The absolute values of the weighted residuals versus the populationpredictions by sex. Each residual is indicated by a blue circle. The red line is asmooth.
This graph is intended to check if the WRESare independent of the population predictionsin each of the covariate categories (similarly to“absval.wres.vs.pred” on page 47). Horizontalsmooths in each panel would suggest that thisis the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("SEX")
> xplot <- absval.wres.vs.pred.by.cov(xpdb,
+ grid = TRUE)
> print(xplot)
65
absval.iwres.vs.ipred.by.cov
|Individual weighted residuals| vs Individual predictions (Run 1)
Individual predictions
|IWR
ES
|
0.0
0.1
0.2
0.3
0 50 100 150
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
AGE
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
● ●
●
●●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●●
●●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
AGE
0 50 100 150
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●● ●
●
●
●
●●
● ●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
●●● ● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●●●●
AGE
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●
●●
●
●
●
●●
●●●●
●●
●● ●●●
●●
●
●●●●
●
●●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●●●●
AGE
0 50 100 150
●
●
●
●●
●
●
●●
●●
●●●
●
●
●●
●
●
●
●
●
●●●
●
●●
●●●●●
●
●
●
●●
●●●●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●●
●
●
●
●
●
●●●
●●
●● ●●●●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●
●
AGE
0.0
0.1
0.2
0.3
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●●●●
●
●
●
●●
●●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●●
●●
●●
●
AGE
Figure 53: The absolute values of the individually weighted residuals versus thepopulation predictions by age. Each residual is indicated by a blue circle. The redline is a smooth.
This graph is intended to check if the IWRESare independent of the population predictionsin each of the interval of the covariate. Hor-izontal smooths in each panel would suggestthat this is the case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("AGE")
> xplot <- absval.iwres.vs.ipred.by.cov(xpdb,
+ grid = TRUE)
> print(xplot)
66
absval.cwres.vs.cov.bw
|Conditional weighted residuals| vs Covariates (Run 1)
|CWRES|
AG
E
[24,28.5]
(28.5,33]
(33,37.5]
(37.5,42]
(42,46.5]
(46.5,51]
(51,55.5]
(55.5,60]
(60,64.5]
(64.5,69]
0 1 2 3
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
Figure 54: Box and whiskers plot of the absolute values of the conditionalweighted residuals for each interval of age.
This graph is intended to check if the CWRESare similarly distributed between the covariatecategories. Black dots (the medians) locatedat the same x-value would suggest that this isthe case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("AGE")
> xplot <- absval.cwres.vs.pred.by.cov(xpdb,
+ grid = TRUE)
> print(xplot)
67
absval.wres.vs.cov.bw
|Weighted residuals| vs Covariates (Run 1)
|WRES|
SE
X
1
2
0 1 2 3 4 5 6
●
●
●● ●●● ●● ●● ●●● ● ●
●●●
Figure 55: Box and whiskers plot of the absolute values of the weighted residualsfor each level of sex.
This graph is intended to check if the WRESare similarly distributed between the covariatecategories. Black dots (the medians) locatedat the same x-value would suggest that this isthe case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("SEX")
> xplot <- absval.wres.vs.cov.bw(xpdb,
+ grid = TRUE)
> print(xplot)
68
absval.iwres.vs.cov.bw
|Individual weighted residuals| vs Covariates (Run 1)
|iWRES|
SE
X
1
2
0.0 0.1 0.2 0.3
●
●
● ●● ●● ●● ●
●● ●●
Figure 56: Box and whiskers plot of the absolute values of the weighted residualsfor each level of sex.
This graph is intended to check if the IWRESare similarly distributed between the covariatecategories. Black dots (the medians) locatedat the same x-value would suggest that this isthe case.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "covariates") <- c("SEX")
> xplot <- absval.iwres.vs.cov.bw(xpdb,
+ grid = TRUE)
> print(xplot)
69
autocorr.cwres
Autocorrelation of Conditional weighted residuals (Run 1)
Conditional weighted residuals(i)
Con
ditio
nal w
eigh
ted
resi
dual
s(i+
1)
−2
−1
0
1
2
3
−2 −1 0 1 2 3
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
Figure 57: The conditional weighted residuals versus the previous conditionalweighted residual. Data points are indicated by blue circles and the red line is asmooth.
This graph is intended to check the degreeof correlation among the residuals. A pro-nounced trend in the smooth around the centerof the data would suggest that there is auto-correlation present.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- autocorr.cwres(xpdb)
> print(xplot)
70
autocorr.wres
Autocorrelation of Weighted residuals (Run 1)
Weighted residuals(i)
Wei
ghte
d re
sidu
als(
i+1)
−2
0
2
4
6
−2 0 2 4 6
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ● ●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
Figure 58: The weighted residuals versus the previous weighted residual. Datapoints are indicated by blue circles and the red line is a smooth.
This graph is intended to check the degreeof correlation among the residuals. A pro-nounced trend in the smooth around the centerof the data would suggest that there is auto-correlation present.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> xplot <- autocorr.wres(xpdb)
> print(xplot)
71
Parameter distribution diagnosticsStatistical comparisons of competing models depends on the correct description of the statis-tical sub-parts of the model, including both the residual error distribution as well as the dis-tributions of the between individual variability (BSV) terms in the model. The distributions ofthe BSV are also important for the simulation properties of the model. NONMEM will generatenormally distributed BSV when simulating from the model. If the BSV distributions estimatedfrom data are asymmetrically distributed, it is likely that the simulated data will not mimic theobserved.
72
parm.hist
Clearance
coun
t
0
5
10
15
20
10 20 30 40
Volume
coun
t
0
5
10
15
50 100 150
Distribution of parameters (Run 1)
Figure 59: Histograms of the posterior Bayes estimates of Clearance and Vol-ume. Each bar represent the count of values in the corresponding interval on thex-axis.
This graph is intended for visualization of thedistribution(s) of the parameters. The parame-ters are typically the data items included in thepatab or using the default xpose definition ofparameters.
The expected shape of the distribution is givenby the way that the parameter has been codedin the NONMEM model file.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "parms") <- c("CL",
+ "V")
> xplot <- parm.hist(xpdb, hidcol = "transparent",
+ type = "count", ylb = "count")
> print(xplot)
73
ranpar.hist
ETA3
Den
sity
0.0
0.2
0.4
0.6
−1 0 1
Figure 60: Histogram of the posterior Bayes estimates of ETA3. The blackdashed line is a smooth of the probability density while the solid black line isthe theoretical density based on mean=0 and sd=0.7652.
This graph function is similar to histogramsof the parameters (Section “parm.hist” on thepreceding page). In this particular case thetheoretical density has been superposed andcan be used to judge the impact of shrink-age. The estimated sd for this parameter was0.7652 and the shrinkage was 17%. From thegraph it can be concluded that the shrinkageis mainly affecting the positive side of the dis-
tribution.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "ranpar") <- c("ETA3")
> xplot <- ranpar.hist(xpdb, math.dens = list(mean = 0,
+ sd = 0.7652), xlim = c(-2,
+ 2), hicol = "gainsboro", main = NULL)
> print(xplot)
74
parm.qq
Quantiles of Normal
Qua
ntile
s of
Cle
aran
ce
10
20
30
40
−2 −1 0 1 2
● ● ●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●
●●●●●
●●
●
●●
●
●
Quantiles of Normal
Qua
ntile
s of
Vol
ume
50
100
150
−2 −1 0 1 2
●●
●●●●●●●●●●●●
●●●●●●●
●●●●●●
●●●●●
●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●
●
● ●
●
Distribution of parameters (Run 1)
Figure 61: The quantiles of the distributions if CL and V are plotted versus thequantiles of the Normal distribution. The diagonal black line is the line of identity.
This graph can be used to compare the distri-bution of the POSTHOC parameters to the Nor-mal distribution. If the parameter is normallydistributed the data points will fall on the lineof identity. This graph is very sensitive and willeasily detect non-normalities. Since the indi-vidual parameters are usually modelled as be-ing non-normal, this display may not be partic-ularly useful (but see “ranpar.qq” on the next
page).
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "parms") <- c("CL",
+ "V")
> xplot <- parm.qq(xpdb)
> print(xplot)
75
ranpar.qq
Quantiles of Normal
Qua
ntile
s of
ETA
2
−0.5
0.0
0.5
−2 −1 0 1 2
●
●
●●●●●●
●●●●●
●●●●●●●●
●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●
●●●●●●●
●●●●
●● ●
●
Quantiles of Normal
Qua
ntile
s of
ETA
3−2.0
−1.5
−1.0
−0.5
0.0
0.5
1.0
−2 −1 0 1 2
●
● ●●
●●●●
●●●
●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●
●●●●●●●●
●● ●●
Distribution of random parameters (Run 1)
Figure 62: The quantiles of the distributions if ETA1 and ETA2 are plotted versusthe quantiles of the Normal distribution. The diagonal black line is the line ofidentity.
This graph can be used to compare the dis-tribution of the POSTHOC parameters or ran-dom effects to the Normal distribution. Ifthe parameter is normally distributed the datapoints will fall on the line of identity. Thisgraph is very sensitive and will easily detectnon-normalities. Since ηs are typically as-sumed normally or symmetrically distributed,this graph is a useful diagnostic for this as-
sumption.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "ranpar") <- c("ETA2",
+ "ETA3")
> xplot <- ranpar.qq(xpdb)
> print(xplot)
76
parm.splom
Scatterplot matrix of parameters (Run 1)
Clearance30
4030 40
10
20
10 20●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
Volume100
120
140
160100120140160
40
60
80
100
40 60 80 100
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
KA
3
43 4
1
2
1 2
Figure 63: The posterior Bayes estimates (blue circles) of the parameters areplotted versus each other. The red line is a smooth.
This graph is used to explore correlations be-tween the parameters. Each pair of parame-ters are plotted twice, once in the upper trian-gle and once in the lower. However, since theparameter regarded as the x-variable is differ-ent for the upper and lower plot, the smoothswill be different. Often, if a trend is only visi-ble with one of the two (upper and lower) plotsthen the trand may be an artifact of the way thesmooth is calculated. Since the individual pa-rameters are typically not normally distributed,this display will look slightly different from thesame one using the POSTHOC ηs (see “ran-
par.splom” on the following page). If covari-ates have already been included in the model,it is more appropriate to use the η-estimatesinstead of the individual parameters for thisgraph.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "parms") <- c("CL",
+ "V", "KA")
> xplot <- parm.splom(xpdb)
> print(xplot)
77
ranpar.splom
Scatterplot matrix of random parameters (Run 1)
ETA10.0
0.5
0.0 0.5
−0.5
0.0
−0.5 0.0
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
ETA20.0
0.50.0 0.5
−0.5
0.0
−0.5 0.0
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
ETA3−0.5
0.0
0.5
1.0−0.5 0.0 0.5 1.0
−2.0
−1.5
−1.0
−0.5
−2.0 −1.0
Figure 64: The posterior Bayes estimates (blue circles) of the ηs are plottedversus each other. The red line is a smooth.
This graph is used to explore the correlationsbetween the random effects, and may be infor-mative with respect to the OMEGA BLOCK struc-ture in NONMEM. Since the distribution of the in-dividual random effects are typically assumednormally or symmetrically distributed, it willlook different to the same graph using the pa-rameters (see “parm.splom” on the previouspage). If covariates have been included inthe model, the individual parameter estimates
should not be used for investigating the corre-lation structure.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "ranpar") <- c("ETA1",
+ "ETA2", "ETA3")
> xplot <- ranpar.splom(xpdb)
> print(xplot)
78
parm.vs.parm
Volume
Cle
aran
ce
10
20
30
40
50 100 150
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
Clearance
Vol
ume
50
100
150
10 20 30 40
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
Parameters vs. parameters (Run 1)
Figure 65: The posterior Bayes estimates (blue circles) of the parameters areplotted versus each other. The red line is a smooth. Extreme data points arelabeled by the corresponding ID-number.
This graph is used to explore correlations be-tween the parameters and fills the same roleas the parm.splom function on page 77. Thebenefit of this display lies in the identification ofextreme individuals. Additionally, if many pa-rameters are compared then this function willproduce a multiple page plot. It is possible todisplay the ηs with this function by first chang-ing the variable definition to the ηs instead of
the parameters.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "parms") <- c("CL",
+ "V")
> xplot <- parm.vs.parm(xpdb)
> print(xplot)
79
tabulate.parameters+---------+-------+-----+
|Parameter| Value | RSE |
+---------+-------+-----+
| TH1 |17.7481|0.059|
+---------+-------+-----+
| TH2 |76.7973|0.052|
+---------+-------+-----+
| TH3 |1.44256|0.108|
+---------+-------+-----+
| OM1:1 | 0.45| 0.13|
+---------+-------+-----+
| OM2:2 | 0.37| 0.19|
+---------+-------+-----+
| OM3:3 | 0.77| 0.29|
+---------+-------+-----+
| SI1:1 | 0.13|0.065|
+---------+-------+-----+
This command extracts the population param-eters estimates and standard errors from theNONMEM list file. It is assumed that the list file isnamed: run#.lst, where # is the run-numberextracted from the Xpose data object. Thestandard error information is presented as the
relative standar errors, i.e. SE/estimate.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> tabulate.parameters(xpdb, dir = "../")
80
parm.summary+----+---------+------+-------+--------+------+--------------+--+
| | Mean| SD| Q1| Median| Q3| Range| N|
+----+---------+------+-------+--------+------+--------------+--+
|ETA3| -0.07386|0.6318|-0.3498| 0.03522|0.3588| -1.935-1.001|64|
+----+---------+------+-------+--------+------+--------------+--+
|ETA2|-0.007865| 0.349|-0.2862| 0.02196|0.1991|-0.7883-0.7406|64|
+----+---------+------+-------+--------+------+--------------+--+
|ETA1| 0.007579|0.4507|-0.4011|-0.00123| 0.37|-0.7097-0.9142|64|
+----+---------+------+-------+--------+------+--------------+--+
| KA| 1.588|0.8672| 1.017| 1.495| 2.065| 0.2083-3.925|64|
+----+---------+------+-------+--------+------+--------------+--+
| V| 80.9| 28.77| 57.68| 78.5| 93.72| 34.91-161.1|64|
+----+---------+------+-------+--------+------+--------------+--+
| CL| 19.81| 9.374| 11.88| 17.73| 25.7| 8.728-44.28|64|
+----+---------+------+-------+--------+------+--------------+--+
This command tabulates the individual param-eter estimates. Only the parameter values onthe first individual data records are used un-less onlyfirst is set to FALSE. It is also pos-sible to re-direct the output to a csv file whichcan be imported into spreadsheet or word pro-
cessing programs.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> parm.summary(xpdb)
81
Model comparisonSimply put, building a model is the task of identifying the least bad model from a set of modelcandidates (remember that “All models are wrong but some are useful”). It is also important tounderstand the relative performance of two or more model candidates (for example, if one ofthe alternatives perform less well at high concentrations then this model should not be carriedforward if it is important to characterise Cmax) or if the apparent benefit in terms of ∆OFV withone model is dependent on only one or two individuals.
82
basic.model.comp
Population predictions (Run 2)
Pop
ulat
ion
pred
ictio
ns (
Run
1)
0
20
40
60
80
0 20 40 60 80
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●
●●
●●●
●●●●
●●●●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●●
●●●
●●
●●
●
●●●
●
●
●●●
●●●●
●●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●
●●
●
●
●
●
●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●
●●●●●
●●●●●●●
●●●●
●●
●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●●●●
●●●●
●●●●
●●
●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●●●
●●
●●
●●●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●
●●●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●●●●●
●●●●
●●
●●
●●●
●●●
●●
●
●
●
●●
●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●●
●●●●
●●
●●●
●●●●
●●
●
●●●
●●●
●●
●●
●●●
Individual predictions (Run 2)In
divi
dual
pre
dict
ions
(R
un 1
)
0
50
100
150
0 50 100 150
●●
●●
●●
●●
●
●●
●●
●●
●
●
●●
●●●●●●●●●●
●●●●
●●
●
●●
●●●●●●
●●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●●●
●●●
●●●
●●
●
●●
●●●
●
●
●●●●●●●
●●
●●●●●●●●
●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●
●●●●
●●
●
●
●
●
●
●
●●●●
●●●
●●●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●
●●
●●●●●●●●●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●●●
●●●●
●●●
●●
●●
●●
●●
●●●●●●●●●●
●●●●●●●
●●●
●●
●●
●●
●
●●●
●●●●●●●●●●●
●●
●●●
●●●●
●●
●
●
●
●
●● ●
●●
●
●●
●
●
●●●●●●●●●●●●●●
●●●●
●●●●
●●●
●●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●
●●●●●
●●
●●
●
●
●
●●●●●
●●●●●
●●●●●●●
●
●
●
●●●
●●●●●●
●●●●●●
●●●●
●●●●●●●●
●●
●●●●●
●●●●●
●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●●●●●●
●●
●●●
●●●●
●●●●
●●
●●
●
●
●●●●●●●●●●
●●●
●●●
●●●
●●●●
●●
●●●
●●●●
●●●●
●●
●●●●●●●●●●
Conditional weighted residuals (Run 2)Con
ditio
nal w
eigh
ted
resi
dual
s (R
un 1
)
−2
−1
0
1
2
3
−4 −2 0 2 4
●
●●
● ●
●●
●
●
●
●●●●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●●●●●●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●●●●
●●●
●
●
●●●
●
●●●
●
●
●
●●●●
●●
●
●
● ●
●
●●●
●
●
●
●
●
●●
●
●●●
●
● ●
●●
●●●●●
●●
●
●
●●
●●●
●
●
●●
●●
●●●
●●●
●●
●
●
●
●●
●●●
●
●● ●
●
●
●●●
●
●●●
●●
●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
● ●
●
●●●●●
●●●●●
●
●
●
●
●
●
●
●●●
●●
●
●●●●
●
●●
●
●●●●●●●
●●
●●
●
●
●
●● ●
●
●
●
●
●●
●
●
●●●
●●●●●
●●
●●●
●
●●●●●
●●●
●
●●
●●●●●●
●
●
●
●●●●
●●
●
●●
●
●●●
● ●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●●●
●
●
●
●●●●
●●
●●
●
●●
● ●●●
●
●
●●●
●
●
●
●
●
●
●●
●●
●●
●
●●
●●
●
●
●●
●●
●
●●●
●●
●●●
●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●●●
●●
●●
●●●
●
●●●●
●
●
●
●●●
●●●
●
●
●
●
●
●●●●●●●
●●●
●●●
●●●
●
●● ●
●●
●
●●
●
●
●●
●
●●●●
●
●
●●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●●
●●
●
●
●●
●●
●●
●●
●●●●●
●●
●
●●
●
●●●
●
●
● ●●
●
●
● ●●●
●
●●
●●
●●●
●●
●
Individual weighted residuals (Run 2)Indi
vidu
al w
eigh
ted
resi
dual
s (R
un 1
)
−0.2
0.0
0.2
−0.4 −0.2 0.0 0.2 0.4
●●
●
●
●
●●
●●
●
●●
●●●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●●●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●●
●●●
●●
●
●
●
●
●●
●●●
●
●
●
●●●●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●●●●●●
●
●
●
●●
●
●●
●●●
●●
●
●
●
● ●
● ●●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●●
●●●
●
●●●●
●
● ●
●●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●●●●●●●
●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●●
●
●●●●
●
●
●●●
●
●●
●
●●
●●
●
●●●●
●●
●
●
●●●
●
●
●
●
●●
●
●●●
●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●
●●
●
●●
● ●●
●
●
●
●
●●
●
●●●
●
●
●●
●
●●
●●●
●
●●●●
●●
●●●
●●
●●
●
●
●●●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●
●●●
●
●
●●
●
●● ●
●●
●
●
●
●●●
●●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●●●
●●●
●
●●
●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●● ●
●
●
●
●●
●
●
●
●
●●●●●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●●●●
●●
Individual weighted residuals (Run 1) vs Individual weighted residuals (Run 2)
Figure 66: Comparison of Run 1 to Run 2. The top panels show the populationand individual predictions for Run 1 and Run 2 versus each other. The bottompanels show the population and individual weighted residuals versus each other.Individual data points are indicated by blue circles and are connected by bluelines. The black diagonal lines are the lines of identity.
This graph is intended for visualization of thedifferences between two runs. In this partic-ular case one OMEGA has been removed (KA),with a corresponding increase in OFV of about110 points. It can be seen that the individualand population predictions are very similar anddoes not seem to depend on the removed η.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> xplot <- basic.model.comp(xpdb2,
+ xpdb1)
> print(xplot)
83
add.model.comp
|Population predictions| (Run 2)
|Pop
ulat
ion
pred
ictio
ns| (
Run
1)
0
20
40
60
80
0 20 40 60 80
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●
●●
●●●
●●●●
●●●●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●●
●●●
●●
●●
●
●●●
●
●
●●●
●●●●
●●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●
●●
●
●
●
●
●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●
●●●●●
●●●●●●●
●●●●
●●
●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●●●●
●●●●
●●●●
●●
●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●●●
●●
●●
●●●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●
●●●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●●●●●
●●●●
●●
●●
●●●
●●●
●●
●
●
●
●●
●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●●
●●●●
●●
●●●
●●●●
●●
●
●●●
●●●
●●
●●
●●●
|Individual predictions| (Run 2)|In
divi
dual
pre
dict
ions
| (R
un 1
)
0
50
100
150
0 50 100 150
●●
●●
●●
●●
●
●●
●●
●●
●
●
●●
●●●●●●●●●●
●●●●
●●
●
●●
●●●●●●
●●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●●●
●●●
●●●
●●
●
●●
●●●
●
●
●●●●●●●
●●
●●●●●●●●
●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●
●●●●
●●
●
●
●
●
●
●
●●●●
●●●
●●●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●
●●
●●●●●●●●●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●●●
●●●●
●●●
●●
●●
●●
●●
●●●●●●●●●●
●●●●●●●
●●●
●●
●●
●●
●
●●●
●●●●●●●●●●●
●●
●●●
●●●●
●●
●
●
●
●
●● ●
●●
●
●●
●
●
●●●●●●●●●●●●●●
●●●●
●●●●
●●●
●●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●
●●●●●
●●
●●
●
●
●
●●●●●
●●●●●
●●●●●●●
●
●
●
●●●
●●●●●●
●●●●●●
●●●●
●●●●●●●●
●●
●●●●●
●●●●●
●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●●●●●●
●●
●●●
●●●●
●●●●
●●
●●
●
●
●●●●●●●●●●
●●●
●●●
●●●
●●●●
●●
●●●
●●●●
●●●●
●●
●●●●●●●●●●
Time
|dC
WR
ES
| (R
un 2
− R
un 1
)
0.0
0.5
1.0
1.5
2.0
2 4 6 8 10 12
●●
●
●
●●
●
●
●
●
●●
●●
● ●
● ●●
●
●●
●●
● ●●
●●
● ●● ●
●● ● ●
●
●●
●●
●
● ●●
●
●
●
●●
●
●●
●
●
●●
● ●●
●
●
●
●●
●
● ●●
●
●
●●
●● ●
● ●
●
●
●●
●●
●
●
● ●●
●
● ● ●●● ●
● ●●
●
●
●
●
●
●
●● ● ●
●●
●●
●
● ●
●
●
● ●
●●
●
●
●
●● ● ●
●●
●
●
●
●● ●
●
● ●● ●
●●
● ●
●
●
●
●● ● ●●
● ● ●● ●
●
●● ●
●
●
●
● ●
● ●
●
●
●
●●
● ●
●
●
●
●
● ●
●● ●
●●
●
●
● ●●
●● ●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●● ● ● ● ●
●●
●
●
●
●●
●
●
●
●
●
●●
● ●
● ● ● ●
●
●
● ● ● ● ● ● ●
● ● ●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●●
●
●
●●
●●
●●
●●
●●
●
●●
●
●● ● ●
●●
●
● ● ● ●● ●
●●
●
●●●
●
●
●●
●
●● ●
●
●
●
●
●
●
●● ●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
● ●●
●●
●
●●
● ●●
●●
● ●
●
●●
● ●
●
●●
●
● ●● ●
●
●
●
●
● ● ● ●●
●
● ●
●
●
● ● ● ●● ● ●
●
●● ●
●●
●●●
●●
●
●●
●
● ● ●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●● ● ●
●●
●
●
● ●
●
● ●● ●
●●
●
●
●●
●
●●
●
●
● ●●
● ●
● ●● ●
●
● ●
●
●
●
●
●
●●
●●
●●
●●
●●
●
● ●
● ●●
●
●
●● ●
●
●
●
●
● ●
●
●●
●
●●
●
● ●
●●
●
●
●
●●
● ●
●●
●
●
●
●
●● ●
●●
●
● ●
●
● ●●
●●
●
●● ● ●
●● ● ● ●●
● ●●
● ● ●●
●
●
●
● ● ●
●
●
●
●
●●
● ●
●
●● ●
● ●
●
Time
|diW
RE
S| (
Run
2 −
Run
1)
0.0
0.1
0.2
0.3
0.4
2 4 6 8 10 12
●
●
● ● ● ● ●●
●
●
●● ●
●● ●
●●
●
●
●● ● ●
● ● ●●
●●●
● ● ● ● ● ● ●
●
●● ● ●
●●
●●
●
●
●●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●●
●
●
● ● ● ● ● ● ● ●
●
●
●
●
●●
● ● ● ● ● ●●
●
●
●●
● ●● ●
●●
●
●
●
● ● ●● ●
●●
●
●
● ●●
● ●●
●●
●
●
●
●
●
●
● ●
● ●
●●
●
● ● ●●
● ●
●●
●
●
● ●●
● ● ●●
●
● ● ● ● ● ● ● ● ● ●
●
●●
● ●●
● ●
●
●
●
●
● ● ●●
●
●
●
●● ●
●●
●
●
●●
● ● ● ● ●● ●
●
●
●
●● ●
●
●
●
●
●
●● ● ● ● ● ● ● ●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
● ● ● ● ● ● ● ● ● ●
●
●●
●
● ●
●
●
●
●
● ●● ● ● ●
● ●●
●
●● ● ● ●
●● ●
●●
● ● ● ● ● ● ● ●●●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
●
● ●
● ●
●
●
●
●
● ● ● ● ● ● ●●
●
●
●
● ●
●
●
●●
●●
● ● ● ● ● ● ● ●
●
●
● ●●
● ●●
●●
●● ● ● ● ● ● ● ● ●
●
●
● ●
●
●
●
●●
● ●●
● ●●
●
●
● ●
● ● ● ●●
●
●● ●
●●
●● ● ● ● ● ● ● ● ●
●
● ●●
●
●
●
● ●●
●●
●●
● ●
●
●
●●
●●
● ●●
●
●●
●●
● ● ● ● ●●
●●
●● ● ● ●
● ● ●
●
●●
● ●● ●
●●
●● ● ● ● ● ● ● ● ● ●
●
●
●
●
●●
●
●
●
●
● ● ● ● ● ● ● ● ●
●
● ●
● ●●
●
●
● ●
● ●●
●
●●
●
●●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●● ●
●●
●
●
●
●
●
● ●● ●
●●
●●●
● ● ● ● ● ● ●●
●
●
●
●●
●● ● ●
●
●●
●●
●
●
●
●●●
● ● ● ● ● ● ● ●●
Additional model comparison plots (Run 2)
Figure 67: Comparison of Run 1 to Run 2. The top panels show the absolutevalues of the population and individual predictions for Run 1 and Run 2 versuseach other. The bottom panels show the absolute values of the differences be-tween the population and individual weighted residuals (CWRES and IWRES) ofeach run versus time. Individual data points are indicated by blue circles and areconnected by blue lines. The black diagonal lines are the lines of identity.
This graph is intended for visualization ofthe differences between two runs. The twotop panels in this display is, in this exam-ple, the same as the two top panels in “ba-sic.model.comp” on the preceding page (allpredictions have the same sign). The twolower panels display the “residuals of the resid-uals”, i.e. the difference between the corre-sponding residual given the two models. Theidea is that it is easier to see patterns in the dif-
ferences, if they are small, than if the residualsare plotted versus each other.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> xplot <- add.model.comp(xpdb2,
+ xpdb1)
> print(xplot)
84
|Population predictions| (Run 2)
|Pop
ulat
ion
pred
ictio
ns| (
Run
1)
0
20
40
60
80
0 20 40 60 80
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●
●●
●●●
●●●●
●●●●●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●●
●●●
●●
●●
●
●●●
●
●
●●●
●●●●
●●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●●●●●
●●●●
●●
●●
●
●
●
●
●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●
●●●●●
●●●●●●●
●●●●
●●
●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●●●●
●●●●
●●●●
●●
●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●●●
●●
●●
●●●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●
●●●●
●●
●●
●●
●
●
●
●
●●
●●●●●●●
●●●●●
●●●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●●
●●
●
●●●
●●
●
●●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●
●●●●
●●
●●
●●
●●●
●
●
●
●
●
●
●
●●
●●●●●
●●●●
●●
●●
●●●
●●●
●●
●
●
●
●●
●
●
●●●
●●
●●
●●
●
●●●
●●●●●●●●
●●●●
●●
●●
●●●
●●●●
●●
●●●
●●●●
●●
●
●●●
●●●
●●
●●
●●●
|Individual predictions| (Run 2)
|Indi
vidu
al p
redi
ctio
ns| (
Run
1)
0
50
100
150
0 50 100 150
●●
●●
●●
●●
●
●●
●●
●●
●
●
●●
●●●●●●●●●●
●●●●
●●
●
●●
●●●●●●
●●●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●●●●●
●●●
●●●
●●
●
●●
●●●
●
●
●●●●●●●
●●
●●●●●●●●
●●
●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●
●●●●
●●
●
●
●
●
●
●
●●●●
●●●
●●●
●●●●●
●●
●●●
●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●●
●●
●●●●●●●●●●
●●●●
●●
●●
●●
●●
●●
●●
●
●●●
●●●●
●●●●
●●●
●●
●●
●●
●●
●●●●●●●●●●
●●●●●●●
●●●
●●
●●
●●
●
●●●
●●●●●●●●●●●
●●
●●●
●●●●
●●
●
●
●
●
●● ●
●●
●
●●
●
●
●●●●●●●●●●●●●●
●●●●
●●●●
●●●
●●●
●●●
●
●
●
●●●●●●●●●●●●●●●●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●●●
●●●●●
●●
●●
●
●
●
●●●●●
●●●●●
●●●●●●●
●
●
●
●●●
●●●●●●
●●●●●●
●●●●
●●●●●●●●
●●
●●●●●
●●●●●
●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●●●●●●
●●
●●●
●●●●
●●●●
●●
●●
●
●
●●●●●●●●●●
●●●
●●●
●●●
●●●●
●●
●●●
●●●●
●●●●
●●
●●●●●●●●●●
Time
|dW
RE
S| (
Run
2 −
Run
1)
0.0
0.5
1.0
1.5
2 4 6 8 10 12
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
● ●
●●
●
●
●
●●
●● ●
●
●
●
● ●
●●
●
● ●●
●
●
●
●●
●
● ● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●● ●
●●
●●
● ●● ● ●
●
●
●
● ●
●
●● ●
●
●
● ● ●
● ●● ● ●
● ● ● ● ●
●
●
●
● ●
● ● ●
●
●
●●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
● ●
●
●
●
●● ●
●●
●●
●
●
●
●
● ● ●
●
●
●
● ● ● ●
●●
●●
●
● ●●
● ●
●
● ●
●● ●
● ●●●
● ●
●
● ●●
●
●●
●● ●
●●
●
●●
●
●
●
● ●● ●
● ●●
●
●
●●
●
●
● ● ●
●
●
●●
● ●
● ●●
●
●
●
● ● ● ● ● ● ●
● ●●
●● ● ●
●
●
●●
●
● ●
● ●
●
●
● ●
●
●
●
●●
●●
●
●
●●
●
● ●●
●● ● ●
●
●
●
● ● ● ●
●
●●
●
●
●
●● ●
●
●● ●
●●
●
●
●● ●
● ●
●
●
●● ●
●● ●
●
●
●● ●
●●
●
●
●
●
●● ●
●
●
●●
●
●
●
●●
●
●
●●
● ●
●
●●
● ●
●●
●
● ●
●
●
●
●●
●
●
●●
● ●● ●
●
●
●
●
●
●●
●● ●
●●
●
●
● ●
●
●●
● ●
●
● ●● ●
●●
●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●
●
●
●
●●
●
● ●
●
●
● ●
●
●
●
●
●
●●
●
●
● ●●
●
●●
● ● ● ●
● ●
●
●
● ●●
●
●
●
●●
●
●
●
●
●●
●
● ●
● ● ●
●
●
●●
● ● ●●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
● ●
●
●●
●
●
● ● ●● ●
● ●
●
●● ●
●
● ● ●●
●
●
●
●
● ●●
●
●
●
●
●
● ●
●
● ●
●
● ●
●
Time
|diW
RE
S| (
Run
2 −
Run
1)
0.0
0.1
0.2
0.3
0.4
2 4 6 8 10 12
●
●
● ● ● ● ●●
●
●
●● ●
●● ●
●●
●
●
●● ● ●
● ● ●●
●●●
● ● ● ● ● ● ●
●
●● ● ●
●●
●●
●
●
●●
●●
●●
●
●
●● ●
●
●
●
●
●
●●
● ●●
●
●
● ● ● ● ● ● ● ●
●
●
●
●
●●
● ● ● ● ● ●●
●
●
●●
● ●● ●
●●
●
●
●
● ● ●● ●
●●
●
●
● ●●
● ●●
●●
●
●
●
●
●
●
● ●
● ●
●●
●
● ● ●●
● ●
●●
●
●
● ●●
● ● ●●
●
● ● ● ● ● ● ● ● ● ●
●
●●
● ●●
● ●
●
●
●
●
● ● ●●
●
●
●
●● ●
●●
●
●
●●
● ● ● ● ●● ●
●
●
●
●● ●
●
●
●
●
●
●● ● ● ● ● ● ● ●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
● ● ● ● ● ● ● ● ● ●
●
●●
●
● ●
●
●
●
●
● ●● ● ● ●
● ●●
●
●● ● ● ●
●● ●
●●
● ● ● ● ● ● ● ●●●
● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●
●
● ●
● ●
●
●
●
●
● ● ● ● ● ● ●●
●
●
●
● ●
●
●
●●
●●
● ● ● ● ● ● ● ●
●
●
● ●●
● ●●
●●
●● ● ● ● ● ● ● ● ●
●
●
● ●
●
●
●
●●
● ●●
● ●●
●
●
● ●
● ● ● ●●
●
●● ●
●●
●● ● ● ● ● ● ● ● ●
●
● ●●
●
●
●
● ●●
●●
●●
● ●
●
●
●●
●●
● ●●
●
●●
●●
● ● ● ● ●●
●●
●● ● ● ●
● ● ●
●
●●
● ●● ●
●●
●● ● ● ● ● ● ● ● ● ●
●
●
●
●
●●
●
●
●
●
● ● ● ● ● ● ● ● ●
●
● ●
● ●●
●
●
● ●
● ●●
●
●●
●
●●
● ● ●
●
●
●
●
●
●●
●
●
●
●
●● ●
●●
●
●
●
●
●
● ●● ●
●●
●●●
● ● ● ● ● ● ●●
●
●
●
●●
●● ● ●
●
●●
●●
●
●
●
●●●
● ● ● ● ● ● ● ●●
Additional model comparison plots (Run 2)
Figure 68: Comparison of Run 1 to Run 2. The top panels show the absolutevalues of the population and individual predictions for Run 1 and Run 2 versuseach other. The bottom panels show the absolute values of the differences be-tween the population and individual weighted residuals (WRES and IWRES) ofeach run versus time. Individual data points are indicated by blue circles and areconnected by blue lines. The black diagonal lines are the lines of identity.
This display is identical to “add.model.comp”on the previous page except for the bottomleft panel, which shows the weighted residualsrather than the conditional weighted residuals.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> xplot <- add.model.comp(xpdb2,
+ xpdb1, force.wres = TRUE)
> print(xplot)
85
absval.dpred.vs.cov.model.comp
WT
|∆P
RE
D|
0
1
2
3
60 80 100 120 140
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●●●●●
●
●
●
●●
●
●●●
●●
●
●●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●●●●●
●●
●
●●●●●●●
●●
●
●●●●●●●
●●
●
●●●●●●●
●●
●
●●●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●●●●
●●
●
●●●●●●●●
●●●●●●●
●
●
●
●●
●●
●
●
●
●●
●●●●●
●
●
●
●●
●●●●●
●
●
●
●●
●●●●●
●
●
●
●●
●●●●●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●●●●●●
●
●
●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●●●●●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●●●●●●
●
●
●
●●
●●●●●
●
●
●
●●
●●●●●
●
●
●
●
●
●
●●●●●●●●●●
●
●
●
●●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●●●●●●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●●●
●
●
●
●●
●
●●●●
●●●●●●●●●●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●●
●
●●●●●
●●
●●●●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●●
●●
●●●●●●●●
●
●
●
●●
●
●●●●
●
●
●●
●●●●
●
●
●
●●
●
●●●●
●
●
●
●●
●
●●●●
SEX
|∆P
RE
D|
0
1
2
3
1 2
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
|PRED_(Run2) − PRED_(Run1)| vs. Covariates
Figure 69: Comparison of Run 1 to Run 2. The absolute differences between thepopulation predictions are plotted verus body weight and sex, respectively. Thered line is a smooth.
This display is used to investigate if the differ-ences in population predictions between twomodels can be explained by covariates. If so,it may be possible to include this covariate ef-fect in the model.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- absval.dpred.vs.cov.model.comp(xpdb2,
+ xpdb1, smooth = TRUE)
> print(xplot)
86
absval.dipred.vs.cov.model.comp
WT
|∆IP
RE
D|
0
5
10
15
60 80 100 120 140
●
●
●●●●●●●
●
●
●●
●
●●●●●●●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●●
●●●●●●●●
●
●
●
●●
●●●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●●
●
●
●
●●●●●●●
●
●
●
●
●
●●●●●
●
●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●
●
●●●●●●
●
●●●●●●●●●
●
●
●
●●●●●
●
● ●
●●●●●●●●●
●
●
●●
●●●●
●
●
●
●●●●●●●●●
●●●●●●●●●●
●
●●
●●●●
●
●
●
●
●
●●
●
●●●●
●
●●●●●●●●●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●●●●●●●●
●
●
●
●●
●
●●
●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●
●
●
●
●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●
●
●
●●●●●●●●
●
●●
●●
●
●
●
●
●
●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●● ●●●●●●●●●●
●
●
●●●●●●●●
●●●●●●●●●
●●●●●●
●
●
●●●●●●●●
●
●●●●●●
●
●
●●
●●●●
●
●
●●●●●●●●
●●●●●●●●●●
●
●●●●●●●●
●
●
●●●●●●●
●
●●
●●●
●●●●
●
●●●●●●●●●
SEX
|∆IP
RE
D|
0
5
10
15
1 2
● ●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
|IPRED_(Run2) − IPRED_(Run1)| vs. Covariates
Figure 70: Comparison of Run 1 to Run 2. The absolute differences between theindividual predictions are plotted verus body weight and sex, respectively. Thered line is a smooth.
This display is used to investigate if the dif-ferences in individual predictions between twomodels can be explained by covariates. If so, itmay be possible to include this covariate effectin the model.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- absval.dipred.vs.cov.model.comp(xpdb2,
+ xpdb1, smooth = TRUE)
> print(xplot)
87
absval.dcwres.vs.cov.model.comp
vs. WT (Run 2)
WT
|∆C
WR
ES
|
0.0
0.5
1.0
1.5
2.0
60 80 100 120 140
●●
●
●
●
●
●
●
●
●
●●●●
●●
●●●
●
●●●
●●●
●
●●
●●●●●●●●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●●●●●
●
●
●●
●●
●
●
●●
●
●
●●●●
●●
●●●●
●
●
●
●
●
●●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●●●
●●
●
●
●
●●●
●
●●●●
●
●
●●
●
●
●
●●●●●
●●●●●
●
●●●
●
●
●
●●
●●
●
●
●
●●●●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●●
●
●
●●●●●●●
●●●
●
●
●
●
●
●
●
●
●
●●●●●
●●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●●●●●
●
●●●●
●●●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●●
●●
●
●●●●●
●●
●●
●
●
●
●●
●
●●
●
●●
●●●
●
●
●
●●●●●
●
●●
●
●
●●●●
●●●
●
●
●●●●
●●
●●
●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●●
●
●
●●
●
●●●●●
●
●
●
●●
●
●●
●
●
●●●
●●
●●
●●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●
●
●●●
●●
●
●●●●
●●●●●●
●●●
●●●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●●●
●●
●
vs. SEX (Run 2)
SEX
|∆C
WR
ES
|0.0
0.5
1.0
1.5
2.0
1 2
● ●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●●
●●
●●
●●
●
|CWRES_(Run2) − CWRES_(Run1)| vs. Covariates
Figure 71: Comparison of Run 1 to Run 2. The absolute differences between theconditional weighted residuals are plotted verus body weight and sex, respec-tively. The red line is a smooth.
This display is used to investigate if the dif-ferences in conditional weighted residuals be-tween two models can be explained by covari-ates. If so, it may be possible to include thiscovariate effect in the model.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- absval.dcwres.vs.cov.model.comp(xpdb2,
+ xpdb1, smooth = TRUE)
> print(xplot)
88
absval.dwres.vs.cov.model.comp
WT
|∆W
RE
S|
0.0
0.5
1.0
1.5
60 80 100 120 140
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●●
●●●
●
●
●
●●
●
●
●●
●
●
●●●
●●
●●●
●●●●●
●
●
●
●●
●●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●●
●●
●
●
●
●●●
●●
●
●●
●
●●●●●
●●●
●
●●●●
●●
●
●●
●
●●
●
●
●
●
●
●●●●●●●●
●
●●
●
●
●●●
●
●
●
●
●●
●●●
●
●
●
●●●●●●●
●●●
●●●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●●
●
●●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●●●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●●●
●●●●
●
●
●●
●
●
●
●●
●
●●●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●●●
●
●●
●●●●
●●
●
●
●●●●
●
●
● ●
●
●
●
●
●
●●
●●
●●●
●
●
●●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●●
●●
●
●
●●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
SEX
|∆W
RE
S|
0.0
0.5
1.0
1.5
1 2
● ●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
|WRES_(Run2) − WRES_(Run1)| vs. Covariates
Figure 72: Comparison of Run 1 to Run 2. The absolute differences betweenthe population weighted residuals are plotted verus body weight and sex, respec-tively. The red line is a smooth.
This display is used to investigate if the dif-ferences in population weighted residuals be-tween two models can be explained by covari-ates. If so, it may be possible to include thiscovariate effect in the model.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- absval.dwres.vs.cov.model.comp(xpdb2,
+ xpdb1, smooth = TRUE)
> print(xplot)
89
absval.diwres.vs.cov.model.comp
WT
|∆IW
RE
S|
0.0
0.1
0.2
0.3
0.4
60 80 100 120 140
●
●
●●●●●
●●
●
●
●●●●●●●
●
●
●
●●●●●●●●●
●●●●●●●●
●
●
●●●●●
●●
●
●
●
●●●
●●
●
●
●●●●
●
●
●
●
●●
●●
●●
●
●●●●●●●●
●
●
●
●
● ●
●●●●●●●●
●
●●
●●
●●●●
●
●
●
●●●
●●●
●
●
●
●●●●●●●●
●
●
●
●
●
●
●●
●●
● ●
●
●●●●●●
●●
●
●
●●●●●●●●
●●●●●●●●●●
●
●●
●●
●
●●
●
●
●
●
●●●
●●
●
●
●
●●●●
●
●
●
●●●●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●●●●●●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●●
●
●●●●●●●●●●
●
●●
●
●●
●
●
●
●
●●
●●●●
●●
●
●
●
●●●●●●●
●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●
●
●●
●●
●
●
●
●
●●●●●●●●
●
●
●
●●
●
●
●
●
●●
●●●●●●●●
●
●
●●
●●●●
●
●
●●●●●●●●●●
●
●
●●
●
●
●
●●●●●●●●
●
●
●●
●●●●
●●
●
●●●●
●
●●●●●●●●●
●
●●●
●
●
●
●●●●
●
●●
●●
●
●
●●●●
●●●●
●
●
●●●●●●●●
● ●
●
●●●●●●●
●
●●
●●
●●●●
●
●●●●●●●●●●
●
●
●
●
●●●
●
●
●
●●●●●●●●●
●
●●
●●
●
●
●
●●
●●
●●
●●
●
●●●●●
●
●
●
●
●
●●●
●
●
●
●●●●●●
●
●
●
●
●●
●●●●●● ●
●●●●●●●●
●
●
●
●●
●●●●
●
●●
●●●
●
●
●●
●
●●●●●●●●●
SEX
|∆IW
RE
S|
0.0
0.1
0.2
0.3
0.4
1 2
● ●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●●●
●
●
●●
●
●
●●●
|IWRES_(Run2) − IWRES_(Run1)| vs. Covariates
Figure 73: Comparison of Run 1 to Run 2. The absolute differences between theindividual weighted residuals are plotted verus body weight and sex, respectively.The red line is a smooth.
This display is used to investigate if the dif-ferences in individual weighted residuals be-tween two models can be explained by covari-ates. If so, it may be possible to include thiscovariate effect in the model.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- absval.diwres.vs.cov.model.comp(xpdb2,
+ xpdb1, smooth = TRUE)
> print(xplot)
90
dOFV.vs.id
Individual influence on change in OFV(Run1 − Run2)
Number of subjects removed
∆OF
V
−100
−50
0
0 10 20 30 40
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
21
54
26
5552 1
● ● ●
● ● ●
∆OFVi < 0∆OFVi > 0Significant ∆OFVTotal ∆OFV
Figure 74: Comparison of Run 1 to Run 2. Individuals are ranked based on thedifference in individual OFV between the two runs. Then the individual that mostbenefits from run 2 (blue) and most benefit from run 1 (magenta) are removedfrom the ∆OFV calculation along the x-axis. The dashed red line indicates thecritical value for a significance at p < 0.05. The dashed grey indicates the totalchange in OFV going from run 1 to run 2.
This display, also known as a “shark plot” isused to investigate how the OFV differencesbetween the new model (run 1 in this case)and the reference model (run 2) is distributedacross individuals. If a few individuals are driv-ing the difference between run1 and run2 thenthe plot will quickly move from below the redline to above the red line as the x-axis in-
creases.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> xplot <- dOFV.vs.id(xpdb2, xpdb1)
> print(xplot)
91
dOFV.vs.cov
WT
∆OF
Vi
−10
−5
0
60 80 100 120 140
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
SEX
∆OF
Vi
−10
−5
0
1 2
●●
●
●
●
●
●●
Individual change in OFV vs. Covariate(s)(Run1 − Run2)
Figure 75: Comparison of the change in individual OFV between Run 1 and Run2 over covariates. The gray dashed line indicates no difference and the red lineis a smooth.
This display can be used to identify and eval-uate if certain covariate related sub-groupsof individuals benefit more from the modelchange.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> change.xvardef(xpdb2, "covariates") <- c("WT",
+ "SEX")
> xplot <- dOFV.vs.cov(xpdb2, xpdb1,
+ smooth = TRUE, max.plots = 4)
> print(xplot)
92
dOFV1.vs.dOFV2
Individual change in OFV1 vs. OFV2(OFV 1 = Run1 − Run2)(OFV 2 = Run3 − Run2)
∆OFV2i
∆OF
V1 i
−10
−5
0
−40 −20 0 20 40
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Figure 76: Comparison of the improvements with Run 1 to Run 2, and Run 3 toRun 2. The individuals differences in OFV for Run 1 to Run 2 are plotted versusthe individual differences in OFV for Run 3 versus Run 2. The grey dashed line isthe line of identity and the red solid line is a linear regression line.
This graph can be used to investigate to whatextent the difference in OFV obtained with onemodel are correlated to the differences ob-tained with another model. A possible conclu-sion can be that the two contending modelsare equivalent.
Code used to generate the graph:
> xpdb1 <- xpose.data(1, directory = "../")
> xpdb2 <- xpose.data(2, directory = "../")
> xpdb3 <- xpose.data(3, directory = "../")
> xplot <- dOFV1.vs.dOFV2(xpdb2,
+ xpdb1, xpdb3)
> print(xplot)
93
Model developmentModel development usually involves trying many alternative candidate models. This can bea tedious and time consuming task. Conversely, if this process can be made more efficient,model development time can be shortened and results and conclusions obtained more quickly.
The time required for developing a model can be reduced by excluding unlikely model candi-dates by graphical examination or by automating model building tasks. The functions in thissection are specifically designed to do both of these tasks.
94
parm.vs.cov
WT
Cle
aran
ce
10
20
30
40
60 80 100 120 140
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
SEX
Cle
aran
ce10
20
30
40
1 2
●
●
●
●
●
Parameters vs. covariates (Run 1)
Figure 77: The individual parameter estimates (blue circles) are plotted versusthe covariates. The red line is a smooth.
This graph is used to investigate if there areany trends between the individual parametersand the covariates. Note that there will betrends in these graphs if the covariates are in-cluded as predictors for the parameters in themodel. To investigate if there are any remain-ing trends between the parameters and covari-ates after the covariates have been included inthe model it is better to use the ranpar.vs.cov
(96). function. Shrinkage may decrease the
usefulness of this graph.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "parms") <- c("CL")
> change.xvardef(xpdb, "covariates") <- c("WT",
+ "SEX")
> xplot <- parm.vs.cov(xpdb)
> print(xplot)
95
ranpar.vs.cov
WT
ETA
1
−0.5
0.0
0.5
60 80 100 120 140
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
SEX
ETA
1
−0.5
0.0
0.5
1 2
●
●
Parameters vs. covariates (Run 1)
Figure 78: The posterior Bayes estimates (blue circles) of the ηs are plottedversus the covariates. The red line is a smooth.
This graph is used to investigate if there areany trends between the posterior Bayes es-timates of the ηs and the covariates. If nocovariates have been included in the model,this graph will provide the same informationas parm.vs.cov on page 95, except for differ-ences caused by the parameterization of therelationship between the individual parameterand the η. Since it is what is remaining afterthe covariates have been accounted for thatis plotted versus the covariate, this graph isuseful if one wish to assess to what extent
the covariate model is appropriate. However,shrinkage may be an issue as with all posteriorBayes estimates.
Code used to generate the graph:
> xpdb <- xpose.data(1, directory = "../")
> change.xvardef(xpdb, "ranpar") <- c("ETA1")
> change.xvardef(xpdb, "covariates") <- c("WT",
+ "SEX")
> xplot <- ranpar.vs.cov(xpdb)
> print(xplot)
96
xpose.gamThe xpose.gam function can be used to run a stepwise GAM analysis (Mandema et al,̇ J Phar-macokinet Biopharm, 20:511-528,1992). It is typically run from the Xpose classic menu inter-face but can also be invoked on the command line.
To run the xpose.gam on the command line:
Looking for NONMEM table files.
Reading ../xptab3
Table files read.
Reading ../run3.phi
Looking for NONMEM simulation table files.
No simulated table files read.
> gobj1 <- xpose.gam(xpdb, parnam = "ETA1")
Start: ETA1 ~ 1; AIC= 48.5707
Trial: ETA1 ~ HT + 1 + 1 + 1 + 1; AIC= 45.8927
Trial: ETA1 ~ 1 + WT + 1 + 1 + 1; AIC= 43.632
Trial: ETA1 ~ 1 + 1 + AGE + 1 + 1; AIC= 49.2928
Trial: ETA1 ~ 1 + 1 + 1 + RACE + 1; AIC= 52.5597
Trial: ETA1 ~ 1 + 1 + 1 + 1 + SEX; AIC= 41.48
Step : ETA1 ~ SEX ; AIC= 41.48
Trial: ETA1 ~ HT + 1 + 1 + 1 + SEX; AIC= 43.4297
Trial: ETA1 ~ 1 + WT + 1 + 1 + SEX; AIC= 39.9721
Trial: ETA1 ~ 1 + 1 + AGE + 1 + SEX; AIC= 41.9986
Trial: ETA1 ~ 1 + 1 + 1 + RACE + SEX; AIC= 43.6867
Step : ETA1 ~ WT + SEX ; AIC= 39.9721
Trial: ETA1 ~ HT + WT + 1 + 1 + SEX; AIC= 41.8201
Trial: ETA1 ~ 1 + ns(WT, df = 2) + 1 + 1 + SEX; AIC= 40.0188
Trial: ETA1 ~ 1 + WT + AGE + 1 + SEX; AIC= 40.1546
Trial: ETA1 ~ 1 + WT + 1 + RACE + SEX; AIC= 42.3149
> gobj1
Call:
gam(formula = ETA1 ~ WT + SEX, data = gamdata, trace = FALSE)
Degrees of Freedom: 63 total; 61 Residual
Residual Deviance: 6.17538
The function returns an Xpose gam object, which can be used for plotting purposes.
The on-screen output from the xpose.gam function is the trace information from the search, fromwhich it is possible to deduce which covariates that were judged to be significant according tothe AIC criterion. However, it is easier to assess this graphically using the xp.plot function(page 99).
By default all covariates defined in the Xpose database will be used in the search. To limit this,one can either change the covariate scope using the xvardef function before running step.gam
or by using the argument covnams, e.g. covnams=c("CL","WT").
97
There are a number of search parameters that can be changed as well. See the xpose.gam
help page (>=?xpose.gam).
98
xp.plot
WT
Res
idua
ls
−0.5
0.0
0.5
−20 0 20 40
1
23
4
5
6
7
8
9
10
12
1314
15
16
17 18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
4445
46
4748
49
50
51
52
53
54
55
56
57
58
59
60
61
62
6364
65
SEX
Res
idua
ls
−0.5
0.0
0.5
1 2
1
23
4
5
6
7
8
9
10
12
1314
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
3233
34
35
36
3738
39
40
41
4243
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
6465
●
●
GAM results for covariates on ETA1 (Run 3)
Figure 79: The partial residual of each term in the final stepwise GAM model ver-sus the covariates. The continuous covariate (WT) is centered around its medianvalue. The blue line in the left panel is a smooth and the blue lines in the rightpanel are the medians in each covariate group. Each data point is labeled by anID number.
This display visualizes the fit of the final step-wise GAM model. The partial residuals, i.e.the residuals obtained when the covariate isremoved from the model, versus each covari-ate are plotted. Continuous covariates arecentered around the median value (to mimicwhat is typically done in the NONMEM model).
The plots can be used to hint at the covariatemodel one might use when testing in NONMEM.
Code used to generate the graph:
> xplot <- xp.plot(gamobj = gobj1)
> print(xplot)
99
xp.ind.inf.fit
Individual influence on the GAM fit for ETA1 (run 3)
Leverage (h/(1−h))
Coo
ks d
ista
nce
0.00
0.05
0.10
0.15
0.05 0.10 0.15 0.20 0.25
1
23
4
5
67
8910
12
13141516 17
18
19
20
2122 23
24
25 2627
28
293031
32
33
34
353637 38
39
4041
42 43
44
45
46
4748
49
505152
53
54
55
56
57 58
59
60
616263
6465
Figure 80: The Cooks distance versus the leverage. Each data point is labeledby the ID number.
This graph is used to identify individuals thatare overly influential in the GAM model. Suchindividuals have a high Cooks distance (largeimpact on parameter estimates when left outof the data set) and a high leverage (a highimpact on the uncertainly of the parameter es-timates when left out of the data set). If thereare such individuals, they can be omitted (us-ing the subset argument to the xp.gam func-
tion) to investigate if they would also influencethe covariate selection. Note that an individu-als may be highly influential without being in-correct.
Code used to generate the graph:
> xplot <- xp.ind.inf.fit(gamobj = gobj1,
+ aspect = 1)
> print(xplot)
100
xp.ind.inf.terms
Index number (ID)
WT
0.0
0.1
0.2
0.3
0 20 40 60
12345678910
121314151617
18
192021222324252627
28
293031323334
353637383940414243
4445464748
49
50515253545556
5758
59
60616263
6465
Index number (ID)
SE
X2
0.0
0.1
0.2
0.3
0 20 40 60
1
23
4
5678910121314151617181920
212223242526
27
28
293031323334
35363738394041
4243444546
4748
49
505152
53
54
55
56
575859606162636465
Inidividual influence (Cooks distance) on each GAM termfor ETA1 (Run 3)
Figure 81: The Cooks distance versus the ID number. Each data point is labeledby the ID number.
The xp.ind.inf.fit graph on page 100 vi-sualizes the individual influence on the over-all fit of the GAM model. In contrast, what thexp.ind.inf.terms function does is to investi-gate the Cooks distances for each term in thefinal GAM model. In other words, it is possible toidentify if there is any specific covariate effect
that is driven by one or a few individuals.
Code used to generate the graph:
> xplot <- xp.ind.inf.terms(gamobj = gobj1,
+ aspect = 1)
> print(xplot)
101
xp.ind.stud.res
Studentized residual of the GAM fit for ETA1 (Run 3)
Studentized residual
ID
532842119404933612265592464187
93114291350633321715605452573747204338242354862530581681036236514426141274656452534123955
−2 −1 0 1 2● ● ● ●●
●● ● ●●●●●●● ● ●●
● ●●●●
● ●●●●●●
●●●●●●●● ● ●●●
●●●●
● ●●●●
● ● ●●● ● ● ● ●●● ● ●
Figure 82: The ID number versus the Studentized residuals. The ID numbers areordered according to the residuals.
Studentized residuals are scaled residualssuch that their standard deviation is 1. Thismeans that we can interpret the scale of theresiduals and judge to what extent any residu-als are much larger that what should be ex-pected. In this graph, the ID numbers, or-dered according to the Studentized residuals,are plotted agains the latter. The scale on thex-axis is in standard deviation meaning that
residuals outside -2 and 2 have a probabilitythat is less than ≈ 2.5%. It may be perfectlyreasonable for an individual to fall outside -2 or2 but it may be worthwhile investigating theseindividuals more closely.
Code used to generate the graph:
> xplot <- xp.ind.stud.res(gamobj = gobj1)
> print(xplot)
102
xp.akaike.plot
AIC values from stepwise GAM search on ETA1 (Run 3)
Akaike value
Mod
els
WT + SEX
ns(WT, df = 2) + SEX
WT + AGE + SEX
SEX
HT + WT + SEX
AGE + SEX
WT + RACE + SEX
HT + SEX
WT
RACE + SEX
HT
1
AGE
RACE
40 45 50
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Figure 83: The best models tested in the stepwise GAM analysis, ordered ac-cording to their Akaike values, versus the Akaike value.
This is a diagnostic for the stepwise GAM anal-ysis. It visualizes the (up to 30) best modelstried during the search. If the final model hasan Akaike value that is well separated fromother models, one may be more confident thatthe GAM search found the most important pre-dictors in teh correct functional form. Con-versely, if the separation is small, one should
probably also regard the models close to thefinal model as likely indicators of which theimportant covariates and/or functional formsmight be.
Code used to generate the graph:
> xplot <- xp.akaike.plot(gamobj = gobj1)
> print(xplot)
103
Getting more helpAll xpose.specific functions have R-help files. They are accessible from the various R GUIs,and how you find then depend on the GUI you are using. A more generic way to find help in R
is to type ?function-name on the R command line, e.g.:
> ?dv.vs.idv
104
What is not included?The intention with this document is to demonstrate all the functions available in xpose.specific.However, this version of the document will not completely fulfill that objective. A majority of thefunctions not covered in this document require simulated data in addition to typical NONMEM tablefiles. These functions require additional NONMEM runs to produce; a topic we will cover in a futureversion of this document.
Below is a list, with a short descripion, of the functions that are not included. They also havethe usual help pages in R so you are not left completely alone should you want to use them.
cat.dv.vs.idv.sb This function generates a goodness of fit graph for ordered categoricaldata. It is basically a stacked bar chart with the independent variable, for example dose orintervals of time, as the x-variable and stacked bars of the observations as the y-variable.
kaplan.plot A function that visualizes the data and VPC from survival type models in NON-MEM.
xpose.VPC This is the function that forms the post-processor of the VPC output from PsN.
xpose.VPC.categorical This is the PsN post-processor for VPCs of categorical variables.
xpose.VPC.both Another VPC post-processor. It is a combination of xpose.VPC and xpose.categorical.
cat.pc Categorical (visual) predictive checks, without the need for PsN.
npc.coverage Implements the graphical back-end to the PsN npc output.
In addition to the above functions, there are also functionality, common to all xpose.specificfunctions that have been left out of this document. Most notably the mirror functionality has notbeen mentioned. Mirror functionality allows xpose to generate a reference plot to a goodnessof fit plot by simulating data. The basic idea is that we don’t really know what a goodness of fitplot should look like if the model is correct (admittedly, some plots we have a fair expectationfor, but there are cases when we don’t), but by simulating data from the model and then makingthe same goodness of fit plot based on the simulated data, we can learn what the plot shouldlook like if the model is correct.
This functionality, which is available for all xpose.specific functions, can be invoked by spec-ifying mirror=T as an argument. It is expected that there are simulated versions of the tablefiles in the same directory as the real table files are. PsN can be asked to set these simulationsup, e.g. execute -mirror_plots='integer', where integer is the number of simulated datasets to generate.
105
How to make NONMEM generate inputto Xpose
How Xpose identifies NONMEM runsXpose recognizes NONMEM runs, and files associated to a particular run, though the run num-ber. This is a number that is used in the name of NONMEM model files, output files and tablefiles.
The fundamental input to Xpose is one or more NONMEM table files. Even though Xpose rec-ognize and can read more than one table file from a single NONMEM run, the most convenientapproach is to use only one table file. This table file should be named xptab followed by therun number, for example xptab1 for run number 1.
The other table files that Xpose reads are sdtab, patab, catab, cotab, mytab, extra and cwtab.The contents of some of these files influence the way that Xpose interprets the data (see below).
Xpose data variablesDefault variable definitions
Within the functions in the Xpose specific library all graphs are defined using Xpose data vari-ables. These variables can be viewed as pointers to one or more columns in the NONMEM tablefile(s). For example, by default the id data variable points to the ID column in the NONMEM ta-ble file. The default definitions of the Xpose data variables are shown in Table “Xpose variabledefinitions” on the following page.
Xpose uses the data variable definitions in the specific functions to determine which data toplot or tabulate. For example, the function dv.vs.idv will plot the column in the NONMEM tablefiles indicated by the dv data variable (default DV) versus the column indicated by the idv datavariable (default TIME). This allows for a great deal of flexibility since the variable definitions canbe changed by the user.
Changing the default variable definitions
One useful thing to remember when using the xpose.specific functions is that they are de-fined in terms of the Xpose variable definitions (Table “Xpose variable definitions” on the nextpage). In other words, the functions dv.vs.idv and pred.vs.idv only deviate from each otherin which Xpose variables they use as the y-variable (dv or pred).
It is possible to change the definition of these variable and, in effect, chang the type of graph aparticular function would generate. For example, the code
106
Table 1: Xpose variable definitionsName Default value Descriptionid ID Used to identify individuals.idlab ID Used to label data points.idv TIME, TAD or CP (in that order) Used as the independent variable.dv DV or the variable in the fourth col-
umn from the right in the table files(checked in the order sdtab, patab,catab and cotab)
Used as the dependent variable.
occ OCC Identifies occasions. Presently notused in any Xpose specific functions.
pred PRED Used as the population predictions.ipred IPRED or IPRE Used as the individual predictions.wres WRES Used as the weighted resduals.iwres IWRES or IWRE Used as the individual weighted resid-
uals.cwres CWRES Used as the conditional weighted
residuals.res RES Identifies the residuals. Presently not
used in any Xpose specific functions.parms CL, V, V1, V2, V3, Q, Q1, Q2, Q3, KA,
ETA1, ETA2, ETA3, ETA4, ETA5, ETA6,ETA7, ETA8, ETA9, ET10, ET11, ET12,ET13, ET14, ET15, ET16, ET17, ET18,ET19 and ET20 + any variables in the2nd to (n-4)th columns in the patab
Used as the parameters.
ranpar ETA1, ETA2, ETA3, ETA4, ETA5, ETA6,ETA7, ETA8, ETA9, ET10, ET11, ET12,ET13, ET14, ET15, ET16, ET17, ET18,ET19 and ET20
Used as the random effects parame-ters.
tvparms TVCL, TVV, TVV1, TVV2, TVV3, TVQ,TVQ1, TVQ2, TVQ3 and TVKA
Used as the typical individual param-eters.
covariates GENO, SEX, RACE, DOSE, FLAG, DAY, PAT,GEND, AGE, WT, HT, CRCL and CLCR +any variables in the 2nd to the (n-4)thcolumns in the patab
Used as the covariates.
107
> change.xvardef(xpdb, "dv") <- c("PRED")
> xplot <- dv.vs.idv(xpdb)
> print(xplot)
would produce a plot that looks like Figure “pred.vs.idv” on page 12 even though the functionusually produces the graph in Figure “dv.vs.idv” on page 11.
108
How Xpose identifies data itemsThe xptab should contain all data items that Xpose will need to produce the desired graphs.
Xpose recognize data items by name and will treat variables as either continuous or categoricalbased on the number of unique values in the variable.
109
How to extend and modify the behaviorof Xpose specific functionsBasically, every aspect of the Xpose plots can be influenced by specifying arguments to thefunction being called. In the examples above, we have tried to examplify some of these aargu-ments.
If you feel that an Xpose plot is close to,but not exactly what you want, check the relevanthelp page (104). Please also note that most standard lattice arguments can be passed asarguments to these functions.
110