models for binary outcomes - indiana university …statmath/stat/all/cat/printable.pdfmodels for...

Models for Binary Outcomes

IntroductionThe simple or binary response (for example, success or failure) analysis models the relationship between a binaryresponse variable and one or more explanatory variables. For a binary response variable Y, it assumes:

where p is Prob(Y=y1) for y1 as one of two ordered levels of Y, is the parameter vector, x is the vector of explanatoryvariables, and g is a function of which p is assumed to be linearly related to the explanatory variables.

The binary response model shares a common feature with a more general class of linear models that a function g=g( )

of the mean of the dependent variable is assumed to be linearly related to the explanatory variables. The function g(

), often referred as the link function, provides the link between the random or stochastic component and thesystematic or deterministic component of the response variable. For the binary response model, logistic and probitregression techniques are often employed among all others.

Logistic RegressionFor a binary response variable Y, the logistic regression has the form:

or equivalently,

The logistic regression models the logit transformation of the ith observation's event probability, pi, as a linear functionof the explanatory variables in the vector xi. The logistic regression model uses the logit as the link function.

Logistic Regression with SAS

LOGISTIC Procedure

Suppose the response variable Y is 0 or 1 binary (This is not a limitation. The values can be either numeric or characteras long as they are dichotomous), and X1 and X2 are two regressors of interest. To fit a logistic regression, you canuse:

proc logistic; model y=x1 x2; run;

SAS PROC LOGISTIC models the probability of Y=0 by default. In other words, SAS chooses the smaller value toestimate its probability. One way to change the default setting in order to model the probability of Y=1 in SAS is tospecify the DESCENDING option on the PROC LOGISTIC statement. That is, use:

proc logistic descending;

Example 1: SAS Logistic Regression in PROC LOGISTIC (individual data)

Categorial Analysis - Part 1

http://pytheas.ucs.indiana.edu/~statmath/stat/all/cat/printable.html (1 of 54) [2/24/2000 3:33:59 PM]

The following data are from Cox (Cox, D. R., 1970. The Analysis of Binary Data, London, Methuen, p. 86). At thespecified time (T) of heating, a number of ingots are tested for some temperature settings and whether an ingot is readyor not (S) for rolling is recorded. S=0 means not ready and S=1 means ready. You want to know if the time of heatingaffects whether an ingot is ready or not for rolling.

T S 1 7 1 2 7 1 . . 55 7 1 1 14 0 2 14 0 3 14 1 4 14 1 . . 157 14 1 1 27 0 2 27 0 . . 7 27 0 8 27 1 9 27 1 . . 159 27 1 1 51 0 2 51 0 3 51 0 4 51 1 . . 16 51 1

With this data set INGOT, you can use:

proc logistic data=ingot; model s=t; run;

As a result, you will have the following SAS output:

Sample Program: Logistic Regression

The LOGISTIC Procedure

Data Set: WORK.INGOT Response Variable: S Response Levels: 2 Number of Observations: 387



Link Function: Logit

Response Profile

Ordered Value S Count

1 0 12 2 1 375

Model Fitting Information and Testing Global Null Hypothesis BETA=0

Intercept Intercept and Criterion Only Covariates Chi-Square for Covariates

AIC 108.988 99.375 . SC 112.947 107.291 . -2 LOG L 106.988 95.375 11.614 with 1 DF (p=0.0007) Score . . 15.100 with 1 DF (p=0.0001)

Analysis of Maximum Likelihood Estimates

Parameter Standard Wald Pr > Standardized Odds Variable DF Estimate Error Chi-Square Chi-Square Estimate Ratio

INTERCPT 1 -5.4152 0.7275 55.4005 0.0001 . . T 1 0.0807 0.0224 13.0290 0.0003 0.442056 1.084

Association of Predicted Probabilities and Observed Responses

Concordant = 59.2% Somers' D = 0.499 Discordant = 9.4% Gamma = 0.727 Tied = 31.4% Tau-a = 0.030 (4500 pairs) c = 0.749

The result shows that the estimated logit is

where p is the probability of having an ingot not ready for rolling. The slope coefficient 0.0807 represents the change inlog odds for a one unit increase in T (time of heating). Its odds ratio 1.084 is the ratio of odds for a one unit change inT. The odds ratio can be computed by exponentiating the log odds, i.e., exp(log odds), which is exp(0.0807)=1.084 inthis example.

If you had used the DESCENDING option:

proc logistic descending; model s=t;



run;

it would have yielded the following estimated logit:

where p is the probability of having an ingot ready for rolling.

You may have the same data set arranged in the following frequency format:

T S F 7 1 55 14 0 2 14 1 155 27 0 7 27 1 152 51 0 3 51 1 13

In this case, to have the same output as above, you can use the syntax:

proc logistic; freq f; model s=t; run;

The LOGISTIC procedure also allows the input of binary response data that are grouped so that you can use:

proc logistic; model r/n=x1 x2; run;

where N represents the number of trials and R represents the number of events.

Example 2: SAS Logistic Regression in PROC LOGISTIC (grouped data)

The data set described in the previous example can be arranged in a different way. At the specified time(T) of heating,the number of ingots (N) tested and the number (R) not ready for rolling can be recorded. Now you have:

T R N 7 0 55 14 2 157 27 7 159 51 3 16

With this data set INGOT2, you can use:

proc logistic data=ingot2; model r/n=t; run;

The SAS output will be:





Data Set: WORK.INGOT2 Response Variable (Events): R Response Variable (Trials): N Number of Observations: 4 Link Function: Logit Response Profile

Ordered Binary Value Outcome Count

1 EVENT 12 2 NO EVENT 375






INTERCPT 1 -5.4152 0.7275 55.4005 0.0001 . . T 1 0.0807 0.0224 13.0290 0.0003 0.442056 1.084



Sometimes you may be interested in the change in log odds, and thus the corresponding change in odds ratio for someamount other than one unit change in the explanatory variable. In this case, you can customize your own oddscalculation. You can use the UNITS option:

proc logistic; model y=x1 x2; units x1=list; run;

where list represents a list of units in change that are of interest for the variable X1. Each unit of change in a list has



one of the following forms:

number SD or -SD number*SD

where number is any non-zero number and SD is the sample standard deviation of the corresponding independentvariable X1.

Example 3: Customized Odds Computation

Using the same data set in Example 2, if you use:

proc logistic data=ingot2; model r/n=t; units t=10 -10 sd 2*sd; run;

you will have the following result in addition to the output in Example 2:

Conditional Odds Ratio

Odds Variable Unit Ratio

T 10.0000 2.241 T -10.0000 0.446 T 9.9361 2.230 T 19.8721 4.971

In this example, you calculated four different odd ratio, each corresponding to change in 10 unit increase, 10 unitdecrease, 1 standard deviation increase, and 2 standard deviation increase in T, respectively.

From the SAS PROC LOGISTIC output, you can also obtain predicted probability values. Suppose you want to knowthe predicted probabilities of having an ingot not ready for rolling (Y=0) at each level of time of heating in the data setfrom Example 2. The predicted probability, p, can be computed from the formula:

Thus, for example, at T=7,

This computation can be easily obtained as a part of the SAS output by using the OUTPUT statement and PRINTprocedure:

proc logistic; model r/n=x1 x2; output out=filename predicted=varname; run; proc print data=filename; run;



where filename is the output data set name and varname is the variable name for predicted probabilities. The SASoutput will show all the predicted probabilities for all observation points.

However, if you need to know the predicted probabilities at some levels of explanatory variables other than levels thedata set provides, you need to do something different. You need to create a new SAS data set with missing values forthe response variable. Then you merge the new data with the original data and run the logistic regression using themerged data set. Because the new data set has missing values for the response variable, they do not affect the model fit.But the predicted probabilities will be also calculated for the new observations.

Example 4: Predicted Probability Computation

Using the data in Example 2, if you use:

proc logistic data=ingot2; model r/n=t; output out=prob predicted=phat; run; proc print data=prob; run;

you will have the following additional result to the output in Example 2:


OBS T R N PHAT

1 7 0 55 0.00777 2 14 2 157 0.01358 3 27 7 159 0.03782 4 51 3 16 0.21422

Now suppose you want to compute the predicted probabilities at T=10,20,30,40,50, and 60. You can use the followingsyntax:

data ingot2; input t r n; cards; 7 0 55 14 2 157 27 7 159 51 3 16 ; data new; input t @@; r=.; n=.; cards; 10 20 30 40 50 60 ; data merged; set ingot2 new; run; proc logistic data=merged;



model r/n=t; output out=prob predicted=phat; run; proc print data=prob; run;

You will have the following additional output to show the predicted probability at each level of T of interest:


OBS T R N PHAT

1 7 0 55 0.00777 2 14 2 157 0.01358 3 27 7 159 0.03782 4 51 3 16 0.21422 5 10 . . 0.00987 6 20 . . 0.02185 7 30 . . 0.04768 8 40 . . 0.10089 9 50 . . 0.20095 10 60 . . 0.36045

PROBIT Procedure

You can even use the PROC PROBIT to fit a logistic regression by specifying LOGISTIC as the cumulativedistribution type in the MODEL statement. To fit a logistic regression model, use:

proc probit; class y; model y=x1 x2 / d=logistic; run;

or

proc probit; model r/n=x1 x2 / d=logistic; run;

depending on your data set. If a single response variable is given in the MODEL statement, it must be listed in aCLASS statement. Unlike the PROC LOGISTIC, the PROC PROBIT is capable of dealing with categorical variablesas regressors as shown in the following syntax:

proc probit; class x2; model r/n=x1 x2 / d=logistic; run;

where X2 is a categorical regressor.

Example 5: SAS Logistic Regression in PROC PROBIT

Using the data in Example 2, you may use:



proc probit data=ingot2; model r/n=t / d=logistic; run;

The resulting SAS output will be:


Probit Procedure

Data Set =WORK.INGOT2 Dependent Variable=R Dependent Variable=N Number of Observations= 4 Number of Events = 12 Number of Trials = 387

Log Likelihood for LOGISTIC -47.68727905

Probit Procedure

Variable DF Estimate Std Err ChiSquare Pr>Chi Label/Value

INTERCPT 1 -5.4151721 0.727541 55.40004 0.0001 Intercept T 1 0.08069587 0.022356 13.02885 0.0003

Probit Model in Terms of Tolerance Distribution

MU SIGMA 67.10594 12.39221

Estimated Covariance Matrix for Tolerance Parameters

MU SIGMA

MU 121.813302 35.655509 SIGMA 35.655509 11.786672

GENMOD Procedure

The GENMOD procedure fits generalized linear models (Nelder and Wedderburn, 1972, "Generalized Linear Models,"Journal of the Royal Statistical Society A, 135, pp. 370-384). Logistic regression can be modeled as a class ofgeneralized linear model where the response probability distribution function is binomial and the link function is logit.To use PROC GENMOD for a logistic regression, you can use:

proc genmod; model y=x1 x2 / dist=binomial link=logit; run;



or

proc genmod; model r/n=x1 x2 / dist=binomial link=logit; run;

Example 6: SAS Logistic Regression in PROC GENMOD

Using the data in Example 2, you may use:

proc genmod data=ingot2; model r/n=t / dist=binomial link=logit; run;

You will have the following SAS output:


The GENMOD Procedure

Model Information

Description Value

Data Set WORK.INGOT2 Distribution BINOMIAL Link Function LOGIT Dependent Variable R Dependent Variable N Observations Used 4 Number Of Events 12 Number Of Trials 387

Criteria For Assessing Goodness Of Fit

Criterion DF Value Value/DF

Deviance 2 1.0962 0.5481 Scaled Deviance 2 1.0962 0.5481 Pearson Chi-Square 2 0.6749 0.3374 Scaled Pearson X2 2 0.6749 0.3374 Log Likelihood . -47.6873 .

Analysis Of Parameter Estimates

Parameter DF Estimate Std Err ChiSquare Pr>Chi

INTERCEPT 1 -5.4152 0.7275 55.4000 0.0001 T 1 0.0807 0.0224 13.0289 0.0003 SCALE 0 1.0000 0.0000 . .



NOTE: The scale parameter was held fixed.

PROC GENMOD is especially convenient when you need to use categorical or class variables as regressors. In thiscase, you can use:

proc genmod; class x2; model y=x1 x2 / dist=binomial link=logit; run;

where X2 is a categorical regressor.

Example 7: SAS Logistic Regression in PROC GENMOD (categorical regressors)

This example is excerpted from a SAS manual (SAS, 1996, SAS/STAT Software Changes and Enhancements throughRelease 6.11, pp. 279-284). In an experiment comparing the effects of five different drugs, each drug was tested on anumber of different's ubjects. The outcome of each experiment was the presence or absence of a positive response in asubject. The following data represent the number of responses R in the N subjects for the five different drugs, labeled Athrough E. The response is measured for different levels of a continuous covariate X for each drug. The drug type andthe covariate X are explanatory variables in this experiment. The number of response R is modeled as a binomialrandom variable for each combination of the explanatory variable values, with the binomial number of trials parameterequal to the number of subjects N and the binomial probability equal to the probability of a response. The followingDATA step creates the data set DRUG:

data drug; input drug$ x r n; cards; A .1 1 10 A .23 2 12 A .67 1 9 B .2 3 13 B .3 4 15 B .45 5 16 B .78 5 13 C .04 0 10 C .15 0 11 C .56 1 12 C .7 2 12 D .34 5 10 D .6 5 9 D .7 8 10 E .2 12 20 E .34 15 20 E .56 13 15 E .8 17 20 ;

A logistic regression for these data is a generalized linear model with response equal to the binomial proportion R/N.PROC GENMOD can be used as follows:

proc genmod data=drug; class drug;



model r/n=x drug / dist=binomial link=logit; run;

You will have the SAS output:



Model Information

Description Value

Data Set WORK.DRUG Distribution BINOMIAL Link Function LOGIT Dependent Variable R Dependent Variable N Observations Used 18 Number Of Events 99 Number Of Trials 237

Class Level Information

Class Levels Values

DRUG 5 A B C D E






INTERCEPT 1 0.2792 0.4196 0.4430 0.5057 X 1 1.9794 0.7660 6.6770 0.0098 DRUG A 1 -2.8955 0.6092 22.5894 0.0001 DRUG B 1 -2.0162 0.4052 24.7628 0.0001 DRUG C 1 -3.7952 0.6655 32.5258 0.0001 DRUG D 1 -0.8548 0.4838 3.1218 0.0773 DRUG E 0 0.0000 0.0000 . .



SCALE 0 1.0000 0.0000 . .


In this example, PROC GENMOD automatically generates five dummy variables for each value of the class variableDRUG. Therefore, the same result could be obtained without using PROC GENMOD, but employing PROCLOGISTIC:

if drug='A' then drugdum1=1; else drugdum1=0; if drug='B' then drugdum2=1; else drugdum2=0; if drug='C' then drugdum3=1; else drugdum3=0; if drug='D' then drugdum4=1; else drugdum4=0; if drug='E' then drugdum5=1; else drugdum5=0; proc logistic data=drug2; model r/n=x drugdum1 drugdum2 drugdum3 drugdum4 drugdum5; run;

where the first five lines must be included in the DATA step to create a new data set DRUG2. Notice that one of thefive dummy variables is redundant.

The resulting output will be:



Data Set: WORK.DRUG2 Response Variable (Events): R Response Variable (Trials): N Number of Observations: 18 Link Function: Logit

Response Profile

Ordered Binary Value Outcome Count

1 EVENT 99 2 NO EVENT 138






NOTE: The following parameters have been set to 0, since the variables are a linear combination of other variables as shown.

DRUGDUM5 = 1 * INTERCPT - 1 * DRUGDUM1 - 1 * DRUGDUM2 - 1 * DRUGDUM3 - 1 * DRUGDUM4



INTERCPT 1 0.2792 0.4196 0.4430 0.5057 . . X 1 1.9794 0.7660 6.6772 0.0098 0.259740 7.238 DRUGDUM1 1 -2.8955 0.6092 22.5895 0.0001 -0.539417 0.055 DRUGDUM2 1 -2.0162 0.4052 24.7628 0.0001 -0.476082 0.133 DRUGDUM3 1 -3.7952 0.6654 32.5336 0.0001 -0.822382 0.022 DRUGDUM4 1 -0.8548 0.4838 3.1218 0.0773 -0.154773 0.425 DRUGDUM5 0 0 . . . . .



CATMOD Procedure

SAS CATMOD (CATegorical data MODeling) procedure fits linear models to functions of response frequencies andcan be used for logistic regression. The basic syntax is:

proc catmod; direct x1; response logits; model y=x1 x2; run;

where X1 is a continuous quantitative variable and X2 is a categorical variable. You must specify your continuousregressors in the DIRECT statement. Because the CATMOD procedure is mainly designed for the analysis ofcategorical data, it is not recommended for use with a continuous regressor with a large number of unique values.

Example 8: SAS Logistic Regression in PROC CATMOD


proc catmod data=ingot; direct t; response logits; model s=t; run;



you will see the result:


CATMOD PROCEDURE

Response: S Response Levels (R)= 2 Weight Variable: None Populations (S)= 4 Data Set: INGOT Total Frequency (N)= 387 Frequency Missing: 0 Observations (Obs)= 387

POPULATION PROFILES Sample Sample T Size 1 7 55 2 14 157 3 27 159 4 51 16

RESPONSE PROFILES

Response S 1 0 2 1

MAXIMUM-LIKELIHOOD ANALYSIS

Sub -2 Log Convergence Parameter Estimates Iteration Iteration Likelihood Criterion 1 2 0 0 536.49592 1.0000 0 0 1 0 152.59147 0.7156 -2.1503 0.0138 2 0 106.76794 0.3003 -3.5040 0.0361 3 0 96.711696 0.0942 -4.6746 0.0633 4 0 95.411914 0.0134 -5.2884 0.0779 5 0 95.374601 0.000391 -5.4109 0.0806 6 0 95.374558 4.5308E-7 -5.4152 0.0807 7 0 95.374558 6.605E-13 -5.4152 0.0807

MAXIMUM-LIKELIHOOD ANALYSIS-OF-VARIANCE TABLE

Source DF Chi-Square Prob -------------------------------------------------- INTERCEPT 1 55.40 0.0000 T 1 13.03 0.0003

LIKELIHOOD RATIO 2 1.10 0.5781



ANALYSIS OF MAXIMUM-LIKELIHOOD ESTIMATES

Standard Chi- Effect Parameter Estimate Error Square Prob ---------------------------------------------------------------- INTERCEPT 1 -5.4152 0.7275 55.40 0.0000 T 2 0.0807 0.0224 13.03 0.0003

LOGISTIC REGRESSION Procedure

Unlike in SAS, the SPSS procedure LOGISTIC REGRESSION models the probability of Y=1 or Y’s higher sortedvalue. Suppose the response variable Y is 0 or 1 binary (This is not a limitation for SPSS either. The values can beeither numeric or character as long as they are dichotomous), and X1 and X2 are two regressors of interest. To run alogistic regression, use:

logistic regression var=y with x1 x2.

Example 9: SPSS Logistic Regression in LOGISTIC REGRESSION procedure (individual data)

Using the data in Example 1, you can use:

logistic regression var=s with t.

You will have the SPSS output:

L O G I S T I C R E G R E S S I O N

Total number of cases: 387 (Unweighted) Number of selected cases: 387 Number of unselected cases: 0

Number of selected cases: 387 Number rejected because of missing data: 0 Number of cases included in the analysis: 387

Dependent Variable Encoding:

Original InternalValue Value .00 0 1.00 1

Dependent Variable.. S

Beginning Block Number 0. Initial Log Likelihood Function

-2 Log Likelihood 106.98843

* Constant is included in the model.



Beginning Block Number 1. Method: Enter

Variable(s) Entered on Step Number1.. T

Estimation terminated at iteration number 6 becauseLog Likelihood decreased by less than .01 percent.

-2 Log Likelihood 95.375 Goodness of Fit 346.446

Chi-Square df Significance

Model Chi-Square 11.614 1 .0007 Improvement 11.614 1 .0007

Classification Table for S Predicted .00 1.00 Percent Correct 0 I 1Observed +-------+-------+ .00 0 I 0 I 12 I .00% +-------+-------+ 1.00 1 I 0 I 375 I 100.00% +-------+-------+ Overall 96.90%

---------------------- Variables in the Equation -----------------------

Variable B S.E. Wald df Sig R Exp(B)

T -.0807 .0224 13.0289 1 .0003 -.3211 .9225Constant 5.4152 .7275 55.4000 1 .0000

The output shows that the estimated logit is

where p is the probability of having an ingot ready for rolling. This is the same result as with the use of theDESCENDING option in SAS PROC LOGISTIC.

Probit RegressionProbit regression can be employed as an alternative to the logistic regression in binary response models. For a binaryresponse variable Y, the probit regression model has the form:

or equivalently,



where is the inverse of the cumulative standard normal distribution function, often referred as probit or normit, and

is the cumulative standard normal distribution function.

The probit regression model can be viewed also as a special case of the generalized linear model whose link function isprobit.

Probit Regression with SAS

LOGISTIC Procedure

The SAS LOGISTIC procedure can be also used for a probit regression. To fit a probit regression use theLINK=NORMIT (or PROBIT) option:

proc logistic; model y=x1 x2 / link=normit; run;

Example 11: SAS Probit Regression in PROC LOGISTIC


proc logistic data=ingot; model s=t / link=normit; run;

You will have the SAS output:

Sample Program: Probit Regression


Data Set: WORK.INGOT Response Variable: S Response Levels: 2 Number of Observations: 387 Link Function: Normit

Response Profile

Ordered Value S Count

1 0 12 2 1 375



AIC 108.988 99.018 . SC 112.947 106.934 .



-2 LOG L 106.988 95.018 11.971 with 1 DF (p=0.0005) Score . . 15.100 with 1 DF (p=0.0001)


Parameter Standard Wald Pr > Standardized Variable DF Estimate Error Chi-Square Chi-Square Estimate

INTERCPT 1 -2.8004 0.3284 72.7050 0.0001 . T 1 0.0391 0.0113 11.9525 0.0005 0.388259



PROBIT Procedure

You can use the PROC PROBIT to fit a probit model. The basic syntax you can use is:

proc probit; class y; model y=x1 x2; run;

or

proc probit; model r/n=x1 x2; run;

or

proc probit; class x2; model r/n=x1 x2; run;

depending on the nature of the data set.

Example 12: SAS Probit Regression in PROC PROBIT


proc probit data=ingot; class s; model s=t; run;





Probit Procedure Class Level Information

Class Levels Values

S 2 0 1

Number of observations used = 387

Probit Procedure

Data Set =WORK.INGOT Dependent Variable=S

Weighted Frequency Counts for the Ordered Response Categories

Level Count 0 12 1 375

Log Likelihood for NORMAL -47.5087804

Probit Procedure


INTERCPT 1 -2.8003508 0.331621 71.30839 0.0001 Intercept T 1 0.0390757 0.011425 11.69807 0.0006


MU SIGMA 71.66476 25.59135


MU SIGMA

MU 186.336614 98.799500 SIGMA 98.799500 55.985053

Example 13: SAS Probit Regression in PROC PROBIT (categorical regressors)


proc probit data=drug; class drug; model r/n=x drug; run;



you will have the result:



Class Levels Values

DRUG 5 A B C D E


Probit Procedure

Data Set =WORK.DRUG Dependent Variable=R Dependent Variable=N Number of Observations= 18 Number of Events = 99 Number of Trials = 237


Probit Procedure


INTERCPT 1 0.19031335 0.24926 0.582954 0.4452 Intercept X 1 1.15885442 0.438333 6.989539 0.0082

DRUG 4 64.33502 0.0001 1 -1.7087998 0.331686 26.5416 0.0001 A 1 -1.2286831 0.239099 26.40741 0.0001 B 1 -2.2309708 0.343196 42.2574 0.0001 C 1 -0.5079719 0.291889 3.028612 0.0818 D 0 0 0 . . E

SAS PROC PROBIT models the probability of Y=0 or of Y’s lower sorted value by default. This default can be alteredby using the ORDER option in the PROC PROBIT statement. For example,

proc probit order=freq;

specifies the sorting order for the levels of the classification variables (specified in the CLASS statement) in adescending frequency count; levels with the most observations come first in the order.

Example 14: Altering Order

You may need to model the probability of the value with the higher count. In Example 12, Y=1 has the count 375 andY=0 has 12. If you use:

proc probit order=freq data=ingot; class s;



model s=t; run;

you will have the following output:



Class Levels Values

S 2 1 0


Probit Procedure

Data Set =WORK.INGOT Dependent Variable=S


Level Count 1 375 0 12


Probit Procedure


INTERCPT 1 2.80035085 0.331621 71.30839 0.0001 Intercept T 1 -0.0390757 0.011425 11.69807 0.0006


MU SIGMA 71.66476 25.59135


MU SIGMA

MU 186.336614 98.799500 SIGMA 98.799500 55.985053

Sometimes you will need to know the predicted probability values. For example, if you need to know the probability ofhaving an ingot not ready for rolling (Y=0) at T=7 from Example 12, you can compute the probability using theformula:



from the standard normal probability distribution table. You can obtain this kind of computation using the OUTPUTstatement and the PRINT procedure:

proc probit; model r/n=x1 x2; output out=filename prob=varname; run; proc print data=filename; run;

where filename is the output data set name and varname is the variable name for predicted probabilities. The SASoutput will show all the predicted probabilities for all observation points.



proc probit data=ingot2; model r/n=t; output out=prob2 prob=phat; run; proc print data=prob2; run;

you will have the following additional result:


OBS T S N PHAT

1 7 0 55 0.00576 2 14 2 157 0.01212 3 27 7 159 0.04047 4 51 3 16 0.20969

GENMOD Procedure

Probit regression can be modeled as a class of generalized linear models in which the response probability function isbinomial and the link function is probit. Therefore you can use the PROC GENMOD to fit a probit model:

proc genmod; model r/n=x1 x2 / dist=binomial link=probit; run;

Example 16: SAS Probit Regression in PROC GENMOD

Using the data as in Example 2, you may use:

proc genmod data=ingot2; model r/n=t / dist=binomial link=probit; run;






Model Information

Description Value

Data Set WORK.INGOT2 Distribution BINOMIAL Link Function PROBIT Dependent Variable R Dependent Variable N Observations Used 4 Number Of Events 12 Number Of Trials 387






INTERCEPT 1 -2.8004 0.3316 71.3084 0.0001 T 1 0.0391 0.0114 11.6981 0.0006 SCALE 0 1.0000 0.0000 . .


Probit Regression with SPSS

Again, unlike in SAS, SPSS models the probability of Y=1 or of Y’s higher sorted value. To fit a probit regression, use:

probit r of n with x1 x2 /model probit.

Example 17: SPSS Probit Regression in PROBIT procedure


compute n = 1. execute.



probit r of n with t /model probit /print none.

The resulting SPSS output will be:

* * * * * * * * * * * * P R O B I T A N A L Y S I S * * * * * * * * * * * *

Parameter estimates converged after 13 iterations. Optimal solution found.

Parameter Estimates (PROBIT model: (PROBIT(p)) = Intercept + BX):

Regression Coeff. Standard Error Coeff./S.E.

T -.03908 .01142 -3.42024

Intercept Standard Error Intercept/S.E.

2.80035 .33162 8.44443

Pearson Goodness-of-Fit Chi Square = 352.383 DF = 385 P = .882

Since Goodness-of-Fit Chi square is NOT significant, no heterogeneity factor is used in the calculation of confidence limits.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

SPSS PROBIT procedure supports the inclusion of categorical variables as explanatory variables. However, it onlyaccepts numerically coded categorical variables. If your categorical variable is string, you need to reassign string valuesto numerical values in a new variable before running PROBIT.

Example 18: SPSS Probit Regression in PROBIT procedure (categorical regressors)


autorecode variables = drug / into drug2. probit r of n by drug2(1 5) with x /model probit /print none /criteria iterate(20) steplimit(.1).

where AUTORECORD reassigns the string values A, B, C, D, and E of the variable DRUG to the consecutive integers1,2,3,4, and 5 in the new variable DRUG2.


* * * * * * * * * * * * P R O B I T A N A L Y S I S * * * * * * * * * * * *

DATA Information

18 unweighted cases accepted. 0 cases rejected because of out-of-range group values.



0 cases rejected because of missing data. 0 cases are in the control group.

Group Information

DRUG2 Level N of Cases Label 1 3 A 2 4 B 3 4 C 4 3 D 5 4 E

MODEL Information

ONLY Normal Sigmoid is requested.

Parameter estimates converged after 18 iterations. Optimal solution found.

Parameter Estimates (PROBIT model: (PROBIT(p)) = Intercept + BX):

Regression Coeff. Standard Error Coeff./S.E.

X 1.15886 .43833 2.64378

Intercept Standard Error Intercept/S.E. DRUG2

-1.51849 .32692 -4.64487 A -1.03837 .26286 -3.95027 B -2.04067 .37455 -5.44829 C -.31766 .33551 -.94678 D .19031 .24926 .76350 E

Pearson Goodness-of-Fit Chi Square = 4.383 DF = 12 P = .975

Since Goodness-of-Fit Chi square is NOT significant, no heterogeneity factor is used in the calculation of confidence limits.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Models for Multiple Outcomes

IntroductionAnalysis for multiple outcomes or choices models the relationship between a multiple response variable and one ormore explanatory variables. There are two broad types of outcome sets, ordered(or ordinal) and unordered. The choiceof travel mode (by car, bus, or train) is unordered. Bond ratings, taste tests (from strong dislike to excellent taste),levels of insurance coverage (none, part, or full) are ordered by design. Two different types of approaches are



employed for the two types of models.

Models for Ordered Multiple ChoicesThe ordered multiple choice model assumes the relationship:

where the response of the variable Y is measured in one of k+1 different categories, are k intercept parameters, and

b is the slope parameter vector not including the intercept term. By construction, holds.This model fits a common slopes cumulative model, which is a parallel lines regression model based on the cumulativeprobabilities of the response categories. Ordered logit and ordered probit models are most commonly used.

Ordered Logit Regression

Ordered logit model has the form:

This model is known as the proportional-odds model because the odds ratio of the event is independent of thecategory j. The odds ratio is assumed to be constant for all categories.

Ordered Logit Regression with SAS

LOGISTIC Procedure

SAS PROC LOGISTIC is a procedure you can use for an ordered multiple outcome model as well as for a binarymodel. All previous discussions about the binary logistic regression estimation in PROC LOGISTIC are also valid forordered logit model. To fit an ordered logit model, you can use:

proc logistic; model y=x1 x2; run;

where Y is the ordinally scaled multiple response variable, and X1 and X2 are two regressors of interest.

Example 19: SAS Ordered Logit Regression in PROC LOGISTIC

The following data are from McCullagh and Nelder (McCullagh and Nelder, 1989, Generalized Linear Models,London, Chapman Hall, p. 175) and used in a SAS manual (SAS, 1996, SAS/STAT Software Changes andEnhancements through Release 6.11, pp. 435-438). Consider a study of the effects on taste of various cheese additives.Researchers tested four cheese additives and obtained 52 response ratings for each additive. Each response was



measured on a scale of nine values ranging from strong dislike (1) to excellent taste (9). The data set CHEESE has fivevariables Y, X1, X2, X3, and F. The variable Y contains the response rating and the variables X1, X2, and X3 aredummy variables, representing the first, second, and third additive, respectively; for the fourth additive, X1=X2=X3=0.F gives the frequency of occurrence of the observation. The following DATA step creates the data set CHEESE:

data cheese; input x1 x2 x3 y f; cards; 1 0 0 1 0 1 0 0 2 0 1 0 0 3 1 1 0 0 4 7 1 0 0 5 8 1 0 0 6 8 1 0 0 7 19 1 0 0 8 8 1 0 0 9 1 0 1 0 1 6 0 1 0 2 9 0 1 0 3 12 0 1 0 4 11 0 1 0 5 7 0 1 0 6 6 0 1 0 7 1 0 1 0 8 0 0 1 0 9 0 0 0 1 1 1 0 0 1 2 1 0 0 1 3 6 0 0 1 4 8 0 0 1 5 23 0 0 1 6 7 0 0 1 7 5 0 0 1 8 1 0 0 1 9 0 0 0 0 1 0 0 0 0 2 0 0 0 0 3 0 0 0 0 4 1 0 0 0 5 3 0 0 0 6 7 0 0 0 7 14 0 0 0 8 16 0 0 0 9 11 ;

Because the response variable Y is ordinally scaled, you can estimate an ordered logit model. You can use:

proc logistic data=cheese; freq f; model y=x1-x3; run;




Sample Program: Ordered Logit Regression


Data Set: WORK.CHEESE Response Variable: Y Response Levels: 9 Number of Observations: 28 Frequency Variable: F Link Function: Logit

Response Profile

Ordered Value Y Count

1 1 7 2 2 10 3 3 19 4 4 27 5 5 41 6 6 28 7 7 39 8 8 25 9 9 12

NOTE: 8 observation(s) having zero frequencies or weights were excluded since they do not contribute to the analysis.

Score Test for the Proportional Odds Assumption

Chi-Square = 17.2868 with 21 DF (p=0.6936)








INTERCP1 1 -7.0802 0.5624 158.4865 0.0001 . . INTERCP2 1 -6.0250 0.4755 160.5507 0.0001 . . INTERCP3 1 -4.9254 0.4272 132.9477 0.0001 . . INTERCP4 1 -3.8568 0.3902 97.7086 0.0001 . . INTERCP5 1 -2.5206 0.3431 53.9713 0.0001 . . INTERCP6 1 -1.5685 0.3086 25.8379 0.0001 . . INTERCP7 1 -0.0669 0.2658 0.0633 0.8013 . . INTERCP8 1 1.4930 0.3310 20.3443 0.0001 . . X1 1 1.6128 0.3778 18.2258 0.0001 0.385954 5.017 X2 1 4.9646 0.4741 109.6453 0.0001 1.188080 143.257 X3 1 3.3227 0.4251 61.0936 0.0001 0.795146 27.735




This result shows eight fitted regression lines as follows:

where p1 is the probability of being strongly disliked, i.e, the probability of Y=1, and so on. Positive coefficients of X1,X2 and X3 indicate that adding those additives is associated with increased probability of the cheese being disliked.The estimated odds are reported 5.017, 143.257 and 27.735 for X1, X2 and X3 respectively. Each odd is constant forall categories.


You can compute the predicted probability at a certain level of independent variables. For example, you can use thefollowing formula to compute the predicted probabilities at X1=1, X2=0 and X3=0 for the model in Example 19:

and so on. However, this computation can be easily obtained for each combination of additives by using:

proc logistic data=cheese;



freq f; model y=x1-x3; output out=prob predicted=phat; run; proc print data=prob; run;

You will have the following additional output:


OBS X1 X2 X3 Y F _LEVEL_ PHAT

1 1 0 0 1 0 1 0.00420 2 1 0 0 1 0 2 0.01198 3 1 0 0 1 0 3 0.03514 4 1 0 0 1 0 4 0.09587 5 1 0 0 1 0 5 0.28746 6 1 0 0 1 0 6 0.51106 7 1 0 0 1 0 7 0.82432 8 1 0 0 1 0 8 0.95713 . . 281 0 0 0 9 11 1 0.00084 282 0 0 0 9 11 2 0.00241 283 0 0 0 9 11 3 0.00721 284 0 0 0 9 11 4 0.02070 285 0 0 0 9 11 5 0.07443 286 0 0 0 9 11 6 0.17242 287 0 0 0 9 11 7 0.48329 288 0 0 0 9 11 8 0.81652

In this output you have 8 observations for each additive-response combination. The observation with _LEVEL_=1shows the predicted probability of Y=1, the observation with _LEVEL_=2 shows the predicted probability of Y=1 or 2,and so on.

PROBIT Procedure

To use the SAS PROC PROBIT to fit an ordered logit model, use the syntax:

proc probit; class y; model y = x1 x2 / d=logistic; run;

where Y is the ordinally scaled multiple response variable, and X1 and X2 are two regressors of interest.

Example 21: SAS Ordered Logit Regression in PROC PROBIT

In this example, you are using the same data set as in Example 19. However, SAS PROC PROBIT does not accept adata set in a frequency format. You need to have the same data set in an individual data format; i.e., you need to have:

X1 X2 X3 Y



1 0 0 3 1 0 0 4 1 0 0 4 . . 1 0 0 4 (7 rows of the same data) 1 0 0 5 1 0 0 5 . . 1 0 0 5 (8 rows of the same data) . . .

With this new data set, CHEESE2, you can use:

proc probit data=cheese2; class y; model y = x1-x3 / d=logistic; run;

The resulting SAS output will be:



Class Levels Values

Y 9 1 2 3 4 5 6 7 8 9


Probit Procedure

Data Set =WORK.CHEESE2 Dependent Variable=Y


Level Count 1 7 2 10 3 19 4 27 5 41 6 28 7 39 8 25



9 12

Log Likelihood for LOGISTIC -355.6739524

Probit Procedure


INTERCPT 1 -7.0801649 0.56401 157.5844 0.0001 Intercept X1 1 1.6127909 0.380544 17.96169 0.0001 X2 1 4.96463991 0.476721 108.4546 0.0001 X3 1 3.32268278 0.42183 62.04439 0.0001 INTER.2 1 1.0551848 0.324654 2 INTER.3 1 2.15474934 0.387165 3 INTER.4 1 3.22336352 0.420573 4 INTER.5 1 4.55961327 0.454216 5 INTER.6 1 5.5116267 0.479248 6 INTER.7 1 7.01328969 0.520899 7 INTER.8 1 8.57313924 0.587685 8

Notice that intercept parameter estimates are computed as:

Ordered Probit Regression

The ordered probit model has the form:

or equivalently,

where

is the inverse of the cumulative standard normal distribution function, often referred to as probit or normit, and



denotes the cumulative standard normal distribution function.

Ordered Probit Regression with SAS

LOGISTIC Procedure

To fit an ordered probit model in PROC LOGISTIC, use the LINK=NORMIT (or PROBIT) option as:

proc logistic; model y=x1 x2 / link=normit; run;

Example 22: SAS Ordered Probit Regression in PROC LOGISTIC

Using the data set CHEESE in Example 19, if you use:

proc logistic data=cheese; freq f; model y=x1-x3 / link=normit; run;

you will have:

Sample Program: Ordered Probit Regression


Data Set: WORK.CHEESE Response Variable: Y Response Levels: 9 Number of Observations: 28 Frequency Variable: F Link Function: Normit

Response Profile

Ordered Value Y Count

1 1 7 2 2 10 3 3 19 4 4 27 5 5 41 6 6 28 7 7 39 8 8 25 9 9 12

NOTE: 8 observation(s) having zero frequencies or weights were excluded since they do not contribute to the analysis.



Score Test for the Equal Slopes Assumption

Chi-Square = 15.0251 with 21 DF (p=0.8217)





Parameter Standard Wald Pr > Standardized Variable DF Estimate Error Chi-Square Chi-Square Estimate

INTERCP1 1 -4.0762 0.2867 202.1202 0.0001 . INTERCP2 1 -3.5087 0.2496 197.6200 0.0001 . INTERCP3 1 -2.8628 0.2248 162.2226 0.0001 . INTERCP4 1 -2.2356 0.2067 117.0124 0.0001 . INTERCP5 1 -1.4641 0.1858 62.0947 0.0001 . INTERCP6 1 -0.9155 0.1730 28.0144 0.0001 . INTERCP7 1 -0.0276 0.1607 0.0296 0.8634 . INTERCP8 1 0.8779 0.1841 22.7341 0.0001 . X1 1 0.9643 0.2119 20.7122 0.0001 0.418540 X2 1 2.8618 0.2508 130.2471 0.0001 1.242206 X3 1 1.9408 0.2296 71.4236 0.0001 0.842421




This result shows eight fitted regression lines as:



PROBIT Procedure

You can use the SAS PROC PROBIT to fit an ordered probit model:

proc probit; class y; model y = x1 x2; run;

Example 23: SAS Ordered Probit Regression in PROC PROBIT

Using the data set CHEESE2 in Example 21, you can use:

proc probit data=cheese2; class y; model y = x1-x3; run;

Your SAS output will be:



Class Levels Values

Y 9 1 2 3 4 5 6 7 8 9


Probit Procedure

Data Set =WORK.CHEESE2 Dependent Variable=Y


Level Count 1 7 2 10 3 19 4 27 5 41



6 28 7 39 8 25 9 12


Probit Procedure


INTERCPT 1 -4.0761911 0.2868 202 0.0001 Intercept X1 1 0.9642503 0.211624 20.761 0.0001 X2 1 2.86184814 0.24956 131.5057 0.0001 X3 1 1.94080682 0.230248 71.05121 0.0001 INTER.2 1 0.56749271 0.166663 2 INTER.3 1 1.21337982 0.200722 3 INTER.4 1 1.84054882 0.216171 4 INTER.5 1 2.61204661 0.229904 5 INTER.6 1 3.16070252 0.241469 6 INTER.7 1 4.04855742 0.263792 7 INTER.8 1 4.95412966 0.298786 8

This result shows eight fitted regression lines as:

which are the same results as we obtained in Example 22.


Predicted probability computation can be easily obtained using:

proc probit data=cheese2; class y; model y = x1-x3; output out=prob2 prob=phat; run; proc print data=prob2; run;

As a result, you will have:


OBS X1 X2 X3 Y _LEVEL_ PHAT

1 1 0 0 3 1 0.00093



2 1 0 0 3 2 0.00547 3 1 0 0 3 3 0.02881 4 1 0 0 3 4 0.10179 5 1 0 0 3 5 0.30857 6 1 0 0 3 6 0.51945 7 1 0 0 3 7 0.82552 8 1 0 0 3 8 0.96728 . . 1657 0 0 0 9 1 0.00002 1658 0 0 0 9 2 0.00023 1659 0 0 0 9 3 0.00210 1660 0 0 0 9 4 0.01269 1661 0 0 0 9 5 0.07158 1662 0 0 0 9 6 0.17997 1663 0 0 0 9 7 0.48898 1664 0 0 0 9 8 0.81001

Models for Unordered Multiple ChoicesThe unordered multiple choice model assumes the relationship:

where the response of the variable Y is measured in one of k+1 different categories, and is the parameter vector foreach j. This model is made operational by a particular choice of the distributional form of g. Although two models,logit and probit could be considered as before, the probit model is practically hard to employ. Two different logitmodels are commonly used; one is multinomial logit or generalized logit model and the other is conditional logit(McFadden, 1974, "Conditional Logit Analysis of Qualitative Choice Behavior," Frontiers in Econometrics, Zarembkaed., New York, Academic Press, pp. 105-142) or discrete choice model (this is also often referred as multinomial logitmodel, resulting in a conflict in terminology). The major difference between the two models is found in thecharacteristics of the vector x. The multinomial logit model is typically (but not necessarily) used for the data in whichx variables are the characteristics of individuals, not the characteristics of the choices. The conditional logit model istypically (but not necessarily) employed in the case where x variables are the characteristics of the choices, often calledattributes of the choices.

Multinomial Logit Regression

The multinomial logit model has the form:

can be set to 0 (zero vector) as a normalization and thus:

As a result, the j logit has the form:



Multinomial Logit Regression with SAS

CATMOD Procedure

The SAS CATMOD procedure is capable of dealing with various types of the multinomial logit model. The basicsyntax to fit a multinomial logit model is:

proc catmod; direct x1; response logits; model y=x1 x2; run;

where X1 is a continuous quantitative variable and X2 is a categorical variable. You must specify your continuousregressors in the DIRECT statement.

The RESPONSE statement specifies the functions of response probabilities used to model the response functions as alinear combination of the parameters. Depending on your model, you can specify other types of responses beside theLOGITS. For example, among all others,

The default is LOGITS (generalized logits) and it models:

Example 25: SAS Multinomial Logit Regression in PROC CATMOD

In this example, you are using a modified version of the data set CHEESE in Example 19. Zero frequency for someobservations causes a sparseness of the data and thus you may have problems in fitting the multinomial logit model. Inorder to avoid the zero frequency, the following is being tried:

X1 X2 X3 Y F

1 0 0 1 1 1 0 0 2 1 1 0 0 3 1 1 0 0 4 5 1 0 0 5 8 1 0 0 6 8 1 0 0 7 19 1 0 0 8 8 1 0 0 9 1 0 1 0 1 6 0 1 0 2 9 0 1 0 3 12 0 1 0 4 11



0 1 0 5 7 0 1 0 6 4 0 1 0 7 1 0 1 0 8 1 0 1 0 9 1 0 0 1 1 1 0 0 1 2 1 0 0 1 3 6 0 0 1 4 8 0 0 1 5 23 0 0 1 6 7 0 0 1 7 4 0 0 1 8 1 0 0 1 9 1 0 0 0 1 1 0 0 0 2 1 0 0 0 3 1 0 0 0 4 1 0 0 0 5 1 0 0 0 6 6 0 0 0 7 14 0 0 0 8 16 0 0 0 9 11

You are supposed to rearrange this data in the way you have used in Example 21. Furthermore, you are collapsing thenine response categories into three for a simpler illustration. That is, you will create a new response variable YLESSsuch that

if y=1 or y=2 or y=3 then yless=1; else if y=4 or y=5 or y=6 then yless=2; else yless=3;

With this new data set CHEESE3, if you use:

proc catmod data=cheese3; direct x1-x4; response logits; model yless=x1-x4 / noiter freq; run;

the resulting output will be:

Sample Program: Multinomial Logit Regression

CATMOD PROCEDURE

Response: YLESS Response Levels (R)= 3 Weight Variable: None Populations (S)= 4 Data Set: CHEESE3 Total Frequency (N)= 208 Frequency Missing: 0 Observations (Obs)= 208



POPULATION PROFILES Sample Sample X1 X2 X3 X4 Size 1 0 0 0 1 52 2 0 0 1 0 52 3 0 1 0 0 52 4 1 0 0 0 52

RESPONSE PROFILES

Response YLESS 1 1 2 2 3 3

RESPONSE FREQUENCIES

Response Number Sample 1 2 3 1 3 8 41 2 8 38 6 3 27 22 3 4 3 21 28


Source DF Chi-Square Prob -------------------------------------------------- INTERCEPT 2 33.46 0.0000 X1 2 7.79 0.0203 X2 2 36.33 0.0000 X3 2 37.07 0.0000 X4 0* . .

LIKELIHOOD RATIO 0 . .

NOTE: Effects marked with '*' contain one or more redundant or restricted parameters.


Standard Chi- Effect Parameter Estimate Error Square Prob ---------------------------------------------------------------- INTERCEPT 1 -2.6150 0.5981 19.12 0.0000 2 -1.6341 0.3865 17.88 0.0000 X1 3 0.3814 0.8525 0.20 0.6546



4 1.3464 0.4824 7.79 0.0053 X2 5 4.8122 0.8532 31.81 0.0000 6 3.6266 0.7267 24.90 0.0000 X3 7 2.9026 0.8058 12.97 0.0003 8 3.4800 0.5851 35.37 0.0000 X4 9 . . . . 10 . . . .

The estimation result shows two regression lines:

Thus, the estimated logit at each combination of X’s is

If you need to know the predicted probabilities you can compute them by applying the formula. For example, at X1=1,X2=0 and X3=0,

This computation can be easily obtained by including the PROB statement in the MODEL command as:

proc catmod data=cheese3; direct x1-x4; response logits;



model yless=x1-x4 / noiter freq prob; run;

In addition to the output above, you will get the following:

RESPONSE PROBABILITIES

Response Number Sample 1 2 3 1 0.05769 0.15385 0.78846 2 0.15385 0.73077 0.11538 3 0.51923 0.42308 0.05769 4 0.05769 0.40385 0.53846

Example 26: SAS Multinomial Logit Regression in PROC CATMOD (categorical regressors)

In Example 25, X1, X2, X3, and X4 are dummy variables to denote each type of additive. The same result can beobtained by employing a categorical variable to represent types of additives. To do this, you can create a new variableX such that

if x1=1 then x=1; else if x2=1 then x=2; else if x3=1 then x=3; else x=4;

Now if you use:

proc catmod data=cheese3; response logits; model yless = x / noiter freq prob; run;

the resulting SAS output will be:

Sample Program: Multinomial Logit Regression

CATMOD PROCEDURE

Response: YLESS Response Levels (R)= 3 Weight Variable: None Populations (S)= 4 Data Set: CHEESE3 Total Frequency (N)= 208 Frequency Missing: 0 Observations (Obs)= 208

POPULATION PROFILES Sample Sample X Size 1 1 52 2 2 52 3 3 52 4 4 52

RESPONSE PROFILES



Response YLESS 1 1 2 2 3 3

RESPONSE FREQUENCIES

Response Number Sample 1 2 3 1 3 21 28 2 27 22 3 3 8 38 6 4 3 8 41

RESPONSE PROBABILITIES

Response Number Sample 1 2 3 1 0.05769 0.40385 0.53846 2 0.51923 0.42308 0.05769 3 0.15385 0.73077 0.11538 4 0.05769 0.15385 0.78846


Source DF Chi-Square Prob -------------------------------------------------- INTERCEPT 2 18.26 0.0001 X 6 78.70 0.0000

LIKELIHOOD RATIO 0 . .


Standard Chi- Effect Parameter Estimate Error Square Prob ---------------------------------------------------------------- INTERCEPT 1 -0.5909 0.2946 4.02 0.0449 2 0.4791 0.2242 4.57 0.0326 X 3 -1.6427 0.5209 9.95 0.0016 4 -0.7668 0.3032 6.39 0.0114 5 2.7881 0.5215 28.59 0.0000 6 1.5133 0.4895 9.56 0.0020 7 0.8786 0.4823 3.32 0.0685 8 1.3667 0.3831 12.73 0.0004

The result shows each estimated logit can be calculated as:



This is exactly the same logit computation as in the previous example.

Multinomial Logit Regression with SPSS

GENLOG Procedure

The SPSS GENLOG procedure conducts the general loglinear analysis and the logit model can be treated as a specialclass of loglinear models. However, SPSS is only capable of dealing with a multinomial logit model with categoricalindependent variables. The basic syntax to fit a multinomial logit model is:

genlog y by x1 x2 /model=multinomial /design y y*x1 y*x2 y*x1*x2.

where Y is response variable and X1 and X2 are categorical regressors.

Example 27: SPSS Multinomial Logit Regression in GENLOG

You are using the same data in Example 25. You can use:

genlog yless by x /model=multinomial /print freq estim /plot none /criteria=cin(95) iteration(20) converge(.001) delta(0) /design yless yless*x.


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - GENERALIZED LOGLINEAR ANALYSIS- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Data Information

208 cases are accepted. 0 cases are rejected because of missing data. 208 weighted cases will be used in the analysis. 12 cells are defined. 0 structural zeros are imposed by design. 0 sampling zeros are encountered.



- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Variable Information

Factor Levels Value

YLESS 3 1.00 2.00 3.00

X 4 1.00 2.00 3.00 4.00

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Model and Design Information

Model: Multinomial LogitDesign: Constant + YLESS + YLESS*X

Note: There is a separate constant term for each combination of levels of the independent factors.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Correspondence Between Parameters and Terms of the Design

Parameter Aliased Term

1 Constant for [X = 1.00] 2 Constant for [X = 2.00] 3 Constant for [X = 3.00] 4 Constant for [X = 4.00] 5 [YLESS = 1.00] 6 [YLESS = 2.00] 7 x [YLESS = 3.00] 8 [YLESS = 1.00]*[X = 1.00] 9 [YLESS = 1.00]*[X = 2.00] 10 [YLESS = 1.00]*[X = 3.00] 11 x [YLESS = 1.00]*[X = 4.00] 12 [YLESS = 2.00]*[X = 1.00] 13 [YLESS = 2.00]*[X = 2.00] 14 [YLESS = 2.00]*[X = 3.00] 15 x [YLESS = 2.00]*[X = 4.00] 16 x [YLESS = 3.00]*[X = 1.00] 17 x [YLESS = 3.00]*[X = 2.00] 18 x [YLESS = 3.00]*[X = 3.00]



19 x [YLESS = 3.00]*[X = 4.00]

Note: 'x' indicates an aliased (or a redundant) parameter. These parameters are set to zero.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Convergence Information

Maximum number of iterations: 20Relative difference tolerance: .001Final relative difference: 2.92779E-14

Maximum likelihood estimation converged at iteration 1.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Table Information

Observed ExpectedFactor Value Count % Count %

X 1.00 YLESS 1.00 3.00 ( 5.77) 3.00 ( 5.77) YLESS 2.00 21.00 ( 40.38) 21.00 ( 40.38) YLESS 3.00 28.00 ( 53.85) 28.00 ( 53.85)




- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Goodness-of-fit Statistics

Chi-Square DF Sig.

Likelihood Ratio .0000 0 . Pearson .0000 0 .

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -



Analysis of Dispersion

Source of Dispersion Entropy Concentration DF

Due to Model 55.4019 35.2404 12Due to Residual 163.2376 97.3462 402Total 218.6395 132.5865 414

Measures of Association

Entropy = .2534Concentration = .2658

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Parameter Estimates

Constant Estimate

1 3.3322 2 1.0986 3 1.7918 4 3.7136

Note: Constants are not parameters under multinomial assumption. Therefore, standard errors are not calculated.

Asymptotic 95% CIParameter Estimate SE Z-value Lower Upper

5 -2.6150 .5981 -4.37 -3.79 -1.44 6 -1.6341 .3865 -4.23 -2.39 -.88 7 .0000 . . . . 8 .3814 .8525 .45 -1.29 2.05 9 4.8122 .8533 5.64 3.14 6.48 10 2.9026 .8058 3.60 1.32 4.48 11 .0000 . . . . 12 1.3464 .4824 2.79 .40 2.29 13 3.6266 .7268 4.99 2.20 5.05 14 3.4800 .5851 5.95 2.33 4.63 15 .0000 . . . . 16 .0000 . . . . 17 .0000 . . . . 18 .0000 . . . . 19 .0000 . . . .

SPSS output shows the following model structure:

Assuming YLESS=3 is the reference category, the estimated logit of YLESS=1 at X=1 is



where m11 is the predicted count for YLESS=1 at X=1 and m31 is the predicted count for YLESS=3 at X=1. Similarly,the estimated logit of YLESS=2 at X=1 is

where m21 is the predicted count for YLESS=2 at X=1. Other possible logit computations at X=2,3 and 4 can bederived in the same manner.

The output provides the following estimation results:

Conditional Logit Regression

The conditional logit model has the form:

In this model, subjects are presented with choice alternatives and asked to choose the most preferred alternative. Theset of alternatives is typically the same for all subjects and the explanatory variables are all choice specific. Unlike inthe multinomial logit model, the parameters are not specific to the choice.

Conditional Logit Regression with SAS

PHREG Procedure

The SAS PHREG procedure performs regression analysis of survival data based on the Cox proportional hazardsmodel. Its likelihood function is similar to that of the conditional logit model.



To fit a conditional logit model with PROC PHREG, you need to rearrange the data set in a form that is consistent withsurvival analysis data. The most preferred choice is said to occur at time 1 and all other choices are said to occur at latertimes or to be censored. You also need to create a status variable to denote whether the observation was censored ornot, i.e., whether the alternative was chosen or not. The censoring indicator variable has the value of 0 if the alternativewas censored (not chosen) and 1 if not censored (chosen). The basic syntax is:

proc phreg; strata strata_varname; model time_varname*status_varname(0) = x1 x2; run;

where strata_varname is the name of variable to specify the variable that determines the stratification, time_varname isthe name of failure time variable (the smaller value means the alternative was chosen), status_varname is the name ofthe censoring indicator variable, of which 0 is the value to indicate censoring, and X1 and X2 are explanatory variables.

Example 28: SAS Conditional Logit Regression in PROC PHREG

This example is from SAS (SAS, 1995, Logistic Regression Examples Using the SAS System, pp. 2-3). Chocolate candydata are generated in which 10 subjects are presented with eight different chocolate candies. The subjects choose onepreferred candy from among the eight types. The eight candies consist of eight combinations of dark(1) or milk(0)chocolate, soft(1) or hard(0) center, and nuts(1) or no nuts(0). The following data step creates the data set CHOCO:

data choco; input subject choose dark soft nuts @@; t=2-choose; cards; 1 0 0 0 0 1 0 0 0 1 1 0 0 1 0 1 0 0 1 1 1 1 1 0 0 1 0 1 0 1 1 0 1 1 0 1 0 1 1 1 2 0 0 0 0 2 0 0 0 1 2 0 0 1 0 2 0 0 1 1 2 0 1 0 0 2 1 1 0 1 2 0 1 1 0 2 0 1 1 1 3 0 0 0 0 3 0 0 0 1 3 0 0 1 0 3 0 0 1 1 3 0 1 0 0 3 0 1 0 1 3 1 1 1 0 3 0 1 1 1 4 0 0 0 0 4 0 0 0 1 4 0 0 1 0 4 0 0 1 1 4 1 1 0 0 4 0 1 0 1 4 0 1 1 0 4 0 1 1 1 5 0 0 0 0 5 1 0 0 1 5 0 0 1 0 5 0 0 1 1 5 0 1 0 0 5 0 1 0 1 5 0 1 1 0 5 0 1 1 1 6 0 0 0 0 6 0 0 0 1 6 0 0 1 0 6 0 0 1 1 6 0 1 0 0 6 1 1 0 1 6 0 1 1 0 6 0 1 1 1 7 0 0 0 0 7 1 0 0 1 7 0 0 1 0 7 0 0 1 1 7 0 1 0 0 7 0 1 0 1 7 0 1 1 0 7 0 1 1 1 8 0 0 0 0 8 0 0 0 1 8 0 0 1 0 8 0 0 1 1 8 0 1 0 0 8 1 1 0 1 8 0 1 1 0 8 0 1 1 1 9 0 0 0 0 9 0 0 0 1 9 0 0 1 0 9 0 0 1 1 9 0 1 0 0 9 1 1 0 1 9 0 1 1 0 9 0 1 1 1 10 0 0 0 0 10 0 0 0 1 10 0 0 1 0 10 0 0 1 1 10 0 1 0 0 10 1 1 0 1 10 0 1 1 0 10 0 1 1 1 ;

where SUBJECT is the subject number, CHOOSE is the status variable, and T is the time variable. Because this dataset is arranged in a survival analysis form you can use the PROC PHREG. You can use the syntax:

proc phreg data=choco; strata subject;



model t*choose(0)=dark soft nuts; run;

As a result, you will have:

Sample Program: Conditional Logit Regression

The PHREG Procedure

Data Set: WORK.CHOCO Dependent Variable: T Censoring Variable: CHOOSE Censoring Value(s): 0 Ties Handling: BRESLOW

Summary of the Number of Event and Censored Values

Percent Stratum SUBJECT Total Event Censored Censored

1 1 8 1 7 87.50 2 2 8 1 7 87.50 3 3 8 1 7 87.50 4 4 8 1 7 87.50 5 5 8 1 7 87.50 6 6 8 1 7 87.50 7 7 8 1 7 87.50 8 8 8 1 7 87.50 9 9 8 1 7 87.50 10 10 8 1 7 87.50 Total 80 10 70 87.50

Testing Global Null Hypothesis: BETA=0

Without With Criterion Covariates Covariates Model Chi-Square

-2 LOG L 41.589 28.727 12.862 with 3 DF (p=0.0049) Score . . 11.600 with 3 DF (p=0.0089) Wald . . 8.928 with 3 DF (p=0.0303)


Parameter Standard Wald Pr > Risk Variable DF Estimate Error Chi-Square Chi-Square Ratio

DARK 1 1.386294 0.79057 3.07490 0.0795 4.000 SOFT 1 -2.197225 1.05409 4.34502 0.0371 0.111 NUTS 1 0.847298 0.69007 1.50762 0.2195 2.333

The result shows the estimation result as:



The positive parameter estimates of DARK and NUTS mean that dark and nuts each increases the preference. Thenegative parameter estimate of SOFT denotes soft center decreases the preference.

For each of eight types of candies, the predicted probabilities can be computed as follows:

This shows that the most preferred type of candy is the dark chocolate with a hard center and nuts.

Conditional Logit Regression with SPSS

COXREG Procedure

With SPSS, you can use the COXREG procedure to fit a conditional logit model. The basic syntax is:

coxreg time_varname with X1 X2 /status=status_varname(1) /strata=strata_varname.

where time_varname is the name of the failure time variable (the smaller value means the alternative was chosen),status_varname is the name of the censoring indicator variable, of which 1 is the value to indicate the event hasoccurred (not censored), strata_varname is the name of variable to specify the variable that determines thestratification, and X1 and X2 are explanatory variables.

Example 29: SPSS Conditional Logit Regression in COXREG procedure


coxreg t with dark soft nuts /status=choose(1) /strata=subject.

you will have the following SPSS output:

C O X R E G R E S S I O N



80 Total cases read 0 Cases with missing values 0 Valid cases with non-positive times 0 Censored cases before the earliest event in a stratum 0 Total cases dropped 80 Cases available for the analysis

Dependent Variable: T

SUBJECT Events Censored

1.00 1 7 (87.5%) 2.00 1 7 (87.5%) 3.00 1 7 (87.5%) 4.00 1 7 (87.5%) 5.00 1 7 (87.5%) 6.00 1 7 (87.5%) 7.00 1 7 (87.5%) 8.00 1 7 (87.5%) 9.00 1 7 (87.5%) 10.00 1 7 (87.5%)

Total 10 70 (87.5%)

Beginning Block Number 0. Initial Log Likelihood Function


Beginning Block Number 1. Method: Enter

Variable(s) Entered at Step Number 1.. DARK NUTS SOFT

Coefficients converged after 5 iterations.


Chi-Square df SigOverall (score) 11.600 3 .0089Change (-2LL) from Previous Block 12.862 3 .0049 Previous Step 12.862 3 .0049

-------------------- Variables in the Equation ---------------------

Variable B S.E. Wald df Sig R Exp(B)

DARK 1.3863 .7906 3.0749 1 .0795 .1608 4.0000NUTS .8473 .6901 1.5076 1 .2195 .0000 2.3333



SOFT -2.1972 1.0541 4.3450 1 .0371 -.2375 .1111

Covariate Means

Variable Mean

DARK .5000NUTS .5000SOFT .5000

The estimation result is exactly the same as what you obtained with SAS.

Permission to use this document is granted so long as the author is acknowledged and notified.Please send comments and suggestions to:[email protected]

Copyright 1995-1999, Indiana University.Last modified: Thursday, 24-Feb-2000 15:28:59 EST

URL /~statmath/stat/all/cat/printable.html



http://pytheas.ucs.indiana.edu/~statmath/contact/index.html

http://www.indiana.edu/

models for binary outcomes - indiana university …statmath/stat/all/cat/printable.pdfmodels for...

Documents