estimating regression models for categorical dependent ... · pdf fileestimating regression...

© 2003-2008, The Trustees of Indiana University Regression Models for Categorical Dependent Variables: 1

http://www.indiana.edu/~statmath

Estimating Regression Models for Categorical Dependent Variables Using SAS,

Stata, LIMDEP, and SPSS* Hun Myoung Park (kucc625)

This document summarizes regression models for categorical dependent variables and illustrates how to estimate individual models using SAS 9.1, Stata 10.0, LIMDEP 9.0, and SPSS 16.0.

1. Introduction 2. The Binary Logit Model 3. The Binary Probit Model 4. Bivariate Logit/Probit Models 5. Ordered Logit/Probit Models 6. The Multinomial Logit Model 7. The Conditional Logit Model 8. The Nested Logit Model 9. Conclusion

1. Introduction

A categorical variable here refers to a variable that is binary, ordinal, or nominal. Event count data are discrete (categorical) but often treated as continuous variables. When a dependent variable is categorical, the ordinary least squares (OLS) method can no longer produce the best linear unbiased estimator (BLUE); that is, OLS is biased and inefficient. Consequently, researchers have developed various regression models for categorical dependent variables. The nonlinearity of categorical dependent variable models (CDVMs) makes it difficult to fit the models and interpret their results. 1.1 Regression Models for Categorical Dependent Variables In CDVMs, the left-hand side (LHS) variable or dependent variable is neither interval nor ratio, but rather categorical. The level of measurement and data generation process (DGP) of a dependent variable determines the proper type of CDVM. Binary responses (0 or 1) are modeled with binary logit and probit regressions, ordinal responses (1st, 2nd, 3rd, …) are formulated into (generalized) ordered logit/probit regressions, and nominal responses are analyzed by multinomial logit, conditional logit, or nested logit models depending on specific circumstances. Independent variables on the right-hand side (RHS) may be interval, ratio, or binary (dummy).

* The citation of this document should read: “Park, Hun Myoung. 2008. Estimating Regression Models for Categorical Dependent Variables Using SAS, STATA, LIMDEP, and SPSS. Technical Working Paper. The University Information Technology Services (UITS) Center for Statistical and Mathematical Computing, Indiana University.”



The CDVMs adopt the maximum likelihood (ML) estimation method, whereas OLS uses the moment based method. The ML method requires an assumption about probability distribution functions, such as the logistic function and the complementary log-log function. Logit models use the standard logistic probability distribution, while probit models assume the standard normal distribution. This document focuses on logit and probit models only, excluding regression models for event count data (e.g., negative binomial regression model and zero-inflated Poisson regression model). Table 1 summarizes CDVMs in comparison with OLS. Table 1. Ordinary Least Squares and CDVMs Model Dependent (LHS) Estimation Independent (RHS) OLS Ordinary least

squares Interval or ratio Moment based method

Binary response Binary (0 or 1) Ordinal response Ordinal (1st, 2nd , 3rd…) Nominal response Nominal (A, B, C …)

CDVMs

Event count data Count (0, 1, 2, 3…)

Maximum likelihood method

A linear function of interval/ratio or binary variables

...22110 XX βββ ++

1.2 Logit Models versus Probit Models How do logit models differ from probit models? The core difference lies in the distribution of errors (disturbances). In the logit model, errors are assumed to follow the standard logistic

distribution with mean 0 and variance 3

2π , 2)1()( ε

ε

ελe

e+

= . The errors of the probit model are

assumed to follow the standard normal distribution, 2

2

21)(

ε

πεφ

−= e .

The probability density function (PDF) of the standard normal probability distribution has a higher peak and thinner tails than the standard logistic probability distribution (Figure 1). The standard logistic distribution looks as if someone has weighed down the peak of the standard normal distribution and strained its tails. As a result, the cumulative density function (CDF) of the standard normal distribution is steeper in the middle than the CDF of the standard logistic distribution and quickly approaches zero on the left and one on the right. The two models, of course, produce different parameter estimates. In binary response models, the estimates of a logit model are roughly 3π times larger than those of the corresponding probit model. These estimators, however, end up with almost the same standardized impacts of independent variables (Long 1997). The choice between logit and probit model is more closely related to estimation and familiarity rather than theoretical and interpretive aspects. In general, logit models reach convergence fairly well. Some (multinomial) probit models may take a long time to reach convergence, although the probit works well for bivariate models.



Figure 1. The Standard Normal and Standard Logistic Probability Distributions

PDF of the Standard Normal Distribution CDF of the Standard Normal Distribution

PDF of the Standard Logistic Distribution CDF of the Standard Logistic Distribution

1.3 Estimation in SAS, Stata, LIMDEP, and SPSS SAS provides several procedures for CDVMs, such as LOGISTIC, PROBIT, GENMOD, QLIM, MDC, PHREG, and CATMOD. Since these procedures support various models, a CDVM can be estimated by multiple procedures. For example, you may run a binary logit model using the LOGISTIC, PROBIT, GENMODE, and QLIM. The LOGISTIC and PROBIT procedures of SAS/STAT have been commonly used, but the QLIM and MDC procedures of SAS/ETS have advantages over other procedures. The QLIM (Qualitative and LImited dependent variable Model) procedure analyzes various categorical and limited dependent variable regression models such as censored, truncated, and sample-selection models. This QLIM procedure also handles Box-Cox regression and bivariate probit and logit models. The MDC (Multinomial Discrete Choice) procedure can estimate conditional logit, and nested logit models. Another advantage of using SAS is the Output Delivery System (ODS), which makes it easy to manage SAS output. ODS enables users to redirect the output to HTML (Hypertext Markup Language) and RTF (Rich Text Format) formats. Once SAS output is generated in a HTML document, users can easily handle tables and graphics especially when copying and pasting them into a wordprocessor document.



Unlike SAS, Stata has individualized commands for corresponding CDVMs. For example, the .logit and .probit commands respectively fit the binary logit and probit models, while .mlogit and .nlogit estimate mulitinomial logit and nested logit models. Stata enables users to perform post-hoc analyses such as marginal effects and discrete changes in an easy manner. Table 2. Procedures and Commands for CDVMs

Model SAS 9.1 Stata 10.0 LIMDEP 9.0 SPSS16.0 OLS (Ordinary least squares) REG .regress Regress$ Regression

Binary logit QLIM, GENMOD, LOGISTIC, PROBIT, CATMOD

.logit,

.logistic Logit$ Logistic regression Binary

Binary probit QLIM, GENMOD, LOGISTIC, PROBIT .probit Probit$ Probit

Bivariate logit QLIM - - - Bivariate Bivariate probit QLIM .biprobit Bivariateprobit$ -

Ordered logit QLIM, PROBIT, LOGISTIC .ologit Ordered$, Logit$ Plum

Generalized logit - .gologit2* - - Ordinal Ordered probit QLIM, PROBIT,

LOGISTIC .oprobit Ordered$ Plum

Multinomial logit CATMOD .mlogit Mlogit$, Logit$ Nomreg Conditional logit MDC, PHREG .clogit Clogit$, Logit$ Coxreg Nested logit MDC .nlogit Nlogit$** - Nominal Multinomial probit .mprobit -

* User-written commands written by Williams (2005) ** The Nlogit$ command is supported by NLOGIT, a stand-alone package, which is sold separately. The LIMDEP Logit$ and Probit$ commands support a variety of CDVMs that are addressed in Greene’s Econometric Analysis (2003). The output format of LIMDEP 9 is slightly different from that of previous version, but key statistics remain unchanged. Note that the nested logit model in LIMDEP is estimated by NLOGIT, a separate package. SPSS supports some CDVMs only and its output is often messy with less useful, if not misleading, information especially for beginners. Table 2 summarizes the procedures and commands for CDVMs. 1. 4 Long and Freese’s SPost Module Stata users may take advantages of user-written modules such as J. Scott Long and Jeremy Freese’s SPost. The module allows researchers to conduct follow-up analyses of various CDVMs including event count data models. See section 2.2 for major SPost commands. In order to install SPost, execute the following commands consecutively. For more details, visit J. Scott Long’s Web site at http://www.indiana.edu/~jslsoc/spost_install.htm. . net from http://www.indiana.edu/~jslsoc/stata/ . net install spost9_ado, replace . net get spost9_do, replace



If you want to use Vincent Kang Fu’s gologit (1998) and Richard Williams’ gologit2 (2005) for the generalized ordered logit model, install their modules. . net search gologit . net install gologit, from(http://www.stata.com/users/jhardin) . net install gologit2, from(http://fmwww.bc.edu/RePEc/bocode/g)



2. The Binary Logit Regression Model The binary logit model is represented as )(

)exp(1)exp()|1(Pr ββ

β xx

xxyob Λ=+

== , where Λ

indicates a link function, the cumulative standard logistic probability distribution function. This chapter examines how car ownership (owncar) is affected by monthly income (income), age, and gender (male). See the appendix for details about the data set.

2.1 Binary Logit in Stata (.logit) Stata provides two equivalent commands for the binary logit model, which present the same result in different ways. The .logit command produces coefficients with respect to logit (log of odds), while the .logistic reports estimates as odd ratios. . logistic owncar income age male Logistic regression Number of obs = 437 LR chi2(3) = 18.24 Prob > chi2 = 0.0004 Log likelihood = -273.84758 Pseudo R2 = 0.0322 ------------------------------------------------------------------------------ owncar | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- income | .9898826 .5677504 -0.02 0.986 .3216431 3.046443 age | 1.279626 .088997 3.55 0.000 1.116561 1.466505 male | 1.513669 .3111388 2.02 0.044 1.011729 2.264633 ------------------------------------------------------------------------------ . logit

In order to get the coefficients (log of odds), simply run the .logit without any argument right after the .logistic command. Or run an independent .logit command with all arguments. . logit owncar income age male Iteration 0: log likelihood = -282.96512 Iteration 1: log likelihood = -273.93537 Iteration 2: log likelihood = -273.84761 Iteration 3: log likelihood = -273.84758 Logistic regression Number of obs = 437 LR chi2(3) = 18.24 Prob > chi2 = 0.0004 Log likelihood = -273.84758 Pseudo R2 = 0.0322 ------------------------------------------------------------------------------ owncar | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- income | -.010169 .5735533 -0.02 0.986 -1.134313 1.113975 age | .2465678 .0695492 3.55 0.000 .1102539 .3828817 male | .4145366 .2055527 2.02 0.044 .0116606 .8174126 _cons | -4.682741 1.474519 -3.18 0.001 -7.572745 -1.792738 ------------------------------------------------------------------------------



Note that a coefficient of the .logit is the logarithmic transformed corresponding estimator of the .logistic. For example, .2465678= log(1.279626). Stata has post-estimation commands that conduct follow-up analyses. The .predict command computes predictions, residuals, or standard errors of the prediction and stores them into a new variable. . predict r, residual The .test and .lrtest commands respectively conduct the Wald test and likelihood ratio test. . test income age ( 1) income = 0 ( 2) age = 0 chi2( 2) = 12.57 Prob > chi2 = 0.0019

The .mfx command computes marginal effects at the reference points (by default, means of independent variables) after estimation of linear and nonlinear regression models. . mfx Marginal effects after logit y = Pr(owncar) (predict) = .65698515 ------------------------------------------------------------------------------ variable | dy/dx Std. Err. z P>|z| [ 95% C.I. ] X ---------+-------------------------------------------------------------------- income | -.0022916 .12925 -0.02 0.986 -.255624 .25104 .61684 age | .0555654 .01553 3.58 0.000 .025119 .086012 20.6911 male*| .0940341 .04673 2.01 0.044 .002451 .185617 .572082 ------------------------------------------------------------------------------ (*) dy/dx is for discrete change of dummy variable from 0 to 1

You may change reference points using the at() option. If a variable is not specified in the option, its mean is used for computation. The predicted probability of owning a car is .9375 for female students who have average income and age 30. . mfx, at(age=30 male=0) warning: no value assigned in at() for variables income; means used for income Marginal effects after logit y = Pr(owncar) (predict) = .9374982 ------------------------------------------------------------------------------ variable | dy/dx Std. Err. z P>|z| [ 95% C.I. ] X ---------+-------------------------------------------------------------------- income | -.0005959 .03361 -0.02 0.986 -.06647 .065278 .61684 age | .0144477 .00471 3.07 0.002 .005218 .023677 30 male*| .0203154 .01606 1.27 0.206 -.011155 .051785 0 ------------------------------------------------------------------------------ (*) dy/dx is for discrete change of dummy variable from 0 to 1



2.2 Using the SPost Module in Stata

The SPost module provides useful follow-up analysis commands (ado files) for various categorical dependent variable models (Long and Freese 2003). The .fitstat command calculates various goodness-of-fit statistics such as log likelihood, McFadden’s R2 (or Pseudo R2), Akaike Information Criterion (AIC), and (Bayesian Information Criterion (BIC). . fitstat Measures of Fit for logit of owncar Log-Lik Intercept Only: -282.965 Log-Lik Full Model: -273.848 D(433): 547.695 LR(3): 18.235 Prob > LR: 0.000 McFadden's R2: 0.032 McFadden's Adj R2: 0.018 ML (Cox-Snell) R2: 0.041 Cragg-Uhler(Nagelkerke) R2: 0.056 McKelvey & Zavoina's R2: 0.059 Efron's R2: 0.040 Variance of y*: 3.495 Variance of error: 3.290 Count R2: 0.638 Adj Count R2: -0.033 AIC: 1.272 AIC*n: 555.695 BIC: -2084.916 BIC': 0.005 BIC used by Stata: 572.015 AIC used by Stata: 555.695

The likelihood ratio for goodness of fit is computed as, . di 2*(-273.848 - (-282.965)) 18.234 The .listcoef command lists unstandardized coefficients (parameter estimates), factor and percent changes, and standardized coefficients to help interpret results. The help option tells how to read the outputs. . listcoef, help logistic (N=437): Factor Change in Odds Odds of: 1 vs 0 ---------------------------------------------------------------------- owncar | b z P>|z| e^b e^bStdX SDofX -------------+-------------------------------------------------------- income | -0.01017 -0.018 0.986 0.9899 0.9982 0.1792 age | 0.24657 3.545 0.000 1.2796 1.4876 1.6108 male | 0.41454 2.017 0.044 1.5137 1.2279 0.4953 ---------------------------------------------------------------------- b = raw coefficient z = z-score for test of b=0 P>|z| = p-value for z-test e^b = exp(b) = factor change in odds for unit increase in X e^bStdX = exp(b*SD of X) = change in odds for SD increase in X SDofX = standard deviation of X

The .prtab command constructs a table of predicted values (events) for all combinations of categorical variables listed. The following example shows that 60 percent of female and 70 percent of male students are likely to own cars, given the mean values of income and age.



. prtab male logistic: Predicted probabilities of positive outcome for owncar ---------------------- male | Prediction ----------+----------- 0 | 0.6017 1 | 0.6958 ---------------------- income age male x= .61683982 20.691076 .57208238

The .prvalue lists predicted probabilities of positive and negative outcomes for a given set of values for the independent variables. Note both the .prtab and .prvalue commands report the identical predicted probability that male students own cars, .6017, holding other variables at their means. . prvalue, x(male=0) rest(mean) logit: Predictions for owncar Confidence intervals by delta method 95% Conf. Interval Pr(y=1|x): 0.6017 [ 0.5302, 0.6732] Pr(y=0|x): 0.3983 [ 0.3268, 0.4698] income age male x= .61683982 20.691076 0

The most useful command is the .prchange, which calculates marginal effects (changes) and discrete changes at the given set of values of independent variables. The help option tells how to read the outputs. For instance, the predicted probability that a male students owns a car is .094 (0->1) higher than that of female students, holding other variables at their mean. . prchange, help logit: Changes in Probabilities for owncar min->max 0->1 -+1/2 -+sd/2 MargEfct income -0.0019 -0.0023 -0.0023 -0.0004 -0.0023 age 0.4404 0.0032 0.0555 0.0893 0.0556 male 0.0940 0.0940 0.0932 0.0462 0.0934 0 1 Pr(y|x) 0.3430 0.6570 income age male x= .61684 20.6911 .572082 sd(x)= .17918 1.61081 .495344 Pr(y|x): probability of observing each y for specified x values Avg|Chg|: average of absolute value of the change across categories Min->Max: change in predicted probability as x changes from its minimum to its maximum



0->1: change in predicted probability as x changes from 0 to 1 -+1/2: change in predicted probability as x changes from 1/2 unit below base value to 1/2 unit above -+sd/2: change in predicted probability as x changes from 1/2 standard dev below base to 1/2 standard dev above MargEfct: the partial derivative of the predicted probability/rate with respect to a given independent variable

The predicted probability of .6570 and the marginal effects are the same as what .mfx produces above. Like .mfx, .prchange can specify reference points other than means of independent variables. Again, the output is the same as that of second .mfx above. . prchange, x(age=30 male=0) logit: Changes in Probabilities for owncar min->max 0->1 -+1/2 -+sd/2 MargEfct income -0.0005 -0.0006 -0.0006 -0.0001 -0.0006 age 0.4838 0.0025 0.0145 0.0234 0.0144 male 0.0203 0.0203 0.0244 0.0120 0.0243 0 1 Pr(y|x) 0.0625 0.9375 income age male x= .61684 30 0 sd(x)= .17918 1.61081 .495344

The SPost module also includes the .prgen, which computes a series of predictions by holding all variables but one interval variable constant and allowing that variable to vary (Long and Freese 2003). . prgen income, from(.1) to(1.5) x(male=1) rest(median) generate(ppcar) logit: Predicted values as income varies from .1 to 1.5. income age male x= .58200002 21 1

The above command computes predicted probabilities that male students own cars when income changes from $100 through $1,500, holding age at its median of 21 and stores them into a new variable ppcar. 2.3 Using the SAS LOGISTIC and PROBIT Procedures SAS has several procedures for the binary logit model such as the LOGISTIC, PROBIT, GENMOD, and QLIM. The LOGISTIC procedure is commonly used for the binary logit model, but the PROBIT procedure also estimates the binary logit. Let us consider LOGISTIC first. PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar = income age male; RUN;



The LOGISTIC Procedure Model Information Data Set MASIL.STUDENTS Response Variable owncar Number of Response Levels 2 Model binary logit Optimization Technique Fisher's scoring Number of Observations Read 437 Number of Observations Used 437 Response Profile Ordered Total Value owncar Frequency 1 1 284 2 0 153 Probability modeled is owncar=1. Model Convergence Status Convergence criterion (GCONV=1E-8) satisfied. Model Fit Statistics Intercept Intercept and Criterion Only Covariates AIC 567.930 555.695 SC 572.010 572.015 -2 Log L 565.930 547.695 Testing Global Null Hypothesis: BETA=0 Test Chi-Square DF Pr > ChiSq Likelihood Ratio 18.2351 3 0.0004 Score 17.4697 3 0.0006 Wald 16.7977 3 0.0008 Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 1 -4.6827 1.4745 10.0855 0.0015



income 1 -0.0102 0.5736 0.0003 0.9859 age 1 0.2466 0.0695 12.5686 0.0004 male 1 0.4145 0.2056 4.0670 0.0437 Odds Ratio Estimates Point 95% Wald Effect Estimate Confidence Limits income 0.990 0.322 3.046 age 1.280 1.117 1.467 male 1.514 1.012 2.265 Association of Predicted Probabilities and Observed Responses Percent Concordant 58.9 Somers' D 0.246 Percent Discordant 34.3 Gamma 0.264 Percent Tied 6.8 Tau-a 0.112 Pairs 43452 c 0.623

The SAS LOGISTIC, PROBIT, and GENMOD procedures by default uses a smaller value in the dependent variable as success. Thus, the magnitudes of the coefficients remain the same, but the signs are opposite to those of the QLIM procedure, Stata, and LIMDEP. The DESCENDING option forces SAS to use a larger value as success. Alternatively, you may explicitly specify the category of successful event using the EVENT option as follows. PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar(EVENT=’1’) = income age male; RUN;

The SAS LOGISTIC procedure computes odds changes when independent variables increase by the units specified in the UNITS statement. The SD below indicates a standard deviation increase in income and age (e.g., -2 means a two unit decrease in independent variables). PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar = income age male; UNITS income=SD age=SD; RUN;

The UNITS statement adds the “Adjusted Odds Ratios” to the end of the outputs above. The following table is selected from SAS output. Note that the odds changes of the two variables are identical to those under the “e^bStdX” of the previous SPost .listcoef output. Adjusted Odds Ratios Effect Unit Estimate income 0.1792 0.998 age 1.6108 1.488

If you want to get the output in the HTML format, use ODS statements before and after a SAS procedure. “ODS HTML;” redirects SAS output to a HTML format.



ODS HTML; PROC LOGISTIC DESCENDING DATA = masil.students; MODEL owncar = income age male; UNITS income=SD age=SD; RUN; ODS HTML CLOSE;

Adjusted Odds Ratios

Effect Unit Estimate

income 0.1792 0.998

age 1.6108 1.488

The PROBIT procedure, which is designed for binary probit model, can estimate the same binary logit model. The PROBIT requires the CLASS statement to list categorical variables. The /DIST=LOGISTIC option indicates the probability distribution to be used in maximum likelihood estimation. PROC PROBIT DATA = masil.students; CLASS owncar; MODEL owncar = income age male /DIST=LOGISTIC; RUN; Probit Procedure Model Information Data Set MASIL.STUDENTS Dependent Variable owncar Number of Observations 437 Name of Distribution Logistic Log Likelihood -273.847577 Number of Observations Read 437 Number of Observations Used 437 Class Level Information Name Levels Values owncar 2 0 1 Response Profile Ordered Total Value owncar Frequency 1 0 153



2 1 284 PROC PROBIT is modeling the probabilities of levels of owncar having LOWER Ordered Values in the response profile table. Algorithm converged. Type III Analysis of Effects Wald Effect DF Chi-Square Pr > ChiSq income 1 0.0003 0.9859 age 1 12.5686 0.0004 male 1 4.0670 0.0437 Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 4.6827 1.4745 1.7927 7.5727 10.09 0.0015 income 1 0.0102 0.5736 -1.1140 1.1343 0.00 0.9859 age 1 -0.2466 0.0695 -0.3829 -0.1103 12.57 0.0004 male 1 -0.4145 0.2056 -0.8174 -0.0117 4.07 0.0437 Unlike LOGISTIC, PROBIT does not have the DESCENDING option. Thus, you have to switch the signs of coefficients when comparing with those of Stata and LIMDEP. The PROBIT procedure also does not have the UNITS statement to compute changes in odds. 2.4 Using the SAS GENMOD and QLIM Procedures The GENMOD provides flexible methods to estimate generalized linear model. The DISTRIBUTION (DIST) and the LINK=LOGIT options respectively specify a probability distribution and a link function. PROC GENMOD DATA = masil.students DESC; MODEL owncar = income age male /DIST=BINOMIAL LINK=LOGIT; RUN; The GENMOD Procedure Model Information Data Set MASIL.STUDENTS Distribution Binomial Link Function Logit Dependent Variable owncar Number of Observations Read 437 Number of Observations Used 437



Number of Events 284 Number of Trials 437 Response Profile Ordered Total Value owncar Frequency 1 1 284 2 0 153 PROC GENMOD is modeling the probability that owncar='1'. Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 433 547.6952 1.2649 Scaled Deviance 433 547.6952 1.2649 Pearson Chi-Square 433 436.4352 1.0079 Scaled Pearson X2 433 436.4352 1.0079 Log Likelihood -273.8476 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 -4.6827 1.4745 -7.5727 -1.7927 10.09 0.0015 income 1 -0.0102 0.5736 -1.1343 1.1140 0.00 0.9859 age 1 0.2466 0.0695 0.1103 0.3829 12.57 0.0004 male 1 0.4145 0.2056 0.0117 0.8174 4.07 0.0437 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed.

If you have categorical (string) independent variables, list the variables in the CLASS statement without creating dummy variables. PROC GENMOD DATA = masil.students DESC; CLASS male; MODEL owncar = income age male /DIST=BINOMIAL LINK=LOGIT; RUN; Users may also provide their own link functions using the FWDLINK and INVLINK statements instead of the LINK=LOGIT option. PROC GENMOD DATA = masil.students DESC; FWDLINK link=LOG(_MEAN_/(1-_MEAN_)); INVLINK invlink=1/(1+EXP(-1*_XBETA_)); MODEL owncar = income age male /DIST=BINOMIAL; RUN;



All three GENMOD examples discussed so far produce the identical result. The QLIM procedure estimates not only logit and probit models, but also censored, truncated, and sample-selected models. You may provide characteristics of the dependent variable either in the ENDOGENOUS statement or the option of the MODEL statement. PROC QLIM DATA=masil.students; MODEL owncar = income age male; ENDOGENOUS owncar ~ DISCRETE(DIST=LOGIT); RUN;

Or, PROC QLIM DATA=masil.students; MODEL owncar = income age male /DISCRETE(DIST=LOGIT); RUN; The QLIM Procedure Discrete Response Profile of owncar Index Value Frequency Percent 1 0 153 35.01 2 1 284 64.99 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable owncar Number of Observations 437 Log Likelihood -273.84758 Maximum Absolute Gradient 9.63219E-6 Number of Iterations 8 AIC 555.69515 Schwarz Criterion 572.01489 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 18.235 2 * (LogL - LogL0) Upper Bound of R (U) 565.93 - 2 * LogL0 Aldrich-Nelson 0.0401 R / (R+N) Cragg-Uhler 1 0.0409 1 - exp(-R/N) Cragg-Uhler 2 0.0563 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0415 1 - (1-R/U)^(U/N) Adjusted Estrella 0.0234 1 - ((LogL-K)/LogL0)^(-2/N*LogL0) McFadden's LRI 0.0322 R / U Veall-Zimmermann 0.071 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.1699 N = # of observations, K = # of regressors



Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > |t| Intercept -4.682741 1.474519 -3.18 0.0015 income -0.010169 0.573553 -0.02 0.9859 age 0.246568 0.069549 3.55 0.0004 male 0.414537 0.205553 2.02 0.0437

Finally, the CATMOD procedure fits the logit model to the functions of categorical response variables. This procedure, however, produces slightly different estimators compared to those of other procedures discussed so far. This procedure is, therefore, less recommended for the binary logit model. The DIRECT statement specifies interval or ratio variables used in the MODEL. The /NOPROFILE suppresses the display of the population profiles and the response profiles. PROC CATMOD DATA = masil.students; DIRECT income age; MODEL owncar = income age male /NOPROFILE; RUN;

2.5 Binary Logit in LIMDEP (Logit$)

The Logit$ command in LIMDEP estimates various logit models. The dependent variable is specified in the Lhs$ (left-hand side) subcommand and a list of independent variables in the Rhs$ (right-hand side). You have to explicitly specify the ONE for the intercept. The Marginal Effects$ and the Means$ subcommands compute marginal effects at the mean values of independent variables. LOGIT; Lhs=owncar; Rhs=ONE,income,age,male; Marginal Effects; Means$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | Binary Logit Model for Binary Choice | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 04:58:39PM.| | Dependent variable OWNCAR | | Weighting variable None | | Number of observations 437 | | Iterations completed 5 | | Log likelihood function -273.8476 | | Number of parameters 4 | | Info. Criterion: AIC = 1.27161 | | Finite Sample: AIC = 1.27183 | | Info. Criterion: BIC = 1.30896 | | Info. Criterion:HQIC = 1.28635 | | Restricted log likelihood -282.9651 | | McFadden Pseudo R-squared .0322214 | | Chi squared 18.23509 |



+----------------------------------------+ | Fit Measures for Binomial Choice Model | | Logit model for variable OWNCAR | +----------------------------------------+ | Proportions P0= .350114 P1= .649886 | | N = 437 N0= 153 N1= 284 | | LogL= -273.848 LogL0= -282.965 | | Estrella = 1-(L/L0)^(-2L0/n) = .04153 | +----------------------------------------+ | Efron | McFadden | Ben./Lerman | | .03963 | .03222 | .56318 | | Cramer | Veall/Zim. | Rsqrd_ML | | .04010 | .07099 | .04087 | +----------------------------------------+ | Information Akaike I.C. Schwarz I.C. | | Criteria 1.27161 1.30896 | +----------------------------------------+ +---------------------------------------------------------+ |Predictions for Binary Choice Model. Predicted value is | |1 when probability is greater than .500000, 0 otherwise.| |Note, column or row total percentages may not sum to | |100% because of rounding. Percentages are of full sample.| +------+---------------------------------+----------------+ |Actual| Predicted Value | | |Value | 0 1 | Total Actual | +------+----------------+----------------+----------------+ | 0 | 21 ( 4.8%)| 132 ( 30.2%)| 153 ( 35.0%)| | 1 | 26 ( 5.9%)| 258 ( 59.0%)| 284 ( 65.0%)| +------+----------------+----------------+----------------+ |Total | 47 ( 10.8%)| 390 ( 89.2%)| 437 (100.0%)| +------+----------------+----------------+----------------+ ======================================================================= Analysis of Binary Choice Model Predictions Based on Threshold = .5000 ----------------------------------------------------------------------- Prediction Success ----------------------------------------------------------------------- Sensitivity = actual 1s correctly predicted 90.845% Specificity = actual 0s correctly predicted 13.725% Positive predictive value = predicted 1s that were actual 1s 66.154% Negative predictive value = predicted 0s that were actual 0s 44.681% Correct prediction = actual 1s and 0s correctly predicted 63.844% ----------------------------------------------------------------------- Prediction Failure ----------------------------------------------------------------------- False pos. for true neg. = actual 0s predicted as 1s 86.275% False neg. for true pos. = actual 1s predicted as 0s 9.155% False pos. for predicted pos. = predicted 1s actual 0s 33.846% False neg. for predicted neg. = predicted 0s actual 1s 55.319% False predictions = actual 1s and 0s incorrectly predicted 36.156% =======================================================================

Note that the marginal effects above are similar to those of the SPost .prchange command in section 2.2. LIMDEP computes discrete changes for binary variables like male.

2.6 Binary Logit in SPSS The SPSS Logistic regression command fits the binary logit model. SPSS generates messy tables, which are often overwhelming beginners. The table below is selected from the output.



LOGISTIC REGRESSION VAR=owncar /METHOD=ENTER income age male /CRITERIA PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .

Variables in the Equation

B S.E. Wald df Sig. Exp(B)

income -.010 .574 .000 1 .986 .990

age .247 .070 12.569 1 .000 1.280

male .415 .206 4.067 1 .044 1.514

Step 1

Constant -4.683 1.475 10.086 1 .001 .009

Note that the “Sig.” in the second column head above is not the significance level but p-value. Table 3 summarizes parameter estimates and goodness-of-fit statistics across procedures and commands for the binary logit model. Estimates and their standard errors produced are almost identical except some rounding errors. As shown in Table 3, the QLIM and LOGISTIC are recommended for categorical dependent variables. Note that the PROBIT procedure returns the opposite signs of estimates. SPSS does not produce Pseudo R2, AIC, Schwarz, and BIC. Table 3. Parameter Estimates and Goodness-of-fit Statistics of the Binary Logit Model

SAS Stata LIMDEP LOGISTIC PROBIT GENMOD QLIM

Intercept -4.6827 (1.4745)

4.6827 (1.4745)

-4.6827 (1.4745)

-4.6827 (1.4745)

-4.6827 (1.4745)

-4.6827 (1.4745)

income -.0102 (.5736)

.0102 (.5736)

-.0102 (.5736)

-.0102 (.5736)

-.0102 (.5736)

-.0102 (.5736)

age .2466 (.0695)

-.2466 (.0695)

.2466 (.0695)

.2466 (.0695)

.2466 (.0695)

.2466 (.0695)

male .4145 (.2056)

-.4145 (.2056)

.4145 (.2056)

.4145 (.2056)

.4145 (.2056)

.4145 (.2056)

Log likelihood 547.695* -273.8476 -273.8476 -273.8476 -273.8476 -273.8476

Likelihood test 18.2351 18.235 18.24 18.2351

Pseudo R2 .0322 .0322 .0322 AIC 555.695** 555.6952** 1.2716

Schwarz 572.015 572.0149 572.0150

BIC 565.9350

* The LOGISTIC procedure reports (-2*log likelihood). ** AIC*N



3. The Binary Probit Regression Model The probit model is represented as )()|1(Pr βxxyob Φ== , where Φ indicates the cumulative standard normal probability distribution function. 3.1 Binary Probit in Stata (.probit) Stata has the .probit command to estimate the binary probit regression model. . probit owncar income age male Iteration 0: log likelihood = -282.96512 Iteration 1: log likelihood = -273.84832 Iteration 2: log likelihood = -273.81741 Iteration 3: log likelihood = -273.81741 Probit regression Number of obs = 437 LR chi2(3) = 18.30 Prob > chi2 = 0.0004 Log likelihood = -273.81741 Pseudo R2 = 0.0323 ------------------------------------------------------------------------------ owncar | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- income | .0005613 .3476842 0.00 0.999 -.6808873 .6820098 age | .1487005 .0409837 3.63 0.000 .068374 .2290271 male | .2579112 .1256085 2.05 0.040 .0117231 .5040993 _cons | -2.823671 .8730955 -3.23 0.001 -4.534907 -1.112435 ------------------------------------------------------------------------------

In order to get standardized estimates and factor changes, run the SPost .listcoef command. . listcoef probit (N=437): Unstandardized and Standardized Estimates Observed SD: .47755228 Latent SD: 1.0371456 ------------------------------------------------------------------------------- owncar | b z P>|z| bStdX bStdY bStdXY SDofX -------------+----------------------------------------------------------------- income | 0.00056 0.002 0.999 0.0001 0.0005 0.0001 0.1792 age | 0.14870 3.628 0.000 0.2395 0.1434 0.2309 1.6108 male | 0.25791 2.053 0.040 0.1278 0.2487 0.1232 0.4953 -------------------------------------------------------------------------------

You may compute the marginal effects and discrete change using either .mfx or the SPost .prchange. . mfx, at(income=1 age=21 male=0) Marginal effects after probit y = Pr(owncar) (predict) = .6177594



------------------------------------------------------------------------------ variable | dy/dx Std. Err. z P>|z| [ 95% C.I. ] X ---------+-------------------------------------------------------------------- income | .0002141 .13261 0.00 0.999 -.259696 .260125 1 age | .0567193 .01557 3.64 0.000 .026211 .087228 21 male*| .093652 .04604 2.03 0.042 .003415 .183889 0 ------------------------------------------------------------------------------ (*) dy/dx is for discrete change of dummy variable from 0 to 1 . prchange, x(income=1 age=21 male=0) probit: Changes in Predicted Probabilities for owncar min->max 0->1 -+1/2 -+sd/2 MargEfct income 0.0002 0.0002 0.0002 0.0000 0.0002 age 0.4900 0.0014 0.0567 0.0912 0.0567 male 0.0937 0.0937 0.0981 0.0487 0.0984 0 1 Pr(y|x) 0.3822 0.6178 income age male x= 1 21 0 sd(x)= .17918 1.61081 .495344

3.2 Using the PROBIT and LOGISTIC Procedures The PROBIT and LOGISTIC procedures estimate the binary probit model. Keep in mind that the coefficients of PROBIT has opposite signs. PROC PROBIT DATA = masil.students; CLASS owncar; MODEL owncar = income age male; RUN; Probit Procedure Model Information Data Set MASIL.STUDENTS Dependent Variable owncar Number of Observations 437 Name of Distribution Normal Log Likelihood -273.8174115 Number of Observations Read 437 Number of Observations Used 437 Class Level Information Name Levels Values owncar 2 0 1



Response Profile Ordered Total Value owncar Frequency 1 0 153 2 1 284 PROC PROBIT is modeling the probabilities of levels of owncar having LOWER Ordered Values in the response profile table. Algorithm converged. Type III Analysis of Effects Wald Effect DF Chi-Square Pr > ChiSq income 1 0.0000 0.9987 age 1 13.1644 0.0003 male 1 4.2160 0.0400 Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 2.8237 0.8731 1.1124 4.5349 10.46 0.0012 income 1 -0.0006 0.3477 -0.6820 0.6809 0.00 0.9987 age 1 -0.1487 0.0410 -0.2290 -0.0684 13.16 0.0003 male 1 -0.2579 0.1256 -0.5041 -0.0117 4.22 0.0400 The LOGISTIC procedure requires a normal probability distribution as a link function (/LINK=PROBIT or /LINK=NORMIT). Note that the ODS statements store the SAS output into a HTML file, c:\temp\logit.html. ODS HTML FILE='c:\temp\probit.html'; PROC LOGISTIC DATA = masil.students DESC; MODEL owncar = income age male /LINK=PROBIT; RUN; ODS HTML CLOSE;

The LOGISTIC Procedure

Model Information

Data Set SM.STUDENTS

Response Variable owncar

Number of Response Levels 2



Model Information

Model binary probit

Optimization Technique Fisher's scoring

Number of Observations Read 437

Number of Observations Used 437

Response Profile

Ordered Value

owncar Total Frequency

1 1 284

2 0 153

Probability modeled is owncar=1.

Model Convergence Status

Convergence criterion (GCONV=1E-8) satisfied.

Model Fit Statistics

Criterion Intercept Only

Intercept and

Covariates

AIC 567.930 555.635

SC 572.010 571.955

-2 Log L 565.930 547.635

Testing Global Null Hypothesis: BETA=0

Test Chi-Square DF Pr > ChiSq

Likelihood Ratio 18.2954 3 0.0004

Score 17.4697 3 0.0006

Wald 17.4690 3 0.0006

Analysis of Maximum Likelihood Estimates

Parameter DF Estimate Standard Error

Wald Chi-Square

Pr > ChiSq




Parameter DF Estimate Standard Error

Wald Chi-Square

Pr > ChiSq

Intercept 1 -2.8237 0.8796 10.3048 0.0013

income 1 0.000548 0.3496 0.0000 0.9987

age 1 0.1487 0.0413 12.9602 0.0003

male 1 0.2579 0.1257 4.2096 0.0402

Association of Predicted Probabilities and Observed Responses

Percent Concordant 57.8 Somers' D 0.249

Percent Discordant 32.9 Gamma 0.274

Percent Tied 9.3 Tau-a 0.113

Pairs 43452 c 0.624

3.3 Using the GENMODE and QLIM Procedures The GENMOD procedure also estimates the binary probit model using the /DIST=BINOMIAL and /LINK=PROBIT options in the MODEL statement. PROC GENMOD DATA = masil.students DESC; MODEL owncar = income age male /DIST=BINOMIAL LINK=PROBIT; RUN; The GENMOD Procedure Model Information Data Set MASIL.STUDENTS Distribution Binomial Link Function Probit Dependent Variable owncar Number of Observations Read 437 Number of Observations Used 437 Number of Events 284 Number of Trials 437 Response Profile Ordered Total Value owncar Frequency



1 1 284 2 0 153 PROC GENMOD is modeling the probability that owncar='1'. Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance 433 547.6348 1.2647 Scaled Deviance 433 547.6348 1.2647 Pearson Chi-Square 433 437.0270 1.0093 Scaled Pearson X2 433 437.0270 1.0093 Log Likelihood -273.8174 Algorithm converged. Analysis Of Parameter Estimates Standard Wald 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 -2.8237 0.8731 -4.5349 -1.1124 10.46 0.0012 income 1 0.0006 0.3477 -0.6809 0.6820 0.00 0.9987 age 1 0.1487 0.0410 0.0684 0.2290 13.16 0.0003 male 1 0.2579 0.1256 0.0117 0.5041 4.22 0.0400 Scale 0 1.0000 0.0000 1.0000 1.0000 NOTE: The scale parameter was held fixed.

The QLIM procedure provides various goodness-of-fit statistics. The DIST=NORMAL option indicates the normal probability distribution used in estimation. Again, the output is generated in the HTML format. ODS HTML; PROC QLIM DATA=masil.students; MODEL owncar = income age male /DISCRETE (DIST=NORMAL); RUN; ODS HTML CLOSE;

The QLIM Procedure Discrete Response Profile of owncar

Index Value Frequency Percent

1 0 153 35.01

2 1 284 64.99

Model Fit Summary



Model Fit Summary

Number of Endogenous Variables 1

Endogenous Variable owncar

Number of Observations 437

Log Likelihood -273.81741

Maximum Absolute Gradient 3.82848E-8

Number of Iterations 10

AIC 555.63482

Schwarz Criterion 571.95456

Goodness-of-Fit Measures

Measure Value Formula

Likelihood Ratio (R) 18.295 2 * (LogL - LogL0)

Upper Bound of R (U) 565.93 - 2 * LogL0

Aldrich-Nelson 0.0402 R / (R+N)

Cragg-Uhler 1 0.041 1 - exp(-R/N)

Cragg-Uhler 2 0.0565 (1-exp(-R/N)) / (1-exp(-U/N))

Estrella 0.0417 1 - (1-R/U)^(U/N)

Adjusted Estrella 0.0235 1 - ((LogL-K)/LogL0)^(-2/N*LogL0)

McFadden's LRI 0.0323 R / U

Veall-Zimmermann 0.0712 (R * (U+N)) / (U * (R+N))

McKelvey-Zavoina 0.0702

N = # of observations, K = # of regressors

Algorithm converged.

Parameter Estimates

Parameter Estimate Standard Error t Value Approx Pr > |t|

Intercept -2.823671 0.873096 -3.23 0.0012

income 0.000561 0.347684 0.00 0.9987



| Estrella = 1-(L/L0)^(-2L0/n) = .04166 | +----------------------------------------+ | Efron | McFadden | Ben./Lerman | | .03984 | .03233 | .56327 | | Cramer | Veall/Zim. | Rsqrd_ML | | .04016 | .07121 | .04100 | +----------------------------------------+ | Information Akaike I.C. Schwarz I.C. | | Criteria 1.27148 1.30882 | +----------------------------------------+ +---------------------------------------------------------+ |Predictions for Binary Choice Model. Predicted value is | |1 when probability is greater than .500000, 0 otherwise.| |Note, column or row total percentages may not sum to | |100% because of rounding. Percentages are of full sample.| +------+---------------------------------+----------------+ |Actual| Predicted Value | | |Value | 0 1 | Total Actual | +------+----------------+----------------+----------------+ | 0 | 5 ( 1.1%)| 148 ( 33.9%)| 153 ( 35.0%)| | 1 | 8 ( 1.8%)| 276 ( 63.2%)| 284 ( 65.0%)| +------+----------------+----------------+----------------+ |Total | 13 ( 3.0%)| 424 ( 97.0%)| 437 (100.0%)| +------+----------------+----------------+----------------+ ======================================================================= Analysis of Binary Choice Model Predictions Based on Threshold = .5000 ----------------------------------------------------------------------- Prediction Success ----------------------------------------------------------------------- Sensitivity = actual 1s correctly predicted 97.183% Specificity = actual 0s correctly predicted 3.268% Positive predictive value = predicted 1s that were actual 1s 65.094% Negative predictive value = predicted 0s that were actual 0s 38.462% Correct prediction = actual 1s and 0s correctly predicted 64.302% ----------------------------------------------------------------------- Prediction Failure ----------------------------------------------------------------------- False pos. for true neg. = actual 0s predicted as 1s 96.732% False neg. for true pos. = actual 1s predicted as 0s 2.817% False pos. for predicted pos. = predicted 1s actual 0s 34.906% False neg. for predicted neg. = predicted 0s actual 1s 61.538% False predictions = actual 1s and 0s incorrectly predicted 35.698% =======================================================================

3.5 Binary Probit in SPSS SPSS has the Probit command to fit the binary probit model. This command requires an additional variable (e.g., n in the following example) with constant 1. COMPUTE n=1. PROBIT owncar OF n WITH income age male /LOG NONE /MODEL PROBIT /PRINT FREQ /CRITERIA ITERATE(20) STEPLIMIT(.1).

The following table is selected from messy SPSS output. Parameter estimates and their standard errors are identical when considering rounding error.



Parameter Estimates

95% Confidence Interval

Parameter Estimate Std. Error Z Sig. Lower Bound Upper Bound

income .001 .348 .002 .999 -.681 .682

age .149 .041 3.628 .000 .068 .229

male .258 .126 2.053 .040 .012 .504

PROBITa

Intercept -2.824 .873 -3.234 .001 -3.697 -1.951a. PROBIT model: PROBIT(p) = Intercept + BX

Table 4 summarizes parameter estimates and goodness-of-fit statistics produced. Note that the LOGISTIC procedure reports slightly different estimates and standard errors. I would recommend the SAS QLIM procedure, Stata, and LIMDEP for the binary probit model. Table 4.Parameter Estimates and Goodness-of-fit Statistics of the Binary Probit Model

SAS Stata LIMDEP LOGISTIC PROBIT GENMOD QLIM

Intercept -2.8237 (.8796)

2.8237 (.8731)

-2.8237 (.8731)

-2.8237 (.8731)

-2.8237 (.8731)

-2.8237 (.8731)

income .0005 (.3496)

-.0006 (.3477)

.0006 (.3477)

.0006 (.3477)

.0006 (.3477)

.0006 (.3477)

age .1487 (.0413)

-.1487 (.0410)

.1487 (.0410)

.1487 (.0410)

.1487 (.0410)

.1487 (.0410)

male .2579 (.1257)

-.2579 (.1256)

.2579 (.1256)

.2579 (.1256)

.2579 (.1256)

.2579 (.1256)

Log likelihood 547.653* -273.8174 -273.8174 -273.8174 -273.8174 -273.8174

Likelihood test 18.2954 18.295 18.30 18.2954 Pseudo R2 .0323 .0323 .0323

AIC 555.635** 555.6348** 1.2715

Schwarz 571.955 571.9546 571.9546 BIC

* The LOGISTIC procedure reports (-2*log likelihood). ** AIC*N



4. Bivariate Probit/Logit Regression Models Bivariate regression models have two equations for the two dependent variables. This chapter explains the bivariate regression model with two binary dependent variables. Like the seemingly unrelated regression model (SUR), bivariate probit/logit models assume that the “independent, identically distributed” errors are correlated (Greene 2003). The bivariate probit model, although consuming relatively much time, is more likely to converge than the bivariate logit model. SAS supports both the bivariate probit and logit models, while Stata and LIMDEP estimate the bivariate probit model. Here we consider a model for car ownership (owncar) and housing type (offcamp). 4.1 Bivariate Probit in Stata (.biprobit) Stata has the .biprobit command to estimate the bivariate probit model. The two dependent variables precede a set of independent variables. . biprobit owncar offcamp income age male Fitting comparison equation 1: Iteration 0: log likelihood = -282.96512 Iteration 1: log likelihood = -273.84832 Iteration 2: log likelihood = -273.81741 Iteration 3: log likelihood = -273.81741 Fitting comparison equation 2: Iteration 0: log likelihood = -54.97403 Iteration 1: log likelihood = -45.919608 Iteration 2: log likelihood = -43.685448 Iteration 3: log likelihood = -43.32265 Iteration 4: log likelihood = -43.309675 Iteration 5: log likelihood = -43.309654 Comparison: log likelihood = -317.12707 Fitting full model: Iteration 0: log likelihood = -317.12707 Iteration 1: log likelihood = -307.15684 Iteration 2: log likelihood = -306.49535 Iteration 3: log likelihood = -306.46018 Iteration 4: log likelihood = -306.45493 Iteration 5: log likelihood = -306.45408 Iteration 6: log likelihood = -306.45395 Iteration 7: log likelihood = -306.45392 Bivariate probit regression Number of obs = 437 Wald chi2(6) = 30.13 Log likelihood = -306.45392 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+----------------------------------------------------------------



owncar | income | -.0017168 .347905 -0.00 0.996 -.6835982 .6801645 age | .1492475 .0409238 3.65 0.000 .0690383 .2294568 male | .2594624 .1255633 2.07 0.039 .0133628 .505562 _cons | -2.834625 .8719679 -3.25 0.001 -4.543651 -1.125599 -------------+---------------------------------------------------------------- offcamp | income | .7519064 .8254937 0.91 0.362 -.8660316 2.369844 age | .5895658 .149221 3.95 0.000 .297098 .8820336 male | .3939644 .2834889 1.39 0.165 -.1616637 .9495925 _cons | -10.34593 2.947501 -3.51 0.000 -16.12293 -4.568938 -------------+---------------------------------------------------------------- /athrho | 2.387522 27.20167 0.09 0.930 -50.92678 55.70182 -------------+---------------------------------------------------------------- rho | .9832658 .9027811 -1 1 ------------------------------------------------------------------------------ Likelihood-ratio test of rho=0: chi2(1) = 21.3463 Prob > chi2 = 0.0000

4.2 Bivariate Probit in SAS The SAS QLIM procedure is able to estimate both the bivariate logit and probit models. You need to provide two equations that may or may not have different sets of independent variables. ODS HTML; PROC QLIM DATA=masil.students; MODEL owncar = income age male; MODEL offcamp = income age male; ENDOGENOUS owncar offcamp ~ DISCRETE(DIST=NORMAL); RUN; ODS HTML CLOSE; Or, simply, ODS HTML; PROC QLIM DATA=masil.students; MODEL owncar offcamp = income age male /DISCRETE; RUN; ODS HTML CLOSE;

The QLIM Procedure

Discrete Response Profile of owncar


1 0 153 35.01

2 1 284 64.99

Discrete Response Profile of offcamp




Discrete Response Profile of offcamp


1 0 12 2.75

2 1 425 97.25

Model Fit Summary

Number of Endogenous Variables 2

Endogenous Variable owncar offcamp



Maximum Absolute Gradient 2.16967E-6


AIC 628.90784



Parameter Estimates


owncar.Intercept -2.834511 0.871964 -3.25 0.0012

owncar.income -0.001723 0.347904 -0.00 0.9960

owncar.age 0.149243 0.040924 3.65 0.0003

owncar.male 0.259462 0.125563 2.07 0.0388

offcamp.Intercept -10.345002 2.947054 -3.51 0.0004

offcamp.income 0.751837 0.825398 0.91 0.3624

offcamp.age 0.589515 0.149197 3.95 <.0001

offcamp.male 0.393859 0.283458 1.39 0.1647

_Rho 0.999990 0 . .



4.3 Bivariate Probit in LIMDEP (Bivariateprobit$) LIMDEP has the Bivariateprobit$ command to estimate the bivariate probit model. The Lhs$ subcommand lists the two binary dependent variables, whereas Rh1$ and Rh2$ respectively indicate independent variables for the two dependent variables. In this model, you may not switch the order of dependent variables (Lhs=owncar,offcamp;) to avoid convergence problems. BIVARIATEPROBIT; Lhs=offcamp,owncar; Rh1=ONE,income,age,male; Rh2= ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | FIML Estimates of Bivariate Probit Model | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 05:07:09PM.| | Dependent variable OFFOWN | | Weighting variable None | | Number of observations 437 | | Iterations completed 35 | | Log likelihood function -306.4539 | | Number of parameters 9 | | Info. Criterion: AIC = 1.44373 | | Finite Sample: AIC = 1.44469 | | Info. Criterion: BIC = 1.52775 | | Info. Criterion:HQIC = 1.47688 | +---------------------------------------------+ +--------+--------------+----------------+--------+--------+----------+ |Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X| +--------+--------------+----------------+--------+--------+----------+ ---------+Index equation for OFFCAMP Constant| -10.3450823 3.65925588 -2.827 .0047 INCOME | .75184070 .85274898 .882 .3780 .61683982 AGE | .58951892 .18572787 3.174 .0015 20.6910755 MALE | .39385995 .29308052 1.344 .1790 .57208238 ---------+Index equation for OWNCAR Constant| -2.83451315 .84825468 -3.342 .0008 INCOME | -.00172310 .34222451 -.005 .9960 .61683982 AGE | .14924263 .03973976 3.755 .0002 20.6910755 MALE | .25946189 .12565094 2.065 .0389 .57208238 ---------+Disturbance correlation RHO(1,2)| .99413116 .733373D+09 .000 1.0000 +-----------------------------------------------------+ | Joint Frequency Table for Bivariate Probit Model | | Predicted cell is the one with highest probability | +-----------------------------------------------------+ | OWNCAR | +-------------+---------------------------------------+ | OFFCAMP | 0 1 Total | |-------------+-------------+------------+------------+ | 0 | 12 | 0 | 12 | | Fitted | ( 0) | ( 0) | ( 0) | |-------------+-------------+------------+------------+ | 1 | 141 | 284 | 425 | | Fitted | ( 0) | ( 437) | ( 437) | |-------------+-------------+------------+------------+ | Total | 153 | 284 | 437 | | Fitted | ( 0) | ( 437) | ( 437) |



|-------------+-------------+------------+------------+ +--------------------------------------------------------+ | Bivariate Probit Predictions for OFFCAMP and OWNCAR | | Predicted cell (i,j) is cell with largest probability | | Neither OFFCAMP nor OWNCAR predicted correctly | | 12 of 437 observations | | Only OFFCAMP correctly predicted | | OFFCAMP = 0: 0 of 12 observations | | OFFCAMP = 1: 141 of 425 observations | | Only OWNCAR correctly predicted | | OWNCAR = 0: 0 of 153 observations | | OWNCAR = 1: 141 of 284 observations | | Both OFFCAMP and OWNCAR correctly predicted | | OFFCAMP = 0 OWNCAR = 0: 0 of 12 | | OFFCAMP = 1 OWNCAR = 0: 0 of 141 | | OFFCAMP = 0 OWNCAR = 1: 0 of 0 | | OFFCAMP = 1 OWNCAR = 1: 284 of 284 | +--------------------------------------------------------+

SAS, Stata, and LIMDEP produce almost the same parameter estimates and standard errors with slight differences after the decimal point.

4.4 Bivariate Logit in SAS The QLIM procedure also estimates the bivariate logit model using the DIST=LOGIT option. Unfortunately, this model does not fit in SAS. PROC QLIM DATA=masil.students; MODEL owncar offcamp = income age male /DISCRETE(DIST=LOGIT); RUN;



5. Ordered Logit/Probit Regression Models Suppose we have an ordinal dependent variable such as the degree of illegal parking (0=none, 1=sometimes, and 2=often). The ordered logit and probit models have the parallel regression assumption, which is violated from time to time. 5.1 Ordered Logit/Probit in Stata (.ologit and .oprobit) Stata has the .ologit and .oprobit commands to estimate the ordered logit and probit models, respectively. . ologit parking income age male Iteration 0: log likelihood = -103.78713 Iteration 1: log likelihood = -92.739147 Iteration 2: log likelihood = -90.036393 Iteration 3: log likelihood = -89.861679 Iteration 4: log likelihood = -89.860105 Iteration 5: log likelihood = -89.860105 Ordered logistic regression Number of obs = 437 LR chi2(3) = 27.85 Prob > chi2 = 0.0000 Log likelihood = -89.860105 Pseudo R2 = 0.1342 ------------------------------------------------------------------------------ parking | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- income | -.5140709 1.283192 -0.40 0.689 -3.029082 2.00094 age | -.7362588 .1894339 -3.89 0.000 -1.107542 -.3649752 male | -1.227092 .4705859 -2.61 0.009 -2.149423 -.3047605 -------------+---------------------------------------------------------------- /cut1 | -12.74479 3.787616 -20.16839 -5.321203 /cut2 | -10.83295 3.801685 -18.28412 -3.381786 ------------------------------------------------------------------------------

Stata estimates mτ , /cut1 and /cut2, assuming 00 =β (Long and Freese 2003). This parameterization is different from that of SAS and LIMDEP, which assume 01 =τ . . oprobit parking income age male Iteration 0: log likelihood = -103.78713 Iteration 1: log likelihood = -90.990455 Iteration 2: log likelihood = -89.496288 Iteration 3: log likelihood = -89.430915 Iteration 4: log likelihood = -89.430754 Ordered probit regression Number of obs = 437 LR chi2(3) = 28.71 Prob > chi2 = 0.0000 Log likelihood = -89.430754 Pseudo R2 = 0.1383 ------------------------------------------------------------------------------ parking | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- income | -.1869839 .6116037 -0.31 0.760 -1.385705 1.011737



age | -.3594853 .0924817 -3.89 0.000 -.540746 -.1782246 male | -.5867871 .2205253 -2.66 0.008 -1.019009 -.1545655 -------------+---------------------------------------------------------------- /cut1 | -6.000986 1.869046 -9.664248 -2.337724 /cut2 | -5.118676 1.862909 -8.769911 -1.467442 ------------------------------------------------------------------------------

5.2 The Parallel Assumption and the Generalized Ordered Logit Model The .brant command of SPost is valid only in the .ologit command. This command tests the parallel regression assumption of the ordinal regression model. The outputs here are skipped. . quietly ologit parking income male . brant not all independent variables can be retained in all binary logits brant test cannot be computed r(999); The parallel regression assumption is often violated. If this is the case, you may use the multinomial regression model or estimate the generalized ordered logit model (GOLM) using either the .gologit command written by Fu (1998) or the .gologit2 command by Williams (2005). Note that Fu’s module does not impose the restriction of )()( 11 −− −≥− jjjj xx βτβτ (Long’s class note 2003). . gologit2 parking income age male, autofit ------------------------------------------------------------------------------ Testing parallel lines assumption using the .05 level of significance... Step 1: male meets the pl assumption (P Value = 0.9901) Step 2: income meets the pl assumption (P Value = 0.8958) Step 3: age meets the pl assumption (P Value = 0.7964) Step 4: All explanatory variables meet the pl assumption Wald test of parallel lines assumption for the final model: ( 1) [0]male - [1]male = 0 ( 2) [0]income - [1]income = 0 ( 3) [0]age - [1]age = 0 chi2( 3) = 0.04 Prob > chi2 = 0.9982 An insignificant test statistic indicates that the final model does not violate the proportional odds/ parallel lines assumption If you re-estimate this exact same model with gologit2, instead of autofit you can save time by using the parameter pl(male income age) ------------------------------------------------------------------------------ Generalized Ordered Logit Estimates Number of obs = 437 Wald chi2(3) = 21.74



Prob > chi2 = 0.0001 Log likelihood = -89.860105 Pseudo R2 = 0.1342 ( 1) [0]male - [1]male = 0 ( 2) [0]income - [1]income = 0 ( 3) [0]age - [1]age = 0 ------------------------------------------------------------------------------ parking | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- 0 | income | -.5140709 1.283192 -0.40 0.689 -3.029082 2.00094 age | -.7362588 .1894339 -3.89 0.000 -1.107543 -.3649752 male | -1.227092 .4705859 -2.61 0.009 -2.149423 -.3047605 _cons | 12.74479 3.787616 3.36 0.001 5.321202 20.16839 -------------+---------------------------------------------------------------- 1 | income | -.5140709 1.283192 -0.40 0.689 -3.029082 2.00094 age | -.7362588 .1894339 -3.89 0.000 -1.107543 -.3649752 male | -1.227092 .4705859 -2.61 0.009 -2.149423 -.3047605 _cons | 10.83295 3.801686 2.85 0.004 3.381785 18.28412 ------------------------------------------------------------------------------ 5.3 Ordered Logit in SAS The QLIM, LOGISTIC, and PROBIT procedures estimate ordered logit and probit models. As shown in Tables 3 and 4, the QLIM procedure is most recommended. Note that the DIST=LOGISTIC indicates the logit model to be estimated. PROC QLIM DATA=masil.students; MODEL parking = income age male /DISCRETE (DIST=LOGISTIC); RUN; The QLIM Procedure Discrete Response Profile of parking Index Value Frequency Percent 1 0 413 94.51 2 1 20 4.58 3 2 4 0.92 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable parking Number of Observations 437 Log Likelihood -89.86011 Maximum Absolute Gradient 8.14046E-7 Number of Iterations 23 AIC 189.72021 Schwarz Criterion 210.11988



Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 27.854 2 * (LogL - LogL0) Upper Bound of R (U) 207.57 - 2 * LogL0 Aldrich-Nelson 0.0599 R / (R+N) Cragg-Uhler 1 0.0618 1 - exp(-R/N) Cragg-Uhler 2 0.1633 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0662 1 - (1-R/U)^(U/N) Adjusted Estrella 0.0418 1 - ((LogL-K)/LogL0)^(-2/N*LogL0) McFadden's LRI 0.1342 R / U Veall-Zimmermann 0.1861 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.6462 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > |t| Intercept 12.744794 3.787615 3.36 0.0008 income -0.514071 1.283192 -0.40 0.6887 age -0.736259 0.189434 -3.89 0.0001 male -1.227092 0.470586 -2.61 0.0091 _Limit2 1.911842 0.468050 4.08 <.0001

The SAS QLIM procedure estimates the intercept and 2τ , assuming 01 =τ . The estimated intercept of SAS is equivalent to (0-/cut1) in Stata. The _Limit2 of SAS is the difference between cut points of Stata, 1.91184=-10.83295-(-12.74479). The SAS LOGISTIC and PROBIT procedures are also used to estimate the ordered logit and probit models. These procedures recognize binary or ordinal response models by examining the dependent variable. PROC LOGISTIC DATA = masil.students DESC; MODEL parking = income age male /LINK=LOGIT; RUN;

Like the Stata .ologit command, The LOGISTIC procedure fits the model, assuming the intercept is zero. The parameter estimates and standard errors are slightly different from those of the QLIM procedure and the .ologit command. Other parts of the output are skipped. Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 2 1 10.8324 3.8112 8.0784 0.0045 Intercept 1 1 12.7444 3.8021 11.2354 0.0008



income 1 -0.5142 1.2908 0.1587 0.6904 age 1 -0.7362 0.1900 15.0221 0.0001 male 1 -1.2271 0.4709 6.7902 0.0092

PROC PROBIT DATA = masil.students; CLASS parking; MODEL parking = income age male /DIST=LOGISTIC; RUN;

The PROBIT procedure returns almost the same results as the QLIM procedure except for the signs of the estimates. Other parts of the output are skipped. Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 -12.7448 3.7876 -20.1684 -5.3212 11.32 0.0008 Intercept2 1 1.9118 0.4680 0.9945 2.8292 16.68 <.0001 income 1 0.5141 1.2832 -2.0009 3.0291 0.16 0.6887 age 1 0.7363 0.1894 0.3650 1.1075 15.11 0.0001 male 1 1.2271 0.4706 0.3048 2.1494 6.80 0.0091 5.4 Ordered Probit in SAS The QLIM procedure by default estimates a probit model. The DIST=NORMAL, the default option, may be omitted. PROC QLIM DATA=masil.students; MODEL parking = income age male /DISCRETE (DIST=NORMAL); RUN; The QLIM Procedure Discrete Response Profile of parking Index Value Frequency Percent 1 0 413 94.51 2 1 20 4.58 3 2 4 0.92 Model Fit Summary Number of Endogenous Variables 1 Endogenous Variable parking Number of Observations 437 Log Likelihood -89.43075 Maximum Absolute Gradient 4.69307E-6 Number of Iterations 17 AIC 188.86151 Schwarz Criterion 209.26117



Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 28.713 2 * (LogL - LogL0) Upper Bound of R (U) 207.57 - 2 * LogL0 Aldrich-Nelson 0.0617 R / (R+N) Cragg-Uhler 1 0.0636 1 - exp(-R/N) Cragg-Uhler 2 0.1682 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.0683 1 - (1-R/U)^(U/N) Adjusted Estrella 0.0439 1 - ((LogL-K)/LogL0)^(-2/N*LogL0) McFadden's LRI 0.1383 R / U Veall-Zimmermann 0.1915 (R * (U+N)) / (U * (R+N)) McKelvey-Zavoina 0.3011 N = # of observations, K = # of regressors Algorithm converged. Parameter Estimates Standard Approx Parameter Estimate Error t Value Pr > |t| Intercept 6.000986 1.869053 3.21 0.0013 income -0.186984 0.611605 -0.31 0.7598 age -0.359485 0.092482 -3.89 0.0001 male -0.586787 0.220526 -2.66 0.0078 _Limit2 0.882310 0.196555 4.49 <.0001

The QLIM procedure and .oprobit command produce almost the same result except for the 2τ estimate. The _Limit2 of SAS is the difference of the cut points of Stata, .88231=-5.118676-(-6.000986). The PROBIT and LOGISTIC procedures also estimate the ordered probit model. Keep in mind that the signs of the coefficients are reversed in the PROBIT procedure. PROC LOGISTIC DATA = masil.students DESC; MODEL parking = income age male /LINK=PROBIT; RUN; Analysis of Maximum Likelihood Estimates Standard Wald Parameter DF Estimate Error Chi-Square Pr > ChiSq Intercept 2 1 5.1181 1.8373 7.7601 0.0053 Intercept 1 1 6.0004 1.8441 10.5872 0.0011 income 1 -0.1869 0.6160 0.0921 0.7615 age 1 -0.3595 0.0908 15.6767 <.0001 male 1 -0.5868 0.2203 7.0941 0.0077



PROC PROBIT DATA = masil.students; CLASS parking; MODEL parking = income age male /DIST=NORMAL; RUN; Analysis of Parameter Estimates Standard 95% Confidence Chi- Parameter DF Estimate Error Limits Square Pr > ChiSq Intercept 1 -6.0010 1.8691 -9.6643 -2.3377 10.31 0.0013 Intercept2 1 0.8823 0.1966 0.4971 1.2675 20.15 <.0001 income 1 0.1870 0.6116 -1.0117 1.3857 0.09 0.7598 age 1 0.3595 0.0925 0.1782 0.5407 15.11 0.0001 male 1 0.5868 0.2205 0.1546 1.0190 7.08 0.0078 5.5 Ordered Logit/Probit in LIMDEP (Ordered$) The LIMDEP Ordered$ command estimates ordered logit and probit models. The Logit$ subcommand runs the ordered logit model. ORDERED; Lhs=parking; Rhs=ONE,income,age,male; Logit$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | Ordered Probability Model | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 05:08:42PM.| | Dependent variable PARKING | | Weighting variable None | | Number of observations 437 | | Iterations completed 13 | | Log likelihood function -89.86011 | | Number of parameters 5 | | Info. Criterion: AIC = .43414 | | Finite Sample: AIC = .43446 | | Info. Criterion: BIC = .48082 | | Info. Criterion:HQIC = .45256 | | Restricted log likelihood -103.7871 | | McFadden Pseudo R-squared .1341883 | | Chi squared 27.85404 | | Degrees of freedom 3 | | Prob[ChiSqd > value] = .3896741E-05 | | Underlying probabilities based on Logistic | +---------------------------------------------+ +---------------------------------------------+ | Ordered Probability Model | | Cell frequencies for outcomes | | Y Count Freq Y Count Freq Y Count Freq | | 0 413 .945 1 20 .045 2 4 .009 | +---------------------------------------------+ +--------+--------------+----------------+--------+--------+----------+



|Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X| +--------+--------------+----------------+--------+--------+----------+ ---------+Index function for probability Constant| 12.7447942 3.78761615 3.365 .0008 INCOME | -.51407086 1.28319229 -.401 .6887 .61683982 AGE | -.73625883 .18943391 -3.887 .0001 20.6910755 MALE | -1.22709196 .47058590 -2.608 .0091 .57208238 ---------+Threshold parameters for index Mu(1) | 1.91184192 .46804996 4.085 .0000 +---------------------------------------------------------------------------+ | Cross tabulation of predictions. Row is actual, column is predicted. | | Model = Logistic . Prediction is number of the most probable cell. | +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | Actual|Row Sum| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | 0| 413| 413| 0| 0| | 1| 20| 20| 0| 0| | 2| 4| 4| 0| 0| +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ |Col Sum| 437| 437| 0| 0| 0| 0| 0| 0| 0| 0| 0| +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ LIMDEP and SAS QLIM produce the same results for the ordered logit model. Note that _Limit2 in SAS is equivalent to Mu(1), the threshold parameter, in LIMDEP. The ordered probit model is estimated by the Ordered$ command without the Logit$ subcommand. The command by default fits the ordered logit model. The output is comparable to that of the QLIM procedure. ORDERED; Lhs=parking; Rhs=ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | Ordered Probability Model | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 05:11:35PM.| | Dependent variable PARKING | | Weighting variable None | | Number of observations 437 | | Iterations completed 11 | | Log likelihood function -89.43075 | | Number of parameters 5 | | Info. Criterion: AIC = .43218 | | Finite Sample: AIC = .43250 | | Info. Criterion: BIC = .47886 | | Info. Criterion:HQIC = .45060 | | Restricted log likelihood -103.7871 | | McFadden Pseudo R-squared .1383252 | | Chi squared 28.71275 | | Degrees of freedom 3 | | Prob[ChiSqd > value] = .2572557E-05 | | Underlying probabilities based on Normal | +---------------------------------------------+ +---------------------------------------------+ | Ordered Probability Model | | Cell frequencies for outcomes |



| Y Count Freq Y Count Freq Y Count Freq | | 0 413 .945 1 20 .045 2 4 .009 | +---------------------------------------------+ +--------+--------------+----------------+--------+--------+----------+ |Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X| +--------+--------------+----------------+--------+--------+----------+ ---------+Index function for probability Constant| 6.00098503 1.86905361 3.211 .0013 INCOME | -.18698360 .61160494 -.306 .7598 .61683982 AGE | -.35948523 .09248209 -3.887 .0001 20.6910755 MALE | -.58678706 .22052578 -2.661 .0078 .57208238 ---------+Threshold parameters for index Mu(1) | .88230960 .19655461 4.489 .0000 +---------------------------------------------------------------------------+ | Cross tabulation of predictions. Row is actual, column is predicted. | | Model = Probit . Prediction is number of the most probable cell. | +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | Actual|Row Sum| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | 0| 413| 413| 0| 0| | 1| 20| 20| 0| 0| | 2| 4| 4| 0| 0| +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ |Col Sum| 437| 437| 0| 0| 0| 0| 0| 0| 0| 0| 0| +-------+-------+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ 5.6 Ordered Logit/Probit in SPSS The Plum command estimates the ordered logit and probit models in SPSS. Threshold points in SPSS are equivalent to the cut points in Stata. Pay attention to link functions used in the /LINK subcommand: The first and second commands respectively use LOGIT and PROBIT, which are indicated in the SPSS output. PLUM parking WITH income age male /CRITERIA = CIN(95) DELTA(0) LCONVERGE(0) MXITER(100) MXSTEP(5) PCONVERGE(1.0E-6) SINGULAR(1.0E-8) /LINK = LOGIT /PRINT = FIT PARAMETER SUMMARY .

Parameter Estimates


Estimate Std. Error Wald df Sig. Lower Bound Upper Bound

[parking = .00] -12.745 3.802 11.235 1 .001 -20.197 -5.292Threshold

[parking = 1.00] -10.833 3.811 8.079 1 .004 -18.303 -3.363income -.514 1.291 .159 1 .690 -3.044 2.016age -.736 .190 15.022 1 .000 -1.109 -.364

Location

male -1.227 .471 6.789 1 .009 -2.150 -.304Link function: Logit. PLUM parking WITH income age male /CRITERIA = CIN(95) DELTA(0) LCONVERGE(0) MXITER(100) MXSTEP(5) PCONVERGE(1.0E-6) SINGULAR(1.0E-8) /LINK = PROBIT /PRINT = FIT PARAMETER SUMMARY .



Parameter Estimates


Estimate Std. Error Wald df Sig. Lower Bound Upper Bound

[parking = .00] -6.001 1.844 10.589 1 .001 -9.616 -2.386Threshold [parking = 1.00] -5.119 1.837 7.762 1 .005 -8.720 -1.518income -.187 .616 .092 1 .761 -1.394 1.020age -.359 .091 15.678 1 .000 -.537 -.182

Location

male -.587 .220 7.093 1 .008 -1.019 -.155Link function: Probit.



6. The Multinomial Logit Regression Model Suppose we have a nominal dependent variable such as the mode of transportation (walk, bike, bus, and car). The multinomial logit and conditional logit models are commonly used; the multinomial probit model is not often used mainly due to the practical difficulty in estimation. However, Stata does have the .mprobit command to fit the model. In the multinomial logit model, the independent variables contain characteristics of individuals, while they are the attributes of the choices in the conditional logit model. In other words, the conditional logit estimates how alternative-specific, not individual-specific, variables affect the likelihood of observing a given outcome (Long 2003). Therefore, data need to be appropriately arranged in advance. 6.1 Multinomial Logit/Probit in Stata (.mlogit and .mprobit) Stata has the .mlogit command for the multinomial logit model. The base() option indicates the value of the dependent variable to be used as the base category for the estimation. You may omit the default option, base(0).

. mlogit transmode income age male, base(0) Iteration 0: log likelihood = -444.84113 Iteration 1: log likelihood = -411.18604 Iteration 2: log likelihood = -406.36474 Iteration 3: log likelihood = -406.3251 Iteration 4: log likelihood = -406.32509 Multinomial logistic regression Number of obs = 437 LR chi2(9) = 77.03 Prob > chi2 = 0.0000 Log likelihood = -406.32509 Pseudo R2 = 0.0866 ------------------------------------------------------------------------------ transmode | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- 1 | income | 4.018021 1.34443 2.99 0.003 1.382986 6.653057 age | .1915917 .1392928 1.38 0.169 -.0814172 .4646006 male | .2582886 .4039971 0.64 0.523 -.5335311 1.050108 _cons | -6.903473 2.97678 -2.32 0.020 -12.73785 -1.069091 -------------+---------------------------------------------------------------- 2 | income | 8.951041 1.338539 6.69 0.000 6.327552 11.57453 age | .1374997 .1451938 0.95 0.344 -.1470749 .4220742 male | .1573179 .4191014 0.38 0.707 -.6641057 .9787415 _cons | -9.091051 3.088123 -2.94 0.003 -15.14366 -3.038442 -------------+---------------------------------------------------------------- 3 | income | 4.210485 1.032024 4.08 0.000 2.187755 6.233215 age | .3457236 .0995071 3.47 0.001 .1506932 .540754 male | .5402549 .2769887 1.95 0.051 -.0026329 1.083143 _cons | -8.388756 2.135792 -3.93 0.000 -12.57483 -4.202681 ------------------------------------------------------------------------------ (transmode==0 is the base outcome)



Let us see if the base outcome changes. As shown in the following, the parameter estimates and standard errors are changed, whereas the goodness-of-fit remains unchanged. The two .mlogit commands with different bases fit the same model but present the result in different manner. The SAS CATMOD procedure in the next section uses the largest value as the base outcome. . mlogit transmode income age male, base(3) Iteration 0: log likelihood = -444.84113 Iteration 1: log likelihood = -411.18604 Iteration 2: log likelihood = -406.36474 Iteration 3: log likelihood = -406.3251 Iteration 4: log likelihood = -406.32509 Multinomial logistic regression Number of obs = 437 LR chi2(9) = 77.03 Prob > chi2 = 0.0000 Log likelihood = -406.32509 Pseudo R2 = 0.0866 ------------------------------------------------------------------------------ transmode | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- 0 | income | -4.210485 1.032024 -4.08 0.000 -6.233215 -2.187755 age | -.3457236 .0995071 -3.47 0.001 -.540754 -.1506932 male | -.5402549 .2769887 -1.95 0.051 -1.083143 .0026329 _cons | 8.388756 2.135792 3.93 0.000 4.202681 12.57483 -------------+---------------------------------------------------------------- 1 | income | -.1924639 1.00606 -0.19 0.848 -2.164305 1.779377 age | -.154132 .1131506 -1.36 0.173 -.3759031 .0676392 male | -.2819663 .3443963 -0.82 0.413 -.9569706 .3930379 _cons | 1.485283 2.430912 0.61 0.541 -3.279216 6.249783 -------------+---------------------------------------------------------------- 2 | income | 4.740556 .9447126 5.02 0.000 2.888953 6.592158 age | -.2082239 .1164954 -1.79 0.074 -.4365507 .0201028 male | -.382937 .3490247 -1.10 0.273 -1.067013 .3011389 _cons | -.7022953 2.460119 -0.29 0.775 -5.52404 4.119449 ------------------------------------------------------------------------------ (transmode==3 is the base outcome)

The SPost .mlogtest command conducts a variety of statistical tests for the multinomial logit model. This command supports not only Wald and likelihood ratio tests, but also Hausman and Small-Hsiao tests for the independence of irrelevant alternatives (IIA) assumption. The .mlogtest command works with the .mlogit command only. . mlogtest, hausman smhsiao base **** Hausman tests of IIA assumption (N=437) Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives. Omitted | chi2 df P>chi2 evidence ---------+------------------------------------ 0 | 0.260 8 1.000 for Ho 1 | -3.307 8 --- --- 2 | -0.319 8 --- --- 3 | 2.315 8 0.970 for Ho ----------------------------------------------



Note: If chi2<0, the estimated model does not meet asymptotic assumptions of the test. **** Small-Hsiao tests of IIA assumption (N=437) Ho: Odds(Outcome-J vs Outcome-K) are independent of other alternatives. Omitted | lnL(full) lnL(omit) chi2 df P>chi2 evidence ---------+--------------------------------------------------------- 0 | -120.685 -116.139 9.092 8 0.335 for Ho 1 | -131.938 -128.574 6.728 8 0.566 for Ho 2 | -155.078 -150.308 9.540 8 0.299 for Ho 3 | -71.735 -67.571 8.327 8 0.402 for Ho -------------------------------------------------------------------

The Stata .mprobit command fits the multinomial probit model. The model took longer time to converge than the multinomial logit model. . mprobit transmode income age male Iteration 0: log likelihood = -407.70755 Iteration 1: log likelihood = -406.39636 Iteration 2: log likelihood = -406.38431 Iteration 3: log likelihood = -406.38431 Multinomial probit regression Number of obs = 437 Wald chi2(9) = 64.47 Log likelihood = -406.38431 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ transmode | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- _outcome_1 | income | -2.751622 .6936632 -3.97 0.000 -4.111177 -1.392067 age | -.2760071 .074178 -3.72 0.000 -.4213933 -.1306208 male | -.4232271 .2086763 -2.03 0.043 -.8322252 -.0142289 _cons | 6.375609 1.598767 3.99 0.000 3.242083 9.509134 -------------+---------------------------------------------------------------- _outcome_2 | income | -.0996732 .6623838 -0.15 0.880 -1.397921 1.198575 age | -.1258603 .0737229 -1.71 0.088 -.2703546 .018634 male | -.2264475 .2305533 -0.98 0.326 -.6783236 .2254285 _cons | 1.30028 1.593781 0.82 0.415 -1.823474 4.424034 -------------+---------------------------------------------------------------- _outcome_3 | income | 3.005989 .6478328 4.64 0.000 1.73626 4.275718 age | -.1541446 .0769925 -2.00 0.045 -.3050471 -.003242 male | -.2569323 .2414529 -1.06 0.287 -.7301712 .2163066 _cons | -.1722656 1.646009 -0.10 0.917 -3.398383 3.053852 ------------------------------------------------------------------------------ (transmode=3 is the base outcome) 6.2 Multinomial Logit in SAS SAS has the CATMOD procedure for the multinomial logit model. In the CATMOD procedure, the RESPONSE statement is used to specify the functions of response probabilities. PROC CATMOD DATA = masil.students; DIRECT income age male;



RESPONSE LOGITS; MODEL transmode = income age male /NOPROFILE; RUN; The CATMOD Procedure Data Summary Response transmode Response Levels 4 Weight Variable None Populations 414 Data Set STUDENTS Total Frequency 437 Frequency Missing 0 Observations 437 Maximum Likelihood Analysis Maximum likelihood computations converged. Maximum Likelihood Analysis of Variance Source DF Chi-Square Pr > ChiSq ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Intercept 3 15.91 0.0012 income 3 45.72 <.0001 age 3 14.66 0.0021 male 3 4.73 0.1927 Likelihood Ratio 1E3 778.33 1.0000 Analysis of Maximum Likelihood Estimates Function Standard Chi- Parameter Number Estimate Error Square Pr > ChiSq ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Intercept 1 8.3888 2.1358 15.43 <.0001 2 1.4853 2.4309 0.37 0.5412 3 -0.7023 2.4601 0.08 0.7753 income 1 -4.2105 1.0320 16.65 <.0001 2 -0.1925 1.0061 0.04 0.8483 3 4.7406 0.9447 25.18 <.0001 age 1 -0.3457 0.0995 12.07 0.0005 2 -0.1541 0.1132 1.86 0.1731 3 -0.2082 0.1165 3.19 0.0739 male 1 -0.5403 0.2770 3.80 0.0511 2 -0.2820 0.3444 0.67 0.4129 3 -0.3829 0.3490 1.20 0.2726

As mentioned before, the CATMOD procedure uses the largest value of the dependent variable as a base outcome. Accordingly, you need to compare the above with the Stata output of the base(3) option. The two outputs are the same except for the likelihood ratio.



6.3 Multinomial Logit in LIMDEP (Mlogit$) In LIMDEP, you may use either the Mlogit$ or simply the Logit$ commands to fit the multinomial logit model. Both commands produce the identical result. Like Stata, LIMDEP by default uses the smallest value as the base outcome. MLOGIT; Lhs=transmod; Rhs=ONE,income,age,male$

Or, use the old style command. LOGIT; Lhs=transmod; Rhs=ONE,income,age,male$ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | Multinomial Logit Model | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 05:12:45PM.| | Dependent variable TRANSMOD | | Weighting variable None | | Number of observations 437 | | Iterations completed 6 | | Log likelihood function -406.3251 | | Number of parameters 12 | | Info. Criterion: AIC = 1.91453 | | Finite Sample: AIC = 1.91622 | | Info. Criterion: BIC = 2.02657 | | Info. Criterion:HQIC = 1.95874 | | Restricted log likelihood -444.8411 | | McFadden Pseudo R-squared .0865838 | | Chi squared 77.03209 | | Degrees of freedom 9 | | Prob[ChiSqd > value] = .0000000 | +---------------------------------------------+ +--------+--------------+----------------+--------+--------+----------+ |Variable| Coefficient | Standard Error |b/St.Er.|P[|Z|>z]| Mean of X| +--------+--------------+----------------+--------+--------+----------+ ---------+Characteristics in numerator of Prob[Y = 1] Constant| -6.90347267 2.97678006 -2.319 .0204 INCOME | 4.01802131 1.34443056 2.989 .0028 .61683982 AGE | .19159167 .13929281 1.375 .1690 20.6910755 MALE | .25828862 .40399708 .639 .5226 .57208238 ---------+Characteristics in numerator of Prob[Y = 2] Constant| -9.09105149 3.08812306 -2.944 .0032 INCOME | 8.95104081 1.33853957 6.687 .0000 .61683982 AGE | .13749967 .14519378 .947 .3436 20.6910755 MALE | .15731789 .41910143 .375 .7074 .57208238 ---------+Characteristics in numerator of Prob[Y = 3] Constant| -8.38875617 2.13579182 -3.928 .0001 INCOME | 4.21048516 1.03202419 4.080 .0000 .61683982 AGE | .34572362 .09950714 3.474 .0005 20.6910755 MALE | .54025494 .27698869 1.950 .0511 .57208238 +--------------------------------------------------------------------+ | Information Statistics for Discrete Choice Model. | | M=Model MC=Constants Only M0=No Model | | Criterion F (log L) -406.32509 -444.84113 -605.81064 |



| LR Statistic vs. MC 77.03209 .00000 .00000 | | Degrees of Freedom 9.00000 .00000 .00000 | | Prob. Value for LR .00000 .00000 .00000 | | Entropy for probs. 406.32509 444.84113 605.81064 | | Normalized Entropy .67071 .73429 1.00000 | | Entropy Ratio Stat. 398.97109 321.93900 .00000 | | Bayes Info Criterion 1.98483 2.16110 2.89780 | | BIC(no model) - BIC .91298 .73670 .00000 | | Pseudo R-squared .08658 .00000 .00000 | | Pct. Correct Pred. 64.98856 .00000 25.00000 | | Means: y=0 y=1 y=2 y=3 y=4 y=5 y=6 y>=7 | | Outcome .1648 .0892 .0961 .6499 .0000 .0000 .0000 .0000 | | Pred.Pr .1648 .0892 .0961 .6499 .0000 .0000 .0000 .0000 | | Notes: Entropy computed as Sum(i)Sum(j)Pfit(i,j)*logPfit(i,j). | | Normalized entropy is computed against M0. | | Entropy ratio statistic is computed against M0. | | BIC = 2*criterion - log(N)*degrees of freedom. | | If the model has only constants or if it has no constants, | | the statistics reported here are not useable. | +--------------------------------------------------------------------+ Frequencies of actual & predicted outcomes Predicted outcome has maximum probability. Predicted ------ -------------------- + ----- Actual 0 1 2 3 | Total ------ -------------------- + ----- 0 5 0 0 67 | 72 1 0 0 1 38 | 39 2 0 0 1 41 | 42 3 6 0 0 278 | 284 ------ -------------------- + ----- Total 11 0 2 424 | 437

Note that the variable name TRANSMOD was truncated because LIMDEP allows up to eight characters for a variable name. LIMDEP and Stata produce the same result of the multinomial logit model. 6.4 Multinomial Logit in SPSS SPSS has the Nomreg command to estimate the multinomial logit model. Like SAS, SPSS by default uses the largest value as the base outcome; you may change the baseline by specifying FIRST or any particular value at the Base= option. When using point-and-click method, you need put categorical variables in a box labeled as “Factor(s)” and continuous variables in the “Covariate(s)” box. NOMREG transmode (BASE=LAST ORDER=ASCENDING) BY male WITH income age /CRITERIA CIN(95) DELTA(0) MXITER(100) MXSTEP(5) CHKSEP(20) LCONVERGE(0) /MODEL /STEPWISE=PIN(.05) POUT(0.1) MINEFFECT(0) RULE(SINGLE) ENTRYMETHOD(LR) REMOVALMETHOD(LR) /INTERCEPT=INCLUDE /PRINT=PARAMETER SUMMARY LRT CPS STEP MFI.

The following table is selected from the SPSS output. Like Stata and LIMDEP, SPSS produces chi-squared 77.032 for the likelihood ratio test and McFadden Pseudo R2 .087. Parameter



estimates and standard errors are the same across statistical software packages. But intercepts produced in SPSS are different from those in other software packages. For example, Stata and SPSS return 1.485 and 1.203 as the intercept of category 1, respectively. In the case of baseline 0, they are -6.903 and -6.645.

Parameter Estimates

95% Confidence Interval for Exp(B)

transmodea B Std. Error Wald df Sig. Exp(B) Lower Bound Upper Bound

Intercept 7.849 2.141 13.438 1 .000

income -4.210 1.032 16.645 1 .000 .015 .002 .112

age -.346 .100 12.071 1 .001 .708 .582 .860

[male=.00] .540 .277 3.804 1 .051 1.716 .997 2.954

0

[male=1.00] 0b . . 0 . . . .

Intercept 1.203 2.435 .244 1 .621

income -.192 1.006 .037 1 .848 .825 .115 5.926

age -.154 .113 1.856 1 .173 .857 .687 1.070

[male=.00] .282 .344 .670 1 .413 1.326 .675 2.604

1

[male=1.00] 0b . . 0 . . . .

Intercept -1.085 2.459 .195 1 .659

income 4.741 .945 25.180 1 .000 114.498 17.974 729.354

age -.208 .116 3.195 1 .074 .812 .646 1.020

[male=.00] .383 .349 1.204 1 .273 1.467 .740 2.907

2

[male=1.00] 0b . . 0 . . . .

a. The reference category is: 3.00. b. This parameter is set to zero because it is redundant.



7. The Conditional Logit Regression Model Imagine a choice of the travel modes among air flight, train, bus, and car. The data set and model here are adopted from Greene (2003). The model examines how the generalized cost measure (cost), terminal waiting time (time), and household income (income) affect the choice. These independent variables are not characteristics of subjects (individuals), but attributes of the alternatives. Thus, the data arrangement of the conditional logit model is different from that of the multinomial logit model (Figure 2). Figure 2. Data Arrangement for the Conditional Logit Model +------------------------------------------------------------------------------+ | subject mode choice air train bus cost time income air_inc | |------------------------------------------------------------------------------| | 1 1 0 1 0 0 70 69 35 35 | | 1 2 0 0 1 0 71 34 35 0 | | 1 3 0 0 0 1 70 35 35 0 | | 1 4 1 0 0 0 30 0 35 0 | | 2 1 0 1 0 0 68 64 30 30 | | 2 2 0 0 1 0 84 44 30 0 | | 2 3 0 0 0 1 85 53 30 0 | | 2 4 1 0 0 0 50 0 30 0 | | 3 1 0 1 0 0 129 69 40 40 | | 3 2 0 0 1 0 195 34 40 0 | … … … … … … … … … … … …

The example data set has four observations per subject, each of which contains attributes of using air flight, train, bus, and car. The dependent variable choice is coded 1 only if a subject chooses that travel mode. The four dummy variables, air, train, bus, and car, are flagging the corresponding modes of transportation. See the appendix for details about the data set. 7.1 Conditional Logit in Stata (.clogit) Stata has the .clogit command to estimate the condition logit model. The group() option specifies the variable (e.g., identification number) that identifies unique individuals. . clogit choice air train bus cost time air_inc, group(subject) Iteration 0: log likelihood = -205.8187 Iteration 1: log likelihood = -199.23679 Iteration 2: log likelihood = -199.12851 Iteration 3: log likelihood = -199.12837 Iteration 4: log likelihood = -199.12837 Conditional (fixed-effects) logistic regression Number of obs = 840 LR chi2(6) = 183.99 Prob > chi2 = 0.0000 Log likelihood = -199.12837 Pseudo R2 = 0.3160 ------------------------------------------------------------------------------ choice | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- air | 5.207443 .7790551 6.68 0.000 3.680523 6.734363 train | 3.869043 .4431269 8.73 0.000 3.00053 4.737555



bus | 3.163194 .4502659 7.03 0.000 2.280689 4.045699 cost | -.0155015 .004408 -3.52 0.000 -.024141 -.006862 time | -.0961248 .0104398 -9.21 0.000 -.1165865 -.0756631 air_inc | .013287 .0102624 1.29 0.195 -.0068269 .033401 --------------------------------------------------------------------------------------

Let us run the .listcoef command to compute factor changes in odds. For a one unit increase in the waiting time for a given travel mode, for example, we can expect a decrease in the odds of using that travel by 9 percent (or a factor of .9084), holding other variables constant. . listcoef clogit (N=840): Factor Change in Odds Odds of: 1 vs 0 -------------------------------------------------- choice | b z P>|z| e^b -------------+------------------------------------ air | 5.20744 6.684 0.000 182.6265 train | 3.86904 8.731 0.000 47.8965 bus | 3.16319 7.025 0.000 23.6460 cost | -0.01550 -3.517 0.000 0.9846 time | -0.09612 -9.207 0.000 0.9084 air_inc | 0.01329 1.295 0.195 1.0134 --------------------------------------------------

7.2 Conditional Logit in SAS SAS has the MDC procedure to fit the conditional logit model. The TYPE=CLOGIT indicates the conditional logit model; the ID statement specifies the identification variable; and the NCHOICE=4 tells that there are four choices of the travel mode. PROC MDC DATA=masil.travel; MODEL choice = air train bus cost time air_inc /TYPE=CLOGIT NCHOICE=4; ID subject; RUN; The MDC Procedure Conditional Logit Estimates Algorithm converged. Model Fit Summary Dependent Variable choice Number of Observations 210 Number of Cases 840 Log Likelihood -199.12837 Maximum Absolute Gradient 2.73152E-8 Number of Iterations 5 Optimization Method Newton-Raphson AIC 410.25674 Schwarz Criterion 430.33938



Discrete Response Profile Index CHOICE Frequency Percent 0 1 58 27.62 1 2 63 30.00 2 3 30 14.29 3 4 59 28.10 Goodness-of-Fit Measures Measure Value Formula Likelihood Ratio (R) 183.99 2 * (LogL - LogL0) Upper Bound of R (U) 582.24 - 2 * LogL0 Aldrich-Nelson 0.467 R / (R+N) Cragg-Uhler 1 0.5836 1 - exp(-R/N) Cragg-Uhler 2 0.6225 (1-exp(-R/N)) / (1-exp(-U/N)) Estrella 0.6511 1 - (1-R/U)^(U/N) Adjusted Estrella 0.6212 1 - ((LogL-K)/LogL0)^(-2/N*LogL0) McFadden's LRI 0.316 R / U Veall-Zimmermann 0.6354 (R * (U+N)) / (U * (R+N)) N = # of observations, K = # of regressors Conditional Logit Estimates Parameter Estimates Standard Approx Parameter DF Estimate Error t Value Pr > |t| air 1 5.2074 0.7791 6.68 <.0001 train 1 3.8690 0.4431 8.73 <.0001 bus 1 3.1632 0.4503 7.03 <.0001 cost 1 -0.0155 0.004408 -3.52 0.0004 time 1 -0.0961 0.0104 -9.21 <.0001 air_inc 1 0.0133 0.0103 1.29 0.1954 Alternatively, you may use the PHREG procedure that estimates the Cox proportional hazards model for survival data and the conditional logit model. In order to make the data set consistent with the survival analysis data, you need to create a failure time variable, failure=1–choice. The identification variable is specified in the STRATA statement. The NOSUMMARY option suppresses the display of the event and censored observation frequencies. ODS HTML; PROC PHREG DATA=masil.travel NOSUMMARY; STRATA subject; MODEL failure*choice(0)=air train bus cost time air_inc; RUN; ODS HTML CLOSE;



The PHREG Procedure

Model Information

Data Set SM.TRAVEL

Dependent Variable failure

Censoring Variable choice

Censoring Value(s) 0

Ties Handling BRESLOW

Number of Observations Read Number of Observations Used

840840

Convergence Status

Convergence criterion (GCONV=1E-8) satisfied.

Model Fit Statistics

Criterion Without Covariates

With Covariates

-2 LOG L 582.244 398.257

AIC 582.244 410.257

SBC 582.244 430.339

Testing Global Null Hypothesis: BETA=0

Test Chi-Square DF Pr > ChiSq

Likelihood Ratio 183.9869 6 <.0001

Score 173.4374 6 <.0001

Wald 103.7695 6 <.0001


Variable DF Parameter Estimate

StandardError

Chi-Square Pr > ChiSq Hazard Ratio

air 1 5.20743 0.77905 44.6799 <.0001 182.625

train 1 3.86904 0.44313 76.2343 <.0001 47.896

bus 1 3.16319 0.45027 49.3530 <.0001 23.646




Variable DF Parameter Estimate

StandardError

Chi-Square Pr > ChiSq Hazard Ratio

cost 1 -0.01550 0.00441 12.3671 0.0004 0.985

time 1 -0.09612 0.01044 84.7778 <.0001 0.908

air_inc 1 0.01329 0.01026 1.6763 0.1954 1.013

While the MDC procedure reports t statistics, the PHREG procedure computes chi-squared (e.g., 12.3671=-3.52^2). The PHREG presents the hazard ratio at the last column of the output, which is equivalent to the factor changes under the e^b column of the SPost .listcoef command. 7.3 Conditional Logit in LIMDEP (Clogit$) LIMDEP fits the conditional logit model using either the Clogit$ or the Logit$ command. The Clogit$ command has the Choices$ subcommand to list the choices available. CLOGIT; Lhs=choice; Rhs=air,train,bus,cost,time,air_inc; Choices=air,train,bus,car$ +---------------------------------------------+ | Discrete choice and multinomial logit models| +---------------------------------------------+ Normal exit from iterations. Exit status=0. +---------------------------------------------+ | Discrete choice (multinomial logit) model | | Maximum Likelihood Estimates | | Model estimated: Oct 18, 2008 at 05:15:03PM.| | Dependent variable Choice | | Weighting variable None | | Number of observations 210 | | Iterations completed 6 | | Log likelihood function -199.1284 | | Number of parameters 6 | | Info. Criterion: AIC = 1.95360 | | Finite Sample: AIC = 1.95557 | | Info. Criterion: BIC = 2.04924 | | Info. Criterion:HQIC = 1.99226 | | R2=1-LogL/LogL* Log-L fncn R-sqrd RsqAdj | | Constants only -283.7588 .29825 .29150 | | Response data are given as ind. choice. | | Number of obs.= 210, skipped 0 bad obs. | +---------------------------------------------+ +---------------------------------------------+ | Notes No coefficients=> P(i,j)=1/J(i). | | Constants only => P(i,j) uses ASCs | | only. N(j)/N if fixed choice set. | | N(j) = total sample frequency for j | | N = total sample frequency. | | These 2 models are simple MNL models. | | R-sqrd = 1 - LogL(model)/logL(other) | | RsqAdj=1-[nJ/(nJ-nparm)]*(1-R-sqrd) | | nJ = sum over i, choice set sizes | +---------------------------------------------+



7.4 Conditional Logit in SPSS Like the SAS PHREG procedure, the SPSS Coxreg command, which was designed for survival analysis data, provides a backdoor way of estimating the conditional logit model. Like SPSS PROBIT, SPSS COXREG for the conditional logit model asks you to create a variable indicating failure as opposed to success. The following Compute command generates a variable failure by subtracting choice from 1 so that success and failure are respectively recoded as 0 and 1. COMPUTE failure = 1 - choice. COXREG failure WITH air train bus cost time air_inc /STATUS=choice(1) /STRATA=subject. The following table is selected from SPSS output. Again, “Sig.” in the column head is the p-value, not the significance level.

Variables in the Equation

B SE Wald Df Sig. Exp(B)

air 5.207 .779 44.680 1 .000 182.627

train 3.869 .443 76.234 1 .000 47.897

bus 3.163 .450 49.353 1 .000 23.646

cost -.016 .004 12.367 1 .000 .985

time -.096 .010 84.778 1 .000 .908

air_inc .013 .010 1.676 1 .195 1.013



8. The Nested Logit Regression Model Consider a nested structure of choices. The first choice is made and the second choice then follows conditional on the first choice. When the IIA assumption is violated, one of the alternatives is the nested logit model. This chapter replicates the nested logit model discussed in Greene (2003). A LIMDEP example is skipped here since the nested logit model is fitted by NLOGIT, a stand-alone package to be purchased separately.

)(*)|(),( branchPbranchchoicePbranchchoiceP = )cos()|( 21321 timetbustrainairPbranchchoiceP child ββααα ++++=

)_()( groundgroundflyflyincomeparent IVIVincairPbranchP ττγ ++= 8.1 Nested Logit in Stata (.nlogit) The Stata .nlogit command estimates the nested logit model using the full information maximum-likelihood (FIML) method. First you need to create a variable based on the specification of the tree using the .nlogitgen command. From the top, the parent-level has fly and ground branches; the fly branch at the child-level has air flight (1); the ground branch has train (2), bus (3), and car (4). Note that fly and ground below are not variable names but arbitrary names you prefer. . nlogitgen tree = mode(fly: 1, ground: 2 | 3 | 4) new variable tree is generated with 2 groups label list lb_tree lb_tree: 1 fly 2 ground The .nlogittree command.displays the tree-structure defined by the .nlogitgen command. . nlogittree mode tree, choice(choice) tree structure specified for the nested logit model tree N mode N k ----------------------------- fly 210 --- 1 210 58 ground 630 --- 2 210 63 |- 3 210 30 +- 4 210 59 ----------------------------- total 840 210 k = number of times alternative is chosen N = number of observations at each level In Stata 10, .nlogit by default uses parameterization consistent with random utility maximization and introduces new syntax different from one in previous edition (Stata 2007:



434). The command is followed by a binary dependent variable, a list of independent variables, specifications of each level, and options. The case() is required to specify identification variable and nonnormalized is needed to request unscaled parameterization. Remind that tree was defined by .nlogitgen above. . nlogit choice air train bus cost time || tree: air_inc || /// mode:, case(subject) nonnormalized nolog noconstant notree The notree option does not show the tree-structure and the nolog suppresses an iteration log of the log likelihood. Note that the /// joins the next command line to the current line. Nonnormalized nested logit regression Number of obs = 840 Case variable: subject Number of cases = 210 Alternative variable: mode Alts per case: min = 4 avg = 4.0 max = 4 Wald chi2(6) = 80.11 Log likelihood = -193.65615 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ choice | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- mode | air | 6.041827 1.198628 5.04 0.000 3.69256 8.391095 train | 5.063954 .6619239 7.65 0.000 3.766607 6.361301 bus | 4.095842 .6150907 6.66 0.000 2.890287 5.301398 cost | -.0315757 .0081541 -3.87 0.000 -.0475575 -.0155938 time | -.1126084 .0141277 -7.97 0.000 -.1402981 -.0849187 ------------------------------------------------------------------------------ tree equations ------------------------------------------------------------------------------ fly | air_inc | .0153323 .0093813 1.63 0.102 -.0030548 .0337193 -------------+---------------------------------------------------------------- ground | air_inc | (base) ------------------------------------------------------------------------------ inclusive-value parameters ------------------------------------------------------------------------------ tree | /fly_tau | .5861148 .1406178 .3105089 .8617207 /ground_tau | .389015 .1236901 .1465869 .6314432 ------------------------------------------------------------------------------ LR test for IIA (tau = 1): chi2(2) = 10.94 Prob > chi2 = 0.0042 ------------------------------------------------------------------------------ If you prefer old style, list a binary dependent or choice variable, utility functions of the parent and child-levels, and options. The group()option is equivalent to case() in version 10. Do not forget to run the .version command to use a previsions version of command interpreter. . version 9 . nlogit choice (mode=air train bus cost time) (tree=air_inc), /// group(subject) notree nolog Nested logit regression Levels = 2 Number of obs = 840



Dependent variable = choice LR chi2(8) = 194.9313 Log likelihood = -193.65615 Prob > chi2 = 0.0000 ------------------------------------------------------------------------------ | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- mode | air | 6.042255 1.198907 5.04 0.000 3.692441 8.39207 train | 5.064679 .6620317 7.65 0.000 3.767121 6.362237 bus | 4.096302 .6151582 6.66 0.000 2.890614 5.30199 cost | -.0315888 .0081566 -3.87 0.000 -.0475754 -.0156022 time | -.1126183 .0141293 -7.97 0.000 -.1403111 -.0849254 -------------+---------------------------------------------------------------- tree | air_inc | .0153337 .0093814 1.63 0.102 -.0030534 .0337209 -------------+---------------------------------------------------------------- (incl. value | parameters) | tree | /fly | .5859993 .1406199 4.17 0.000 .3103894 .8616092 /ground | .3889488 .1236623 3.15 0.002 .1465753 .6313224 ------------------------------------------------------------------------------ LR test of homoskedasticity (iv = 1): chi2(2)= 10.94 Prob > chi2 = 0.0042 ------------------------------------------------------------------------------

8.2 Nested Logit in SAS The SAD MDC procedure fits the conditional logit model as well as the nested multinomial logit model. For the nested logit model, you have to use the UTILITY statement to specify utility functions of the parent (level 2) and child level (level 1), and the NEST statement to construct the decision-tree structure. Note that “2 3 4 @ 2” reads that there are three nodes at the child level under the branch 2 at the parent-level. ODS HTML; PROC MDC DATA=masil.travel; MODEL choice = air train bus cost time air_inc /TYPE=NLOGIT CHOICE=(mode); ID subject; UTILITY U(1,) = air train bus cost time, U(2, 1 2) = air_inc; NEST LEVEL(1) = (1 @ 1, 2 3 4 @ 2), LEVEL(2) = (1 2 @ 1); RUN; ODS HTML CLOSE; The MDC Procedure Nested Logit Estimates


Model Fit Summary

Dependent Variable choice



Model Fit Summary


Number of Cases 840


Maximum Absolute Gradient 0.0000147


Optimization Method Newton-Raphson

AIC 403.31230


Discrete Response Profile

Index mode Frequency Percent

0 1 58 27.62

1 2 63 30.00

2 3 30 14.29

3 4 59 28.10

Goodness-of-Fit Measures

Measure Value Formula

Likelihood Ratio (R) 194.93 2 * (LogL - LogL0)

Upper Bound of R (U) 582.24 - 2 * LogL0

Aldrich-Nelson 0.4814 R / (R+N)

Cragg-Uhler 1 0.6048 1 - exp(-R/N)

Cragg-Uhler 2 0.6451 (1-exp(-R/N)) / (1-exp(-U/N))

Estrella 0.6771 1 - (1-R/U)^(U/N)

Adjusted Estrella 0.6485 1 - ((LogL-K)/LogL0)^(-2/N*LogL0)

McFadden's LRI 0.3348 R / U

Veall-Zimmermann 0.655 (R * (U+N)) / (U * (R+N))

N = # of observations, K = # of regressors

Parameter Estimates



Parameter DF Estimate StandardError

t Value Approx Pr > |t|

air_L1 1 6.0423 1.1989 5.04 <.0001

train_L1 1 5.0646 0.6620 7.65 <.0001

bus_L1 1 4.0963 0.6152 6.66 <.0001

cost_L1 1 -0.0316 0.008156 -3.87 0.0001

time_L1 1 -0.1126 0.0141 -7.97 <.0001

air_inc_L2G1 1 0.0153 0.009381 1.63 0.1022

INC_L2G1C1 1 0.5860 0.1406 4.17 <.0001

INC_L2G1C2 1 0.3890 0.1237 3.15 0.0017

The /fly and /ground in the Stata output above are equivalent to the INC_L2G1C1 and INC_L2G1C2 in the SAS output. SAS and Stata produce the same result.



9. Conclusion The appropriate type of categorical dependent variable model (CDVM) is determined largely by the measurement level of a dependent variable of interest. The level of measurement should be, however, considered in conjunction with your theory and research questions (Long 1997). You must also examine the data generation process (DGP) of a dependent variable to understand its behavior. Experienced researchers pay special attention to censoring, truncation, sample selection, and other particular patterns of the DGP, although these limited dependent variable issues are addressed here. If your dependent variable is a binary variable, you may use the binary logit or probit regression model. For ordinal responses, try to fit the ordered logit/probit regression models. If you have a nominal response variable, investigate the DGP carefully and then choose one of the multinomial logit, conditional logit, and nested logit models. In order to use the conditional logit and nested logit, you need to reshape the data set in advance. You should check the key assumptions of CDVMs when fitting the models. Examples are the parallel regression assumption in the ordered logit model and the independence of irrelevant alternatives (IIA) assumption in the multinomial logit model. You may conduct the Brant test and Hausman test for these assumptions. Since CDVMs are nonlinear, they produce estimates that are difficult to interpret intuitively. Consequently, researchers need to spend more time and effort interpreting the results substantively. Reporting parameter estimates and goodness-of-fit statistics is not sufficient. J. Scott Long (1997) and Long and Freese (2003) provide good examples of meaningful interpretations using predicted probabilities, factor changes in odds, and marginal/discrete changes of predicted probabilities. Regarding statistical software for CDVMs, I would recommend the SAS QLIM and MDC procedures of SAS/ETS (see Table 3 and 4). SAS has other procedures such as LOGISTIC, GENMODE, and PROBIT, but the QLIM procedure seems best for binary and ordinal response models, and the MDC procedure is good for nominal dependent variable models. ODS is another advantage of using SAS. I also strongly recommend Stata since it provides handy ways to fit various CDVMs and also can be assisted by SPost, which has various useful commands such as .prchange, .listcoef, and .prtab. I encourage SAS Institute to develop additional statements similar to those SPost commands. LIMDEP supports various CDVMs addressed in Greene (2003) but does not seem as user-friendly and stable as SAS and Stata. Thus, I recommend LIMDEP for CDVMs that SAS and Stata do not support. SPSS is least recommended mainly due to limited support for CDVMs and messy syntax and output.



Appendix: Data Sets The first data set is a subset of data provided for David H. Good’s class in the School of Public and Environmental Affairs (SPEA). The data were manipulated for the sake of data security.

• owncar: 1 if a student owns a car • parking: Illegal parking (0=none, 1=sometimes, and 2=often) • offcamp: 1 if a student lives off-campus • transmode: the mode of transportation (0=walk, 1=bike, 2=bus, 3=car) • age: students’ age • income: monthly income • male: 1 for male and 0 for female

. tab male owncar | owncar male | 0 1 | Total -----------+----------------------+---------- 0 | 76 111 | 187 1 | 77 173 | 250 -----------+----------------------+---------- Total | 153 284 | 437 . tab male offcamp | offcamp male | 0 1 | Total -----------+----------------------+---------- 0 | 7 180 | 187 1 | 5 245 | 250 -----------+----------------------+---------- Total | 12 425 | 437 . tab male parking | parking male | 0 1 2 | Total -----------+---------------------------------+---------- 0 | 170 13 4 | 187 1 | 243 7 0 | 250 -----------+---------------------------------+---------- Total | 413 20 4 | 437 . tab male transmode | transmode male | 0 1 2 3 | Total -----------+--------------------------------------------+---------- 0 | 38 18 20 111 | 187 1 | 34 21 22 173 | 250 -----------+--------------------------------------------+---------- Total | 72 39 42 284 | 437



. sum income age Variable | Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- income | 437 .6168398 .17918 .4 1.227 age | 437 20.69108 1.610812 18 29

The second data set on travel mode choice is adopted from Greene (2003). You may get the data from http://pages.stern.nyu.edu/~wgreene/Text/tables/tablelist5.htm

• subject: identification number • mode: 1=Air, 2=Train, 3=Bus, 4=Car • choice: 1 if the travel mode is chosen • time: terminal waiting time, 0 for car • cost: generalized cost measure • income: household income • air_inc: interaction of air flight and household income, air*income • air: 1 for the air flight mode, 0 for others • train: 1 for the train mode, 0 for others • bus: 1 for the bus mode, 0 for others • car: 1 for the car mode, 0 for others • failure: failure time variable, 1-choice

. tab choice mode | mode choice | 1 2 3 4 | Total -----------+--------------------------------------------+---------- 0 | 152 147 180 151 | 630 1 | 58 63 30 59 | 210 -----------+--------------------------------------------+---------- Total | 210 210 210 210 | 840 . sum time income air_inc Variable | Obs Mean Std. Dev. Min Max -------------+-------------------------------------------------------- time | 840 34.58929 24.94861 0 99 income | 840 34.54762 19.67604 2 72 air_inc | 840 8.636905 17.91206 0 72



References Allison, Paul D. 1991. Logistic Regression Using the SAS System: Theory and Application. Cary,

NC: SAS Institute. Fu, V. Kang. 1998. “Estimating Generalized Ordered Logit Models,” Stata Technical Bulletin,

STB-44: 27-30. Greene, William H. 2003. Econometric Analysis, 5th ed. Upper Saddle River, NJ: Prentice Hall. Greene, William H. 2007. LIMDEP Version 9.0 Econometric Modeling Guide. Plainview, New

York: Econometric Software. Long, J. Scott, and Jeremy Freese. 2003. Regression Models for Categorical Dependent

Variables Using Stata, 2nd ed. College Station, TX: Stata Press. Long, J. Scott. 1997. Regression Models for Categorical and Limited Dependent Variables:

Advanced Quantitative Techniques in the Social Sciences. Sage Publications. Maddala, G. S. 1983. Limited Dependent and Qualitative Variables in Econometrics. New York:

Cambridge University Press. SAS Institute. 2004. SAS/STAT 9.1 User's Guide. Cary, NC: SAS Institute. SPSS Inc. 2007. SPSS 16.0 Command Syntax Reference. Chicago, IL: SPSS Inc. Stata Press. 2007. Stata Base Reference Manual, Release 10. College Station, TX: Stata Press. Stokes, Maura E., Charles S. Davis, and Gary G. Koch. 2000. Categorical Data Analysis Using

the SAS System, 2nd ed. Cary, NC: SAS Institute. Williams, Richard, 2005. Glogit2: A Program for Generalized Logistic Regression/Partial

Proportional Odds Models for Ordinal Dependent Variables. North American Stata Users’ Groups Meeting 2005. http://www.stata.com/meeting/proceedings.html

Acknowledgements I am grateful to Jeremy Albright and Kevin Wilhite at the UITS Center for Statistical and Mathematical Computing for comments and suggestions. I also thank J. Scott Long in Sociology and David H. Good in the School of Public and Environmental Affairs, Indiana University. A special thanks to many readers around the world who have eagerly provided constructive feedback and encouraged me to keep improving this document. Revision History

• 2003. First draft • 2004. Second draft • 2005. Third draft (Added bivariate logit/probit and nested logit models) • 2008. Fourth draft (Added SAS ODS and SPSS output)

estimating regression models for categorical dependent ... · pdf fileestimating regression...

Documents