Strategic Management JournalStrat. Mgmt. J., 26: 881–886 (2005)

Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/smj.479

RESEARCH NOTES AND COMMENTARIES

RESPONSE TO McGAHAN AND PORTER’SCOMMENTARY ON ‘INDUSTRY, CORPORATE ANDBUSINESS-SEGMENT EFFECTS AND BUSINESSPERFORMANCE: A NON-PARAMETRIC APPROACH’

TIMOTHY W. RUEFLI1* and ROBERT R. WIGGINS2

1 McCombs School of Business and The IC2 Institute, University of Texas at Austin,Austin, Texas, U.S.A.2 Fogelman College of Business and Economics, University of Memphis, Memphis,Tennessee, U.S.A.

In the comment on Ruefli and Wiggins (2003), a number of points are made supporting thevariance component analysis approach to determining the importance of industry, corporate,and business segment factors on business segment performance. This response addresses inmore detail the nature of the methodological and statistical assumptions made by variancecomponents analysis or ANOVA and their implications for the ‘puzzling’ results obtained whenthese techniques are employed. The response then contrasts the variance-based methodologieswith a non-parametric approach used in Ruefli and Wiggins (2003) that makes fewer and weakerassumptions and yields more robust and more internally consistent results. The response alsoexamines the limitations of employing an autoregressive approach to measuring persistence ofabnormal profits and contrasts it with a non-parametric methodology presented in the article.Copyright 2005 John Wiley & Sons, Ltd.

INTRODUCTION

The comment (McGahan and Porter, 2005) onRuefli and Wiggins (2003) raises a number ofissues relevant to not only our article, but tothe broader area of research on the importanceof industry, corporate, and business segment fac-tors to business segment performance. Rather thansequentially address each of the points noted in thecomment, this response begins with an overview

Keywords: non-parametric analysis; variance componentsanalysis; ceteris paribus assumption*Correspondence to: Timothy W. Ruefli, McCombs School ofBusiness, University of Texas at Austin, CBA 5.202, Austin,TX 78712, U.S.A. E-mail: tim.ruefli@mccombs.utexas.edu

of the methodology employed in Ruefli and Wig-gins (2003) as a way of providing a context inwhich to speak to the issues that were raised.In our research, time series performance data onentities at the levels of business segments, cor-porations, and industries were stratified on eachlevel and in each period by the iterative applicationof the non-parametric Kolmogorov–Smirnov (IKShereafter) test (Ruefli and Wiggins, 2000). Thistechnique clusters entities into strata that are statis-tically significantly different in their performancefrom all other strata in that period. In using IKSthe researcher does not specify a priori or ex post(as in most methods of clustering) the number ornature of the strata—the IKS technique determinesthose parameters from the data. Our application

Copyright 2005 John Wiley & Sons, Ltd. Received 12 October 2004Final revision received 3 December 2004

882 T. W. Ruefli and R. R. Wiggins

of IKS analysis in each period, for each levelof organization, found a modal group comprisingapproximately 60 percent of the target entities ateach organizational level: business segment, cor-porate, and industry (Ruefli and Wiggins, 2003;Tables 2, 3 and 4). On average, another 20 per-cent of the entities were in superior performancestrata and 20 percent were in inferior performancestrata.

For the methodology in Ruefli and Wiggins(2003), for each organizational level the sets ofstrata above the modal stratum were combined toyield one statistically significantly superior stratumand those below the modal stratum were com-bined to yield one statistically significantly infe-rior performance stratum. As noted in McGahanand Porter (2005), these three performance stratayield an ordinal ranking; however, contrary to thecomment, this tripartite partitioning is statisticallybased and stands in contrast to arbitrary divisionsof entities by performance into halves, quarters,deciles, and such. For the data in Ruefli and Wig-gins (2003) (and also in McGahan and Porter,1999), a partitioning of the entities into sets aboveand below the mean (to test for persistence ofabnormal profit, for example) would result in eachpartition having 60 percent of its entities that werenot statistically different from 60 percent of thosein the other partition—or from the mean.

Having identified the performance position ofeach entity at each level in each period, thesepositions were then used in an ordinal regressionto determine how well knowledge of performanceposition at each level predicted performance posi-tion at the business segment level. In order todetermine this information for business segmentposition as predictor, the independent variableswere lagged one period. In so doing, no assump-tions were made about causality regarding man-agement action or inaction—or the effects of otherforces. The general assumption was mutatis mutan-dis and the context was aimed at addressing aversion of Schmalensee’s (1985: 349) notion ofcontingent information, to whit: what does know-ing about A tell you about B? In our case A wasthe performance position of industry, corporation,or business segment and B was the subsequent per-formance position of the business segment. Thusour measure of the importance of a factor wasthe degree to which knowledge of the performanceposition of the factor aided prediction of a businesssegment’s performance position.

ASSUMPTIONS

One of the points made in various ways in McGa-han and Porter (2005) concerned the nature ofassumptions. One of the most important of theseis the assumption of ceteris paribus. This is abasic assumption in much of economic and strate-gic management research—so basic that it is oftentaken for granted and not invoked (Bierens andSwanson, 2000). But it does not have to be invokedto be applicable. In discussing ceteris paribusBlack (1997: 58) notes, ‘All statements in eco-nomics include such a clause, either explicitly orimplicitly: it is impossible to list all the thingswhich could alter.’ The ceteris paribus assump-tion is also often embedded in the models andtechniques we use in research, so the choice oftechnique includes the (hidden) baggage of adopt-ing the assumption. In the discussion at hand, theceteris paribus assumption is important becauseVCA embodies this assumption. The reason thatgenetics research using VCA employs a variety ofgenetic relations to provide a context for interpret-ing VCA results (Brush and Bromiley, 1997) is totry to better satisfy the ceteris paribus assumption.Interpretation of VCA results in strategic man-agement research usually eschews the contextualrelations and directly equates importance of a fac-tor with the size of its variance component. Thisequation is valid under the assumption of ceterisparibus and not necessarily otherwise.

A second area in which assumptions aresmuggled into strategic management researchby our choice of methodologies is with regardto the assumptions that accompany wholeclasses of statistical techniques. Parametricstatistical techniques (to which class VCAbelongs), the most commonly used, requireassumptions about the nature of distributionsand other population parameters. With regardto parametric statistical techniques Siegel andCastellan (1988: 3) state, ‘Such techniques produceconclusions which contain qualifiers, e.g., “If theassumptions regarding the shape of the populationdistribution(s) are valid, then we may conclude that. . .”’ These assumptions about parameters (e.g.,distributions are Gaussian) and the subsequentqualifiers on results are very often not stated—forexample in the reporting of VCA results. As aparametric technique a possible problem in theuse of VCA is violation of normality (Allisonet al., 1999). When the analysis is run on samples,

Copyright 2005 John Wiley & Sons, Ltd. Strat. Mgmt. J., 26: 881–886 (2005)

Research Notes and Commentaries 883

this is less likely—because of the central limittheorem—but may arise when population data areemployed.

An alternative to using parametric statisticaltechniques and accepting their assumptions isto employ non-parametric statistical techniques.‘Since populations do not always meet the assump-tions underlying parametric tests, we frequentlyneed inferential procedures whose validity does notdepend on rigid assumptions. Non-parametric sta-tistical procedures fill this need in many instances,since they are valid under very general assump-tions’ (Daniel, 1978: 15). For these reasons non-parametric techniques were employed in Ruefliand Wiggins (2003) and thus our methodology,in contrast to most antecedent studies in the area,involved no parametric assumptions.

ORDINAL CATEGORICAL ANALYSIS

McGahan and Porter (2005) are correct in not-ing that, in comparison to using cardinal data,employing ordinal categories as in Ruefli and Wig-gins (2003) results in the loss of some informa-tion. However, it should be noted that the cardinalto ordinal transformation also results in the lossof some noise—so while ordinal techniques maybe less detailed, they may be more accurate. InMcGahan and Porter (2005) in point seven it isstated that ‘This approach is fundamentally inac-curate because an observation with exactly thesame level of profitability year after year may becategorized differently in each year. For exam-ple, an industry with exactly the same perfor-mance level in two years can be classified indifferent categories just because of differences inthe other industries.’ A problem with the notionespoused in this quote is that to have an indus-try maintain its position vis a vis other industriessimply by keeping its performance level constantstrongly invokes an assumption of ceteris paribus.In the world of managers, where mutatis mutan-dis rules, constant levels of cardinal performancemay yield improved, worsened, or constant rank-ing—depending on the performance of other busi-ness segments/corporations/industries. This notionof relative position is, in fact, central to strategicmanagement (Porter, 1980, 1985) in the form ofcomparative advantage, relative cost position, etc.The use of ordinal categories in Ruefli and Wiggins(2003) was made in the spirit of the importance of

relative position in strategic management and toavoid having to assume ceteris paribus.

VARIANCE EXPLAINED AS AMEASURE OF IMPORTANCE

McGahan and Porter (2005) assert that variance isan appropriate method of measuring performancedifferences. We agree with that—but with quali-fications. First, if the objective is to measure theimportance of a factor to the performance of a sys-tem, then dimensions involved must be clearly andconsistently stated, particularly with relation to thedependent variable. Researchers employing VCAand ANOVA in the research area at issue have,at times, suggested that a larger coefficient on avariance component is an indication that the coef-ficient is more important to profitability when whatis meant is that in the model the factor accountsfor a higher proportion of variance in profitabil-ity. For example in the abstract of McGahan andPorter (2002: 834) it is stated that, ‘The purpose ofthe analysis is to identify the importance of year,industry, corporate parent, and business-specificeffects on accounting profitability among operatingbusinesses across sectors.’ On occasion the con-fusion is compounded by other researchers sum-marizing prior research. For example, Chang andSingh (2000) in discussing Schmalensee (1985)state, ‘He found that industry effects were the mostimportant factor in explaining a firm’s profitability. . .’ In these studies, while the dependent variablein the equations is profitability, the results are interms of variance of profitability—but are confus-ingly stated in terms of effect on profitability.

Even when is it clear that what is being dis-cussed is amount of variance explained, there isa presumption that more variance explained ismore important than less variance explained. (Andbecause lower variance in the dependent variableassociated with a factor means lower varianceexplained, there is a relationship between vari-ance and variance explained.) Even if we grantthat explaining more variance is preferred byresearchers, we still must recognize that, for man-agers, more variance in profitability is not alwaysdesirable.

Examples of the confusion that the usualinterpretation of variance components causes areevident in the strategic management literatureinvolving VCA. For example, Rumelt (1991: 182)

Copyright 2005 John Wiley & Sons, Ltd. Strat. Mgmt. J., 26: 881–886 (2005)

884 T. W. Ruefli and R. R. Wiggins

states, ‘Given the extent of the literature oncorporate strategy, corporate culture, the numberof consulting firms that specialize in corporatemanagement, and the focus on senior corporateleaders in the business world, it is surprisingto find only vanishingly small corporate effectsin these data. This result, first observed bySchmalensee, remains a puzzle and deservesfurther investigation.’ This quote makes theassumption that if a factor, e.g., corporate effects,is important, it should have a large variancecomponent when, in fact, if some of the items citedin the first part of Rumelt’s quote are effective, thefactor in question should have a small variancecomponent.

In the same vein, McGahan and Porter (2005)note that including only diversified firms in thisarea of research biases results because diversifiedfirms have lower variance in profitability. Revealedpreference would suggest that a possible decreasein variance is likely one of the outcomes thatmotivates managers to diversify. If, as Rumeltobserved, managers spend extensive amounts oftime and resources managing corporate factors toreduce the variance of performance and if, as inpoint nine of McGahan and Porter (2005), they aresuccessful, is there not a logical conclusion that formanagers a small variance coefficient (reflectiveof lower variance in performance) on corporateeffects is more important than a large coefficient?If researchers via a statistical technique attachmore importance to higher variance in performancewhile managers of corporation attach more impor-tance to lower variance in performance, we arethen left with the dilemma of whose measure ofimportance should be employed in this area ofresearch: that of researchers or that of managers.

Our reservations about variance analysis are notbased only on the assumptions of random effectsin VCA. Contrary to the assertion in McGahan andPorter (2005), point six, we do cite and acknowl-edge the contributions of Rumelt (1991) as well asMcGahan (1999) (which we erroneously referredto as McGahan and Porter, 1999) and McGahanand Porter (1997, 2002, 2003)—although, in Rue-fli and Wiggins’ (2003) article, we cited the lattertwo references as 2002a1 and 2002b, respectively.However, ANOVA techniques depend on the same

1 What we cited as McGahan and Porter (2002a) was a workingpaper that is correctly referenced here as McGahan and Porter(2003).

assumptions of ceteris paribus, and rely on similarparametric assumptions, and therefore are subjectto producing results with the same interpretationproblems. Amount of variance explained by a fac-tor in an ANOVA as a measure of importanceraises the same ‘puzzle’ (to use the term in Rumelt,1991) as does the size of a variance component.

MANAGERIAL EFFECTS

Our emphasis on allowing for the effects ofmanagerial efficacy in our models was both forverisimilitude and because antecedent studies inthis area had assumed them away with implicitceteris paribus assumptions. We do not claim to beable to identify managerial drivers of performance(nor do our models rely on such drivers)—the non-parametric techniques and ordinal categories weemployed merely allow for the possibility of man-agerial action, i.e., mutatis mutandis is assumed.Whether managerial actions (or inactions) increaseor lower variance is not germane to our method-ology—what is important is that our methodologyaccommodates such circumstances and generatesthe results in a robust and less ambiguous format.

PERSISTENCE

McGahan and Porter (2005) raise the issue of per-sistence in its third point. In Ruefli and Wiggins(2003), to determine the persistence of abnormalperformance at each level, a relatively straight-forward approach was adopted. Only abnormallyhigh profits were examined and the probability ofan entity leaving the superior performance stratain each time period was calculated—the lowerthat probability, the higher the persistence andvice versa. In contrast, the more usual autoregres-sive techniques (McGahan and Porter, 1999, 2003;Mueller, 1986; Waring, 1996) implicitly assume apriori that the objective is to determine the rateof decay of abnormal (both high and low) profits.Such techniques confound the behavior of supe-rior, average, and inferior performers; i.e., they donot separate the rate of convergence of superiorperformers from that of inferior performers. So, inthese traditional studies we don’t know, for exam-ple, how much of the persistence reported is dueto poor performers continuing to underperform.Autoregressive techniques also require parametric

Copyright 2005 John Wiley & Sons, Ltd. Strat. Mgmt. J., 26: 881–886 (2005)

Research Notes and Commentaries 885

Table 1. Mean ROA for Compustat business segments

Year 84 85 86 87 88 89 90 91 92 93 94 95

Mean ROA 0.136 0.109 0.077 0.068 0.058 0.032 0.035 0.039 0.032 0.016 0.018 0.004

assumptions, and do not (without adjustment ofthe data) compensate for overall trends in per-formance (i.e., they cannot tell whether inferior(superior) performers are converging to the meanor whether the mean is declining (rising) to meetthe inferior (superior) performers). With respect tothat last point, Barber and Lyon (1996) showedthat the accounting performance data for corpo-rate level in the COMPUSTAT database has beentrending down over time. Table 1 shows the sametrend for the data employed in Ruefli and Wiggins(2003) and in, among other studies, McGahan andPorter (1999).

AGGREGATION AND DIVERSIFI-CATION

In the comment’s eighth point it is stated that Rue-fli and Wiggins (2003) ‘aggregate their data to the3-digit level despite the availability of Compustatdata at the 4-digit level.’ The point goes on to notethat higher levels of aggregation in industry defi-nition make it more likely that industry effects areobscured. In fact, as both the text and the results inTable 72 (Ruefli and Wiggins, 2003: 874) indicate,we analyzed the data at both 3- and 4-digit levels.Further, due to the robustness of our methodology,aggregation had only a slight effect on the size ofour industry log-odds coefficient and no effect onits relative position or significance, with no sig-nificant effect on the model’s performance. Also,apropos of the ninth point in McGahan and Porter(2005) (though it correctly exempts Ruefli andWiggins, 2003), wherein it is stated that using onlydiversified firms inflates the corporate effect, as thetext and Table 7 (Ruefli and Wiggins, 2003: 874)indicate, we analyzed both diversified only andmixed samples and our methodology yielded noimpact on the industry coefficient but a decrease inthe size of the corporate log-odds coefficient—but,

2 This table contained a reference to McGahan and Porter (2003)which was an error; the reference should have been to McGa-han and Porter (2002b) in that article, which here is correctlyreferenced as McGahan and Porter (2002).

again, no change in its relative position or signifi-cance nor impact on the performance of the model.Thus our methodology is robust, not only in termsof level of industry aggregation, but also in termsof level of diversification in the sample.

CONCLUSION

Arriving at a determination of the relative impor-tance of industry, corporate, and business segmenteffects of performance should be an importantobjective in strategic management research. Weagree, for somewhat different reasons, with McGa-han and Porter (2003b: 850) in their conclusionregarding further replications of VCA studies inthis area: ‘While there are ways to continue tolearn from this research, its limits suggest that thetime has come to explore whole new approaches.’In fact, Ruefli and Wiggins (2003) were clearlyin this spirit, and clearly indicate there are bothadvantages and costs to employing whole newapproaches that avoid the drawbacks of traditionalmethodologies. We avoided some confusion inher-ent in antecedent approaches and gained robustnessand directness but lost the precision of cardinalmeasures and had to settle for much rougher ordi-nal categorical results. We join with McGahan andPorter in their call for additional new approachesthat will shed more light on this important topic.

ACKNOWLEDGEMENTS

The first author would like to acknowledge thesupport of the Daniel Stuart Endowment and theHerb Kelleher Center for Entrepreneurship at theMcCombs School of Business. The second authorwould like to acknowledge funding support fromthe Fogelman College of Business and Economics.

REFERENCES

Allison DB, Neale MC, Zannolli R, Schork NJ, AmosCI, Blangero J. 1999. Testing the robustness of

Copyright 2005 John Wiley & Sons, Ltd. Strat. Mgmt. J., 26: 881–886 (2005)

886 T. W. Ruefli and R. R. Wiggins

the likelihood ratio test in a variance-componentquantitative-trait locimapping procedure. AmericanJournal of Human Genetics 65: 531–544.

Barber BM, Lyon JD. 1996. Detecting abnormal operat-ing performance: the empirical power and specifica-tion of test statistics. Journal of Financial Economics41(3): 359–399.

Bierens HJ, Swanson NR. 2000. The econometricconsequences of the ceteris paribus condition ineconomic theory. Journal of Econometrics 95(2):225–253.

Black J. 1997. A Dictionary of Economics . OxfordUniversity Press: Oxford.

Brush TH, Bromiley P. 1997. What does a smallcorporate effect mean? A variance componentssimulation of corporate and business effects. StrategicManagement Journal 18(10): 825–835.

Chang S-J, Singh H. 2000. Corporate and industryeffects on business unit competitive position. StrategicManagement Journal 21(12): 739–752.

Daniel WW. 1978. Applied Nonparametric Statistics .Houghton Mifflin: Boston, MA.

McGahan AM 1999. The performance of U.S. corpo-rations: 1981–1994. Journal of Industrial Economics47: 373–398.

McGahan AM, Porter ME. 1997. How much doesindustry matter really? Strategic Management Journal ,Summer Special Issue 18: 15–30.

McGahan AM, Porter ME. 1999. The persistence ofshocks to profitability. Review of Economics andStatistics 81(1): 143–153.

McGahan AM, Porter ME. 2002. What do we knowabout variance in accounting profitability? Manage-ment Science 48(7): 834–851.

McGahan AM, Porter ME. 2003. The emergence and sus-tainability of abnormal profits. Strategic Organization1(1): 79–108.

McGahan AM, Porter ME. 2005. Comment on ‘Industry,corporate and business-segment effects and businessperformance: a non-parametric approach’. StrategicManagement Journal 26(9): 873–880.

Mueller DC. 1986. Profits in the Long Run . CambridgeUniversity Press: Cambridge, U.K.

Porter ME. 1980. Competitive Strategy: Techniques forAnalyzing Industries and Competitors . Free Press:New York.

Porter ME. 1985. Competitive Advantage: Creating andSustaining Superior Performance. Free Press: NewYork.

Ruefli TW, Wiggins RR. 2000. Longitudinal performancestratification: an iterative Kolmogorov–Smirnov ap-proach. Management Science 46(5): 685–692.

Ruefli TW, Wiggins RR. 2003. Industry, corporate andbusiness-segment effects and business performance:a non-parametric approach. Strategic ManagementJournal 24(9): 861–879.

Rumelt R. 1991. How much does industry matter?Strategic Management Journal 12(3): 167–185.

Schmalensee R. 1985. Do markets differ much? AmericanEconomic Review 75: 341–351.

Siegel S, Castellan NJ Jr. 1988. Nonparametric Statisticsfor the Behavioral Sciences (2nd edn). McGraw-Hill:New York.

Waring G. 1996. Industry differences in the persistence offirm-specific returns. American Economic Review 86:1253–1265.

Copyright 2005 John Wiley & Sons, Ltd. Strat. Mgmt. J., 26: 881–886 (2005)