the chicago guide to writing about multivariate analysis, 2nd edition. implementing...

The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.

Implementing “generalization, example, exception”:

Behind-the-scenes work for summarizing a pattern

Jane E. Miller, PhD


Overview

• Steps– Display the data in a chart– Indentify the dimensions of the comparison– Choose a representative example– Characterize the pattern– Identify exceptions– Write the description

• “Generalization, example, exception” useful for– Three-way patterns– Interaction patterns calculated from multivariate coefficients


Objectives of a GEE

• GEE = “generalization, example, exception”• Aims to describe the forest, not the trees, in a complex

pattern involving a lot of numbers– Which aspects can be summarized?– Which aspects deviate from that summary?

• In a three-way association, describe the pattern among– Two independent variables (IVs) and a dependent variable

(DV)– One IV and a series of DVs


Display the data in a chart

• Create a chart of the association• Charts are the easiest way to see patterns– Which values are highest and by how much compared to

other values?– Whether trends are up, down, or level


Table and corresponding chart

Mode of transmission

Language spoken at home/language used on

questionnaire

English

Spanish /English ques.

Spanish /Spanish

ques. Likely modes of transmission

Sexual intercourse with an infected person 93.6 87.5 95.0

Shared needles for IV drug use 92.4 90.6 65.0

Pregnant mother to baby 89.5 75.0 80.0

Blood transfusion from infected person 87.5 81.3 60.0

Knowledge about AIDS transmission, by language spoken at home and ability to speak English, New Jersey, 1998


Design issues for your chart

• Draw it to scale, so you can assess – Directionality – Size of differences and trends across values

• Organize variables in the order you will describe them– Theoretical criteria– Empirical order– See podcast on organizing data in tables and charts


Use a spreadsheet to create your chart

• A spreadsheet can – Test different ways of organizing the variables in your

chart– Calculate magnitude of differences across values for

use in your description• Difference (subtraction)• Ratios (e.g., relative risks)• Percentage change


Identify dimensions of the comparison

• One dimension for each variable or set of variables in your table or chart

• In a table, one dimension for each– Row– Column – Panel

• In a chart, one dimension for each – Axis – Legend item


Dimensions in a three-way table• Moving down the rows within one column• Moving across the columns within one row

Language spoken at home/language used on questionnaire

Mode of transmission EnglishSpanish

/English ques. Spanish

/Spanish ques. Likely modes of transmission Sexual intercourse with an infected person 93.6 87.5 95.0 Shared needles for IV drug use 92.4 90.6 65.0 Pregnant mother to baby 89.5 75.0 80.0 Blood transfusion from infected person 87.5 81.3 60.0



Dimensions in a clustered bar chart• Moving across clusters within one bar color• Moving across bar colors within one cluster


Dimensions in a multiple-line chart• Moving left to right along one line• Moving vertically across lines at one value of the x

variable


Choose a representative example

• Identify one representative example as the basis for each generalization about the shape of the pattern– Must characterize many if not most values in the pattern– Not an arbitrary choice!

• Find one value of the first independent variable to use to describe the general shape of the association between the second IV and the DV. E.g.,– Hold one IV constant while you vary the other– Don’t vary values of both IVs at the same time


What does “representative” mean?

• A representative example of a general pattern will be characteristic of the association between the second IV and the DV for many if not all subgroups of that first IV. E.g., – A region that shows a time trend in prices that is similar to

that observed in several other regions– A topic for which the rank order of language groups in test

performance is the same


Possible starting points for identifying representative examples

• Theoretically based– A value of particular interest for your research question– A summary measure that combines results for the various

component variables• Useful if comparing across a series of related outcomes

• Empirically based– Start at one end of an empirically organized axis, column, or

row– Overall sample, for a comparison across groups– Modal (most common) group


Checking for representativeness of example in a three-way table

• Describe the pattern across columns within one row• Test whether the direction and magnitude of that pattern

also applies to other rows




/Spanish ques. Likely modes of transmission Sexual intercourse with an infected person 93.6 87.5 95.0 Shared needles for IV drug use 92.4 90.6 65.0 Pregnant mother to baby 89.5 75.0 80.0 Blood transfusion from infected person 87.5 81.3 60.0Mean percentage of “likely” questions correct 91.7 83.6 75.0



Checking for representativeness of example in a three-way table

• Describe the pattern down the rows within one column• Test whether the direction and magnitude of that pattern

also applies to other column




/Spanish ques. Likely modes of transmission Sexual intercourse with an infected person 93.6 87.5 95.0 Shared needles for IV drug use 92.4 90.6 65.0 Pregnant mother to baby 89.5 75.0 80.0 Blood transfusion from infected person 87.5 81.3 60.0



Checking representativeness of an example in a multiple-line chart

• Describe the pattern cutting vertically across lines (regions) at one value of the x variable (year)

• Check whether that rank order of regions also applies to the other dates


Checking representativeness of an example in a multiple-line chart

• Describe the pattern along one line moving left to right• Test whether that pattern applies to the other lines


Trial and error in identifying a representative example

• Might have to try several groups or values to find one that has a pattern that is similar to that in other groups

• If your initial choice isn’t similar to any of the others in terms of direction or magnitude, it might be an exception

• See whether other values have a pattern that can be generalized


Characterize the pattern

• For your representative value of one independent variable, characterize these aspects of the association between the other independent variable and the dependent variable:– Direction– Magnitude– Statistical significance


Characterizing direction in across-sectional comparison

• Which categories of the independent variable (IV) have the highest and lowest values of the dependent variable (DV)?

• For multiple category IVs, describe the rank order of categories according to values of the DV


Characterizing direction for trends

• Describe whether each line slopes – Upward– Downward– Is approximately level– Follows a standard statistical distribution • e.g., a normal curve

– Asymptotically approaches some value


Characterizing magnitude

• Do some behind-the-scenes math to measure the size of differences between values you are comparing. E.g.,– Difference (subtraction or OLS coefficient)– Ratio (division, or odds ratio from logit model)– Percentage difference or change

• See chapters on types of quantitative comparisons to help you choose the kind of calculation that best suits your objectives


Take notes about direction and magnitude

• On a hardcopy chart, make notes about:– Direction: Use “>,” “<,” “=,” or arrows – Size: Jot results of calculations to compare values

• To help you later write a correct, complete description, mention– Units– Type of calculation

– Shape: Verbs, adjectives, adverbs, analogies, or metaphors that capture the shape of the pattern

– Statistical significance: of specific contrasts, marked with • Symbols• Shading (highlighter)

• Don’t worry about full sentences at this point


Identify exceptions

• If parts of your table or chart depart substantially from the general pattern you have identified, they are exceptions

• Exceptions can occur in terms of– Direction– Size– Statistical significance


Exceptions in direction of a trend• The median sales prices dipped in the early 1990s in

the West, but continued upward or remained level in each of the other three regions– The West was the exception


Exceptions in magnitudeof a cross-sectional difference

• The difference between odds ratios across racial groups is much larger among the non-poor than in the other two income groups (compare brackets 1 and 2 to bracket 3)– Generalize based on the two

income groups for which the racial gap in ER use is similar (poor and near poor)

– Non-poor are the exception


Exceptions in statistical significance

• Examples of this type of exception– All but one of the coefficients in a regression model are

statistically significant at p < 0.05– Only one of the coefficients in a regression model is

statistically significant• The generalization is lack of statistical significance

– In stratified models of the same outcome for several groups, the coefficient on one or more specific independent variables are statistically significant only for one group


Write the description

• Organize the material into paragraphs– One paragraph for each major pattern to be described

• Write topic sentences for each paragraph– Introduce the substantive concepts to be discussed in that paragraph

• Write evidentiary sentences to describe the– General pattern– Any exceptions

• In those descriptions, mention – Direction– Magnitude– Statistical significance


Wording for a generalization• If a pattern characterizes many of the numbers you are

summarizing, introduce the generalization with– “In general, . . .” – “Typically, . . .” – “By and large . . .”

– If the pattern characterizes all groups, say so!– “In every region studied, . . .” – “Throughout the entire period of observation . . .” – “Regardless of [characteristic], . . .”


Wording to present the example

• To document a pattern– Introduce numeric evidence– Refer to the table or chart that reports all of the numbers

• E.g., – “For instance, . . . [numeric contrast, with the W’s specified].”– “For example, as shown in Figure X . . . [numeric fact, with its

W’s attached].”

• Reminder: “the W’s” refer to when, where, who, and what


Wording to introduce exception(s)

• To introduce exceptions to a general pattern, write– “An exception [to that pattern] is . . . ” – “On the other hand, . . .” – “However, . . .”

• E.g., “In seven out of 10 years studied, [general pattern and example]. However, in the other three years, [contrasting pattern = exception].”


Wording for exceptions in direction

• Some exceptions are literally the opposite of the pattern described in the generalization. E.g., – A falling rather than rising trend– A deficit instead of excess for one group compared to another

• Having described the direction of the general pattern, introduce such exceptions with phrases such as:– “On the contrary, . . .” – “Conversely, . . .”


Summary

• Use a systematic, step-by-step approach to identify which parts of a broad pattern are– Similar to one another– Different from one another

• In terms of direction, magnitude, and statistical significance

• Create a chart or table that you refer to in your prose description of that pattern

• Organize material into substantively cohesive paragraphs• Use vocabulary and phrasing to differentiate general

patterns from exceptions


Suggested resources• Chapter 2 and appendix A in

– Miller, J.E., 2004. The Chicago Guide to Writing about Numbers. (“WA#”)

OR– Miller, J.E., 2013. The Chicago Guide to Writing about Multivariate

Analysis, 2nd edition. (“WAMA II”)

• Chapter 7 in WA# or chapter 6 in WAMA II on creating effective charts

• Chapter 5 in WA# or chapters 8 and 9 in WAMA II on types of quantitative comparisons

• Chapter 9 of WA#, section on three-way associations • Chapter 16 of WAMA II, on interactions


Suggested online resources

• Podcasts on– Reporting one number– Comparing two numbers or series of numbers– Summarizing a pattern with many numbers– Organizing data in tables and charts– Visualizing shapes of interaction patterns– Writing prose to present results of an interaction


Suggested practice exercises

• Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.– Question #9 in problem set for chapter 2– Question #9 in problem set for chapter 6– Suggested course extensions for • chapter 2

– “Reviewing” exercise #7– “Writing” and “revising” exercise #4

• chapter 14– “Applying statistics and writing” exercises #3 and 4– “Revising” exercise #3


Contact information

Jane E. Miller, [email protected]

Online materials available athttp://press.uchicago.edu/books/miller/multivariate/index.html

the chicago guide to writing about multivariate analysis, 2nd edition. implementing...

Documents