NATIONAL CENTER FOR EDUCATION STATISTICS

Technical Report May 1997

National Household Education Survey

Measuring Participation in Adult Education

U.S. Department of EducationOffice of Educational Research and Improvement NCES 97-341

NATIONAL CENTER FOR EDUCATION STATISTICS

Technical Report May 1997

National Household Education Survey

Measuring Participation in Adult Education

Mary A. CollinsJ. Michael BrickKwang KimWestat, Inc.

Peter Stowe, Adult Education Project OfficerNational Center for Education Statistics

U.S. Department of EducationOffice of Educational Research and Improvement NCES 97-341

U.S. Department of EducationRichard W. RileySecretary

Office of Educational Research and ImprovementRamon C. CortinesActing Assistant Secretary

National Center for Education StatisticsPascal D. Forgione, Jr.Commissioner

The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, andreporting data related to education in the United States and other nations. It fulfills a congressional mandate tocollect, collate, analyze, and report full and complete statistics on the condition of education in the United States;conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist stateand local education agencies in improving their statistical systems; and review and report on education activitiesin foreign countries.

NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete,and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S.Department of Education, the Congress, the states, other education policymakers, practitioners, data users, andthe general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a variety ofaudiences. You, as our customer, are the best judge of our success in communicating information effectively. Ifyou have any comments or suggestions about this or any other NCES product or report, we would like to hearfrom you. Please direct your comments to:

National Center for Education StatisticsOffice of Educational Research and ImprovementU.S. Department of Education555 New Jersey Avenue, NWWashington, DC 20208-5574

May 1997

Suggested Citation

U.S. Department of Education. National Center for Education Statistics. National HouseholdEducation Survey. Measuring Participation in Adult Education, NCES 97-341, by Mary A. Collins,J. Michael Brick, and Kwang Kim. Adult Education Project Officer, Peter Stowe. Washington,DC: 1997.

Contact:Peter Stowe(202) 219-2099(e-mail) nhes@ed.govhttp://www.ed.gov/NCES/NHES

iii

TABLE OF CONTENTS

Introduction ................................ ................................ ................................ ..................... 1

The National Household Education Survey................................ .................... 1The Current Population Survey................................ ................................ ..... 2What is Adult Education?................................ ................................ .............. 3

Adult Education Participation Rates................................ ................................ .................. 4

Population Coverage and Selection Methods................................ ................................ ...... 6

Telephone Coverage and Estimates of Participation................................ ....... 7Selection of Adults................................ ................................ ........................ 11

Survey Nonresponse................................ ................................ ................................ .......... 13

Proxy Reports of Adult Education Participation................................ ................................ . 15

Survey Context................................ ................................ ................................ ................. 20

Questionnaire Wording................................ ................................ ................. 20Interview Status: Sponsorship and Survey Focus................................ .......... 22

Summary ................................ ................................ ................................ ..................... 24

References ................................ ................................ ................................ ..................... 28

Appendix A ................................ ................................ ................................ ..................... A-1

iv

LIST OF TABLES

1 Estimated number of adults and percentage of those participating inadult education from the Current Population Survey and the NationalHousehold Education Survey: 1984-1995 ................................ .................... 5

2 Comparisons of AE participation rates from NHES:91, CPS:92,and NHES:95 Splice sample................................ ................................ .......... 6

3 Estimated percent of persons by telephone status and estimatedcoverage bias for adults................................ ................................ ................. 10

4 Number and percent of sampled adults whose participation statuschanged from screener to extended interview completion byScreener respondent: NHES:95................................ ................................ .... 13

5 Number of sampled adults, by self/proxy reporting status anddemographic characteristics: CPS:92................................ ............................ 17

6 Percent of adults participating in adult education activities, bysource of report: 1992................................ ................................ .................. 18

7 Logistic regression parameters for a main effects model of AEparticipation, excluding full-time degree programs................................ ......... 19

8 Observed and predicted participation rates if all adults hadself-reported, by type of participation: CPS:92................................ .............. 19

9 Summary of factors associated with possible biases in estimates ofadult education participation, by survey and possible size of bias................... 25

1

Introduction

The measurement of participation in adulteducation (AE) activities presents a number ofdefinitional and methodological challenges.Zemsky and Shapiro (1993) note that estimatesof job-related training differ "not just widely, butwildly," and liken the attempt to provide accuratetraining estimates to "measuring a mirage."Estimates of participation in a broad range ofadult education activities from the NationalHousehold Education Survey and the CurrentPopulation Survey also vary (Hill 1987; Korb,Chandler, and West 1991; Kim et al. 1995). Theways in which a given research study definesadult education and the methods used to samplepersons and collect information about theireducational experiences may have a substantialimpact on the observed rates of participation.

The purpose of this report is to compare twostudies that have yielded disparate rates of AEparticipation at several points in time, and toexamine the definitional and methodologicalfactors that may lead to these different results.The two studies on which this analysis focusesare the National Household Education Survey(NHES) and the Current Population Survey(CPS). The NHES included adult educationcomponents in 1991 and 1995. The CPScollected information on participation in adulteducation every three years from 1969 through1984, and the 1984 participation rate is discussedin this report. In addition, the CPS included abrief set of questions on adult education in theOctober 1992 educational supplement; thoseitems replicated items used to estimate the AEparticipation rate in the NHES:91. Included inthis study are examinations of populationcoverage and the selection of adults forinterviews, survey nonresponse, the use of proxyreports of adult education participation, andsurvey context. In each of these investigations,the appropriate data sets are used to examine thespecific issue at hand, because not all of theselected data sets lend themselves to each of themethodological factors explored in this research.Before proceeding further, brief overviews ofeach of these data collection systems arepresented.

The National Household EducationSurvey

The National Household Education Survey(NHES) is a data collection system of theNational Center for Education Statistics (NCES),which has as its legislative mission the collectionand publication of data on the condition ofeducation in the Nation. The NHES isspecifically designed to support this mission byproviding information on those educational issuesthat can be addressed by contacting householdsrather than schools or other educationalinstitutions. The NHES provides descriptivedata on the educational activities of the U.S.population and offers policy makers, researchers,and educators a variety of statistics on thecondition of education in the U.S. By focusingspecifically on educational topics of interest, theNHES permits the collection of more in-depthinformation than can be collected in supplementsto existing household surveys.

The NHES is a telephone survey of thenoninstitutionalized civilian population of theU.S. Households are selected for the surveyusing random digit dialing (RDD) methods, anddata are collected using computer-assistedtelephone interviewing (CATI) procedures.About 45,000 to 64,000 households are screenedfor each administration, and individuals withinhouseholds who meet predetermined criteria aresampled for more detailed or extended interviews.The data are weighted to make estimates of theentire population. The NHES survey for a givenyear typically consists of a Screener, whichcollects household composition and demographicdata, and extended interviews on two substantivecomponents addressing education-related topics.In order to assess data item reliability and informfuture NHES surveys, each administration alsoincludes data quality activities, such as areinterview with a subsample of respondents.The primary purpose of the NHES is to conductrepeated measurements of the same phenomenaat different points in time and one-time surveyson topics of interest to the Department ofEducation. Throughout its history, the NHEShas collected data in ways that permit estimates

2

to be tracked across time. This includesrepeating topical components on a rotating basisin order to provide comparative data acrosssurvey years. In addition, each administration ofthe NHES has benefited from experiences withprevious cycles, resulting in enhancements to thesurvey procedures and content. Thus, while thesurvey affords the opportunity for trackingphenomena across time, it is also dynamic inaddressing new issues and including conceptualand methodological refinements.

Full-scale implementations of the NHES havebeen conducted in 1991, 1993, 1995, and 1996.Topics addressed by the NHES:91 were earlychildhood education and adult education. TheNHES:93 collected information about schoolreadiness and school safety and discipline. The1991 components were repeated for theNHES:95, addressing early childhood programparticipation and adult education. Bothcomponents underwent substantial redesign toincorporate new issues and develop newmeasurement approaches. In the NHES:96, thetopical components were parent/familyinvolvement in education and civic involvement,and an expanded screening approach providedstate-level estimates of public library use byhousehold.

In the NHES:91, 60,314 households werescreened and the total number of completed adulteducation interviews was 12,568, which includedinterviews with 9,774 adults who participated inadult education in the previous 12 months and2,794 interviews with adults who did notparticipate. In the NHES:95, of the 45,465households with completed screener interviews,19,722 adult education interviews werecompleted, including 11,713 AE participants and8,009 AE nonparticipants. The NHES:95 alsocontained a methodological experiment to helpevaluate whether any changes between 1991 and1995 were the result of changes in the design ofthe interview or were due to changes in theparticipation rate over time. This methodologicalexperiment, called the NHES:95 Splice sample,used the interview design of the NHES:91. Atotal of 4,082 Splice sample interviews wascompleted in the NHES:95. More details on the

NHES:95 data collection and public use datafiles are given in National Household EducationSurvey of 1995: Adult Education Data FileUser’s Manual (Collins et al. 1996).

The Current Population Survey

The Current Population Survey (CPS) is amonthly survey of households conducted by theU.S. Bureau of the Census. The main purpose ofthe CPS is to provide estimates of employment,unemployment, and other characteristics of thelabor force. Through the use of supplements tothe main CPS interview, estimates are obtainedfor other topics of interest. Among these aresupplements pertaining to educational issues.For example, the October School EnrollmentSupplement provides specific information on theenrollment status of individuals in the populationby demographic and socioeconomiccharacteristics. A supplement on participation inadult education activities was conducted everythree years from 1969 through 1984 (Hill 1987).Questions on adult education were also includedin October 1992, and this brief set of questionsreplicated the items used to estimate theparticipation rate in the NHES:91; the samequestions were used in the NHES:95 AE Splicesample interview described above.

The CPS is a sample of the civilian,noninstitutionalized population of the UnitedStates, and estimates from the survey excludeactive duty military personnel. The CPS is basedon a multistage stratified probability sample.The first stage involves sampling PrimarySampling Units (PSUs) from homogeneous stratawhere the PSUs are counties or groups ofcounties. In the second stage a cluster ofaddresses is sampled from each of the sampledPSUs. The persons living in the sampledaddresses are interviewed in a rotation scheme.The addresses are included in the sample for fourconsecutive months, excluded for eight months,then included for four more months. Typically,the first interview and the fifth interviews aredone in person and the other interviews are doneby telephone, where possible. More details onthe design of the CPS is given in U.S.Department of Commerce (1978).

3

The total monthly sample size in 1992 wasapproximately 60,000 housing units, of whichabout 57,000 were interviewed. The householdrespondent must be a knowledgeable householdmember aged 14 years or more; this respondentprovides information for each household member.In some cases, individuals are selected to respondto supplements about their activities orcharacteristics. However, in some supplementsother knowledgeable household members (proxyrespondents) may be allowed to respond if thetargeted person is not available to complete thesupplementary interview. In the October CPSsupplement on adult education, 42 percent of theresponses were by proxies. About two-thirds ofall CPS interviews are conducted by telephone;the remainder are conducted by personalinterviews. In the October CPS:92, the month inwhich the adult education supplement wasconducted, 70 percent of the interviews wereconducted by telephone.1

The triennial Adult Education Supplement to theCPS (1969-1984) was collected for adults whowere reported by the household respondent ashaving participated in adult education activitiesin the previous 12 months. Persons who were notreported as participants were not asked tocomplete the AE supplement. Furthermore, thesampled adult became ineligible if that adult saidthey had not participated in adult education in thepast year. In the AE supplement itself, adultswere asked about their participation in a varietyof educational activities. Those who hadparticipated in activities other than, or in additionto, full-time enrollment were asked to describethe four most recent courses in which theyparticipated (Hill 1987).

As noted above, in October 1992, data fromadults about their participation in AE activitieswere again collected in the education supplementto the CPS. Adults were defined as civilian,noninstitutionalized persons 16 years or olderwho were not enrolled full-time in elementary or

1Special tabulation from the 1992 Current Population

Survey.

secondary school. In this survey, theparticipation items were those used in theNHES:91 AE component. The householdrespondent answered for all adult householdmembers in most cases, so some data are proxyresponses. The question only included the itemsdefining participation and excluded the moredetailed information on adult education collectedin the NHES. All adults were eligible for thissupplement without asking the householdrespondent any preliminary screening questionsabout the adult education participation of otheradults in the household.

What is Adult Education?

The measurement of adult education participationis problematic in part because the learningactivities that fall within the realm of adulteducation can be defined in a number of ways.Darkenwald and Merriam (1982) define adulteducation as participation in systematic learningactivities for the purpose of acquiring newknowledge or skills by persons who haveassumed adult social roles. This definitionpotentially encompasses a large range ofactivities that may include basic skills education,activities leading to academic credentials, work-related courses or training, and courses orprograms taken for personal development,general interest, or recreation. In addition, themethod of participation may include traditionalschooling; participation in classes, workshops orseminars; self-directed learning throughcorrespondence courses or computer- or video-based tutorials; or courses offered by televisionor on video or audio tapes (Knowles 1980).

Some studies of adult education focusspecifically on basic skills education (Andersonand Darkenwald 1979), while others examineonly work-related training (Frazis, Herz, andHorrigan 1995). Still others look at educationalactivities that include participation in structured,formal learning activities with an instructor,excluding self-teaching activities using books,television, video tapes, or audio tapes.

4

The estimate of the percentage of adults whoparticipate in adult education activities, (i.e.,participation rates, may differ by whether theyinclude full-time or part-time activities, or both,and by whether they include courses leading tocollege degrees or other postsecondarycredentials)(Hill 1987; Korb, Chandler, and West1991; Kim et al. 1995). The broadest definitionincludes academic, vocational, work-related, anda vocational courses taken in classroom settings,in informal settings, and those involving the useof television, radio, video or audio tapes, or self-directed learning. The studies under investigationhere, the NHES and the CPS, each took broadand inclusive approaches to defining adulteducation. Details about how adult educationwas defined and how those definitions affectedestimates of participation rates are discussed inlater sections of this report.

First, some estimates of AE participation fromthe NHES (1991 and 1995) and CPS (1984 and1992) components on adult education arepresented. Within the context of these disparateparticipation rates, subsequent sections of thereport examine the definitions of adult educationused and other methodological and operationaldifferences that could account for some of thedifferences in the survey estimates.

Adult Education Participation Rates

Table 1 shows the total number of adults and theadult education participation rates for five datasources, in chronological order:

The 1984 Adult Education Supplement tothe CPS (CPS:84);

The 1991 National Household EducationSurvey (NHES:91);

The 1992 Adult Education Supplement tothe CPS (CPS:92);The 1995 National Household EducationSurvey (NHES:95); andA special methodological study includedin the NHES:95 in which a Splice sample(a separate, supplementary sample) wasused to examine whether any differencesin observed NHES rates were due toquestionnaire changes made to the AEquestionnaire between the 1991 and 1995surveys (NHES:95 Splice).

Table 1 shows participation rates based on twodifferent sets of AE activities: participation ratesin any adult education activity and participationrates in adult education activity excluding full-time credential programs. The second type ofparticipation rate includes participation incredential programs on a part-time basis or inactivities other than credential programs on apart-time or full-time basis. The AEparticipation rates from these five data sources,in general, show a trend toward greaterparticipation over time. This observed growth isconsistent with the findings of the triennial adulteducation supplement to the CPS conducted from1969 through 1984, which reflected modest butsignificant growth over the 15-year period (Hill1987). However, the estimates from the CPS:84and the CPS:92 are markedly lower than thosefrom the NHES:91 and the NHES:95. Onemight be tempted to attribute the differencesbetween the 1984 CPS participation rate of 14percent and the NHES rates of 33 and 40 percentas reflecting continued growth in AEparticipation. However, the CPS:92 estimatedrate of 20.5 percent, when considered in thischronological context, suggests that differencesare not due to growth alone, and that there areaspects of the measurement of AE participationthat are contributing to these disparate estimates.

5

Table 1.—Estimated number of adults1 and percentage of those participating in adult education from theCurrent Population Survey and the National Household Education Survey: 1984-1995

Participation Rate CPS:84 NHES:91 CPS:92 NHES:95NHES:95

Splice

Number of adults (in thousands)

Any adult education activity

Adult education activity excludingfull-time degree programs

172,583

--2

14.0% (0.1)

181,800

37.9% (0.7)

33.0% (0.7)

169,772

24.0% (2.5)

20.5% (0.5)

189,576

44.3% (0.5)

40.2% (0.5)

189,912

44.6% (1.0)

40.0% (1.0)

1Adults include persons age 16 and older not enrolled in elementary or secondary school and not on activity duty in the ArmedForces. Data from the 1984 survey (Hill 1987) include persons age 17 and older.2Not available from the published report (Hill 1987).NOTE: Standard error in parentheses.SOURCES: U.S. Department of Commerce, Bureau of Census, Current Population Survey (CPS), 1984 Adult EducationSupplement, 1992 Adult Education Supplement; U.S. Department of Education, National Center for Education Statistics,National Household Education Survey (NHES), spring 1991, spring 1995, spring 1995 Splice Sample Interview.

The specific types of adult education that wereasked about in the NHES and CPS wereexamined in order to identify the types ofactivities that contributed to the differences inoverall participation estimates. Table 2 showsthe participation rates for the specific types offull-time and part-time AE activities. TheCPS:92 estimates are significantly lower than theNHES:91 estimates for full-time participation inliteracy programs and "other educationalprograms." In addition, the CPS:92 estimates offull-time participation are significantly lowerthan NHES:95 Splice estimates for associate'sdegree programs and ESL programs. Moredifferences in participation rates are observedwhen part-time AE activities are examined. TheCPS:92 estimates are significantly lower than the

NHES:91 and NHES:95 Splice estimates forpart-time continuing education, mail or televisioncourses, private instruction or tutoring, andemployer training. In addition, the CPS:92estimates are lower than NHES:95 Spliceestimates for part-time basic skills education and"other" part-time educational activities.

These findings indicate that the reporting ofactivities outside of formal schooling is moreproblematic than reports of formal schooling.This is consistent with the results of theNHES:95 reinterview program and a responsebias study examining reports of AE participationin the NHES:95 AE interview (Brick et al.1996a, Brick et al. 1996b).

6

Table 2.—Comparisons of AE participation rates from NHES:91, CPS:92 , and NHES:95 Splice Sample

Participation items Participation rates in

NHES:91

Participation rates in

CPS:92

Participation rates in

NHES:95 Splice

Full-time educational programs

An elementary, junior high, or high school 11.7% (1.9) 13.3% (1.4) 13.6% (3.9)

An associate's degree program 23.1 (1.2) 18.6 (1.7) 29.4 (2.9)

A bachelor's or advanced degree program 63.4 (1.6) 61.1 (3.0) 58.7 (2.9)

A vocational/occupational training program 17.2 (1.1) 15.2 (1.5) 18.8 (2.6)

An adult literacy or basic skills program 3.6 ( .6) 1.2 ( .4) 5.6 (2.1)

An English as a Second Language program 2.3 ( .4) 1.1 ( .4) 4.4 (1.1)

Other educational program 2.4 ( .6) 5.0 ( .9) 2.1 (1.0)

Part-time educational programs

Continuing education 12.0% ( .4) 6.4% (1.0) 13.8% ( .5)

Courses by mail, TV, radio 1.6 ( .1) .8 ( .3) 2.7 ( .4)

Private instruction 2.9 ( .2) .9 ( .4) 4.5 ( .5)

Training given by employer 22.5 ( .6) 11.1 ( .3) 29.9 (1.0)

Instruction in basic skills 1.3 ( .1) .6 ( .3) 3.5 ( .4)

Instruction in ESL .6 ( .5) .6 ( .3) 1.6 ( .3)

Other educational activity .6 ( .1) 1.7 ( .5) .1 ( .1)

NOTE: Standard error in parentheses.SOURCES: U.S. Department of Commerce, Bureau of Census, Current Population Survey (CPS) 1992 Adult EducationSupplement; U.S. Department of Education, National Center for Education Statistics, National Household Education Survey(NHES), spring 1991, spring 1995 Splice Sample Interview.

In the sections that follow, we discuss severalaspects of the methods used in these five surveysand their potential effects on the observedparticipation rates. The specific aspects that areconsidered include:

population coverage and selectionmethods;nonresponse;proxy reporting of adult educationparticipation; and

issues related to the context of thesurveys.

Population Coverage and SelectionMethods

In this section, factors associated with surveycoverage of households and persons withinhouseholds and the selection of adults withinhouseholds for adult education data collection arediscussed. The estimates for both the NHES andthe CPS include all civilian, noninstitutionalized

7

adults, age 16 and older,2 in the 50 states and theDistrict of Columbia who were not enrolled inelementary or secondary school. However, thereare differences between the CPS and the NHES,and differences between years within eachsurvey, in the sampling of households and theselection of adults for the collection ofinformation on AE participation. The NHES isan RDD telephone survey and includes onlytelephone households, whereas the CPS is asample of both telephone and nontelephonehouseholds. Thus, the CPS covers adults livingin households without telephones who are notsampled in the NHES.

Population coverage is not related to the coverageof households alone. The coverage of personswithin households is also of concern. Aninvestigation of within household coverage,comparing in-person and telephone surveys, wasconducted by Maklan and Waksberg (1988).They found that within-household coverage is nomore problematic in telephone surveys than in-person household surveys.

In addition to population coverage, there are alsovariations in the selection of adults within thehouseholds. In the CPS:84, only those adultsidentified as participants by the householdrespondent who completed the CPS AE screeningitems were sampled for the AE supplement. Inthe CPS:92, participation data were collected forall adults, but proxy reporting was used for mostadults other than the household respondent. Inthe NHES:91 and NHES:95, adults wereclassified as participants and nonparticipants atthe screening stage and both types of adults weresampled for AE extended interviews, but atdifferent sampling rates. In the NHES:91,participants were sampled at rates up to 14

2The 1984 CPS estimates are for adults age 17 and older.

The NHES:91 sample includes only 17 eligible 16-year-olds out of a total adult sample of 12,568, and theNHES:95 sample includes only 13 eligible 16-year-oldsout of a sample of 19,722. The NHES:95 participationrate is 40 percent regardless of whether 16-year-olds areincluded or not. Thus, the absence of 16-year-olds fromthe 1984 estimates is not problematic in consideringdifferences in rates.

times greater than nonparticipants. In theNHES:95, both participation status and highschool completion were considered in sampling,but there was less variation in the sampling rateswith rates for participants and nonparticipantsvarying by a factor of five at a maximum.

Below, the implications of differences betweenthe CPS and the NHES in terms of the coverageof households and the potential for lack ofcoverage of nontelephone households in theNHES as explanatory factors in the estimates ofAE participation are examined. Following this,the implications of differences in selecting adultswithin households are addressed.

Telephone Coverage and Estimates ofParticipation

The telephone coverage of adults in the NHES:95is discussed in some detail in Brick (1996).Some of the important findings of that work aresummarized here. Approximately 94 percent ofall persons live in households with telephones,according to data from the March 1992 CPS.The percentage of persons who live in householdswith telephones varies somewhat bycharacteristics of the populations considered.For example, while 95 percent of all adults age16 years and older live in telephone households,only 87 percent of black adults and 88 percent ofHispanic adults live in telephone households,according to these CPS estimates. The differencein telephone coverage rates by characteristics ofthe population is one of the factors that leads tobiases in statistics based on data collected frompersons in telephone households only.

The term bias has a specific technical definitionin this context; bias is the expected differencebetween the estimates from the survey and theactual population value. For example, if alltelephone households were included in the surveyand responded to the interview, the differencebetween the estimate from the survey and theactual population value (which includes theresponses of persons living in nontelephonehouseholds) is the bias due to incompletecoverage. Since the NHES is based on a sample,

8

the bias is defined as the expected or averagevalue of this difference over all possible samples.

Coverage bias, the bias due to failure to give allpersons a chance to be selected in the sample, canbe substantial when two conditions hold. First,the differences between the characteristics in thecovered population and the uncovered populationmust be relatively large. For example, considerestimating the percentage of persons enrolled in agiven type of program. If the percentage enrolledis nearly identical in both the covered anduncovered populations, then the bias for theestimate will be negligible.

Second, the proportion of the population notcovered by the survey must be large compared tothe size of the estimates. If only 2 percent of thepopulation is not covered, estimates thatcomprise 20 or 30 percent of the population willnot be greatly affected, even if the differences inthe characteristics between the covered anduncovered populations are relatively large. It isimportant to realize that this condition requiresthat the proportion not covered must be largerelative to the size of the estimates. If theestimate is for a small domain or subgroup, theneven a small undercoverage problem can result inimportant biases if the differences in thecharacteristic between the covered and uncoveredpopulations are large. Statistics for dropoutsfrom high school, a small subgroup, suffer fromthis problem (Brick, Burke, and West 1992).

The bias of an estimate can be expressedmathematically to show the relationships betweenthe bias and the two factors discussed above.The bias is given by

{ }Bias Y P E Y Yt n t n ( ) = ( )− (1)

where Yt is the estimated characteristic based onthe telephone households only, Pn is theproportion of nontelephone households, Yn is theestimated characteristic based on thenontelephone households, and E is theexpectation operator for averaging over allpossible samples.

The undercoverage rate or proportion of adults inhouseholds without telephones, Pn, can beestimated directly from the CPS since it coversboth telephone and nontelephone households.For adults eligible for the AE component of theNHES:91 and the NHES:95 (civilians who are16 years old or older and not currently enrolled inelementary or secondary school), 5.6 percentlived in nontelephone households, according tothe October 1992 CPS.

The differences in the characteristics of personsin telephone and nontelephone households havebeen explored for a number of topics by differentauthors. In general, these studies have shownthat having a telephone is highly related tosocioeconomic status and lifestyles (Smith 1990).Thornberry and Massey (1988) assessedestimates of health characteristics and foundmany health and health-related characteristics ofpersons in nontelephone households weresignificantly different from those of persons intelephone households. Brick, Burke, and West(1992), Brick (1992), and Brick (1996) studied avariety of estimates for education statistics.They found the differences between persons intelephone and nontelephone households forenrollment statistics were typically smaller thanthose reported by Thornberry and Massey(1988). However, for some statistics, such asthose for high school dropouts, the differenceswere very large.

In order to examine the impact that excludingnontelephone households in the NHES could haveon differences in estimates of participation inadult education, the adult education supplementto the October 1992 CPS was used to estimatethe differences in the characteristics of persons intelephone and nontelephone households. Thesedifferences and the estimates of the percent ofadults not covered can then be used in equation(1) to estimate the bias. However, this mayoverestimate the bias in the NHES because someweighting adjustments are made in the NHES toreduce the coverage bias. In particular, theNHES estimates are raked to population totals ofadults living in both telephone and nontelephone

9

households.3 By applying the NHES rakingadjustment procedures to CPS data for telephonehouseholds, the amount of residual bias can beexamined after the weighting adjustments.

Percentage distributions for selectedcharacteristics of adults were tabulated from theOctober 1992 CPS supplement (table 3). Thetabulation for adults was limited to those adultsmeeting the eligibility criteria for the AEcomponents of the NHES, that is, civilians whoare 16 years old or older and not currentlyenrolled in elementary or secondary school. Thefirst three columns of table 3 show the estimatedpercentage distributions for persons in telephonehouseholds, persons in nontelephone households,and persons in all households. The fourthcolumn in the table is the estimated coveragebias, or the difference between the estimate forpersons in telephone households and the estimatefor persons in all households; this is the algebraicequivalent of the bias given by equation (1).

The estimates in table 3 show that while thecharacteristics of adults living in telephone andnontelephone households may be very different,the resulting biases are generally small becausethe undercoverage rates (Pn) are relatively low.Thus, even though adults in telephone householdswere twice as likely as those in nontelephonehouseholds to have been involved in any activitiesin the last year (24.8 percent and 11.8 percent,respectively, from the row labeled “Involved inadult education activity in the past year” in table

3Raking is an iterative procedure that ensures that survey

weights sum to known population totals. It is a calibrationprocedure and is closely related to poststratification.

3), the bias in the estimate due to undercoveragefor this statistic is only 0.6 percent (the estimatefor all adults, 24.2 percent, and the estimate forany adults in telephone households is 24.8percent). Only 5 of the statistics in the table haveestimated biases greater than 0.5 percent.

Due to the potential biases resulting fromundercoverage, the standard practice in theNHES is to make statistical adjustments ofsurvey weights to compensate, to the extentpossible, for undercoverage. The NHESadjustments that are specifically developed tocompensate for the undercoverage are raking orpoststratification to known control totals thatcontain counts of persons living in both telephoneand nontelephone households. The goal of theseadjustments is to make the estimates from thesurvey consistent with known totals (referred toas control totals), to partially correct forundercoverage bias, and to reduce the variance ofthe estimates.

Control totals of the number of persons in bothtelephone and nontelephone households were firstproduced from the CPS file for adults eligible forthe AE interview. The weights for the CPSrespondents from telephone households were thenraked to these control totals to produce adjustedweights that summed to the total number ofpersons in both telephone and nontelephonehouseholds. The responses from persons intelephone households were used with theseadjusted weights to produce adjusted estimates.More details are given in Brick (1996).

10

Table 3.—Estimated percent of persons by telephone status and estimated coverage bias for adults

CharacteristicTelephonehouseholds

Non-telephone

householdsAll

householdsCoverage

bias

Adjustedtelephone

households

Adjustedcoverage

biasHighest education

Less than 12th 17.2 40.8 18.6 -1.4 17.9 -0.712th grade 36.6 36.7 36.6 0.0 36.5 -0.11 or 2 years college 18.2 9.8 17.7 0.5 18.0 0.33-4 years of college 15.0 3.4 14.5 0.5 14.7 0.2More than 4 years of college 7.6 1.2 7.3 0.3 7.4 0.1

Persons aged 15 to 24 yearsHigh school graduate 88.1 53.7 85.4 2.7 87.6 2.2Complete high school by equivalency test 4.8 8.7 5.1 -0.3 4.9 -0.2Speak language other than English at home 13.5 19.6 14.0 -0.5 14.0 0.0Do not speak English well 19.4 39.8 21.6 -2.2 20.1 -1.5Ever had condition affecting ability to learn 3.2 3.3 3.2 0.0 3.5 0.3

Of those in college, enrolled full-time 65.1 69.7 65.2 -0.1 65.5 0.3Now taking business, vocational, technical, 2.3 1.8 2.2 0.1 2.3 0.1Reported for person

Self 52.6 60.8 53.0 -0.4 52.8 -0.2Parent 7.7 4.2 7.6 0.1 7.8 0.2Spouse 23.8 13.9 23.3 0.5 23.4 0.1Other relative 5.6 7.3 5.7 -0.1 5.6 -0.1Nonrelative 2.7 6.4 2.9 -0.2 2.8 -0.1

Involved in adult education activity in last yearAny activity 24.8 11.8 24.2 0.6 24.8 0.6Any activity, excluding full-time college 20.0 9.9 19.6 0.4 19.9 0.3Any activity, excluding college 17.6 8.1 17.2 0.4 17.5 0.3

Enrolled full-time in last year 6.1 4.4 6.0 0.1 6.3 0.3In elementary or high school program 12.8 13.5 12.8 0.0 12.7 -0.1In associated degree program 17.9 21.7 18.1 -0.2 18.0 -0.1In bachelor's or advanced program 60.5 31.2 59.5 1.0 60.2 0.7In vocation or occupational program 14.2 26.8 14.7 -0.5 14.4 -0.3In literacy or ABE program 0.9 5.1 1.1 -0.2 1.0 -0.1In ESL program 1.0 2.1 1.0 0.0 1.1 0.1In other program 4.8 7.0 4.8 0.0 4.8 0.0

Enrolled part-time in college in last year 3.9 1.8 3.8 0.1 3.9 0.1In other continuing ed or noncredit program 6.2 2.1 6.0 0.2 6.1 0.1In mail, TV, radio, or newspaper courses 0.8 0.4 0.8 0.0 0.8 0.0In private instruction 0.9 0.4 0.9 0.0 0.9 0.0In program by employer, union, community

organization 10.8 3.5 10.5 0.3 10.7 0.2In basic math or English 0.6 1.0 0.6 0.0 0.6 0.0In ESL 0.5 0.7 0.5 0.0 0.5 0.0In other organized activity 1.6 0.8 1.6 0.0 1.6 0.0

11

Table 3.—Estimated percent of persons by telephone status and estimated coverage bias for adults—Continued

CharacteristicTelephonehouseholds

Non-telephone

householdsAll

householdsCoverage

bias

Adjustedtelephone

households

Adjustedoverage

biasInvolved in AE activity in last year, by race

Hispanic adultsAny activity 20.2 11.5 19.2 1.0 20.0 0.8

Black, non-Hispanic adultsAny activity 19.3 11.2 18.3 1.0 19.1 0.8Any activity, excluding full-time college 14.5 8.7 13.8 0.7 14.3 0.5Any activity, excluding college 12.4 6.4 11.6 0.7 12.2 0.5

Nonblack, non-Hispanic adultsAny activity 25.9 12.3 25.4 0.4 26.0 0.5Any activity, excluding full-time college 21.0 10.3 20.7 0.3 21.0 0.3Any activity, excluding college 18.6 8.9 18.3 0.3 18.6 0.3

NOTE: In this analyses, persons with missing values on participation variables are treated as nonparticipants.SOURCE: Special tabulations from the 1992 Current Population Survey (CPS).

The adjusted estimates are shown in the next tolast column in table 3. The adjusted bias in thesestatistics is given in the last column, where thebias is the difference between the adjustedestimate for those in telephone households andthe estimate from all households. As before, anegative coverage bias indicates that the estimateis smaller than the estimate based on allhouseholds.

These estimates indicate that telephone coveragebias accounts for at most a very minorcomponent of the differences in the observedadult education participation rates for the NHESAE components and the CPS AE supplements.In particular, the coverage bias for the percentageof adults participating in any adult educationexcluding full-time college is estimated to be only0.6 percent (in bold in table 3). Thus, thepercentage of the differences in the ratesattributable to coverage bias is very small,especially when compared to the very largedifferences in the estimates from the CPS andNHES AE surveys.

Selection of Adults

Although the major focus is on differencesbetween the CPS:92 and the NHES estimates, thedifferences between the CPS:84 and theNHES:91 estimates initiated this type ofinvestigation and do have some implications forthe comparison of the CPS:92 and the NHESdifferences. A factor that might account forsome of the difference between the 1984estimates and later estimates is the variation inprocedures for selecting adults withinhouseholds, that is, person-level samplingprocedures. In both the NHES:91 and theCPS:84, a knowledgeable adult member of thehousehold was asked a set of screening itemsabout the adult education participation of eachadult in the household; a single screening itemwas used for the NHES:95. In the CPS:84, theAE supplement was administered only to thoseadults who were classified at the screening stageas participating in adult education (Hill 1987).In the CPS:92, information was collected aboutall adults, but proxy reporting was permitted.

12

The CPS:92 can also be considered a screeninginterview like the CPS:84, but without a follow-up interview with any of the adults. Both theseapproaches are considered later. In the NHES:91and NHES:95, adults screened as participantsand as nonparticipants were sampled for adulteducation interviews with varying probabilities.For the Splice sample of adults in the NHES:95,participation status was not a factor in either thescreening or the sampling procedures.

In some cases, the household respondent mayhave answered the screening items incorrectly forother members of the household because of lackof knowledge or because of recall error. Becausenonparticipants were not selected for the CPS:84AE supplement, these adults did not have anopportunity to report on their own activities, ifany. Some screener respondents who reportedthat they did not participate might have recalledactivities while completing the full interview ifthey had been given an opportunity to respond.This would have resulted in the adults beingreclassified as participants.

The results of the NHES:91 and the NHES:95support the notion that the estimatedparticipation rate is reduced by excluding adultsfrom the supplement based on participationstatus as reported in screening questions. In theNHES, both participants and nonparticipantswere reclassified in terms of their participationstatus based on their responses at the extendedinterview level. We refer to the percentagechange in status as the "switching rate."

In the NHES:91, about 11 percent of thosesampled as AE nonparticipants who completedextended interviews were found to be AEparticipants and about 14 percent of personssampled as participants who completed extendedinterviewers were identified as nonparticipants.In the CPS:84, the persons who were identifiedas nonparticipants were not sampled, so they hadno chance to switch to participants in theextended interview. This scheme results in alower estimate of the participation rate. Bothweighted and unweighted estimates show thatself-reporters were more likely to remain in thesame status from the screener to the extended

interview, whether they were initially identifiedas participants or nonparticipants.

Higher percentages of sampled adults were"switchers" in the NHES:95 (table 4). Of thosesampled as participants, 17 percent were foundto be nonparticipants. Of those sampled asnonparticipants, 23 percent were found to beparticipants. This higher rate of change is mostlikely related to a change in the screeningquestion. In the NHES:95, a single briefscreening question was used compared with alonger list of types of AE activities used in theNHES:91. This change in the screeningprocedure was made as a result of objectionsfrom screener respondents in the NHES:91 tohaving to respond to the full list of activitiesrepeatedly for each adult in the household.Because both participants and nonparticipantswere sampled in the NHES collections, the lossof accuracy associated with the single screeningitem was not considered a major issue, especiallyin light of the opportunity to reduce respondentannoyance that might otherwise lead tononparticipation in the study.

The NHES:95 provides some additional insighton the "switching" phenomenon. Table 4 givesboth weighted and unweighted estimates ofswitching, by the type of switching and thescreener respondent (self or other). Notsurprisingly, the extent to which switchingoccurred in the NHES:95 was associated withwhether the sampled adult was the screenerrespondent or whether another adult in thehousehold had answered the screeningparticipation question about the sampled adult,with the switching rate greater for otherhousehold responders than for self responders.

The observed switching of status from thescreening level to the extended interview levelsuggests that the CPS:84 participation rate wouldhave been higher if persons who were initiallyidentified by the household respondent asnonparticipants had an opportunity to report forthemselves. Applying the NHES:91 switchingrate to the CPS:84, the participation rate for theCPS:84 would have been about 23 percent. Thisincrease of 9 percent from 14 to 23 percent

13

results when the estimated 86 percent ofnonparticipants in the CPS:84 are assumed to beswitchers at the same rate as found in theNHES:91 (11 percent times 86 percent = 9percent). The estimate of 23 percent is stillsubstantially below the NHES:91 estimate of 32percent, but accounts for a very large percentageincrease in the participation rate.

Evidence from the CPS:92 indicates that the"switching" phenomenon alone does not accountfor the differences between the NHES and CPSestimates. As we saw earlier, the CPS:92estimate is 20.5 percent if no adjustments aremade to the estimates. If the NHES:91 switchingrates are applied to the CPS:92 estimates, theadjusted estimate is 26.4 percent (20.5 percenttimes 86 percent plus 79.5 percent times 11percent =26.4 percent). The difference betweenthe adjusted estimate of 23 percent for theCPS:84 and the adjusted estimate of 26.4 percent

suggests there was not a substantial increase inparticipation rates in the 8 years between theCPS surveys.A different approach to adjusting the CPS:92 isto model the reporting of those who did not reportfor themselves. This proxy adjustment isdiscussed later, but it leads to an adjustedCPS:92 estimate of participation of 25 percent,which is very similar to the switching adjustedestimate. Both approaches to adjusting theCPS:92 reveal little substantial increase in adulteducation participation between 1984 and 1992.This contradicts the common impression thatthere has been an increase during this period.These results raise doubts about the validity ofthe estimate, especially the unadjusted estimate,from the CPS:92, even in comparison to theCPS:84 estimates, which have othermethodological problems.

Table 4.—Number and percent of sampled adults whose participation status changed from screener toextended interview completion by Screener respondent: NHES:95

Estimates Sample size

Number(in thousands)

Percentswitching

Standarderror Number

Percentswitching

All adults

Sampled as participantsSampled as nonparticipants

68,971120,605

1622

.38

.5811,8907,832

1723

Screens respondents (self)

Sampled as participantsSampled as nonparticipants

39,68473,979

1119

.41

.687,0294,778

1121

Not screener respondent (other)

Sampled as participantsSampled as nonparticipants

29,28746,626

2426

.65

.804,8613,084

2527

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey(NHES), spring 1995.

Survey Nonresponse

Just as undercoverage can result in biases in theestimates, nonresponse, or the failure to obtain aresponse from a sampled unit, may also cause the

estimates from a survey to differ from thepopulation values (Kalton and Kasprzyk 1986;Lessler and Kalsbeek 1992). Both the NHESand the CPS suffer from nonresponse, althoughthe rates of responding are different in the

14

surveys. The potential impact of nonresponse onthe differences in participation rate estimates inthe surveys is investigated below.

Nonresponse is often classified as either unitnonresponse or item nonresponse. Unitnonresponse occurs when all or nearly all thedata from the sampled unit (person, household)can not be obtained. Unit nonresponse mayhappen for several reasons, such as being unableto contact the unit, the respondent refusing, or theinability to complete the interview in the datacollection time period. Item nonresponse occurswhen most of the data are obtained, but somequestions are not answered. As with unitnonresponse, there may be many reasons for itemnonresponse, such as the respondent not wantingto answer a particular question like income,interviewer error in not asking the question, orinconsistent responses to items that are detectedin later editing. Because item nonresponse isgenerally small compared to unit nonresponse inboth the NHES and CPS, unit nonresponse is thefocus of this examination of nonresponse as apotential source of differences between thesurvey estimates. However, those cases forwhich all AE items are missing in the CPS:92(about 5.2 percent) are treated as unitnonresponse for this analysis.

The bias due to nonresponse can be expressed inthe same way as the bias due to undercoverage.The bias of the estimate due to nonresponse canbe written as equation (1), by simply changingthe subscript t (for telephone households) to r(for respondents), and the subscript n nowindicates nonrespondents while in equation (1) itreferred to nontelephone households. Thus, thenonresponse bias equation is

Bias y P E y yr n r n( ) { ( )}= − . (2)

As with coverage, this bias is a function of boththe nonresponse rate ( Pn ) and the difference incharacteristics of the respondents and thenonrespondents ( )y yr n− . As the nonresponserate becomes larger, the bias increases even if thedifference between the characteristics for

respondents and nonrespondents remainsconstant.

The response rates for the NHES:91 and theNHES:95 AE components were 69 percent and59 percent respectively, while for the NHES:95Splice sample the response rate was 64 percent.These response rates are weighted by theappropriate probabilities of selection and accountfor all levels of unit nonresponse (Screener andextended interviews). The National HouseholdEducation Survey of 1995: Adult EducationData File User’s Manual (Collins et al. 1996)contains more information on the response ratesfor the NHES.

The data from CPS are also subject tononresponse at two levels: failure to complete thecore interview and failure to complete thesupplement. The response rate to the coreinterview for the CPS is about 95.6 percent (U.S.Department of Commerce 1993). The percent ofadults with completed AE supplement data is94.8 percent. This yields a net response rate of90.6 percent for the CPS:92 AE data. Thisresponse rate will be assumed for the adulteducation supplement for purposes of thisassessment.

The second component of nonresponse bias is thedifferences in characteristics between therespondents and nonrespondents. For bothsurveys, it is very hard to quantify thesedifferences because so little is known about thenonrespondents. Efforts to identify groups ofhouseholds and adults that responded at differentrates in the NHES (Brick and Broeneforthcoming) did not reveal any majordifferences, but these efforts were limited by thelack of data on nonrespondents.

Although it is not possible to obtain muchinformation on differences between respondentsand nonrespondents at the screening stage, somedata are available on those sampled for theextended interviews. It is clear from theNHES:91 that response to the AE interview wasrelated to whether or not the householdrespondent said the sampled adult was aparticipant in adult education activities. In the

15

NHES:91, the completion rate (the percent ofsampled adults who completed the extended AEinterview) for participants was 90 percent andthe completion rate for nonparticipants was 82percent. In the NHES:95, the participantcompletion rate was 84 percent, while it was 77percent for nonparticipants.

The difference in completion rates byparticipation status could result in a significantbias if nothing were done to address thedifferential bias. However, the differentialresponse rates were recognized in the NHES:91and the NHES:95 and nonresponse adjustmentswere made to the weights to account for thedifference and minimize the potential bias. Nosuch adjustments were made in the NHES:95Splice sample because the participation status ofthe sampled adults was not asked in the Screener.In order to make this type of adjustment, data forboth respondents and nonrespondents are needed,and if participation status is not collected in theScreener, this adjustment cannot be made. In theCPS, no special weighting adjustments weremade for the supplement, but this may not havebeen as necessary due to the higher response ratein that study.

To quantify the potential nonresponse bias, anexample assuming a large difference between thecharacteristics of respondents andnonrespondents is explored to provide an upperbound on this source of bias. Assume that thepercent of respondents that are participants is 40percent and the percent of nonrespondents thatare participants is 20 percent. This is probably alarge difference given the estimates from thevarious surveys. Applying equation (2) to theseestimates with the NHES:95 nonresponse rate of40 percent results in an estimated bias of 8percent {40%(40%-20%)=8%}, implying thatthe participation rate from the NHES:95 could beas low as 32 percent (40% - 8 % =32%). Thesame approach can be applied to the CPS,assuming the same difference betweenrespondents and nonrespondents, but with anonresponse rate of 15 percent. This results in abias in the CPS:92 estimate of 3 percent{15%(40%-20%)=3%}, implying an even lowerparticipation rate of 21 percent (24% - 3% =

21%). The lower bias in the CPS is due to thehigher response rate and is the rationale for thestatement above that special weightingadjustments to account for the different responserates of participants and nonparticipants may notbe as important in the CPS.

This exercise sheds some light on the potentialfor nonresponse bias in the estimates of theNHES and CPS. Estimates from both surveysmay be biased, but the nonresponse bias in theNHES is probably larger than that in the CPSestimates. However, even assuming a very largedifference between the characteristics ofrespondents and nonrespondents as was done inthe example, the differential bias is probably 5percent or less (8 percent in the NHES minus 3percent in the CPS). While differential bias maybe a large contributing factor, this alone does notaccount for the large disparity in the estimatesfrom the two surveys.

Proxy Reports of Adult EducationParticipation

Proxy reporting of adults' participation in AEactivities is allowed for some purposes in each ofthe data sets under consideration in this report,with the exception of the NHES:95 Spliceinterview. However, the surveys differ as towhether the proxy report is used only to selectadults for interviewing or for classifying the AEparticipation status of the sampled adult. Asnoted above, in the CPS:84, a single householdrespondent reported on the participation status ofall adults in the households and adults wereselected for the AE supplement based on theseproxy responses. Only those reported to beparticipants in the screening items were sampledfor the supplement. The interview was conductedwith the sampled adult, but proxy respondentscould also respond. In the CPS:92, participationitems were asked about all adults, but proxyresponses were accepted for adults who were notavailable for interviews.In the NHES:91 and the NHES:95, a householdrespondent to the screener reported on theparticipation of all adults in the household;however, the sample for the AE interviews

16

included both participants and nonparticipants.The participation status of the adults wasdetermined by responses to the extendedinterview that could only be answered by thesampled adult. No proxy responses were allowedfor extended interviews. In the NHES:95 Spliceinterview, one adult was sampled per householdwithout first asking about participation, so proxyreporting was not involved. Only the sampledadult was eligible to answer the AE Spliceinterviews; proxy responses were not accepted.Thus, the use of proxy respondents in the NHESwas very different from that in the CPS.

Previous research on the reports of proxyrespondents indicates that there may be somereporting error associated with this type ofprocedure (Moore 1988), but the results are notconsistent or based on studies that permit validestimation of the impact of proxy responses. In astudy of reports of health status and satisfactionwith medical care, Epstein et al. (1989) founddifferences between self and proxy reports, andfound variation in the extent and direction ofdiscrepancies based on such factors as theamount of time the proxy spent with the subject.In the field test of the NHES survey system, ananalysis of proxy reports of the educationalstatus of 14- to 21-year-olds indicated that thecorrespondence of proxy reports and self reportsof dropout rates varied according to the timing ofthe dropout event (e.g., whether it had occurredin the previous year) and the age of the youth(Mohadjer et al. 1992). Thus, the issue of proxyreporting may be an important area to examinewhen assessing differences in AE participationrates.

In the CPS:84, persons who were identified asnonparticipants at the screening stage were notselected for the AE supplement. As a result, it isnot possible to compare proxy and self-reports asa possible source of error.

In the NHES:91, if the AE interview respondentwas the screener respondent, the participationquestions (which were the same in the screenerand extended interview) were not repeated, sothat the adult did not have the opportunity tochange his or her status under these conditions.

All of the "switching" in the NHES:91 (describedearlier) occurred when someone other than theadult sampled for the AE interview completed thescreening items. In the NHES:95, a single briefscreening item on participation in adult educationwas used for sampling purposes, and this itemdid not provide detail on what was included asadult education. As a result, the data collected inthe NHES:95 does not support an analysis of theimpact of proxy reporting on participation rates.

Only the CPS:92 provides an opportunity toobserve differences in participation rates basedon proxy and self reports. In an assessment ofmethodological evidence concerning the qualityof proxy reports in surveys, Moore (1988)questioned the adequacy of extant work,primarily on the basis of measurement problemsand lack of random assignment to self report orproxy report status. The CPS:92 data indicatethat self and proxy reporting does not occurrandomly, supporting Moore's concerns. About58 percent of adults answered the AE itemsthemselves, and 42 percent of cases had proxyrespondents. In about 25 percent of proxyresponse cases, the proxy respondent was thespouse of the adult for whom he or she wasreporting.4

Table 5 shows the percentages of adults forwhom proxy responses were obtained for theadult education items, by several characteristics.Males, younger adults, Hispanic adults, thosewith more education, those who were employed,and those with higher incomes were more likelyto have proxy respondents than females, olderadults, non-Hispanic adults, those with lesseducation, those not employed, and those withlower incomes. Because age, education, andlabor force status are also associated with AEparticipation (Korb, Chandler, and West 1991;Kim et al. 1995) these differences in rates ofproxy reporting may confound any reportingerror associated with the failure to obtainresponses from the adult himself or herself. Forexample, higher rates of participation for thosewith proxy respondents could be related to

4 Special tabulations from the 1992 Current Population

Survey (CPS).

17

differences in reporting by proxies, or to the factthat proxy reports are more likely for highlyeducated adults who participate in adult

education at a higher rate than adults with lesseducation.

Table 5.—Number of sampled adults, by self/proxy reporting status and demographic characteristics:CPS:92

Characteristics Number of self-reportersNumber with proxy

reporters Percent proxy reportingTotal

Age Less than 30 years 30 to 49 years 50 years or older

Sex Male Female

Race Black Hispanic Non-black, non-Hispanic

Highest education Less than high school diploma High school diploma Some college/vocational school Bachelor's degree or more

Employment status Not employed Employed

Household income (1) Less than $10,000 $10,000 or more

Household income (2) Less than $25,000 $25,001 to $50,000 More than $50,000

56,425

10,24824,12322,054

20,29936,126

5,512 3,31247,601

11,31519,11213,34512,653

19,91736,508

9,43346,992

25,71416,83710,468

40,645

11,41516,42011,810

24,83715,808

3,475 3,00934,161

6,71315,87511,157 6,900

16,40324,242

3,61637,029

13,73813,63210,645

42

534137

5530

394842

37454635

4540

2844

354550

NOTE: Numbers equal the unweighted number of adults.SOURCE: Special tabulations from the 1992 Current Population Survey (CPS), Adult Education Supplement.

Table 6 shows the participation rates of adultswho responded for themselves and those who hadproxy respondents. The differences in

participation rates for adults who responded forthemselves and those with proxy respondents aresmall (3 to 4 percent). Self-reporters were

18

slightly more likely to be participants than thosewith proxy respondents. However, as notedabove, these small differences might be the resultof the nonrandom chance of proxy status(persons for whom proxy respondents reported

on AE participation may be more likely to beparticipants than nonparticipants) rather than thelack of difference between the two groups ofadults.

Table 6.—Percent of adults participating in adult education activities, by source of report: 1992

Percent participating in

Participation status Number

(in thousands)

Any activity Any activity excludingfull-time college

Any activityexcluding college

All adults 169,722 24.0 20.5 17.0

Self reporter 97,947 24.0 21.7 18.8

Proxy reporter 71,758 24.0 18.9 14.5

NOTE: Excludes about 8 percent of the sample, for whom the source of the report (self/proxy) was not identified.SOURCE: Special tabulations from the 1992 Current Population Survey (CPS), Adult Education Supplement.

To examine the extent to which estimates ofparticipation are affected by differences in thecharacteristics of self-reports and proxy reports,a logistic regression analysis was conducted. Inthe regression models, the variable beingpredicted was participation status (either aparticipant or not a participant). The predictorvariables were those related to eitherparticipation status or self/proxy status. Inaddition to the predictor variables, an indicatorvariable for proxy status (1 if a proxy and zero ifnot) was included in the models. Table 7 showsthe predictor variables in the logistic regressionmodel where the variable being predicted wasparticipation in any type of adult educationactivity, excluding full-time college enrollment.

The first step was to evaluate whether or notproxy status was an important predictor for AEparticipation when the other predictor variableswere included in the models. For all three typesof adult education (any activity, excluding full-time college, and excluding any college activity),the proxy indicator was statistically significant,

confirming the importance of proxy responsestatus in estimating AE participation.

The regression analysis confirms the importanceof proxy reporting and suggests that the smalldifferences in participation rates in table 6 maybe due to the nonrandom assignment of proxyreporting status. To examine this more carefullyand quantify the proxy reporting effect, theparameter estimates from the logistic regressionmodels can be used to estimate what theparticipation rate would have been in the CPS:92if all adults had reported for themselves. This isdone in two steps. First, the probability ofparticipating in adult education was estimated foreach sampled adult in the CPS:92 using theestimated regression parameters given in table 7,but the self/proxy indicator was always set toself. This resulted in an estimated probability ofparticipation if all adults self-reported. Second,the estimated probabilities were summed over allthe sampled adults using the CPS:92 estimatorweights to produce a predicted participation rate.

19

Table 7.—Logistic regression parameters for a main effects model of AE participation, excluding full-timedegree programs

Model parameter Parameter estimate Standard error

Intercept 1.82 .04

Whether self-reported or proxy-reported -.28 .02

Whether had a high school diploma or not 1.36 .04

High school diploma .96 .02

Some college or associate's degree .33 .02

Whether employed or not -.41 .02

Family income under $25,000 .51 .02

Family income between $25,000 and $49,999 .20 .02

Gender -.06 .02

Age 16 to 25 years old -1.04 .03

Age 26 to 55 years old -.95 .03

SOURCE: Special tabulations from the 1992 Current Population Survey (CPS), Adult Education Supplement.

Table 8 shows the observed participation ratesand the rates predicted if all adults had reportedfor themselves. The participation rate for anyactivity excluding full-time college is estimated tobe 2.2 percent greater if all the adults hadreported for themselves (22.7 - 20.5 = 2.2). Forthe other types of participation, the rates increasenearly 3 percent, using the same modelingprocedures.

Thus, these results indicate that proxy reportingin the CPS:92 probably resulted in a bias of

nearly 11 percent in the percent of adults whoparticipated in adult education activities (2.7percent is 11.3 percent of the observedparticipation rate of 24.0 percent for activitiesexcluding full-time college). If all adults hadresponded for themselves, it is estimated that theparticipation rate for any AE activity would beabout 27 percent. While this is a significantsource of error, proxy reporting accounts for arelatively small part of the difference between theCPS and NHES estimates.

Table 8.—Observed and predicted participation rates if all adults had self-reported, by type ofparticipation: CPS:92

Type of participationObserved participation

ratePredicted self-reported

participation rate

Any adult education activity 24.0 26.7

Activity excluding full-time college 20.5 22.7

Activity excluding college 17.0 20.1

SOURCE: Special tabulations from the 1992 Current Population Survey (CPS), Adult Education Supplement.

20

Survey Context

The context in which a survey takes place canhave a substantial impact on the resultingestimates. Many readers may think of surveycontext effects in terms of question order; indeedthis is related to the issues we address. But thiseffort takes a broader view of the context inwhich the survey takes place. In this section, weaddress three aspects of survey context that couldhave an effect on estimates of AE participation:questionnaire wording, sponsorship, andinterview supplement status. Each of theseaspects of the survey create a context that mayaffect the interviewers’ and respondents’perception of the content and import of thesurvey.

Two other aspects of survey context wereconsidered, but are not addressed in detail here,because we do not believe that they have animpact on estimates of AE participation. Thefirst of these is the mode of administration.While mode effects have been documented in theliterature (Sudman and Bradburn 1974;Bradburn 1983; Dillman and Tarnai 1991), thisdoes not seem to be a plausible source ofdifferences in the NHES and CPS estimates ofparticipation. The majority of CPS interviewsare conducted by telephone, which is the onlyNHES mode of administration. A second areathat was considered is the timing of the surveys.The NHES AE components were conductedduring the winter and early spring months, whereas the earlier triennial CPS AE supplements wereconducted in May, and the CPS:92 wasconducted in October. The time of year mighthave had an important impact on recall if AEactivities were tied to a traditional school year,and this could be used to prompt recall.However, many AE activities are not structuredaround a traditional school year, and all surveysunder consideration had 12-month recall periods.

Questionnaire Wording

The way questions are asked in a survey canhave a major impact on responses, as is welldocumented in the literature (Groves 1989;Converse and Presser 1986; Schwarz and

Hippler 1991; Kalton and Schuman 1982).Therefore, it is reasonable to expect that the wayin which a question about AE participation isasked could have an impact on estimates ofparticipation. Converse and Presser (1986; p.18) note the considerable difficulty inherent inattempting to convey a frame of reference to arespondent and having the respondent adhere tothat frame of reference in the interview.Questionnaire items on participation can attemptto convey the frame of reference in a number ofways, for example:

provide examples or definitions thatclarify the concept of adult education andwhat it includes;

set exclusionary criteria to prevent "falsepositives"; and

provide time parameters that assist therespondents' recall.

In each of the data collections discussed in thisreport, specific questions were asked that weredesigned to provide the respondent with promptsand recall aids so that all appropriate educationalactivities would be reported. However, thequestions were not the same in each of the datacollections. The specific questions for each ofthe five data sources appear in appendix A. Thissection provides a summary of wordingdifferences and interpretation of some of theimplementation. Details about how adulteducation was defined and how parameters affectestimates of participation rates will be discussedlater.

In the CPS:84, the NHES:91, the CPS:92, andthe NHES:95 Splice interview, the words "adulteducation" were used in at least some of thequestions or to introduce a series of questions.Cognitive laboratory work conducted for theNHES:95 indicated that the phrase "adulteducation" has different meanings for differentpeople. Some persons think of formalinstruction, and some think of classroom settingsonly. Many adults exclude courses, classes, orlessons taken for recreational purposes whenthinking about adult education. While a list of

21

specific activities followed the words "adulteducation," some respondents may have beenfocusing on their own conception of "adulteducation" when responding to the items. Anintensive reinterview Bias Study conducted forthe NHES:95 supports this notion, and suggeststhat respondents may form their own paradigm asto what is included in the survey very early in theinterview, especially when the initial questionsfocus on "traditional" classroom activities suchas basic skills programs, English as a SecondLanguage (ESL) programs, and postsecondarycredential programs, as in the NHES:95 (Brick etal. 1996).

Defining the Parameters of Adult Education .Different approaches were used in the variousdata collections to inform respondents about thenature of the interview and set up the responseparadigm. Often this was done by providing theadults with inclusionary and exclusionaryinformation on adult education. In the CPS:84, aflashcard was provided that gave a definition ofadult education, and provided examples of typesof activities or courses that would or would notbe included in the definition. The examples givenin the flashcard did not specifically mentionEnglish as a Second Language (ESL) or GEDpreparation courses, which were explicitlyincluded in the other surveys in this analysis,although these activities could have been includedby respondents under categories such as: “alladult and continuing education and noncreditcourses or activities,” or “basic instruction foradults who have never attended school or whohave interrupted their formal schooling at theelementary or secondary level.” It is difficult toknow exactly how this flashcard was used,because 70 percent of the interviews wereconducted by telephone and therefore mostrespondents did not see the flashcard.Interviewers were instructed to read orsummarize the information on the card for therespondents. However, because the amount ofinformation contained on the card was fairlyextensive, many respondents may not haveabsorbed and retained all of the information, evenif the interviewers did read it. In addition, it isnot known how interviewers were trained tosummarize the card.

In the CPS:92, the questions were included as apart of the October education supplement, and noAE-specific introduction was provided. Thiscould be partly responsible for the largedifferences between the NHES and the CPS:92estimates.

The introduction in the NHES:91 and theNHES:95 Splice interviews was very differentfrom that of the CPS:84 and the NHES:95 AEinterview. In the NHES:91 and the NHES:95Splice sample, respondents were told that thesurvey covered "three kinds of educationalactivities over the past year: full-time school,part-time school, and all other kinds of adulteducation." Respondents were also informed thateducational activities for job-related purposes,personal enjoyment, or recreation were included.No detailed information was provided prior to thespecific questions about AE participation.Because these items were designed specificallyfor telephone administration, lengthy introductoryexplanations were avoided.

In the NHES:95 interview, the introductoryapproach was substantially modified based onexperience from previous data collections and theresults of cognitive laboratory work andtelephone field testing. Adult education was notspecifically mentioned in the introduction.Rather, the questionnaire was structured so as tohave a section on each type of adult education.Each type of activity was asked about separately,with appropriate explanation or examples. Thus,each section defined a type of activity andprovided examples or exclusions regarding thatspecific type of activity. (The specific itemsappear in appendix A.)

The different NHES approaches to introducingthe adult education items did not result indifferences in participation rates between theNHES:95 AE and the Splice sample (which wasthe same as the NHES:91).

Exact Wording. In their work on the disparitiesin training estimates, Zemsky and Shapiro (1993)note that items from different surveys often usedifferent wording and formats to collectinformation on participation. The use of

22

different items in various data sources, includingthe National Longitudinal Survey of Youth, theSurvey of Income and Program Participation, thetriennial CPS surveys, and High School andBeyond, present the researcher with aconsiderable challenge in developing a systematicmeasure of participation. A similar challenge isfound in comparing CPS and NHES items.

In addition to differences in the introductions anddefinitions of activities in the CPS and NHESinterviews, there were also differences in thespecific activities that were included. TheNHES:95 differs significantly from the CPS:84,the NHES:91, the CPS:92, and the NHES:95Splice interviews, all of which are quite similar toone another. For example, the CPS:84 includedcourses taken by "correspondence, radio,television, or newspaper...." The NHES:91,CPS:92, and NHES:95 Splice interviewsincluded "courses by mail, television, radio, ornewspaper." Also, all of the instruments exceptthe NHES:95 AE interview asked aboutcontinuing education courses or noncreditcourses; private instruction or tutoring; educationor training given by employers, labororganizations, neighborhood centers, churches, orcommunity groups; or any other "organizededucational activity." While the CPS:84 askedabout courses for adults who had not finishedhigh school, the NHES:91, CPS:92, andNHES:95 Splice interviews asked aboutinstruction in basic skills such as math andEnglish, and also asked about instruction inEnglish as a Second Language.

In the NHES:95, items and questionnaire sectionsfocusing on six types of AE were developed:English as a Second Language; basic skillsincluding adult basic education (ABE) andpreparation for the General EducationalDevelopment (GED) examination; credentialprograms to earn college degrees or vocational,technical, or occupational certificates ordiplomas; apprenticeship programs; work-relatedcourses or training; and other formal structuredcourses, such as personal development,recreation, and so on. The NHES:95 did notspecifically ask about correspondence courses orthose taken by mail, radio, television, or

newspaper, and the section on other formalstructured courses (also called personaldevelopment courses) specifically asked forcourses "where there was an instructor."

The new approach in the NHES:95 AE interviewcreated a concern about the impact restructuringof the instrument and changes in wording mighthave on estimates of participation. TheNHES:95 Splice interview was implementedprimarily to compare estimates using theNHES:91 questions in the Splice interview andthe new NHES:95 AE questions in a concurrentdata collection. This permitted an assessment ofwhether any change in participation rates couldbe attributed to changes in the items themselves,and the extent of the instrument effect on theestimates. In fact, however, the participationrates for the NHES:95 AE interview and theNHES:95 Splice interview were virtuallyidentical -- 40.2 percent and 40.0 percent.

Because the NHES:95 items and the NHES:95Splice items were collected at the same timeusing the same survey procedures and yieldednearly identical estimates of participation, itappears unreasonable to attribute the largedifferences between CPS and NHES estimates todifferences in item wording. Also, this evidencesuggests that the apparent increase in adulteducation between the NHES:91 and theNHES:95 should not be attributed to wordingdifferences in items concerning participation.However, there are other factors related to thesurvey context that must be considered.

Interview Status: Sponsorship andSurvey Focus

Two other context differences between the NHESand the CPS are so closely related as to beinseparable — sponsorship of the survey and theprimary focus of the survey activity. In theNHES, the respondents are told that this is aninterview conducted for the U.S. Department ofEducation and is related to education issues. TheCPS is a survey conducted by the Bureau of theCensus for the U.S. Department of Labor toestimate characteristics of the labor force. The

23

questions about adult education are only includedin a supplement to the CPS and are not the mainfocus of the study. Thus, the overall context ofthe interviews with adults about education aredifferent from the perspective of both therespondent and the interviewer.

These differences may manifest themselves innumerous ways. For example, the NHESinterviewers undergo 16 hours of specifictraining to conduct the NHES interviews,including practice interviews under the directsupervision of trainers and telephone center staff.The training for supplements in the CPS isminimal, most often consisting of a home-studypackage that the interviewers are supposed toread. The level of training can make animportant difference in the quality of theinterviews, as suggested by Brick et al. (1995) ina comparison of interviewer effects in twosurveys. Perhaps the training or lack of trainingsends a more direct message to the interviewersabout the importance of the interview. If theinterview is a supplement with little or notraining required, then the interviewers mayperceive the data to be less important.

It is possible to speculate on how the overallcontext of the survey might affect the responsesto the survey, but these speculations cannot bequantified due to the lack of data. For example,it seems likely the responses to standard items,such as college enrollment, that are asked everyyear in the March CPS, would be more reliablethan the questions about other types of adulteducation. In fact, the AE Bias Study (Brick etal. 1996) showed that even in the NHES:95 manyof the adults under-reported less formal types ofadult education activity (in particular, work-related and personal development courses). It isvery likely that this type of under-reporting waseven more prevalent in the CPS for the reasonsdiscussed above.

As suggested above, the status of a survey asbeing a supplement to another survey could havean effect on the quality of the data beingcollected. Although no references in theliterature were found that addressed this specificissue, it is interesting to note that Shapiro (1987)

did investigate the opposite: the effect asupplement might have on the estimates from thecore study. In a review of four surveys, he foundthat the core estimates were affected whensupplementary questions were asked later in theinterview (after the core items were collected).He speculated that the observed differences mightbe the result of the interviewer’s anticipation ofthe supplemental questions that might alter theprobing that is done in the core study.

Shapiro’s results suggest that it is very possiblethat the supplement estimates may also beaffected by the core study, even though he did notexamine this situation. For example, it ispossible that the interviewer and/or respondentmay place less attention on the items in asupplement than on the core items, leading to lessthoughtful administration and response to thesupplementary items. Although it is not possibleto quantify the difference without a follow-up orreinterview study to the CPS specificallydesigned for this purpose, it is feasible that thecontext of the survey as a supplement could haveresulted in under-reporting of participation ofadult education in the CPS.

Findings of the NHES:95 Bias Study cited above(Brick et al. 1996) and the supplement status ofthe CPS items on AE participation may interactto produce the observed estimates. The BiasStudy results suggest that the reporting of adulteducation activities, particularly those that falloutside of formal schooling, is a cognitivelydemanding task. It may be that respondents aremore successful at rising to this task in a surveythat focuses specifically on adult education, ascompared to one in which adult education is asupplementary issue. The CPS has a longhistory of successfully collecting information onformal schooling, which appears from theNHES:95 findings to be a less demanding taskfor the respondent.

Summary

Reliable estimates of participation in adulteducation are important for a number of reasons.One of the key roles of these participation rates is

24

to inform the public about the NationalEducational Goal (National Education GoalsPanel 1994), which states that “By the year2000, every adult American will be literate andwill possess the knowledge and skills necessaryto compete in a global economy and exercise therights and responsibilities of citizenship.”Participating in various forms of adult educationis an activity that may support achieving thisobjective.

The focus of this study is an assessment ofreasons that estimates from the NHES and CPS,two important studies that measure adulteducation participation, are so different. Asnoted in table 1, the estimates from the CPS aresubstantially lower than those from the NHES.The report examined a number ofmethodological, definitional, and operationalfactors that might have had an impact on thedifferences in the estimates. Some of thesefactors could be quantified so as to estimate thepotential impact they had on the estimates, whileother factors evade measurement given theavailable data. Relevant findings aresummarized in table 9 and discussed below.

The 1984 CPS estimate of participation in adulteducation excluding full-time college was 14percent and for 1992 was 21 percent. Anincrease of 7 percent in adult education over the8 years is reasonable given the efforts made bygovernment and business to increase the skills ofadults during this time. However, the two CPSstudies used very different approaches (inparticular, different approaches to selectingadults) that should have had a substantialinfluence on the participation estimates. Asexplained earlier, if the CPS:84 had sampled alladults regardless of the responses of thehousehold respondent, it is estimated that the

‘switching rate’ would have caused theparticipation rate in 1984 to be 23 percent. Thisis close to 26 percent adjusted estimate from theCPS:92. This very modest increase inparticipation rates in adult education between1984 and 1992 is not consistent with whatexperts in adult education expect (Kim et al.1996), nor is it consistent with the higher ratesestimated in the NHES. This raises somequestions about the validity of the CPS:92estimates.

Comparing the CPS and the NHES estimates, anumber of factors were considered. One of themost obvious factors is the bias in the NHESestimates due to the failure to sample adults inhouseholds without telephones. However, theempirical evidence clearly indicates that theundercoverage bias in the NHES estimates isvery small (less than one percent), especiallywhen compared to the differences between theCPS and NHES estimates that range from 13 to20 percent.

A second methodological difference is theestimated bias due to nonresponse that affectsboth surveys, but has a larger impact on theNHES because of the lower response rate in thisstudy. The nonresponse biases could not beestimated directly because of the lack of data onthe differences between respondents andnonrespondents to the surveys. Nevertheless, bymaking some assumptions about thesedifferences, an upper bound on the differentialnonresponse bias was estimated. The boundshows that the NHES may overestimate theparticipation rate by up to 5 percent more thanthe CPS. This is a potentially important sourceof bias, but it is an upper bound and the actualbias due to nonresponse is probably less than 5percent.

25

Table 9.—Summary of factors associated with possible biases in estimates of adult education participation,by survey and possible size of bias

Factor Survey Possible size of bias1

Sampling only participants CPS:84 -9%

Sampling only telephone households NHES:91, NHES:95, and Splice sample <1%

Unit nonresponse CPS:84 and CPS:92NHES:91, NHES:95, and Splice sample

<3%2

<8%2

Proxy respondents CPS:84 and CPS:92 -3%3

Context effects Introductory statements All surveys No measurable effect Wording All surveys No measurable effect Supplemental status All surveys Unmeasurable4 Interviewer training All surveys Unmeasurable4 Survey sponsorship All surveys Unmeasurable4

1Bias is the difference between the survey estimate and expected population value. A negative bias indicates the surveyestimate is less than the population value.2Biases estimated for the CPS:92 and NHES:95.3Bias estimated for the CPS:92.4These context effects can not be estimated from the available data, but could be substantial.SOURCES: U.S. Department of Commerce, Bureau of Census, Current Population Survey (CPS), 1984 Adult EducationSupplement, 1992 Adult Education Supplement; U.S. Department of Education, National Center for Education Statistics,National Household Education Survey (NHES), spring 1991, spring 1995, spring 1995 Splice Sample Interview.

A third potential source of error was the use ofproxy respondents, particularly the practice inthe CPS where other adults could respond for thesampled adult. In the NHES, the sampled adultwas the only person who could complete theextended interview and the responses of otheradults in the household were only used forsampling purposes. The use of proxyrespondents was found to be an importantpredictor of AE participation of the adults in thesample, with those adults who respondedthemselves being more likely to participate. Byusing logistic regression analysis, the potentialimpact of proxy reporting was estimated for theCPS:92. These findings show the estimates fromthe CPS:92 are biased downward by about 3percent. No corresponding bias exists in the

NHES estimates, since proxy reporting was notpermitted.

The next aspect of the surveys considered wasthe overall context of the surveys. As noted, theliterature on context effects is diverse, and insome cases context effects can be substantial.The first two aspects of context that wereconsidered were the introductory approach thatsets the groundwork for the survey andquestionnaire wording. The NHES:95 Splicesample, a methodological study especiallydesigned and implemented to address theseissues, reveals that these two factors are not asource of the differences between the NHES andCPS estimates.

26

Other context issues were considered but couldnot be quantified. One of these was the fact thatthe adult education questions in the CPS were asupplement to the core CPS survey. This clearlyresulted in different operational approaches thatmay have influenced the interviewers and therespondents. The training of interviewers for theadult education interviews was also obviouslydifferent. The question is whether this affectedthe estimates and, if so, by how much. Inaddition, supplemental status may result in thequestions holding a place of lesser importance orfocus in the minds of respondents, and perhapsinterviewers. The second related issue is thesponsorship of the study. Sponsorship has beenshown to have an influence on the responses to asurvey, but again this cannot be quantified forthese studies.

Putting these sources of error together andassuming the errors are additive, much of thedifference between the NHES and CPS estimatesof participation could be explained. Forexample, the NHES:91 estimate of 33 percentmay be an overestimate by as much as 5 percentdue to undercoverage and nonresponse bias,while the CPS:92 estimate of 21 percent may bean underestimate by as much as 3 percent due toproxy reporting and nonresponse. The remainingdifference of 4 percent (28% - 24% = 4%) couldeasily be accounted for by context effects.

While this approach does account for much ofthe difference, it is not a very satisfying solutionfor several reasons. First, the CPS:92 estimatesappear to be out of line with the other estimates ,including the CPS:84 estimates. As noted above,the CPS:84 estimates adjusted for not samplingnonparticipants who might have participatedresults in an estimate of 23 percent, only veryslightly lower than the 26 percent adjustedCPS:92 estimate. Second, the differentialnonresponse bias in the two surveys is really anupper bound and the difference between theNHES and CPS due to this source is probablymuch less than 5 percent. Third, it is unlikelythat the error sources are actually completelyadditive. The errors due to coverage,nonresponse, use of proxy respondents, andcontext effects almost certainly interact in some

complex ways. However, approximating theinfluence of these effects as additive is all thatcan be done, given the data, and appears to be areasonable approach.

The evidence that most conflicts with this simpleapproach to summing the errors is the Bias Studyconducted in the NHES:95 (Brick et al. 1996).All of the sampled participants (based on theoriginal NHES:95 interview) who were includedin that intensive reinterview study proved to beparticipants, while 41 percent of the samplednonparticipants reported an adult educationactivity that they had not reported in the originalinterview. The activities reported by thesenonparticipants were examined and did appear tobe real activities undertaken during the timeframe of the survey. The findings in Bias Studystrongly suggest that even the NHES:95estimates, which are larger than any previousestimates of participation, may stillunderestimate participation. That study suggeststhat the estimates of participation rates might besubstantially higher than 40 percent.

These issues make it difficult to arrive at aconclusion without some ambiguity, althoughsome points appear to be based on solidfoundation. The NHES estimates are probablysubject to some bias due to nonresponse , butthere is little bias coming from telephonecoverage bias. The evidence from the Bias Studyindicates that respondents are underreporting , sothe overall estimates of participation in adulteducation are probably even larger than theestimates produced from the NHES. The CPSestimates from 1984 and earlier were biaseddownward due to several sources, most notablythe failure to sample adults classified asnonparticipants by the household respondent,proxy response, and context effects associatedwith the survey being a supplement to a laborforce survey. The underreporting of activitiesnoted in the Bias Study for the NHES:95 areprobably even greater in the CPS due to the lackof training with respect to the adult educationsupplement.Although the results of this study are subject toother interpretations, a reasonable conclusion isthat the NHES estimates of participation are

27

more accurate representations of the percentageof adults who participate in some type of adulteducation than the CPS estimates. As the BiasStudy suggests, the participation rate is likely tobe even higher than the NHES estimates due torespondent underreporting activities. The CPSestimates are depressed by the series of factorsnoted above. Furthermore, the Splice sampleshows that change in the participation ratebetween 1991 and 1995 is a reflection of anactual change in the rates and not an artifact ofthe survey methodology.

Since the results of this study are not definitive, itis interesting to consider options that could beused to further investigate the differencesbetween the NHES and CPS estimates if futurestudies were to be conducted. A study of theCPS might be most useful to test the hypothesisthat the CPS is severely biased downward. Toconduct such a study, another supplement to theCPS could be conducted using the sameapproach used in 1992. An intensive reinterview,similar to the NHES Bias Study, could then beconducted with a sample of the respondents (bothproxy and sampled adults) to ascertain the size ofthe bias in the estimates. This approach is ratherexpensive and should only be considered if futureadult education surveys might be conducted assupplements to the CPS.

The biggest potential source of error in theNHES, other than respondent underreportingwhich has already been at least partiallyexamined in the Bias Study, is unit nonresponse.This source of error is a significant problem forall NHES administrations, and for that matter alltelephone surveys, because telephone surveysoften result in lower response rates than in-person surveys. A number of methods could beused to address this concern. First and foremost,the importance of conducting the surveys usingtechniques to increase the response rates shouldalways be considered. Given the best efforts toimprove response rates, studies of nonresponsemight be undertaken. Different study designsshould be considered, such as record checkstudies, where appropriate and feasible. Follow-up studies could also be undertaken, but thesestudies tend to be expensive, especially if theyinvolve matching telephone numbers to addressesand conducting in-person interviews. Lessexpensive methods, such as matching telephonenumbers to files with demographic data onhouseholds, are feasible, but limited because thedata in the files is often of poor quality and alarge percentage of the telephone numbers maynot be matched correctly. Despite thesedifficulties, some information from these studiesmight improve the understanding of the natureand size of nonresponse bias in telephonesurveys.

28

REFERENCES

Anderson, R.E., and Darkenwald, G.G. (1979). Participation and Persistence in American AdultEducation. New York: College Entrance Examination Board.

Bradburn, N.M. (1983). Response Effects. In P.H. Rossi, J.D. Wrights, and A.B. Anderson. (Eds.).Handbook of Survey Research. New York: Academic Press.

Brick, J.M. (1992). Undercoverage Bias in Estimates of Characteristics of 3- to 7-year-olds, Appendix H .National Household Education Survey Methodology Report. Unpublished report prepared for theNational Center for Education Statistics. U.S. Department of Education, National Center forEducation Statistics.

Brick, J.M. (1996). Undercoverage Bias in Estimation of Characteristics of Adults and 0- to 2-Year-Olds in the 1995 National Household Education Survey. NCES Working Paper 96–29.Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Brick, J.M., and Broene, P. (forthcoming). Unit and Item Response Rates, Weighting, and ImputationProcedures in the 1996 National Household Education Survey. Washington, DC: U.S. Departmentof Education, National Center for Education Statistics.

Brick, J.M., Burke, J., and West, J. (1992). National Household Education Survey Technical Report No.2: Telephone Undercoverage Bias of 14- to 21-year-olds and 3- to 5-year-olds. NCES 92–101.Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Brick, J.M., Kim, K., Nolin, M.J., and Collins, M.A. (1996a). Estimation of Response Bias in theNHES:95 Adult Education Survey. NCES Working Paper 96–13. Washington, DC: U.S.Department of Education, National Center for Education Statistics.

Brick, J.M., McGuinness, R., Lapham, S., Cahalan, M., Owens, D., and Gray, L. (1995). InterviewerVariance in Two Telephone Surveys . Proceedings of American Statistical Association, Section onSurvey Research Methods, 447-452.

Brick, J.M., Tubbs, E., Collins, M.A., and Nolin, M.J. (forthcoming). Telephone Coverage Bias andRecorded Interviews in the 1993 National Household Education Survey. NCES Working Paper97–02. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Brick, J.M., Wernimont J., Montes, M. (1996). The 1995 National Household Education Survey:Reinterview Results for the Adult Education Component. NCES Working Paper 96–14.Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Collins, M.A., Brick, J.M., Kim, K., Gilmore, S., and Stowe, P. (1996). National Household EducationSurvey of 1995: Adult Education Data File User’s Manual. NCES Publication No. 96–926.Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Converse, J.M., and Presser, S. (1986 ). Survey Questions: Handcrafting the StandardizedQuestionnaire. Newbury Park, CA: Sage Publications.

Darkenwald, G.G., and Merriam, S.B. (1982). Adult Education: Foundations of Practice. New York:Harper & Row.

29

Dillman, D.A., and Tarnai, J. (1991). Mode Effects of Cognitively Designed Recall Questions: AComparison of Answers to Telephone and Mail Surveys. In P.P. Biemer, R.M. Groves, L.E.Lyberg, N.A. Mathiowetz, and S. Sudman. (Eds.). Measurement Error in Surveys. New York:John Wiley & Sons.

Epstein, A.M., Hall, J.A., Tognetti, J., Son, L.H., and Conant, L. (1989). Using Proxies to EvaluateQuality of Life. Medical Care, 27:S91–S98.

Frazis, H.E., Herz, D.E., and Horrigan, M.W. (1995). Employer-Provided Training: Results from a NewSurvey. Monthly Labor Review, 4.

Groves, R.M. (1989) Survey Errors and Survey Costs. New York: John Wiley & Sons.

Hill, S. (1987). Trends in Adult Education: 1969–1984. Washington, DC: U.S. Department ofEducation, Center for Education Statistics.

Kalton, G., and Kasprzyk, D. (1986). The Treatment of Missing Survey Data . Survey Methodology, 12(1), 1–16.

Kalton, G., and Schuman, J. (1982). The Effect of the Question on Survey Responses: A Review .Journal of the Royal Statistical Society, 145:42–73.

Kim, K., Collins, M.A., Stowe, P., and Chandler, K. (1995). Forty percent of Adults Participate in AdultEducation Activities: 1994–95. NCES Publication No. 95–823. Washington, DC: Department ofEducation, National Center for Education Statistics.

Kim, K., Loomis, L.S., Collins, M.A., and Chandler, K. (1996). Comparison of Estimates from the 1995National Household Education Survey. NCES Working Paper 96–30. Washington, DC: U.S.Department of Education, National Center for Education Statistics.

Knowles, M.S. (1980). The Modern Practice of Adult Education: From Pedagogy to Andragogy. NewYork: Cambridge.

Korb, R., Chandler, K., and West, J. (1991). Adult Education Profile for 1990–91. NCES PublicationNo. 91-222. Washington, DC: U.S. Department of Education, National Center for EducationStatistics.

Lessler, J.T., and Kalsbeek, W.D. (1992). Nonsampling Error in Surveys. New York: John Wiley &Sons, Inc.

Maklan, D. and Waksberg, J. (1988). Within-household Coverage in RDD Surveys. In R.M. Groves,P.P. Biemer, L.E. Lyberg, J.T. Massey, W.L. Nicholls II, and J. Waksberg. (Eds). TelephoneSurvey Methodology. New York: John Wiley & Sons.

Mohadjer, L., Brick, J.M., and West, J. (1992 ). Proxy Reports of Dropout Status in the NHES FieldTest. NCES Publication No. 92–103. Washington, DC: U.S. Department of Education, NationalCenter for Education Statistics.

Moore, J.C. (1988). Self/Proxy Response Status and Survey Response Quality . Journal of OfficialStatistics, 4:155–172.

30

National Education Goals Panel. (1994). The National Education Goals Report: Building a Nation ofLearners. Washington, DC.

Schwarz, N., and Hippler, H.J. (1991). Response Alternatives: The Impact of Their Choice andPresentation Order. In P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowetz, and S. Sudman.(Eds.). Measurement Error in Surveys. New York: John Wiley & Sons.

Shapiro, G. (1987). Interviewer-Respondent Bias Resulting from Adding Supplemental Questions .Journal of Official Statistics, 3:155–168.

Smith, T.W. (1990). Phone Home? An Analysis of Household Telephone Ownership. InternationalJournal of Public Opinion Research, 2 (4).

Sudman, S., and Bradburn, N.M. (1974). Response Effects in Surveys. Chicago: Aldine.

Thornberry, O.T., and Massey, J.T. (1988). Trends in United States Telephone Coverage Across Timeand Subgroups, In R.M. Groves, et al. (Eds). Telephone Survey Methodology. New York: JohnWiley and Sons.

U.S. Department of Commerce. (1978). The Current Population Survey: Design and Methodology.Technical Report, No. 40. Washington, DC: U.S. Bureau of Census.

U.S. Department of Commerce. (1993). Current Population Survey, October 1992: School EnrollmentTechnical Documentation. Washington, DC: U.S. Bureau of the Census.

Zemsky, R., and Shapiro, D. (1993). On Measuring a Mirage: Why U.S. Training Numbers Don't AddUp. Washington, DC: U.S. Department of Education, Office of Educational Research andImprovement.

Appendix A

Adult Education Participation Items

A-1

This appendix presents the adult education participation questions that were used in 1984 CurrentPopulation Survey Adult Education Supplement (CPS:84), in the 1991 National Household EducationSurvey Adult Education component (NHES:91), in the 1992 Current Population Survey Adult EducationSupplement (CPS:92), in the 1995 National Household Education Survey Adult Education component(NHES:95), and in the 1995 National Household Education Survey Splice Sample Adult Educationinterview.

A-2

CPS:84 Adult Education Screening Items

31. Is ... attending or enrolled in a high school, college, or university?

� Yes..................................... (verify: � High school, � College or Univ.)� No

32. Is ... enrolled as a full-time or part-time students?

� Full-time� Part-time

33. Is ... now attending or enrolled as a full-time students in a vocational or occupational program?

� Yes � No

34. Now I would like to ask some questions about ...'s participation in adult education activities.(Read or show definition and list of examples on flashcard.)(Excluding full-time school attendance) during the past 12 months (that is, since May 1 a yearago) has ... (Ask and mark each category)

a. Taken any adult or continuing education or noncredit courses or educational activities?

� Yes � No � DK

b. Taken any courses for credit as a part-time students in high school, college, vocational,or other school?

� Yes � No � DK

c. Taken a course by correspondence, television, radio, or newspaper, or taken any privateinstruction or tutoring?

� Yes � No � DK

d. Taken any courses or educational activities given by an employer, a labor organization,neighborhood center, a church, or another community group?

� Yes � No � DK

e. Taken any instruction for adults who have not finished high school?

� Yes � No � DK

f. Taken any other organized educational activities or courses during the past 12 months?

� Yes � No � DK

A-3

CPS:84 Adult Education SupplementParticipation Items

1. Are you now attending or enrolled as a full-time student in a school of any type?

� Yes (Ask 2) � No (Skip to 5)

2 Are you in a program leading toward the completion of elementary school or high school?

� Yes (Skip to 6) � No

3. Are you in a college degree program (such as an associate’s, bachelor’s, master’s, or doctoraldegree program)?

� Yes (Skip to 6) � No

4. Are you in a vocational or occupational program?

� Yes (Skip to 6) � No (Skip to 6)

5. Are you now enrolled in a college degree program or university as a part-time student, in coursesfor credit toward a degree (such as an associate’s, bachelor’s, master’s, or doctoral degreeprogram)?

� Yes � No

6. (Excluding full-time attendance in school) during the past 12 months, that is since May 1, 1983,have you taken part in any organized adult education courses or activities such as ....?(Read or show definition and list of examples on flashcard.)

� Yes (Ask 7) � No (End questions for this person)

7. How many of these courses or activities did you participate in, including any that you did notcomplete?

A-4

FORM CPS-681 U.S. DEPARTMENT OF COMMERCE(3-31-81) BUREAU OF THE CENSUS

FLASHCARDEXAMPLES OF COURSES AND ACTIVITIES

SURVEY OF ADULT EDUCATION

Adult education, for the purposes of this survey, includes all courses and other organized educationalactivities, taken by persons 16 years of age and over other than courses taken as a full-time' student in aprogram leading toward a high school diploma or a college degree or as a full-time 1 student in avocational or occupational program.

SOME EXAMPLES OF COURSES OR ORGANIZED INSTRUCTION TO BE REPORTED IN THISSURVEY ARE SHOWN BELOW. INCLUDE COURSES THAT WERE STARTED BUT DROPPED.

Automobile mechanics Foreign languages PhotographyBible study Health care PsychologyBiology How to stop smoking Reading or writing skillsBridge Mathematics Real estateChild care Music lessons Tennis lessonsDriver education Needlework Typing

To Be Reported Not To Be Reported

1. All adult or continuing education andnoncredit courses or activities.

2 Courses taken for credit as a part-time2

student in a high school, college, vocational,or other school.

3. Courses given by correspondence,television, radio, newspaper, tutor, orprivate instructor.

4. Courses or educational activities given byan employer, a labor organization, aneighborhood center, a church, or othercommunity group.

5. Basic instruction for adults who have neverattended school or who have interruptedtheir formal schooling at the elementary orsecondary levels.

1. Courses taken as a full-time1 student in aprogram leading toward completion ofelementary, junior, or high school (includinga work study program), or toward anacademic degree (such as, an Associate ofArts, Bachelor of Arts, Master of Science,or Doctor of Philosophy).

2. Courses taken as a full-time' student in avocational or occupational program.

3. Courses or educational activities in anapprenticeship or internship program.

4. Self-directed learning activities without theguidance of a teacher or sponsoring agency.

5. Individual counseling (such as,psychotherapy, marriage counseling, andvocational counseling) or group therapy.

6. Religious activities that are primarily forworship.

A-5

NHES:91 Adult Education Participation Items

Screening Items

AINTRO. We are also interested in knowing about the educational activities of people 16 years old and older.

Now I am going to ask you some questions about ((NAME/AGE/SEX'S)/YOUR) participation in three kinds ofeducational activities over the past year: full-time school, part-time school, and all other kinds of adulteducation.

S12INTRO. First, (has (NAME)/have you) been enrolled as a full-time student in any kind of school, trainingprogram, or other educational program in the past 12 months, that is since (INTERVIEW MONTH), 1990?

YES...................................................................................1NO ...................................................................................2

IF S12=1 THEN ASK S12A, OTHERWISE SKIP TO S13

S12A. In which of the following types of full-time educational programs (was (NAME/AGE/SEX)/were you) enrolled?(VERIFY: PAST 12 MONTHS ONLY)

YES NOa An elementary school, junior high school,

or high school program?...................................................... 1 2b. A program leading to an associate's degree?....................... 1 2c. A program leading to a bachelor's or

more advanced degree?...................................................... 1 2d. A vocational or occupational training program

after completing or leaving high school?.............................. 1 2e. An adult literacy or basic skills program............................... 1 2f. An English as a Second Language (ESL)

program............................................................................... 1 2g. Some other program

(SPECIFY) _________________________............................. 1 2

IF S12A-CATEGORY A=1 THEN ASK S12B, OTHERWISE SKIP TO S13

S12B. (Is (NAME/AGE/SEX)/Are you) currently enrolled as a full-time student in an elementary school, junior high school, or high school program?

YES...................................................................................1NO ...................................................................................2

IF S12B=1 THEN DISCONTINUE SERIES

A-6

S13. [In addition to (his/her/your) full-time studies] (Has (NAME/AGE/SEX)/Have you) been enrolled as a part-time student at a college or university in an associate's, bachelor's, or graduate degree program, in the past 12 months?

YES...................................................................................1NO ...................................................................................2

S14. Now I'm going to read you a short list of other kinds of adult education. [Not counting (full-time school) (or) (part-time courses taken for credit toward a degree)] Please tell me whether ((NAME/AGE/SEX) has/you have) been involved in any of these in the past 12 months.

YES NOa. Continuing education courses or noncredit

courses?.............................................................................. 1 2b. Courses by mail, television, radio or

newspaper?......................................................................... 1 2c. Private instruction or tutoring? ............................................. 1 2d. Educational or training activities given by

an employer, labor organization, neighborhoodcenter, church, or community group?................................... 1 2

e. Instruction in basic skills such as math,or reading and writing English?............................................ 1 2

f. Instruction in English as a SecondLanguage?........................................................................... 1 2

g. Or, any other organized educational activities?Please describe: _____________....................................... 1 2

Extended Interview Items

IF ADULT IS ALSO THE SCREENER RESPONDENT THEN READ MODIFIED VERSION OF THE INTRODUCTION AND SKIP TOTHE BOX AFTER A1B

Hello, my name is (INTERVIEWER). I am calling on behalf of the U.S. Department of Education. We areconducting a voluntary and confidential national study about the educational activities of adults, and would like toask you a couple of questions about your participation in three kinds of educational activities over the past year:full-time school, part-time school, and all other kinds of adult education. Keep in mind that we are interested notonly in classes taken for a degree or certificate, but also in educational activities that are job related, and thosetaken for personal enjoyment or recreation.

A1INTRO. First, have you been enrolled as a full-time student in any kind of school, training program, or other educational program in the past 12 months, that is since (INTERVIEW MONTH) 1, 1990?

FTANY YES...................................................................................1NO ...................................................................................2

IF A1INTRO=1 THEN ASK A1A, OTHERWISE SKIP TO A4

A-7

A1A. In which of the following types of full-time educational programs were you enrolled?(VERIFY: PAST 12 MONTHS ONLY)

YES NOFTSCHOOL a. An elementary school, junior high school, or

high school program?.......................................................... 1 2FTASSOC b. A program leading to an associate's degree?....................... 1 2FTBACHLR c. A program leading to a bachelor's or

more advanced degree? ...................................................... 1 2FTVOCA d. A vocational or occupational training program after

completing or leaving high school?...................................... 1 2FTBASIC e. An adult literacy or basic skills program............................... 1 2FTENGLSH f. An English as a Second Language (ESL) program............... 1 2FTOTHER g. Some other program (specify) ____________...................... 1 2FTGED h. GED .................................................................................... 1 2FTCERT i. Professional certification/recertification................................. 1 2

A4. [In addition to your full-time studies] Have you been enrolled as a part-time student at a college or university in an associate's, bachelor's, or graduate degree program, in the past 12 months?

PTENROLL YES...................................................................................1NO ...................................................................................2

A8. Now I'm going to read you a short list of other kinds of adult education. [Not counting (full-time school) (or) (part-time courses taken for credit toward a degree)] Please tell me whether you have been involved in any of these in the past 12 months.

YES NOOTNONCR a. Continuing education courses or noncredit courses?............. 1 2OTCORRS b. Courses by mail, television, radio or newspaper?.................. 1 2OTPRIV c. Private instruction or tutoring?............................................... 1 2OTGROUP d. Educational or training activities given by an employer,

labor organization, neighborhood center, church,or community group? ............................................................ 1 2

OTBASIC e. Instruction in basic skills such as math, or readingand writing English?.............................................................. 1 2

OTENGL f. Instruction in English as a Second Language?....................... 1 2OTOTHER g. Or, any other organized educational activity?

Please describe: _____________......................................... 1 2OTGED h. GED...................................................................................... 1 2OTCOMP i. Computers (unspecified)....................................................... 1 2

A-8

CPS:92 Adult Education Supplement

55A. Has .... been enrolled as a full-time student in any kind of school, training program, or other educational program in the past 12 months, that is since October 1991?

YES...................................................................................1NO ...................................................................................2

55B. In which of the following types of full-time educational programs was enrolled during the past 12 months, that is, since October 1991? (Ask and fill each one.)

YES NOa. An elementary school, junior high school, or

high school program?.......................................................... 1 2b. A program leading to an associate's degree?....................... 1 2c. A program leading to a bachelor's or more advanced degree? 1 2d. A vocational or occupational training program after

completing or leaving high school?...................................... 1 2e. An adult literacy or basic skills program............................... 1 2f. An English as a Second Language (ESL) program............... 1 2g. Some other program (specify) ____________...................... 1 2

55C. (In addition to ...’s full-time studies,) has ...been enrolled as a part-time student at a college or university in an associate's, bachelor's, or graduate degree program, in the past 12 months, that is, since October 1991?

YES...................................................................................1NO ...................................................................................2

55D. Now I'm going to read you a short list of other kinds of adult education. [(Not counting full-time school) (or) (part-time courses taken for credit toward a degree)] Please tell me whether you have been involved in any of these in the past 12 months. (Ask and fill each one.)

YES NOa. Continuing education courses or noncredit courses?............. 1 2b. Courses by mail, television, radio or newspaper?.................. 1 2c. Private instruction or tutoring?............................................... 1 2d. Educational or training activities given by an employer,

labor organization, neighborhood center, church,or community group? ............................................................ 1 2

e. Instruction in basic skills such as math, or readingand writing English?.............................................................. 1 2

f. Instruction in English as a Second Language?....................... 1 2g. Or, any other organized educational activity?

Please describe: _____________......................................... 1 2

A-9

NHES:95 Adult Education Participation Items

INTRO1. [READ DISPLAY IF RESPONDENT WAS NOT SCREENER RESPONDENT.] Hello, this is (INTERVIEWER) calling for the U.S. Department of Education. We are conducting a voluntary and confidentialnational study about the education of adults.[SCREENER RESPONDENTS: The purpose of our study is to learn what kinds of educational activities adults take part in and why some adults do not participate. These questions usually take about 15 to 20 minutes.

B1. These first questions are about English as a Second Language only. Please do notESLANG include other classes here. During the past 12 months, did you have a tutor or take any

classes to learn English as a Second Language?

YES ..........................................1NO ..........................................2

Ask C1 if A2 NE 1 (no high school diploma) orA4 = 1 (received high school diploma

in the last 12 months).If A3 NE 1 (foreign high school diploma), ask C1.

Else, go to D1.

C1. Not including regular day-time high school classes, during the past 12 months, did you have atutor or take any classes:

YES NOBSIMPROV a. To improve your basic reading, writing, and math skills?...... 1 2BSGED b. To prepare to take the General Educational

Development, or GED?........................................................ 1 2BSHSEQUV c. In some other high school equivalency program or

adult high school program?.................................................. 1 2

D1. (Not including the classes you told us about earlier,) During the past 12 months, did you take any courses that are part of a program, or a series of courses associated with a programleading toward...

YES NOCRDEGREE a. A college or university degree, such as an

associate's, bachelor's, or graduate degree?........................ 1 2CRVOCDIP b. A diploma or certificate from a vocational or technical

school after high school or a formal vocationaltraining program ................................................................. 1 2

A-10

E1. During the past 12 months, were you in a formal apprenticeship program leading to journeyman status in a skilled trade or craft?

APPRENTI YES ..........................................1NO ...........................................2

F1. Now, I'd like to ask about courses related to a job or career, whether or not you had a job when you took the courses. (Please don't include courses you already told me about.) Some examples are courses taken at your job, courses taken somewhere else that relate to your job or a new career, or courses for a license or certification you need for your job. Have you taken any of these in the past 12 months?

WRACTY YES ..........................................1NO ...........................................2

G1. Now, I am going to ask about any other courses where there was an instructor. (Please don't and crafts, sports or recreation, first aid or childbirth, Bible study, or any other types of

courses we haven't talked about yet. Did you take any of these or other courses in the past 12months?

SAACTY YES ..........................................1NO ...........................................2

A-11

NHES:95 Adult Education Interview(Splice Sample)

Now I would like to ask you a couple of questions about your participation in three kinds of educationalactivities over the past year: full-time school, part-time school, and all other kinds of adult education. Keep inmind that we are interested not only in classes taken for a degree or certificate, but also in educationalactivities that are job related, and those taken for personal enjoyment or recreation. These questions usuallytake about 5 minutes.

A1INTRO. First, have you been enrolled as a full-time student in any kind of school, training program, or other educational program in the past 12 months, that is since (INTERVIEW MONTH )1, 1994?

YES............................................................................... 1NO ................................................................................ 2REFUSED...................................................................... -7DON'T KNOW ................................................................. -8

A1. In which of the following types of full-time educational programs were you enrolled?(VERIFY: PAST 12 MONTHS ONLY)

YES NO R DKa. An elementary school, junior high school,

or high school program? ................................................1 2 -7 -8b. A program leading to an associate's degree?.................1 2 -7 -8c. A program leading to a bachelor's or

more advanced degree? ................................................1 2 -7 -8d. A vocational or occupational training program

after completing or leaving high school? ........................1 2 -7 -8e. An adult literacy or basic skills program? .......................1 2 -7 -8f. An English as a Second Language (ESL)

program?.......................................................................1 2 -7 -8g. Some other program?....................................................1 2 -7 -8

SPECIFY

A-12

A9. Now I'm going to read you a short list of other kinds of adult education. [Not counting (full-time school) (or) (part-time courses taken for credit toward a degree)] Please tell me whether you have been involved in any of these in the past 12 months.

YES NO R DKa. Continuing education courses or noncredit

courses?........................................................................1 2 -7 -8b. Courses by mail, television, radio or newspaper?...........1 2 -7 -8c. Private instruction or tutoring? .......................................1 2 -7 -8d. Educational or training activities given by an

employer, labor organization, neighborhood center,church, or community group?.........................................1 2 -7 -8

e. Instruction in basic skills such as math,or reading and writing English?......................................1 2 -7 -8

f. Instruction in English as a Second Language?...............1 2 -7 -8g. Or, any other organized educational activities?..............1 2 -7 -8

SPECIFY