Collecting Practice-level Data in a Changing Physician Office-based Ambulatory Care Environment: A Pilot Study Examining the Physician Induction Interview Component of the National Ambulatory Medical Care Survey

Vita

l and

Hea

lth S

tatis

tics

Serie

s 2,

Num

ber

176

N

ovem

ber

201

7

Copyright information

All material appearing in this report is in the public domain and may be reproduced or copied without permission; citation as to source, however, is appreciated.

Suggested citation

Halley MC, Rendle KA, Gillespie KA, et al. Collecting practice-level data in a changing physician office-based ambulatory care environment: A pilot study examining the physician induction interview component of the National Ambulatory Medical Care Survey. National Center for Health Statistics. Vital Health Stat 2(176). 2017.

Library of Congress Cataloging-in-Publication Data

Names: National Center for Health Statistics (U.S.), issuing body.Title: Collecting practice-level data in a changing physician office-based ambulatory care environment : a pilot study examining the physician induction interview component of the National Ambulatory Medical Care Survey.Other titles: Vital and health statistics. Series 2, Data evaluation and methods research ; no. 176 | DHHS publication ; no. 2018-1376. 0276-4733Description: Hyattsville, Maryland : U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics, November 2017. | Series: Vital and health statistics. Series 2, Data evaluation and methods research ; number 176 | Series: DHHS pub. ; number 2018-1376 | Includes bibliographical references.Identifiers: LCCN 2017043093| ISBN 9780840606839 (pbk.) | ISBN 0840606834 (pbk.)Subjects: | MESH: National Ambulatory Medical Care Survey (U.S.) | Ambulatory Care | Health Care Surveys | Data Accuracy | Physicians | Group Practice | United StatesClassification: LCC RA974 | NLM W2 A N148vb no.176 2017 | DDC 362.12072/7--dc23LC record available at https://lccn.loc.gov/2017043093

For sale by the U.S. Government Printing Office Superintendent of Documents Mail Stop: SSOP Washington, DC 20402–9328 Printed on acid-free paper.

Vital and Health Statistics

Series 2, Number 176

Collecting Practice-level Data in a Changing Physician Office-based Ambulatory Care Environment: A Pilot Study Examining the Physician induction interview Component of the National Ambulatory Medical Care Survey

Data Evaluation and Methods Research

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Centers for Disease Control and Prevention National Center for Health StatisticsHyattsville, Maryland November 2017 DHHS Publication No. 2018–1376

National Center for Health StatisticsCharles J. Rothwell, M.S., M.B.A., DirectorJennifer H. Madans, Ph.D., Associate Director for Science

Division of Health Care StatisticsDenys T. Lau, Ph.D., Acting DirectorAlexander Strashny, Ph.D., Associate Director for Science

iii

Contents

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2Sampling and Recruitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Theme 1—Inconsistent Understanding of the Definition of “Location” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Theme 2—Lack of Familiarity With Administrative Matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Theme 3—Primary Care-oriented Questions Not Relevant to Specialty Care Providers . . . . . . . . . . . . . . . . . . . . . .6

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Text TablesA Physicians randomly selected to participate in pilot study, by physician and field

representative identifier and physician department type: 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3B Emergent theme on inconsistent understanding of the definition of “location” from interviews

with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

C Emergent theme on lack of familiarity with administrative matters from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013 . . . . . . . . . . . . . . .7

D Emergent theme on primary care-oriented questions not relevant to specialty care providers from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8

iv

Acknowledgments

The authors would like to acknowledge and thank Suepattra May, M.P.H., Ph.D., and Dominick Frosch,Ph.D., for their role in developingthe original design of this study. Theauthors would also like to thank Ms.Katherine Stanley for her assistancein data management and analysis.Finally, the authors would like to thankthe participants, and in particular, thefield representatives who allowed theethnographers of this study to shadowthem while they did their work.

Page 1

Background

The landscape of office-based ambulatory medical care is changing rapidly. Over the past 3 decades, a decline has been observed in independent, solo, and small group practices, and an increase has been observed in the proportion of physicians in large multispecialty group practices (1). For example, only 18% of physicians were in solo practices in 2012, down from 41% in 1983 (2). Conversely, more than 22% of all physicians were working in multispecialty group practices in 2012, with an even larger proportion of internal medicine (36%) and family practice (28%) physicians in multispecialty groups (2). National Ambulatory Medical Care Survey (NAMCS) data during this period found that 26% of physicians were in multispecialty group practices in 2013, up from 20% in 2008 (3,4). Although single-specialty practice remains the

most common model at just over 45% of ambulatory care practices (2), care is increasingly being delivered in larger group settings.

Recognizing this changing environment, the National Center for Health Statistics (NCHS) has undertaken multiple quality improvement initiatives and methodological changes to ensure that NAMCS remains the leading source of data for understanding office-based ambulatory health care in the United States. These initiatives include: (a) the development, rollout, and refinement of a computer-assisted, in-person data collection instrument; (b) a program of reabstraction to assess interrater reliability of NAMCS patient record forms (PRFs); and (c) pilot studies that compare NAMCS in-person data abstraction techniques with alternatives, such as electronic standardized summaries of care obtained from electronic health records (EHRs).

Collecting Practice-level Data in a Changing Physician Office-based Ambulatory Care Environment: A Pilot Study Examining the Physician Induction Interview Component of the National Ambulatory Medical Care Surveyby Meghan C. Halley, Ph.D., M.P.H., Palo Alto Medical Foundation Research Institute; Katharine A. Rendle, Ph.D., M.S.W., Palo Alto Medical Foundation Research Institute and University of Pennsylvania; Katherine A. Gillespie, M.A., M.P.H., Palo Alto Medical Foundation Research Institute; Brian Gugerty, D.N.S., M.S., R.N., and Denys T. Lau, Ph.D., National Center for Health Statistics; and Harold S. Luft, Ph.D., Palo Alto Medical Foundation Research Institute and University of California, San Francisco

ObjectiveThis report examines ways to

improve National Ambulatory Medical Care Survey (NAMCS) data on practice and physician characteristics in multispecialty group practices.

MethodsFrom February to April 2013, the

National Center for Health Statistics (NCHS) conducted a pilot study to observe the collection of the NAMCS physician interview information component in a large multispecialty group practice. Nine physicians were randomly sampled using standard NAMCS recruitment procedures; eight were eligible and agreed to participate. Using standard protocols, three field representatives conducted NAMCS physician induction interviews (PIIs) while trained ethnographers observed and audio recorded the interviews. Transcripts and field notes were analyzed to identify recurrent issues in the data collection process.

ResultsThe majority of the NAMCS

items appeared to have been easily answered by the physician respondents. Among the items that appeared to be difficult to answer, three themes emerged: (a) physician respondents demonstrated an inconsistent understanding of “location” in responding to questions; (b) lack of familiarity with administrative matters made certain questions difficult for physicians to answer; and (c) certain primary care‑oriented questions were not relevant to specialty care providers.

ConclusionsSome PII survey questions

were challenging for physicians in a multispecialty practice setting. Improving the design and administration of NAMCS data collection is part of NCHS’ continuous quality improvement process.

Keywords: physician practice • ambulatory medical care • NAMCS

Abstract

Page 2  Series 2, No. 176

NAMCS is intended to provide estimates on the characteristics of office-based physicians and their practices, the specific health care services ordered or provided to patients during office-based visits, and changes in health care delivery and service utilization over time (5,6). NAMCS has collected data periodically since 1973 and annually since 1989 to better understand ambulatory medical care services in the United States (7,8). NAMCS consists of two components: the physician induction interview (PII) and PRF. The PII component collects data on physician and practice characteristics. Physician characteristics include demographics (e.g., age, sex, and ethnicity), professional demographics (e.g., specialty and board certification), professional activities (e.g., patient care, teaching, and research), office location(s), and patient volume. Practice characteristics include ownership, size, provider types, ancillary services available to the patient (e.g., electrocardiogram, lab, and radiology), office workforce roles (e.g., medical assistant and nurse), tasks performed, and EHR information, including capabilities. Visit-level data come from manually abstracting information about the patient and visit from medical and administrative charts using PRF. The NAMCS PRF component collects patient visit-level data, including demographics, vital signs, reason for visit, diagnoses, continuity of care, diagnostic and therapeutic testing or services, and medications and immunization. Researchers sometimes use data from both sources (e.g., to examine whether physicians who have lab facilities in their offices are more likely to order tests) (9). This report focuses on the PII component of NAMCS and examines how accurately the data collected from a set of physicians in a large multispecialty group reflect their practice.

To examine potential improvements in the PII component of NAMCS, NCHS recruited an independent research partner to conduct a small, qualitative evaluation of the NAMCS PII. The goal was to examine how NAMCS data collection on practice and physician characteristics may be improved given the increasing

proportion of multispecialty physician group practices.

Methods

Study DesignThis study used the participant

observation method for collecting data based on the usual NAMCS PII process conducted in a large multispecialty group ambulatory care setting. Participant observation is particularly well suited for understanding human behavior in context and is also commonly used to better understand how everyday practices shape the health care system (10) as well as to improve survey administration and development (11). It can take a variety of forms, from “detached observer” to “full participation,” and the form is tailored to whatever behavior the researcher is trying to understand (10). Because the goal of this study is to understand issues that spontaneously emerge in collecting NAMCS PII data in a particular setting, the researcher’s role was that of a detached observer.

As much as possible, this pilot study followed NAMCS’ standard protocols in which data are collected from a nationally representative sample of nonfederal, office-based physicians. (Data on community health centers are also collected as part of NAMCS, but that is outside the scope of this report.) Field representatives (FRs), trained interviewers representing NCHS, visit a stratified sample of physicians to collect both a range of information on each physician’s practice and patient characteristics and visit data for a sample of up to 30 randomly selected encounters during a designated reporting week.

NCHS employed an ethnographic approach to document apparent difficulties and inconsistencies arising in the NAMCS PII data collection process. An ethnographer with advanced training in qualitative research (authors MH or KR) silently observed FRs administering the NAMCS PII to physician respondents. To minimize any impact of this observation, ethnographers provided no verbal or nonverbal

feedback on the data collection process or responses to individual questions from FRs or physicians. FRs were instructed to conduct the data collection using standard NAMCS data collection procedures, even though ethnographers were present. Data obtained as part of this study were excluded from NAMCS datasets. Further, as changes are made annually to NAMCS data collection protocols, it is important to note that the NAMCS 2013 data collection protocol and NAMCS 2013 PII form were used in this study.

Sampling and RecruitmentThis study was conducted in a large

multispecialty group ambulatory care setting (referred to as a “reference clinic” in this report). The reference clinic has sites located in multiple counties, and this study took place at one of these locations. The site consists of several buildings on adjacent blocks housing a full range of primary care and specialty departments and administrative support services.

Specifically, for this study, a member of the site’s management team first provided NCHS with a complete list of all ambulatory care physicians (approximately 400) practicing at the reference site. Using SAS SURVEYSELECT (12), NCHS staff used this list to randomly identify nine physicians. The FRs then approached the nine physicians using standard NAMCS PII survey procedures, which involved an initial telephone contact followed by an appointment for the interview (13). The FRs explained to the physicians that during the PII data collection process, an ethnographer would also be present in the room to observe the FRs interviewing and elicitation style and to identify any issues that might arise when fielding the NAMCS survey in a multispecialty group practice environment. Once the physicians consented, the FRs notified the ethnographers of the appointment times and locations.

Of the nine physicians approached, eight were deemed eligible to participate in NAMCS; the ninth engaged exclusively in administrative duties as opposed to clinical work, which is among NAMCS’ eligibility criteria. Each

Series 2, No. 176  Page 3

of the eight physicians was given an information sheet and provided verbal consent to be observed and recorded by an ethnographer prior to the interview. In total, three FRs participated in this study; two FRs conducted three interviews each and one FR conducted two interviews. The institutional review boards of the participating institution and NCHS approved this protocol.

Data CollectionIn addition to audio recording

the entire NAMCS PII data collection process, the ethnographers took field notes of their observations, documenting any components of NAMCS PII that appeared to be particularly challenging for physicians to answer as well as the strategies the FRs used to attempt to address these challenges during the interview. Following completion of each physician interview, the ethnographer met with the FR alone, and the FR provided feedback about how they felt the interview went. The FRs also described any components of the survey they found difficult to administer. The ethnographers took additional field notes but did not provide any feedback to the FRs in order to not affect any subsequent data collection.

Audio recordings of the interviews were transcribed verbatim and then destroyed. During transcription, all identifying information—including

physician and FR names, physician addresses, and department names—was deleted to ensure that neither the physicians nor the FRs were identifiable.

Data AnalysisData were analyzed using a

multiphase, structured thematic approach designed to identify difficulties and inconsistencies in the NAMCS data collection process (14,15). The analytic team included two research assistants trained in coding qualitative data and two ethnographers involved in data collection. First, the research assistants used Microsoft Excel 2011 to organize the eight transcribed interviews by survey question and physician response. This allowed the team to examine the FRs’ questions, the physicians’ responses, and any subsequent discussion for each item individually and then compare them across the eight interviews.

The number of questions the physicians answered depended on their responses and the skip patterns embedded in the survey instrument. For each of the potential 187 questions and associated responses and conversations, each research assistant independently reviewed and created a “memo” summarizing three domains: (a) whether physicians expressed any confusion or hesitation regarding how to answer the question and the reason physicians appeared to be confused (e.g., they were unclear whether a question referred to their department or the entire multispecialty group); (b) guidance given by the FRs in answering the questions, noting any differences in advice given for the same question; and (c) the range of responses physiciansgave, particularly noting inconsistencieswhen the answers provided wereexpected to have been the same across allrespondents.

Next, the research assistants presented recurrent issues within these domains to the ethnographers. The ethnographers independently reviewed all transcripts and memos generated by the research assistants, along with their own field notes, and developed a list of common emergent themes. The entire team of research assistants

and ethnographers then convened and reviewed the identified themes together. Through consensus, they finalized the list of key overarching themes.

Results

Of the eight participating physicians, two were primary care providers and six were specialty care providers (Table A). Relative to the overall organization, the reference site had a disproportionate number of specialists, which was reflected in this sample. There were 50 key questions on the 2013 NAMCS PII administered by the FRs. Some questions were skipped based on the embedded skipping pattern of the survey, and about one-half were multipart questions (e.g., 25a, 25b, 25c, etc.). The majority of the questions appeared to have been easily answered by the physician respondents. Because the number of questions asked varied by physician and many were interdependent, quantifying the proportion of questions observed to be difficult or confusing to answer was not straightforward. The results presented in this report focus on three central themes emerging from reviewing the observations, field notes, and transcript analyses. They include: (a) an inconsistent understanding of thedefinition of “location”; (b) a lack offamiliarity with administrative matters;and (c) primary care-oriented questionsnot relevant to specialty care providers.These themes are described below, andexamples of physicians’ responses to theNAMCS survey questions illustratingthese themes are provided in Tables B–D.

Theme 1—Inconsistent Understanding of the Definition of “Location”

One issue that arose in all eight interviews was the lack of clarity in the NAMCS questions referencing the practice location of the respondents. Physicians expressed confusion about whether a given question was asking about a specific department, a specific address within the larger clinic, the main multispecialty clinic (which had multiple

Table A. Physicians randomly selected to participate in pilot study, by physician study identifier, physician department type, and field representative identifier: 2013

Physician study identifier (department type)

Field representative study identifier

P101 (specialty care) FR01P102 (primary care) FR02P103 (specialty care) FR02P104 (administrator) ---P105 (specialty care) FR01P106 (primary care) FR03P107 (specialty care) FR03P108 (specialty care) FR03P109 (specialty care) FR02

--- Data not available, because physician was ineligible for study.

NOTES: Among nine randomly selected physicians to participate in the pilot study, eight physicians were considered eligible and provided data via an in-person interview. An ethnographer also recorded the interview (see Data Collection for more information). The study participants were sampled in the United States.

Page 4  Series 2, No. 176

street addresses), or even the entire health care organization, which spanned multiple counties. Table B illustrates the observed confusion among respondents while answering survey questions about practice location.

At the beginning of the survey, an “in-scope location(s)” question was asked to serve as the reference location for the remainder of the survey (see question 17b in Table B). Physicians expressed confusion as to how they should respond to this question. While seven of the eight physicians ultimately selected “group practice” as part of their response (selecting multiple categories was allowed), six asked for clarification or stated that the categories were not an obvious fit for the reference site, which included multiple groups of physicians in various departments organized by specialty and spread across multiple buildings on the site’s campus. The

question did not provide details about whether the in-scope location refers to the entire campus for the group practice, only a specific address for the respondent’s office, or only a respondent’s department (e.g., pediatrics or family medicine).

The uncertainty about the reference location made some subsequent survey questions difficult to answer. For example, question 19b asks, “How many physicians are associated with you (at this/that in-scope location)?” FRs varyingly replaced “at this/that in-scope location” with “at this address,” “at the clinic,” “in this group practice,” “in this [type] department,” “at this location,” or simply “here.” In response, five of the eight physicians initially answered by referencing the entire multispecialty clinic and responding with large numbers, such as, “I guess about 150, 200, I don’t know,” and “probably 800.” Two physicians responded by referencing

their specific department at that site, with one reporting that the practice included only one other physician. During two physician interviews, the FR explicitly stated to the respondents that question 19b generated a lot of confusion in these settings. One physician stated that he/she did not know. Among six physicians who asked for clarification regarding the reference location (see examples in Table B), three were told by the FRs to report the number of physicians in their department, one was told to provide the number for the whole building, and one was told to give the number for the entire address. Although these physicians worked at the same site, physician responses varied from 2 to 800 “physicians practicing at the reference site,” and one answered, “I don’t know.”

Physicians also experienced difficulty answering other survey questions, such as the capabilities of the

Table B. Emergent theme on inconsistent understanding of the definition of “location” from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013

NAMCS survey question Sample participant response

17b. Choose ALL of the type(s) of settings that describe each location where you work. For each location, enter all setting types that apply. For each location, also enter the appropriate "scope" status. If any even numbered settings are entered, then enter location as out-of-scope.

1. Private solo or group practice

2. Hospital emergency department

3. Freestanding clinic/urgicare center (not part of a hospital outpatient department)

4. Hospital outpatient department

5. Community health center (e.g., federally qualified health center)

6. Ambulatory surgicenter

7. Mental health center

8. Institutional setting (e.g., school infirmary, nursing home, prison)

9. Nonfederal government clinic (e.g., state, county, city, maternal and child health, etc.)

10. Industrial outpatient facility

11. Family planning clinic (including Planned Parenthood)

12. Federal government operated clinic

13. Health maintenance organization or other prepaid practice (e.g., Kaiser Permanente)

14. Laser vision surgery

15. Faculty practice plan

● FR01: Okay [typing], this is the current office. Um, let’s see…looking at this list, choose all the types of settings that describe your office here at [this address].

P101: …[reviewing list]…best describes this practice…uh…it’s a free standing clinic.

FR01: Okay, so number three?

P101: What’s the difference between that and number nine, which is a nonfederal government clinic?

FR01: Um…is, well first question is, the private, solo, or group practice—does that characterize this office or your practice?

P101: I don’t…so, we’re considered a multispecialty group. Not for profit. I'm not quite sure what to [inaudible].

FR01: We’ll do specialty group, okay, not for profit. [typing] So I'm just entering this as a note because it’s really not um—

P101: It’s not directly—it, it usually characterizes us as a multispecialty group.

FR01: Okay. Um, you can select um up to four.

P101: I can?

FR01: Yeah.

P101: Okay. I would probably go with free standing clinic, number three. Nonfederal government clinic, number nine…we have an ambulatory surgery center but… .

FR01: Let’s just, your—

P101: But it’s just my practice?

FR01: Right.

P101: But think those two are probably best characterizing this.

See footnotes at end of table.

Series 2, No. 176  Page 5

Table B. Emergent theme on inconsistent understanding of the definition of “location” from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013

NAMCS survey question Sample participant response

19b. How many physicians are associated with you (at this/that in-scope location)?

● FR02: Okay, so not solo. How many physicians are associated with you at this address?

P102: At [this address]?

FR02: At [this address].

P102: Well, it’s actually the group. I'm, I'm a member of is probably 800 physicians.

FR02: Okay…[laughs].

P102: [laughs].

FR02: [inaudible]…invalid number, okay. How many just at this practice, in this building, within this [inaudible]?

P102: Well, within this [name of building] practice that I'm practicing with now, there’s four others, so a total of five.

● FR01: Okay. And how many physicians are associated with you in this group practice?

P105: So, not just my department but everybody?

FR01: This is one of the questions that keeps, we have a struggle with. So, I would say in this department. This is the [department] department.

P105: Okay, and just here, right? So, um, there’s probably like 15 of us. It doesn’t have to be exact, right? ‘Cause I mean, you could find out how many exactly.

● FR02: Okay, so that’s nonsolo. How many physicians are associated with you here [at this location]?

P109: One. One for this office, but we are a multispecialty group, so I don’t know how to answer your question adequately in terms of what the question is asking.

FR02: Sure. Well, it's how many physicians are associated with you at this address, so it’s 795. So, it would be the total physicians.

P109: Oh, I don’t know the answer to that question. I don’t know.

FR02: Okay, I put “don’t know.”

P109: It’s a large number…

19c. Is this a single- or multispecialty (group) practice (at this/that in-scope location)?

● P102: Multi. Well, well if you're just saying the four I practice with, that’s single because we’re all practicing [department]. So, I don’t know how you, how you would do that. But, but this—since I'm just talking about the people I practice with in [this building] that is strictly a primary care [department] practice.

● FR01: No. And is this a single or multispecialty group practice?

P105: Well, it’s multispecialty, that’s like a couple hundred, but what I'm talking about not is just single specialty I guess. So…

FR01: Okay, because it’s just [department].

P105: Yeah.

19g. (1–5) Does your practice have the ability to perform any of the following on site (at this/that in-scope location)? 1) EKG/ECG, 2) Lab testing, 3) Spirometry, 4) Ultrasound, 5) X-ray?

● FR01: Okay. And does your practice have the ability to perform any of the following: EKG/ECG?

P105: So, now we’re talking about just our department?

FR01: Yeah.

P105: Okay, no.

● FR02: Lab testing?

P102: Well, that’ll be confusing. We draw lab here but we send it to the main lab. So, uh, so I work with [name of health care organization], we have our own lab, but we don’t actually have a lab on this property, but our nurses draw, they draw the blood. But uh… .

FR02: Okay, so we’ll, I think for lab testing we would put… .

P102: Uh…well I think I have to consider the whole group. We do have lab testing, yeah.

NOTES: NAMCS is National Ambulatory Medical Care Survey. FR is field representative. P is responding physician. EKG and ECG are electrocardiography. Among nine randomly selected physicians to participate in the pilot study, eight physicians were considered eligible and provided data via an in-person interview. An ethnographer also recorded the interview. During transcription, all identifying information was destroyed (see Data Collection for more information). The study participants were sampled in the United States.

–Con.

Page 6  Series 2, No. 176

in-scope location. For example, question 19g (1–5) asks whether the physician’s practice has the ability to perform various types of tests, such as labs, x-rays, or ultrasounds “on site at this/that in-scope location.” At the site, thesetests are not performed within individualdepartments but are provided in a centralunit at the site, which may not be inthe same building as a given physicianrespondent’s office. When physicianrespondents asked for clarificationabout the reference location for thesequestions, FRs’ responses included:(a) only their department, (b) thewhole building, or (c) all the buildingsat that address. For example, wheninterviewing three physician respondents,the FRs did not state “on site” whenthey asked whether the location hadelectrocardiogram capabilities. For twophysician respondents, the FR told themto respond for just their department,leading both physicians to answer “no.”The remaining six respondents did notrequest clarification and all answered“yes.” When asked about laboratorytesting, six of the seven physicianslocated in the same building as the lab(which is separate from the physicians’ departments) responded “yes” theyhave lab testing and one answered “no.”The eighth physician, whose practice islocated on the same site, but in a buildingwith a different address, responded “no.”Responses were even more divergentwhen asked about x-ray and ultrasound,with four physicians reporting that theyhave this capability and four reportingthat they do not, though all had access tothe same facilities at the site.

The ambiguity of the reference location also affected question 19c, which asks, “Is this a single- or multispecialty (group) practice (at this/that in-scope location).” For seven of the eight physicians, the FRs asked the question without any substitutions for “(at this/that in-scope location).” Three physicians did not request clarification before answering “multispecialty clinic.” The other four physicians initially stated that it was a multispecialty practice, but then noted that if the question was referencing their department, their answers would be “single specialty.” For one physician, the

FR did not ask the question and instead stated, “This is a multispecialty practice,” to which the physician respondent did not object.

Theme 2—Lack of Familiarity With Administrative Matters

In this study’s large multispecialty group, administrative matters were often handled by specialized office staff. For certain NAMCS questions related to administrative matters, physicians were either unable to answer or provided only general estimates with low accuracy (Table C). For example, question 19i asks, “What is your national provider identifier?” None of the eight physician respondents could provide the number from memory. Five provided a number after looking it up and three did not answer the question. Similarly, for question 19j—“What is your federal tax ID number?”—only one physician provided a response after looking it up on their phone. Physicians were either unable to answer or expressed uncertainty when answering administrative questions characterizing their practice. For example, question 42 asks, “Roughly, what percentage of the patient care revenue received by this practice comes from managed care contracts?” Half of the physician respondents stated that they did not know. The remaining four provided answers only after expressing some uncertainty.

To at least one administrative question asked during the interviews, six physicians stated that someone in the organization other than themselves (e.g., a staff member, an office manager, or a higher-level administrator) would be better able to provide an accurate answer. In one case, the ethnographer observed the FR attempting to follow up with office staff, but the FR was unsuccessful in obtaining the information.

Certain questions about the practice should elicit the same response from all physician respondents at the reference site. For example, question 32b asks, “In which year did you install your EHR/EMR system?” One physician provided the correct year; six provided an incorrect year; and one responded, “I don’t know.”

Theme 3—Primary Care‑oriented Questions Not Relevant to Specialty Care Providers

The final emergent theme from these interviews was that certain questions were perceived by specialty care providers as oriented toward primary care providers (Table D). For example, question 24 includes a series of items asking about the type of providers in the practice most responsible for performing specific clinical tasks. Some specialty care providers thought that a number of these tasks were not applicable to their practices. For example, question 24a says, “Records body measurements (such as height and weight) and vital signs (such as BP, temperature, heart rate);” 24d says, “Provides immunizations (includes both childhood and adult);” 24g says, “Provides counseling services (such as diet/nutrition, weight reduction, tobacco cessation, stress management);” and 24h says, “Manages the routine care of patients with chronic conditions (such as hypertension, asthma, and diabetes).” In five instances, physicians tried to adapt a question to their department (e.g., an ophthalmologist described taking “eye vitals” or a surgeon adapting preoperative counseling for the category of counseling). FRs did not encourage respondents to do this or provide additional examples beyond those listed in the questions. For many of these questions, specialty care providers stated the question was not applicable, including one respondent who stated, “This wasn’t written for us.”

Discussion

This study provides insights for NCHS to further improve NAMCS PII data collection and instrument development as well as for quantitative researchers using NAMCS PII data. Key themes include: (a) an inconsistent understanding among physician respondents of “location” as used within the survey, (b) physician respondents’ lack of knowledge of administrative

Series 2, No. 176  Page 7

Table C. Emergent theme on lack of familiarity with administrative matters from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013

NAMCS survey question Sample participant response

19i. What is your national provider ID at each office location? ● FR02: And what is your national provider identification number?

P109: Uh…I don’t know [laughs]. I'm sorry, I always ask people to fill it in for me, I'm sorry.

19j. What is your federal tax ID at each office location? ● P101: Don’t know that either, my staff—

FR01: Your staff might know that?

P101: My staff would know that.

FR01: Okay. Is it okay if I uh, ask them for that information?

P101: Yes.

● FR02: Okay, let me put that in there. And do you know your federal tax ID number?

P109: No. I don’t know any of these things. [laughs]

29. What is the tax identification number that you use? ● P103: It’s the organization number, and they know it out front but I don’t.

● P108: Um, I'm sure it’s the [Clinic] TIN, which um, I don’t know if I have it in my phone. I can look it up.

FR03: Okay, thank you. [looking up number]

P108: No, I don’t have it, sorry.

FR03: Okay, is it okay if I put “don’t know” for this question?

P108: Don’t know, yeah.

● FR03: Okay. A tax identification number, or TIN, is required by payors, such as Medicare to pay physicians’ claims. What is your tax identification number that you use? Okay, we can put “don’t know,” that’s fine.

P106: It’s why I like being in a big group [inaudible].

32b. In which year did you install your EHR/EMR system? ● FR03: Okay, and so you remember what year uh, the system was installed?

P106: Oh, it was a couple years prior to my arrival, and I’ve been here for 10 years, so probably about 12 years ago?

FR03: Okay, so 2001? Does that seem about right?

P106: I guess, probably. Yeah.

FR03: Does it?

P106: Yeah.

● P108: I’ve been here since 2008, and it was installed at that time, so probably a couple years prior to that, but I can’t tell you for sure. So as long as I’ve been here, that’s the only answer I … . I think it got installed in 2000 something, but I can’t tell you what year.

FR03: Okay, so I’ll just put that “don’t know” and then in the notes I’ll put 2000—

P108: My experience is ever since I’ve been here.

FR03: Okay, I’ll put at least through 2008.

P108: Yeah.

● FR02: Okay. And in which year did you install the EHR or EMR system?

P109: A long time ago, it was before I started working here.

FR02: And when was that?

P109: I started working here in 2006.

FR02: Okay, um, I can also put, put “don’t know.” Because you're not really sure when it was installed?

P109: Yeah.

42. Roughly, what percentage of the patient care revenue received by this practice comes from managed care contracts?

● P106: HMO and PPO? Combined?

FR03: Yeah

P106: Yes, it says HMO/PPO but not Medi, Medicaid

FR03: Right. Comes from—yeah, ‘cause I don’t think, well, do you have a managed care contract with uh, [our state’s Medicaid program]?

P106: Um…I'm not, I don’t know the answer to that question. Um, but um, we do see [Medicaid] patients here, so there must, there’s gotta be some contractual relationship there. Um, so that, I would say probably 100%. I mean, all of our, all of our PPO and HMO, there’s contractual relationships, yeah.

● P107: Managed care contracts, um…I don’t think any. I, I honestly don’t know.

FR03: Okay, I can put “don’t know”—is that okay?

P107: Yeah, I don’t know.

NOTES: NAMCS is National Ambulatory Medical Care Survey. FR is field representative. P is responding physician. EHR is electronic health record. EMR is electronic medical record. HMO is health maintenance organization. PPO is preferred provider organization. Among nine randomly selected physicians to participate in the pilot study, eight physicians were considered eligible and provided data via an in-person interview. An ethnographer also recorded the interview. During transcription, all identifying information was destroyed (see Data Collection for more information). The study participants were sampled in the United States.

Page 8  Series 2, No. 176

matters, (c) and primary-care-oriented questions not relevant to specialty care physicians. In addition, there were inconsistencies in the ways FRs asked survey questions or helped respondents answer them. These findings suggest that changes to the NAMCS instrument and its administration may improve the accuracy of data collected from large multispecialty groups.

Inconsistent understanding of the reference location was a key source of confusion in the sampled large multispecialty group practice setting. Survey items asking for the respondents’ practice location did not have a category for a multispecialty group practice environment. Despite this, seven of the eight physician-FR pairs in this study provided the “best” response, group practice, from the available list. Subsequent survey items referred to the reference location using different terms, such as “clinic,” “department,” “location,” and “practice.” In a large multispecialty clinic setting, however, “clinic” may be interpreted as one department, all departments within a

multispecialty clinic, or all buildings located on a given clinic campus.

NAMCS PII instructions and data collection protocol regarding location could be further improved. For example, when the physicians requested clarification about the referent location, inconsistent guidance was given both within a single interview and among different FRs. In some cases, physicians answered the same question using a different referent location. Addressing this issue may require NCHS to clarify the referent location for each survey question as well as provide further training to FRs on how best to guide physicians who ask for clarification. For some questions, the appropriate locational reference might be the physicians’ geographic site (e.g., a small satellite clinic), which may have limited EHR capabilities. However, if an option was provided to also select partnership with a much larger organization, respondents may indicate they have greater EHR capabilities (16).

Some researchers, however, may be interested in the economic incentives

the physician may have to order the test, which reflect not physical, but organizational and ownership structures. In small “traditional” physician offices, these details are usually irrelevant, but they may be critical in interpreting data from large, complex practices. Eliciting such detail would add further complexity to the questionnaire, but it might be critical for researchers seeking to explore certain hypotheses (9,17).

After this study was conducted in 2013, ongoing quality improvement efforts led to changes made in the 2015 NAMCS PII survey instrument, which states, “The next set of questions pertain to characteristics of the sampled physicians' practice. How many physicians, including you, are associated with this practice? Please include physicians at [fill in address of sampled location], and physicians at any other locations in this practice.” This revision was intended to help FRs and providers better understand what is meant by reference location.

Lack of familiarity with administrative matters was another theme of confusion.

Table D. Emergent theme on primary care-oriented questions not relevant to specialty care providers from interviews with physicians participating in pilot study, by NAMCS survey question and sample participant responses: 2013

NAMCS survey question Sample participant response

24a. Records body measurements (such as height and weight) and vital signs (such as BP, temperature, heart rate)

● P101: We don’t take those sort of vitals, but, but doing core eye vitals would be medical assistants.

FR01: Eye vitals?

P101: Uh, vision, eye pressure, um, visual field testing, things like that.

24d. Provides immunizations (includes both childhood and adult) ● P108: Nope, we don’t do that.

FR03: Okay, you don’t do that. They [NCHS] are working on letting us have an N/A category, yeah.

P108: Mhm, mhm, that makes sense. I mean, you're testing the feasibility of multispecialty clinics for these kinds of questions, it just sounds like most of these don’t apply. You go to [other department], they're going to tell you the same thing, also.

24g. Provides counseling services (such as diet/nutrition, weight reduction, tobacco cessation, stress management)

● P101: Not applicable [laughs]. This wasn’t written for us.

● P107: Counseling…as in what kind of counseling?

FR03: Well, this one has such as for diet, nutrition, weight reduction?

P107: Well, we have sort of a preoperative counseling.

FR03: That would be the physicians that do that?

P107: No, actually it’s our um, it’s the, [name] the RN and then the nurse practitioner usually do kind of these preoperative consultation, you know, what they expect from—surgery kind of thing.

24h. Manages the routine care of patients with chronic conditions (such as hypertension, asthma, and diabetes)

● P109: We do not do that, we’re a specialty.

● P107: Uh, we don’t have—so, I'm in a surgical practice, so you know, we don’t typically see patients chronically here. They come in either for a pre-op consultation and a post-op visit and that’s usually it.

NOTES: NAMCS is National Ambulatory Medical Care Survey. FR is field representative. P is responding physician. BP is blood pressure. N/A is not applicable. RN is registered nurse. Among nine randomly selected physicians to participate in the pilot study, eight physicians were considered eligible and provided data via an in-person interview. An ethnographer also recorded the interview. During transcription, all identifying information was destroyed (see Data Collection for more information). The study participants were sampled in the United States.

Series 2, No. 176  Page 9

The NAMCS PII asks about a variety of administrative matters, some are specific to each individual department and others are relevant to the entire clinic organization. In large group practices, the sampled physicians responsible for clinical duties may not be familiar with the administrative concerns of their practice. Further, for those questions that physicians could accurately answer, some could have been answered by other staff, thus reducing the amount of time needed from physician respondents (e.g., question 47b, which asks, “Does your practice set time aside for same day appointments?”).

Identifying the person, such as an administrator or department manager, who is most knowledgeable and can answer the administrative questions can be challenging. For example, questions about managing department-specific schedules may be best answered by a clinic manager, but questions about EMR implementation may be best answered by a central administrator. A different NAMCS workflow might determine that consistent administrative sources provide organization or department-level information. This information could then be confirmed by the physician respondent during the interview. This method was used in subsequent NAMCS FR training (as described below). Reducing the number of questions the physician is required to answer may increase physician participation rates and reduce physicians’ frustration, which was evident at certain points during these interviews, over the long list of questions they are unable to answer.

As the U.S. health care delivery system undergoes change, NCHS conducts various ongoing assessments of its survey questions and methods to examine and ensure quality in capturing accurate and reliable health care data. Ongoing quality improvement activities led to changes in 2016, when NAMCS FRs were trained to try to collect administrative information from a knowledgeable person if the sampled physician was unable to answer. The latest FR training material states, “It is unlikely you will actually conduct a full Induction Interview with the physician;

on almost every occasion you will interview the office manager or some other designee.”

The last emergent theme causing confusion was primary care-oriented questions not relevant to specialty care providers. When physician respondents perceived questions as not pertaining to them, they responded in various ways, with some attempting to adapt the question to aspects of their specialty, while others simply stated that the questions were not applicable to them. For example, one question asks, “Which type of provider [at your office location] most commonly…records body measurement (such as height and weight) and vital signs (such as BP, temperature, heart rate)?” Because some specialty providers rarely, if ever, take these kinds of body measurements or vital signs, this question could be removed for specialty providers. Alternatively, this question could be tailored to different specialties. For example, when interviewing an ophthalmologist, the following question might be asked, “Which type of provider [at your office location] most commonly…records body measurement (e.g., visual acuity)?” Furthermore, ensuring that FRs communicate that “not applicable” is a response option might help further reduce confusion. The 2013 form actually allowed a response of “Tasks not performed in this office” for the workforce items, but it does not appear as if that response category was encouraged by the FRs. Though this theme applied only to a subset of questions within the NAMCS instrument, given the length of the interviews and challenges of recruiting busy physicians, addressing this issue could have a significant impact on the substantial percentage of NAMCS physician participants who identify as specialty care providers (45%).

Users of NAMCS PII surveys fielded prior to 2015 should consider the findings from this pilot study. Researchers interested in conducting analyses of large multispecialty practices should be aware that there may be variation in responses regarding location as well as less familiarity with administrative issues among certain physicians. This

may be noted as a possible limitation for analysis. For questions that may be most appropriate for primary care physicians (e.g., collection of vital signs), researchers may consider conducting sensitivity analyses of the impact of including or excluding responses by specialty physicians.

As part of NCHS’ continuing quality improvement effort, this small qualitative study was conducted to describe observations of the NAMCS PII data collection process at a single large multispecialty, multisite group practice, and findings may not apply to all similar practices. Based on a sample of eight participants, results from this pilot study are not generalizable. Furthermore, physicians in this pilot study were part of a practice setting with a very large number of physicians, as opposed to most NAMCS physicians who typically worked in either solo or group practices with an average of three to five physicians (18). Additionally, data in this study were collected in 2013 and may not reflect more recent adaptations of the NAMCS PII data collection process. Although this study examined data collected in 2013, the findings did result in specific changes to the collection of the 2015 NAMCS data. Further, findings in this report raise issues relevant to the hundreds of journal articles already published using data from the NAMCS 2013 survey and earlier versions, and therefore should be considered by those using these data for analyses.

Conclusions

Although based on only one large multispecialty practice, this study suggests that, as fielded in 2013, aspects of the NAMCS PII survey questions may be challenging for physicians in these practice settings to answer. Since then, as part of ongoing quality improvement efforts, NCHS has made modifications to FR trainings and NAMCS data collection procedures, including protocols to collect administrative information from an office manager or another designee rather than the sampled physician. These efforts are designed to improve the accuracy of

Page 10  Series 2, No. 176

the data, reduce respondent burden, and increase data collection efficiency.

References

1. Hsiao CJ, Cherry DK, BeattyPC, Rechtsteiner EA. NationalAmbulatory Medical Care Survey:2007 summary. National healthstatistics reports; no 27. Hyattsville,MD: National Center for HealthStatistics. 2010. Available from:https://www.cdc.gov/nchs/data/nhsr/nhsr027.pdf.

2. Kane CK, Emmons DW. New dataon physician practice arrangements:Private practice remains strongdespite shifts toward hospitalemployment. American MedicalAssociation. 2013. Available from:http://www.nmms.org/sites/default/files/images/2013_9_23_ama_survey_prp-physician-practice-arrangements.pdf.

3. National Center for Health Statistics.National Ambulatory Medical CareSurvey: 2008 summary tables.2011. Available from: https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2008_namcs_web_tables.pdf.

4. National Center for Health Statistics.National Ambulatory Medical CareSurvey: 2013 state and nationalsummary tables. 2016. Availablefrom: https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2013_namcs_web_tables.pdf.

5. National Center for Health Statistics.About health care surveys. 2015.Hyattsville, MD. Available from:https://www.cdc.gov/nchs/dhcs/about_dhcs.htm.

6. National Center for Health Statistics.2012 NAMCS micro-data filedocumentation. 2012. Availablefrom: ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NAMCS/doc2012.pdf.

7. National Center for Health Statistics.National Ambulatory Medical CareSurvey (NAMCS) and NationalHospital Ambulatory MedicalCare Survey (NHAMCS) list ofpublications. 2015. Available from:https://www.cdc.gov/nchs/data/ahcd/namcs_nhamcs_publication_list.pdf.

8. McLeod CC, Klabunde CN, WillisGB, Stark D. Health care providersurveys in the United States,2000–2010: A review. Evaluation &the Health Professions 36(1):106–26.2013.

9. Bishop TF, Federman AD, Ross JS.Laboratory test ordering at physicianoffices with and without on-sitelaboratories. J Gen Intern Med(10):1057–63. 2010.

10. Allen D. Chapter 18: Fieldworkand participant observation. In:Bourgeault I, Dingwall R, De VriesR (editors). Qualitative methods inhealth research. Thousand Oaks,CA: SAGE Publications, 353–72.2010.

11. Fowler FJ, Jr. Chapter 2: Codingthe behavior of interviewers andrespondents to evaluate surveyquestions. In: Madans J, MillerK, Maitland A, Willis G (editors).Question evaluation methods:Contributing to the science of dataquality. Hoboken, NJ: John Wiley& Sons, Inc. 5–21. 2011. Availablefrom: http://onlinelibrary.wiley.com/doi/10.1002/9781118037003.ch2/summary.

12. SAS Institute, Inc. SAS (Release 9)[computer software]. 2012.

13. National Center for Health Statistics.National Ambulatory Medical CareSurvey: 2012 state and nationalsummary tables. 2015. Availablefrom: https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2012_namcs_web_tableCouple of issuesstill, the "abstract"Abstracts.pdf.

14. Glaser BG, Strauss AL. Thediscovery of grounded theory:Strategies for qualitative research.New York, NY: TransactionPublishers. 2009.

15. Strauss AL, Corbin JM. 1990.Basics of qualitative research:Grounded theory procedures andtechniques. Thousand Oaks, CA:Sage Publications. 1990. Availablefrom: http://books.google.com/books?id=nvwOAQAAMAAJ&pgis=1.

16. Decker SL, Jamoom EW, Sisk JE.Physicians in nonprimary care andsmall practices and those age 55and older lag in adopting electronichealth record systems. Health Aff(Project Hope) 31(5):1108–14. 2012.

17. Luft HS. Policy-oriented researchon improved physician incentivesfor higher value health care. HealthServices Research 50(S2):2187–215.2015.

18. Hing E, Rui P, Palso K. NationalAmbulatory Medical Care Survey:2013 state and national summarytables. National Center for HealthStatistics. 2013. Available from:https://www.cdc.gov/nchs/data/ahcd/namcs_summary/2013_namcs_web_tables.pdf.

Vital and Health Statistics Series Descriptions

Active SeriesSeries 1. Programs and Collection Procedures

Reports describe the programs and data systems of the National Center for Health Statistics, and the data collection and survey methods used. Series 1 reports also include definitions, survey design, estimation, and other material necessary for understanding and analyzing the data.

Series 2. Data Evaluation and Methods ResearchReports present new statistical methodology including experimental tests of new survey methods, studies of vital and health statistics collection methods, new analytical techniques, objective evaluations of reliability of collected data, and contributions to statistical theory. Reports also include comparison of U.S. methodology with those of other countries.

Series 3. Analytical and Epidemiological StudiesReports present data analyses, epidemiological studies, and descriptive statistics based on national surveys and data systems. As of 2015, Series 3 includes reports that would have previously been published in Series 5, 10–15, and 20–23.

Discontinued SeriesSeries 4. Documents and Committee Reports

Reports contain findings of major committees concerned with vital and health statistics and documents. The last Series 4 report was published in 2002; these are now included in Series 2 or another appropriate series.

Series 5. International Vital and Health Statistics ReportsReports present analytical and descriptive comparisons of U.S. vital and health statistics with those of other countries. The last Series 5 report was published in 2003; these are now included in Series 3 or another appropriate series.

Series 6. Cognition and Survey MeasurementReports use methods of cognitive science to design, evaluate, and test survey instruments. The last Series 6 report was published in 1999; these are now included in Series 2.

Series 10. Data From the National Health Interview SurveyReports present statistics on illness; accidental injuries; disability; use of hospital, medical, dental, and other services; and other health-related topics. As of 2015, these are included in Series 3.

Series 11. Data From the National Health Examination Survey, the National Health and Nutrition Examination Surveys, and the Hispanic Health and Nutrition Examination SurveyReports present 1) estimates of the medically defined prevalence of specific diseases in the United States and the distribution of the population with respect to physical, physiological, and psychological characteristics and 2) analysis of relationships among the various measurements. As of 2015, these are included in Series 3.

Series 12. Data From the Institutionalized Population SurveysThe last Series 12 report was published in 1974; these reports were included in Series 13, and as of 2015 are in Series 3.

Series 13. Data From the National Health Care SurveyReports present statistics on health resources and use of health care resources based on data collected from health care providers and provider records. As of 2015, these reports are included in Series 3.

Series 14. Data on Health Resources: Manpower and FacilitiesThe last Series 14 report was published in 1989; these reports were included in Series 13, and are now included in Series 3.

Series 15. Data From Special SurveysReports contain statistics on health and health-related topics from surveys that are not a part of the continuing data systems of the National Center for Health Statistics. The last Series 15 report was published in 2002; these reports are now included in Series 3.

Series 16. Compilations of Advance Data From Vital and Health StatisticsThe last Series 16 report was published in 1996. All reports are available online; compilations are no longer needed.

Series 20. Data on MortalityReports include analyses by cause of death and demographic variables, and geographic and trend analyses. The last Series 20 report was published in 2007; these reports are now included in Series 3.

Series 21. Data on Natality, Marriage, and DivorceReports include analyses by health and demographic variables, and geographic and trend analyses. The last Series 21 report was published in 2006; these reports are now included in Series 3.

Series 22. Data From the National Mortality and Natality SurveysThe last Series 22 report was published in 1973. Reports from sample surveys of vital records were included in Series 20 or 21, and are now included in Series 3.

Series 23. Data From the National Survey of Family GrowthReports contain statistics on factors that affect birth rates, factors affecting the formation and dissolution of families, and behavior related to the risk of HIV and other sexually transmitted diseases. The last Series 23 report was published in 2011; these reports are now included in Series 3.

Series 24. Compilations of Data on Natality, Mortality, Marriage, and DivorceThe last Series 24 report was published in 1996. All reports are available online; compilations are no longer needed.

For answers to questions about this report or for a list of reports published in these series, contact:

Information Dissemination StaffNational Center for Health StatisticsCenters for Disease Control and Prevention3311 Toledo Road, Room 4551, MS P08Hyattsville, MD 20782

Tel: 1–800–CDC–INFO (1–800–232–4636)TTY: 1–888–232–6348Internet: https://www.cdc.gov/nchsOnline request form: https://www.cdc.gov/infoFor e-mail updates on NCHS publication releases, subscribe online at: https://www.cdc.gov/nchs/govdelivery.htm.

U.S. DEPARTMENT OF HEALTH & HUMAN SERVICES

Centers for Disease Control and Prevention National Center for Health Statistics 3311 Toledo Road, Room 4551, MS P08 Hyattsville, MD 20782–2064

OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300

FIRST CLASS MAIL POSTAGE & FEES PAID

CDC/NCHS PERMIT NO. G-284

DHHS Publication No. 2018–1376, Series 2, No. 176CS282973