the oversight of medical care: a proposal for reform · the oversight of medical care: a proposal...

POSITION PAPER

The Oversight of Medical Care: A Proposal for Reform American College of Physicians*

W The oversight of medical care, in the form of peer review, has traditionally been used to ensure that the highest standards of care are maintained. What is relatively new is the external oversight of medical practice carried out by a growing number of indepen- dent entities (government, third-party payers, for-profit firms, for example), overseeing care in uncoordinated ways. Tensions arise when reviews of utilization and reviews of quality are conducted by different organizations with conflicting goals. The review instruments are still crude and have neither been adequately tested nor validated. Future attention to developing reliable and valid measures of efficiency and quality is essential. Evidence suggests that the principal process of review, the case-by-case review, may not be cost-effective and may not be conducive to improving quality. It should be replaced by profiles of practice patterns at institutional, regional, or national levels. We propose a model of oversight that emphasizes the appropriate balance between internal mechanisms of quality improvement and external accountability. In this model, internal and external reviews have specific, complementary roles that promote efficiency and quality. Detailed monitoring of quality and problem solving are left to providers who are intimately involved with care. In return, they be- come accountable to payers and the public through the surveillance of patterns of practice.

Ann Intern Med. 1994;120:423-431.

I t is a reality of the American health care system: The practice of medicine is being closely watched. Although the oversight of medical care, in the form of peer review, has traditionally been used to ensure that the highest standards of care are maintained (l-4), what is relatively new is the external oversight of medical practice being carried out by a growing number of indepen- dent entities such as government and third-party payers (5, 6). Health care expenditures have continued to grow, and society is looking for ways to slow this rate

* This paper, authored by Anne-Marie Audet, MD, MSc, SM, and H. Denman Scott, MD, MPH, was developed for the Health and Public Policy Committee: Clifton R. Cleveland, MD, Chair; Cecil 0. Samuel- son, MD, Mce-Chair; Christine K. Cassel, MD; David J. Gullen, MD; Ernest L. Mazzaferri, MD; Quentin D. Young, MD; Whitney Adding ton, MD; Robert A. Berenson, MD; John M. Eisenberg, MD; Nancy E. Gary, MD; P. Preston Reynolds, MD; Gerald E. Thomson, MD; Mack V. Traynor, Jr., MD; and Sankey V. Williams, MD. Approved by the Board of Regents on 29 March 1993.

of rise. Many studies have identified wide geographic variations in clinical management strategies (7, 8). Oth- ers have shown that some of the care delivered is done so inappropriately (9, 10). These findings, coupled with a sense of frustration regarding escalating costs, have prompted those who pay for health care to resort to intensive review of clinical decisions to control costs by getting rid of unnecessary and inappropriate care.

The problems with the current system of medical oversight are as follows: First, too many external parties are overseeing narrowly defined aspects of care in uncoordinated ways. The growth of external oversight has occurred with little systematic thinking and has led to unnecessary duplication and to conflicting oversight activities, some focused on costs and some focused on quality-the so-called “hassle factor.” As a result, many programs have been poorly conceived and have relied on unscientific and nonvalidated methods. Our nation’s health care system has no organized framework to evaluate, monitor, and promote cost-effectiveness and quality (11). Finally, little evidence is available about the overall effect of these various review activities on cost, quality, and efficiency.

Within this context, physicians, patients, and payers have voiced growing concerns about the oversight of care and, in particular, about utilization review programs. (A glossary of frequently used terms can be found in Appendix 1 at the end of the text.) Physicians doubt that utilization review improves the appropriateness of health care. Payers are also beginning to question its value and to require from firms that provide utilization review services quantitative evidence that costs are being controlled. Further, because quality is neither a priority nor an element for which utilization review organizations have been held accountable, questions must be raised about the effects of utilization review on the public’s health. Patients may be denied access to services they need when cost control, not quality, is the main priority.

This paper expands on the concepts in “Universal Insurance for American Health Care,” the American College of Physicians’ proposal for the reform of the U.S. health care system (12). In that proposal, the Col- lege recommended that the external oversight of care be restructured and that “the primary locus of quality assurance (Appendix 1) be returned to the medical profession and to health institutions. Clinicians should be responsible for providing high-quality, cost-effective care, and be accountable for efficient use of resources.”

As the nation seeks solutions to the concerns of the medical profession, the payers of health care, and the public, it must carefully consider rational approaches to utilization review and quality improvement (Appendix 1). The goal of medical oversight should be to promote quality and efficiency. At a time when pressure to con-

01994 American College of Physicians 423

trol costs is overwhelming, it is important to ensure that incentives do not compromise the quality of care and that a balance among cost, access, and quality is maintained. The following recommendations address the changes needed to create the credibility and promote the high standards that are crucial to any system of oversight.

The Oversight of Utilization in the Private and Public Sectors

In the past decade, the oversight of medical care has predominantly focused on utilization of health care services (for example, length of stay, number of inpatient days per enrollee, number of ancillary tests) and much less on quality. The growth in the number of organizations that perform utilization review (both private and public) has been unprecedented. In 1983, 14% of cor- porate benefit plans required prior approval for non- emergency admissions. By 1988, this number had risen to 95% (11). In 1989, the utilization review market included about 125 private review organizations, and in January 1992, this number approached 350. Most of these organizations are freestanding, are for profit, often do not have contra< :Y other formal relationships with the providers of health care, and frequently use unilaterally developed, secret criteria to evaluate care. Some insurance companies have developed their own utilization review programs. Utilization review also oc- curs as a provider-based activity, where a health care institution reviews its own experience as an alternative to having an external party provide these services. In the public sector, the care delivered to Medicare beneficiaries is subject to review by the federally funded, nonprofit peer review organizations.

Despite this flurry of activity, little is known about the effect of utilization review on costs, on the quality of care, and on the practice of medicine. The limited number of available studies thus underscores the impor- tance of improving our knowledge about how and to what extent the current system of utilization review affects our nation’s health care system. Most studies have dealt with issues of costs (13, 14), and very few have addressed issues such as clinical outcomes or quality of care (15). Because of methodologic limita- tions, most studies are unable to isolate the effects of utilization review from other factors (for example, the Prospective Payment System, the Resource-based Rel- ative Value System, clinical guidelines (Appendix l), or other policies and professional activities) (16). Never- theless, the available data suggest that utilization review activities have played a role in decreasing inpatient hospital use, such as in decreasing the number of inpatient days per 1000 enrollees and inpatient hospital costs. The effect of utilization review appears to be greater when utilization is high.

Several critics have argued that these effects rep& sent one-time cost savings and have little to do with the overall rate of rise of health care costs (11). They argue that because most utilization review programs have focused on inpatient care, the net financial effect will be blunted by the shift of care to outpatient settings. The most important limitation of most studies, however, is

their failure to evaluate the overall effect on societal costs and on quality. Administrative costs, or the costs borne by patients or physicians as a result of utilization review activities, are usually not included in cost cal- culations (for example, benefit-to-cost ratios), and they may increase net costs. This creates a bias that favors the cost-saving effect of utilization review. A recent General Accounting Office report suggests the magni- tude of the financial burden imposed on the health care system. In 1991, gross revenues reported by utilization review firms ranged from $100 000 to $62 million, with an average of $4.8 million per firm (17). Clearly, these costs must be added to the nation’s health care bill.

The intrusion of uiilization review into the practice of medicine varies according to a physician’s specialty. A 1990 survey by the American Medical Association revealed that approximately 50% of internists had under- gone utilization review in the month before the survey. In addition, the average number of reviews was 3.5 per week. For surgeons, the corresponding figures were 70% and 5.0 reviews per week (IS). Of course, these figures do not reflect the time that physicians may need to spend personally in response to utilization review. Whereas some physicians may be able to delegate work to personnel hired specifically for such administrative tasks, others may have to substitute clinical time to fulfill these obligations.

Reviews of utilization have continued to be the dom- inant method used to evaluate care and to guide decision making. Nevertheless, the issues related to quality of care have been seriously neglected. In a 1989 study, “Controlling Costs and Changing Patient Care? The Role of Utilization Management,” the Institute of Medicine defined utilization review as “a set of techniques used by or on behalf of purchasers of health benefits to manage health care costs by influencing patient care decision-making through case-by-case assessments of the appropriateness of care before its pro- vision” (Appendix 1) (11). Quality assurance was defined as “organized programs to protect and improve the quality of care, by evaluating medical care, correct- ing problems and monitoring corrective actions.” These working definitions clearly dissociate, at the operational level, reviews of utilization from those of quality. They create tensions because quality (for example, the de- crease in breast cancer mortality) is directly affected by the use of services (for example, screening mammogra- phy). Problems arise because utilization review organizations use cost as the central criterion to evaluate the delivery of care, whereas those involved in quality- improvement activities focus on clinical outcomes to evaluate the same health services. Conflicts can easily occur when reviews of utilization and reviews of quality are separate activities conducted by different external bodies. In this situation, it is understandable that uncoordinated evaluations could result in care being judged as appropriate according to one set of criteria and inappropriate according to another. These arrange- ments may also create incentives that favor underutilization of services, especially if cost is emphasized over quality.

424 1 March 1994 l Annals of Internal Medicine l Volume 120 l Number 5

An Integrated Approach to the Oversight of Utilization and Quality

A reformed system of medical oversight should focus equally on cost, quality, and efficiency. In an integrated system, those who make decisions that affect costs should also be accountable for the clinical conse- quences of such decisions. Similarly, those who make decisions that affect clinical outcomes and quality of care should be accountable for the financial implications of their decisions.

The oversight of care cannot be entrusted solely to groups external to the clinical setting. Appropriate roles and relationships are essential to ensure that external and internal reviews complement and support and do not undermine quality-improvement activities. The medical profession can and must also be involved and be accountable to payers and patients for both efficiency and quality. Only those who directly provide care will thoroughly understand its complexities and be able to define effective solutions to utilization and quality problems.

Reforming the system of oversight also requires that methods and processes match the objectives and goals. The current methods of reviews, especially the case-by- case, implicit reviews, are deficient in that they are unreliable or have not been validated, or both. In many cases, the ititruments that are needed have not yet been fully developed (19). Measures of utilization and quality are also still in their infancy and require a ded- icated investment in research.

Science-based, Explicit Review Criteria in Lieu of Secret, Implicit Criteria

Many utilization review organizations currently evaluate health care services by applying secret rules that they claim to be proprietary; others make decisions based on implicit, ad hoc judgments by reviewers. There is good evidence to suggest that implicit reviews are neither reliable nor valid predictors of problems of utilization or quality (20, 21). Many explicit criteria have been developed without any defined methodology; only a few are based on rigorous methods (ll), and organizations do not always assess whether they are applied uniformly by reviewing nurses and physicians.

Ad hoc rules should be replaced by explicit criteria that have been developed through a formal and accepted process. Implicit reviews by physicians can still have a role but should be done within a structured process that has been shown to be reliable and valid (22). Review criteria should be open to public scrutiny and subject to regular update as new knowledge be- comes available. Different organizations may develop guidelines to assist physicians in the diagnosis or treat- ment of specific clinical conditions. Sometimes their recommendations will disagree. For example, the age threshold for a yearly mammogram in the screening for breast cancer is 40 years according to the American Cancer Society but 50 years according to the American College of Physicians, the U.S. Preventive Services Task Force, and the National Cancer Institute. A fair and valid process to resolve conflicts can only occur in

an environment of openness. Secrecy around review criteria hinders the flow of information and only defeats efforts to improve quality.

Explicit Methods for Translating Guidelines into Utilization and Quality Criteria

One study of the Medicare Peer Review Organization program (23) underscores the need for an explicit process for developing review criteria that adequately reflect the aspects of care they evaluate. It showed the wide state-by-state variation in the criteria used to judge the need for certain surgical interventions. Criteria dif- fered despite the fact that they were presumed to be based on the same guidelines developed by national organizations such as the American College of Physi- cians (for carotid endarterectomy), by the American College of Cardiology (for cardiac pacemaker implanta- tion), and by the American Academy of Ophthalmology (for cataract removal).

Clinical practice guidelines that assist physicians and patients in their clinical decisions and review criteria that evaluate those same clinical decisions should be developed simultaneously, using the same knowledge base and decision-making heuristic to ensure consis- tency. Simply translating guidelines into review criteria without using an explicit process may lead to criteria that do not reflect the clinical recommendations of the guidelines and that may even contradict them (23).

In the last decade, the methodology for developing guidelines has been the focus of several health services research initiatives. Although the science has pro- gressed, several questions regarding their validity and their effects on quality and costs still remain (Eisenberg JM, Foster NE. Using clinical guidelines: impact on public policy, clinical policy, and individual care decisions. Presented at the National Quality of Care Forum, 15-16 April 1993, Lansdowne, Pennsylvania) (24, 25). On the other hand, the development of measures of utilization and quality has not received the same degree of attention (26). Crude measures of utilization (unad- justed lengths of stay, yearly number of clinic visits, number of diagnostic tests ordered, number of medica- tions prescribed, and so forth) are still being used. Al- though they may serve as rough indicators of short-term costs, they remain unsatisfactory as measures of true societal and long-term health care costs. Measures of quality are even less refined and in many cases consist of indicators of resource use as proxies for quality or of crude measures, such as in-hospital mortality rates.

Just as the medical profession has taken a leadership position in the development of clinical practice guidelines, it should lead in the development of review criteria. Defining reliable and valid measures of utilization and quality, however, is not an easy task. Guidelines can more easily address a continuum of complex clinical circumstances and offer alternative diagnostic and therapeutic options. Review criteria, on the other hand, usually apply only to discrete aspects of care (such as the percentage of patients with an acute myocardial infarction who receive thrombolytic therapy). A narrow focus is often chosen so that data can be abstracted and statistically analyzed for profiling (Appendix 1). The

1 March 1994 l Annals of Internal Medicine l Volume 120 l Number 5 425

consequence is that without careful attention to how they are defined, review criteria may not capture the important aspects of clinical care.

If not thoughtfully developed, review criteria may inadvertently conflict with the general principles of the guidelines from which they are derived. For example, a recently published guideline stressed that the hemoglobin threshold (traditionally set at 5100 g/L) for red blood cell transfusions should be abandoned (27). Deci- sions to transfuse should be based on a patient’s clinical status (the presence of certain signs and symptoms such as angina, syncope in the presence of acute or chronic blood loss) and not on any arbitrary hemoglobin level. The guideline also mentioned that studies indicate that asymptomatic patients tolerate hemoglobin levels as low as 75 g/L very well. This statement was misinterpreted by one reviewer as meaning that a lower hemoglobin threshold of 75 g/L should replace the traditional one (28), in effect violating the new paradigm proposed in the guideline. However, beyond the possibility of pure misunderstanding of guidelines, this example also raises the question of whether review criteria based on a sim- ple statistical measure will encourage practitioners to satisfy the requirements of the review screen (how many transfusions are given to patients with hemoglobin levels above 75 g/L) rather than promote practice based on careful clinical evaluation. It is also possible that one guideline could be used to generate different review criteria, depending on the goals pursued-cost control or improved outcomes. Issues such as this one need careful study.

Profiles of Patterns of Care in Lieu of Case-by-Case Reviews

We have discussed how reviews of care can be cen- tered around economic criteria or around criteria of quality. We have also stressed that the credibility of any system of oversight depends on the validity of the criteria used to review and evaluate care. But the process of review is equally important. Reviews can focus on the individual patient managed by a particular physician, the case-by-case reviews that are currently the principal method of review. Reviews can also be based on the analysis of large numbers of patients managed by groups of physicians. This results in profiles of institutional, regional, or national patterns of practice.

Evidence suggests that case-by-case reviews are inefficient in identifying problems of utilization or quality. The process disrupts the physician-patient relationship in ways that may impede quality instead of promoting it. Studies have raised substantial concerns about the inadequate levels of sensitivity (the ability of case-by- case reviews to identify true problems of quality) and specificity (the ability of case-by-case reviews not to falsely label as quality problems those cases that are not) of such reviews done in both the private sector and by the peer review organizations (20, 21, 29). Because of the low sensitivity of such reviews, large numbers of medical records must be evaluated before problems are identified and confirmed, making this an unduly costly method. For example, of the 9445 prior-authorization cases reviewed by the Maine Peer Review Organization

in 1990, only 6 (0.06%) were finally denied (Health Care Review Incorporated. First Annual Report. State of Maine; 1990). Of the 11 294 inpatient quality-of-care cases reviewed, 71 (0.63%) were confirmed as true quality problems. The initial screening process by nurses has also been shown to be inefficient in identifying quality problems; as a result, physician reviewers spend needless time and resources reviewing cases in which adequate care was delivered. In a pilot study involving 21 312 randomly selected medical records in seven peer review organizations, the overall referral rate for quality from nurses to physicians was 35%, but only 6% of the referred cases had confirmed quality problems (Phase I Results of the Uniform Clinical Data Set [UCDS; Ap- pendix l] Implementation, September 1991 to January 1992. Health Care Financing Administration. Office of Peer Review. Health Standards and Quality Bureau. Baltimore, Maryland).

Evidence also suggests that the case-by-case review is not a reliable method for judging quality of care. A recent analysis of published studies of peer review methods concluded that in most cases, peer agreement concerning quality was only slightly better than that expected by chance (21). Such poor inter-rater reliability is not surprising because typical case-by-case reviews focus on a single episode of care within a limited time frame. They cannot reflect the complex series of management decisions that constitute a patient’s overall care. These observations underscore that constructive oversight will succeed only if methods of review are improved so that they can reliably and efficiently identify suboptimal care.

On the basis of these data, the American College of Physicians recommends that routine case-by-case reviews be abandoned and replaced by profiling of patterns of care. Profiling, as defined by the Physician Payment Review Commission (30), “uses epidemiologic methods to compare patterns of providers on the dimensions of cost, service use, or quality (process and outcomes) of care. Theoretically at least, profiling can be an effective and relatively non-intrusive way to identify over- and underutilization of services, to uncover problems with the efficiency and quality of care, and to assess provider performance.” The percentage of diabetic patients referred for yearly eye examinations, analyzed at an institutional, regional, or national level, is an example of a profile that could be used to assess the quality of care of diabetic patients. It could identify practice patterns that merit detailed scrutiny and selected individual review.

Interest in profiling is growing, and its applications are promising. Several statewide profiling projects have been implemented and are currently being evaluated. For several years, New York State has had a profile- based system for evaluating the care of patients who have had coronary artery bypass grafting. This voluntary reporting system is based on pattern analysis and uses validated measures of quality. The program has been well accepted and successful in improving care. Recent data indicate that New York state has achieved low surgical mortality rates for coronary artery bypass grafting compared with other states (31). This program benefited from the substantial amount of information


that has accumulated over the years from many studies about coronary artery bypass surgery. Expanding this model to other clinical conditions will not be easy and will require major investment in research and evaluation.

The Minnesota Clinical Comparison and Assessment Program is another statewide initiative to collect data, analyze, and disseminate comparative information about patterns of care and outcomes. Trends in care and variations in practice are monitored and used to suggest areas for improvement and research. This program is a voluntary consortium of the Council of Hospital Corpo- rations, County Medical Societies, the State Medical Association, and the Minnesota School of Public Health. Preliminary results are encouraging (32), but further evaluation is essential to assess fully its effects on quality and efficiency.

The Maine Medical Assessment Foundation is another example of successful population-based profiling. For 15 years, study groups in 10 specialties have used small-area analysis to evaluate surgical and medical care across the state. The focus has been mostly inpatient care, with more attention recently being paid to ambulatory services. This initiative has shown that physicians can overcome data problems, conduct important research, and work toward improving the quality of medical care. For example, at one point in time in the past decade, the state’s regional variation in the rate of hysterectomy was almost threefold. Physicians agreed that this variation was inappropriate, but they lacked the data to understand the reasons why. This led to a community-based prospective cohort study of treat- ment alternatives for benign uterine conditions. As a result of this process, the rates of hysterectomy in two geographical areas dropped markedly.

Many other organizations are beginning to explore the value of profiling in continuous quality-improvement efforts. These include, for example, the Community- based Ambulatory Care Network _in Rochester, New York, the University Hospital Consortium (national consortium of more than 50 teaching hospitals), and the Academic Medical Center Consortium, composed of 12 medical centers in the United States, whose primary goal is to show how the results of collaborative studies can be integrated into the management and quality- improvement practices of the member institutions. Pri- vate insurers and managed-care organizations are also starting to examine practice patterns (for example, the percentage of eligible women who received a mammogram in the previous 12 months). Profiling is thus be- coming an important way for providers of care to gather relevant information about the quality and the efficiency of the services they offer, both for internal purposes and for reporting to external parties such as payers and the public.

Some areas of clinical practice will be simpler to monitor and evaluate, such as when processes like the use of a medication (for example, penicillin) have a direct effect on the clinical outcome (cure of a strepto- coccal throat infection) or when the system-level processes of care are relatively easy to trace. However, in many instances, this will not be the case. The complexity of the medical care that internists provide to patients

who have several coexisting conditions of various de- grees of severity requires that the methods for profiling reach an appropriate level of sophistication. The benefits of profiling depend on valid disease-specific models of care. These models need to go beyond the current ones that emphasize process measures and begin to include clinical outcomes. Knowledge about clinical effectiveness and outcomes is beginning to accrue, and results from studies should, in the coming years, inform and find important applications to quality-improvement programs.

Expectations for profiling should therefore be realis- tic. Even if it includes several dimensions of care, profiling will not provide a detailed understanding of the entire episode of care. This is especially true for the practice of internists. The anticipated patient outcomes (improvement or worsening of disease-specific symptoms, of functional state, and of social functioning; satisfaction with care), the availability of alternative management strategies, patients’ preferences as to these alternatives, uncertainty about risks and benefits of many clinical interventions, and the availability of resources are all criteria that affect clinical decision making. These multiple and interrelated factors make clinical practice complex. This complexity cannot be easily captured in statistical profiles, which, for practical reasons, can only measure specific and rather narrow aspects of care. Because all aspects of care cannot be evaluated, it is essential to select the most important ones. For example, processes of care that should be profiled include those that have posed problems with quality as identified through studies of practice variations or studies of outcomes. Processes of care that have a substantial effect on resource utilization and on clinical outcomes should also be given priority.

A Stepwise Model for Medical Oversight Using Practice Profiling

We propose a systematic approach to the oversight of care. The review process consists of a series of steps that lead to continuous improvements in quality and efficiency. Each step has a specific role and provides information that is necessary to the next step. Coordi- nation is therefore essential to the overall cycle of oversight. This model outlines the different but complementary roles for external and internal reviews. It proposes a balance between these two forms of review activities, a balance that is needed to promote quality improvement within our health care system (Table 1).

At each level, the type and intensity of the monitoring activity differ. The type of profiles that need to be generated, the data required for such profiles, and the appropriate locus for conducting the oversight also vary accordingly. Using the early management of patients with myocardial infarction as an example, Table 2 outlines a mode1 for evaluating quality. The focus is on one specific process of care, the use of thrombolytic therapy. This process of care was selected as an important measure of quality because there is good evidence that the use of thrombolytic agents improves survival (33, 34). A comprehensive oversight program would also consider other relevant processes of care (such as dose,

1 March 1994 l Annals of Internal Medicine l Volume 120 l Number 5 427

Table 1. Medical Oversight: The Major Elements of Reform

Current System

The oversight of care is governed primarily by parties external to the organizations providing health care (Joint Commission on Accreditation of Healthcare Organizations, the Peer Review Organizations, major insurers, private for-profit utilization review organizations). Internal programs are often given priority to complying with external standards, due to resource constraints.

The oversight of care is done by several organizations, some focusing on quality, others on costs. A lack of coordination leads to duplication and to conflicting activities. In such an environment, quality and efficiency are not promoted.

Cost control is one of the primary goals, quality is not. Reviews of utilization (length of stay, use of ancillary services, for example) are often used as proxies for quality.

Implicit and secret review criteria are used to assess care.

Review criteria are often developed on an ad hoc basis or without defined methods. There are no mechanisms to ensure that criteria reflect current knowledge.

Reliable and valid measures of quality and efficiency are few and focus on narrowly defined aspects of care (for example, immunizations, some acute conditions). Measures are usually not tested for their reliability and validity before implementation.

Medical oversight still focuses primarily on structure and process, within a limited time frame.

Reformed System

A balance between external and internal oversight programs is achieved. Both forms of oversight are given adequate priority and are interdependent in the overall system that leads to continuous quality improvement. The medical profession is primarily involved in quality-improvement activities and is accountable to the public for quality and efficiency.

A systematic, stepwise model for the oversight of care can promote quality and efficiency (Table 2). Each step in the oversight process is essential to the next. External and internal reviews have specific and complementary roles.

Quality and efficiency are the primary goals. Instruments for reviews of utilizatiorrand reviews of quality are valid and specifically measure either utilization or quality.

Explicit, open review criteria are used to assess care.

Review criteria are developed through explicit, accepted methods. This ensures that criteria reflect current, evidence-based knowledge of clinical practice and that regular updates are scheduled and are consistent with new knowledge.

Quality measures have shown reliability and validity. A more comprehensive assessment of care includes care for acute conditions but also care for chronic conditions, home care, long-term care, and preventive care. Selective assessments of health care delivery can focus on conditions and treatments of broad public interest (for example, high incidence, high resource use, and high burden of illness).

Medical oversight encompasses structure, process, and the outcomes of care over a long-term time frame. Attention is given to outcomes such as quality of life, return to work, functional status, and satisfaction with care.

timing, and the associated care given to patients with acute myocardial infarction) and profile them. Of course, focusing on processes of care is not sufficient, and clinical outcomes, such as symptom relief, return to function, or survival also need to be profiled.

At least four levels of review can be identified: sur- veilling possible problems, developing and testing a hypothesis about the existence and the causes of a quality problem, confirming or rejecting the hypothesis, and monitoring the corrective responses to the problem.

Surveillance consists of a search for unusual variations in patterns of care and of the monitoring of trends over time. In the case of thrombolytic use, surveillance is done by reviewing the crude rate of use for the nation, for states, for regions, and for institutions and health plans within these regions. This information iden- tifies areas that warrant further evaluation. But surveillance profiles only raise questions that need explora- tion; they should not be used to make final judgments about cost-effectiveness or quality. Generating such profiles should require few, easily available data. It is through information obtained at this level that external parties will be able to gauge how well the delivery system is working-the accountability check that ensures efficiency and quality.

To reach more definitive conclusions (the evaluative component), it is necessary to gather and analyze additional data, discuss the issues with physicians and oth-

ers involved to define the problem, to identify possible causes, to test theories, and to monitor responses to problem-solving strategies. This may require highly specific data that can only be obtained through physician or patient surveys or through more in-depth analysis of institutional processes of care (for example, the time from order by the physician to actual delivery to the patient of thrombolytic agents in an emergency situation). Although the initial surveillance may occur both at a regional and the national level, this more intensive review must happen at the level of local health care institutions. It is only through provider-based review systems that a comprehensive understanding of the complex interactions in the delivery of care can be achieved and effective solutions developed.

How might this work in practice? Using the care of patients with myocardial infarction as an example, we can imagine that surveillance profiles revealed a low rate of thrombolytic use in a specific geographic region. This alone does not mean that a quality problem exists. For instance, the low rate may be related to the patient population in the region (a high percentage of patients may have major contraindications to the use of thrombolytic therapy or may not fit into the recognized cat- egories of patients that benefit from this intervention). On the other hand, the low rate of use may represent underutilization, suggesting a quality problem. This could be due to physicians’ lack of knowledge about the


Table 2. A Stepwise Model for the Oversight of Care: Thrombolytic Therapy in the Care of Patients with Acute Myocardial Infarction

Vpe of Review

Primary Role for Data

(Informative or Evaluative)

Type of Profile

Surveillance

Confirming the existence of a quality problem

Testing hypotheses about causes and monitoring responses to identified problems

Informative Rates of thrombolytic therapy use

Evaluative

Evaluative

Rates of appropriate and inappropriate use of thrombolytic therapy

Physician knowledge

Delivery system processes of care

-

Data Require-

ment

Availability of Data

Primary Responsibility for Review

Geo- Organiza- graphic tional

LOW Easily available: Claims data Large databases (for

example, UCDS*) Medical records National,

regional, or local

High Not easily available: Medical records Condition-specific Local

encounter forms Very high Not easily available:

Physician surveys Analysis of institu- Local

tional processes of care

External or internal

Internal

Internal

* UCDS = Uniform Clinical Data Set.

appropriateness of thrombolytic therapy or to inadequate coordination of care within the delivery system, resulting in delays and the inability to administer thrombolytic therapy within the time frame required for it to be effective.

Additional and more specific data would then be needed, data that could only be obtained by abstracting medical records. At this stage, the review process should include the active participation of those involved in the local delivery of care. An assessment of institutional processes of care might even be necessary and should involve medical, nursing, pharmacy, and administrative staff.

Let us suppose that, in our example, the cause of the low rate of thrombolytic use was found to be lack of knowledge. In response, educational programs would be implemented and the use of thrombolytic therapy closely monitored by both the institution and by the local and regional (state) authorities charged with monitoring patterns of care.

The validity of models for medical oversight such as the one proposed here; their acceptability for clinicians, patients, and payers; and their effect on medical practice itself need to be tested. Profiles that compare practices at the level of the community, the region, or the nation must also be fair and relevant. As the factors that come into play in the diagnosis and management of specific clinical conditions increase in number, so will the challenges to the medical profession in finding profiles that can effectively lead to quality improvement. For example, seminal work has confirmed that patient-mix, that is, patients’ sociodemographic status, illness severity, and functional status, is a significant determinant of utilization and patient outcomes (35, 36). Valid profiling thus requires sophisticated adjustment methods that take into account patient and provider factors that independently affect processes and outcomes of care.

Conclusions

Our recommendations have focused on perfecting methods and processes used in the oversight of medical care. As health care professionals, we need to continue to learn and develop innovative approaches to improve the quality and the efficiency of the care we deliver (37). Population-based, longitudinal profiles, based on valid measures of utilization and quality, can contribute substantially to these goals. The balance between external and internal reviews must also be finely adjusted to promote positive changes in practice (see Table 1).

The environment in which these instruments and methods are implemented is also important to the suc- cess of quality-improvement initiatives. The dissemina- tion of information regarding utilization and quality in the form of profiles, however reliable and valid they may be, can only enhance knowledge and improve practice if the principles and the goals of this activity are credible to the profession (38). One of the greatest challenges to the profession is therefore its willingness to develop creative models of quality improvement and to test their effectiveness. This process entails looking at how to incorporate oversight processes, such as those described here, into various health care programs; de- termining factors that affect organizational readiness to adopt quality-improvement methods; and finding out what works and does not work in settings as different as acute care, long-term care, primary care or tertiary care settings, in rural areas and in the inner cities. These activities should undoubtedly be a priority of our nation’s health care research agenda.

Appendix 1. Glossary of Frequently Used Terms

Practice guidelines: Systematically developed state- ments to assist practitioner and patient decisions about

1 March 1994 * Annals of Infernal Medicine * Volume 120 l Number 5 429

appropriate health care for specific clinical circumstances (2.5).

Profile analysis: Use of aggregate statistical data at the level of a practitioner, a region, or the nation to compare and assess various characteristics of practice patterns.

Quality of care: The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge (5).

Quality assessment: The measurement of the techni- cal and interpersonal aspects of health care and the outcomes of that care (5).

Quality assurance: A formal and systematic exercise in identifying problems in medical care delivery, design- ing activities to overcome the problems, and carrying out follow-up monitoring to ensure that no new problem has been introduced and that corrective steps have been effective (38). Quality assurance programs usually focus on individuals (outliers) and consist of short-term problem-solving activities. Such programs are usually structured along separate clinical departments within an organization and do not necessarily foster their integra- tion.

Quality improvement: A set of techniques for the continuous study and improvement of the processes of delivering health care services and products to meet the needs and expectations of the customers of those services and products (5, 38). Quality-improvement programs focus on systems (as opposed to individuals only) and emphasize continuous, long-term improvement. Programs involve organization-wide, interdisciplinary, and interdepartmental collaboration.

Review criteria: Rules and measures by which the necessity and the appropriateness of a medical service are evaluated (11).

Uniform Clinical Data Set (UCDS): A computerized data collection system, containing approximately 1800 elements that describe patient demographic characteristics, clinical history, clinical findings, and therapeutic interventions. The UCDS is the proposed new national data set for Medicare’s Quality Assurance program. It will be used to evaluate the quality of care of Medicare beneficiaries. It is currently in a developmental and testing phase (19).

Utilization review: A set of techniques used by or on behalf of purchasers of health benefits to manage health care costs by influencing patient care decision making through case-by-case assessments of the appropriateness of care before it is provided (11).

Requests for Reprints: Linda Johnson White, Director, American Col- lege of Physicians, Independence Mall West, Sixth Street at Race, Philadelphia, PA 19106-1572.

References

1. Brook RH. Quality of care: do we care? [Editorial]. Ann Intern Med. 1991;115:486-90.

2. Schroeder SA, Kabcenell AI. Do bad outcomes mean substandard care? JAMA. 1991;265:1995.

3. Linton AL, Peachey DK. Utilization management: a medical responsibility. Can Med Assoc J. 1989;141:283-6.

4. Richardson FM. Peer review of medical care. Med Care. 1972;lO: 29-39.

5. Lohr KN; ed. Medicare: A Strategy for Quality Assurance. Institute of Medicine. Washington, DC: National Academy Press; 1990.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18. 19.

20.

21.

22.

23.

24.

Bohr KN, Schroeder SA. A strategy for quality assurance in Medi- care. N Engl J Med. 1990;322:707-12. Chassin MR, Brook RH, Park RR, Keesey J, Fink A, KosecotT J, et al. Variations in the use of medical and surgical services by the Medicare population. N Engl J Med. 1986;314:285-90. Paul-Shaheen P, Clark JD, Williams D. Small area analysis: a review and analysis of the North American literature. J Health Polit Policy Law. 1987;12:741-809. Winslow CM, Kosecoff JB, Chassin M, Kanouse DE, Brook RH. The appropriateness of performing coronary artery bypass surgery. JAMA. 1988;260:505-9. Chassin MR, Kosecoff JB, Park RE, Winslow CM, Kahn KL, Merick NJ, et al. Does inappropriate use explain geographic variations in the use of health care services? JAMA. 1987;258:2533-7. Gray BH, Field MJ; eds. Controlling Costs and Changing Patient Care? The Role of Utilization Management. Washington, DC: Na- tional Academy Press; 1989. American College of Physicians. Universal Insurance for American Health Care. A proposal of the American College of Physicians. Ann Intern Med. 1992;117:511-9. Wicklzer TM. Private cost containment. The effect of utilization review on hospital use and expenditures: a review of the literature and an update on recent findings. Med Care Rev. 1990,47:327- 63. Feldstein PJ, Wicklzer TM, Wheeler JR. The effects of utilization review programs on health care use and expenditures. N Engl J Med.1988;318:1310-4. Imperlale TF, Siegal AP, Crede WB, Kamens EA. Preadmission screening of Medicare patients. The clinical impact of reimburse- ment disapproval. JAMA. 1988;259:3418-21. Thorpe KE. Health care cost containment: results and lessons from the past 20 years. In: Shortell SM, Reinhardt UE; eds. Improving Health Policy and Management. Nine Critical Research Issues for the 1990s. Ann Arbor, Michigan: Health Administration Press; 1992. Utilization Review. Information on External Review Oreanizations. Washington, DC: General Accounting Office Report; 1993. GAO/ HRD-93-22FS. American Medical News. 1992;March 2:11-3. Audet AM, Scott HD. The uniform clinical data set: an evaluation of the proposed national database for Medicare’s quality review program. Ann Intern Med. 1993;119:1209-13. Goldman RL. The reliability of peer assessments of quality of care. JAMA. 1992;267:958-60. Rubin HR, Rogers WH, Kahn KL, Rubenstein LV, Brook RH. Watching the doctor watchers: how well do peer review organization methods detect hospital care quality problems? JAMA. 1992; 267:2349-54. Rubenstein LV. Kahn KL. Reinisb EJ. Sherwood M. Rogers WII, Brook RH. St&wed Im&icit Revie&‘Analysis of thk Method and Quality of Care Results. Santa Monica, California: RAND; 1991. N-3033-HCFA. Kellie SE, Kelly JT. Medicare Peer Review Organization preproce- dure review criteria. An analysis of criteria for three procedures. JAMA. 1991;265:1265-70. Greenfield S, Gottlieb L, Dukes K, Coltin K. The impact of local physicians’ participation in guideline development on physician ad- herence to guidelines and on patient outcomes of guideline directed care [Abstract]. Clin Res. 1993;41:539A.

25. Field MJ, Lobr KN, eds. Guidelines for Clinical Practice. From Development to Use. Institute of Medicine. Washington, DC: Na- tional Academy Press; 1992.

26. Relman AS. Controlline costs bv “managed comoetition”-would

27.

28.

29.

30.

31.

it work? [Editorial]. N-Engl J Med. 1993:328:133:5. American College of Physicians. Practice strategies for elective red blood cell transfusion. Ann Intern Med. 1992;116:403-6. Practice strategies for elective red blood cell transfusion. Abstracts of Clinical Care Guidelines. 1992;4:1-4. Strmnwasser I, Paraajpe NV, Ronis DL, Share D, Sell IJ. Reliability and validity of utilization review criteria. Appropriateness Evalua- tion Protocol, Standardized Medreview Instrument, and Intensity- Severity-Discharge criteria. Med Care. 1990;28:95-111. Physician Payment Review Commission. Annual report to Congress. 1992. [Available from the Physician Payment Review Commission, 2120 L Street, N.W., Suite 510, Washington, DC 20037.1 Coronary Artery Bypass Graft Surgery in New York State, 1989- 1991. 1992. [Available from the New York State Department of Health, Albany, New York.]

32.

430 1 March 1994 l Annals of Internal Medicine l Volume 120

Borbas C, McLaughlin DB, Schultz A. Bridging the Gap between Theory and Practice: Exploring Clinical Practice Guidelines. Chi- cago: Hospital Research and Educational Trust; 1993. Gruppo Italian0 per lo Studio della Streptochlnasi Nell’Infarto Mio- cardico (GISSI). Effectiveness of intravenous thrombolytic treat- ment in acute myocardial infarction. Lancet. 1988;1:397-402. ISIS (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin,

l Number 5

both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988;2:349-60.

35. Greenfield S, Nelson EC, Zubkoff M, Manning W, Rogers W, Krav- its RL, et al. Variations in resource utilization among medical specialties and systems of care. JAMA. 1992;267:1624-30.

36. Kravitz RL, Greenheld S, Rogers W, Manning WG Jr, ZubkoU M, Nelson EC, et al. Differences in the mix of patients among medical

specialties and systems of care. Results from the Medical Outcomes Study. IAMA. 1992;267:1617-23.

37. Berwick DM, Godfrey AR, Roessner J. Curing Health Care. New Strategies for Quality Improvement. San Francisco: Jossey-Bass; 1990.

38. Rritchevsky SB, Simmons BP. Continuous quality improvement. Concepts and applications for physician care. JAMA. 1991;266: 1817-23.

1 March 1994 - Annals of Internal Medicine l Volume 120 l Number 5 431

the oversight of medical care: a proposal for reform · the oversight of medical care: a proposal...

Documents