the australian capital territory injury surveillance and prevention project

ACTISP Project, McClure. Burnside 529

Australian National L'niversity, Canberra. Australian Capital Territory. Australia

Sational Ccntre for Epidemiology and Population Health (RJMcC)

Formerly. Project Officer ACTISPP ACT Health. Canberra. Australian Capital Territory (JB)

Received: April 4. 1994 Revision received: June 10. 1994 Accepted: June 12, 1994

Prior presentation: In part att the 13th International Epidemiological Associatian Scientific Meeting. Sydney. Australia, Septcmher 1993.

Address for correspondence and reprints:

Roderick J. McClure. MBBS. BA Sational Centre for Epidemiology and Population Health Canberra. Australian National University ACT Australia OXK)

Key words: injury: injury surveillance: injury prevention: emergency department

I SPECIAL CONTRIBUTIONS

The Australian Capital Territory Injury Surveillance and Prevention Project

Roderick J . McClure, MBBS, BA, Joan Burnside, RN

I ABSTRACT

Objective: To identify the representativeness and sensitivity of the data collected by the Australian Capital Territory Injury Surveillance and Prevention Project (ACTISPP) for ED injury cases seen in the Australian Capital Territory (ACT).

Methods: A retrospective descriptive study was conducted using a systematic sample of people injured in the ACT between January 20 and September 19, 1992, and who received their first treatment for this injury in an ACT ED. The ACTISPP surveillance system is dependent on ED patient or family completion and return of a comprehensive injury survey. The following population variables in the ACTISPP database were compared with those of the injured ED population, as determined by ED and hospital record review: age, gender, postal code of residence, nature of injury, body part injured, and whether the patient had been admitted to hospital.

Results: The sensitivity of the surveillance system for injuries seen in the ED was 31% (95% CI = 30-32%). The distribution of the ACTISPP sample did not differ from the expected distribution for sex or socioeconomic status. The distributions differed by age only in the 5-14-year age group (p < 0.001). With respect to the nature of injury and body part injured, the difference was significant (p < 0.001) but was no greater than 7% for any of the categories. Two percent of the surveillance sample were admitted to hospital, compared with an estimated 4% of the target population (p < 0.001).

Conclusions: Approximately one-third of all incident injuries managed in the EDs of the ACT are included in the ACTISPP database. Although some sampling bias is introduced in the surveillance process, it does not appear to be of sufficient magnitude to compromise the project's goals.

Acad. Emerg. Med. 1995; 2529-534.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . , . . . . . . , . . . . . . . . . , . . . . , , . . , . . . , . . . . . . , .

~~ ~~

530 ACADEMIC EMERGENCY MEDICINE JUN 1995 VOL 2/NO 6

I It is well recognized that local injury surveillance data are essential to a community’s effort to control injury.’ What has not yet been clar- ified, however, is the most appropriate form this surveillance should take. On the one hand, it is advo- cated that injury surveillance systems should be designed with the primary objective of “collecting detailed information about the se- quence of events that lead to injuries and the reasons for the breakdown of the previously stable system of interaction” between the host, agent, and environment.* On the other hand, it is suggested that the necessarily low inclusion rates of such systems negate their usefulness and that they should be replaced by routine electronic collections from “general purpose hospital outpatient case management systems.” ’

Like most outpatient injury surveillance systems currently operating in Australia, the Australian Cap- ital Territory Injury Surveillance and Prevention Project (ACTISPP) maintains a database that contains a wealth of detailed information about the etiology of injuries occurring within the community it serves. Unfortunately, in 1991 it had be- come clear that only a small pro- portion of the targeted sample was being included in the database. The value of persisting with a purpose specific surveillance system that had low sensitivity for all injuries seen in the ED was questioned, and con- sideration was given to replacing it with an electronic system that moTe completely recorded a simpler data set. To resolve this dilemma a study was conducted to measure the sensitivity of the passive injury surveillance system operating in the ACT during 1992 and to evaluate the ef- fect of this low inclusion rate on the representativeness of the data collected. This report describes the study and discusses the findings with respect to the requirements of satisfactory surveillance. The results of this evaluation will benefit other

health services currently reassessing their injury surveillance needs.

METHODS

Study Design A retrospective descriptive study

of injuries managed in EDs of the Australian Capital Territory (ACT) was conducted using information obtained from hospital records. The characteristics of the injury population thus identified were compared with the surveillance system data for the sample population over the same period to identify deficien- cies in the ACTISPP database.

Definitions Case definition. Injured per-

son-an ACT resident who pre- sented to an ACT ED for first medical assessment of a new condition that could be classified according to the ICD-94 codes 800 to 995.

independent variables. Age- Number of whole years between date of birth and date of injury. Sex- Male .or female as indicated by subject. Nature of injury-Diagnosis specified by the treating medical practitioner and coded by researchers to sprains, lacerations, breaks, and other. Body part injured-Di- agnosis specified by the treating medical practitioner and coded by researchers to upper limb, lower limb, head and face, trunk, and other. Postal code of residence-A four-digit code that identifies areas for the purpose of facilitating postal services. These codes have been shown to equate to broad socioeconomic Categories.’ Whether admitted-Whether the patient had been treated in the ED and sent home or admitted to the wards as an inpa- tient for care.

Outcome measures: Sensj- tivify-“The measure of a system’s ability to identify all of the true cases of injury in the surveyed population.” ’ Representativeness-The

extent to which the injured people detected by the surveillance system are typical of those people in the target population.

Population and Study Site The study was conducted in the

ACT between January 20 and Sep- tember 19,1992. The ACT is a geo- graphically circumscribed area of 250 square kilometers in southeastern Australia that includes the nation’s capital city, Canberra. It contains a mostly urban community of 296,000 residents whose socioeconomic status tends to be higher than the national average.s Two EDs and 273 general practitioners serve this community. Woden Valley Hospital is a teaching hospital located in the south of Canberra and is the trauma ten- ter for the ACT and surrounding re- gion. Calvary Hospital, located in Canberra’s northern suburbs, typi- cally sees as many injuries as does Woden Valley but of the less severe categories. Of the 46,000 persons with incident injuries who receive medical care in the ACT per year, just over half present to EDs (un- published data, 1992). The remainder receive initial care from general practitioners in the community. Pa- tients can attend specialist physi- cians for treatment for an injury only if they have been first assessed by an ED medical officer or a general practitioner. The study population was defined as those ACT residents who were injured and who pre- sented for first assessment of the injury at one of the EDs in the ACT.

Study Sample (Criterion Sample)

The characteristics of the study population were estimated from a sample of hospital records. Ap- proval was obtained from the ACT Health Institutional Ethics Com- mittee for the data collection. The study sample served as the criterion against which the ACTISPP data were compared.

ACTISP Project, McClure, Burnside 53 1

Sample selection. The hospital ED attendance registers were used as a sampling frame for the criterion sample. On every eighth day from January 20 to September 19, 1992, those people presenting with a complaint that could have resulted from an acute injury were identified from the register and had their hospital medical records drawn to confirm their compliance with the case definition. This method of identifying injuries presenting to EDs is similar to that used in previous injury incidence studies6 and has been shown to be a highly effective means of identifying cases.’ The effectiveness of the register screen was retrospec- tively checked by randomly select- ing 10% of the sample days and drawing the medical records for all people presenting on that day, whether or not their presenting complaint was related to injury. The number of presenting injuries that had originally been missed on these days was noted. All first-presentation injuries for the residents of the A m attending the ED on these days were included in the incidence sample.

A one-day-in-eight sample was chosen to cover all weeks in the study period, with each day of the week being equally represented. This approach was necessary to adequately cope with the well-recognized cycli- cal patterns in the presentation of injury. The other specification to be met by the sample selection was one of sample size. On the basis of about 24,000 hospitalized injured per year, it was estimated that the chosen selection strategy would ensure that the sample contain enough cases of those injuries that occurred as rarely as one in 20 to be 95% sure of pre- dicting their incidence to within 1% of the true value.

Data eollection. The following variables were obtained during record review: nature of injury, body part injured, age, sex, postal code of residence of injured person, and

whether admitted to hospital. The ED and hospital medical records were reviewed when the subject’s age and sex had been documented by hospital clerical staff and the relevant details concerning the nature of injury and body part injured had been documented by medical staff. The principal researcher and two trained assistants coded the medical records into ISISx codes and entered it into SPSS’ statistical package for analysis.

ACTISPP Database

In 1992, the ACTISPP was an all-age, all-injury surveillance system that collected data from people attending EDs for treatment for their injuries. The system concentrated on identifying:

the location where the injuries were occurring, the activities in which people were engaged at the time they sustained the injuries, the way in which the injuries occurred, the factors involved in the injury event, the character of the injuries sustained, and the type of immediate ED care.

The surveillance data were collected by means of a questionnaire completed by both the injured person and the treating physician, at the time of presentation to the ED. It was the responsibility of the ED staff to ensure that each patient who attended the hospital for first treatment of an injury be given a questionnaire. The patient was asked to complete the form while waiting for treatment. In the case of children, the information sheet was completed by the parent or guardian and for those patients whose injuries prevented their being able to re- spond, accompanying friends or rel- atives were invited to assist. After the form was distributed, no further

attempt was made to encourage compliance and no record was kept of those patients to whom the form was, in fact, given.

Information about how, where, when, and why the injury occurred was requested by the questionnaire. Signed consent for further follow-up also was obtained at this time. The patients were expected to return the questionnaire to their treating medical officers, who recorded clinical details of the injury and management arranged. The completed forms were returned to a central point for collection by the ACTISPP project officer three times a week. The survey information was coded to 36 variables (and text strings) that en- capsulated information about the injured person, the injury, the injury event, and the treatment instigated. This information was entered into a desktop computer using ISIS soft- ware that enabled immediate analysis and the printing of reports. Staffing of ACTISPP involved one full-time position.

Data Analysis

The study sample was used to estimate the total number of ACT residents presenting to EDs in the ACT for treatment for a new injury over the study period. This was achieved by first calculating a mean and standard error for the number of injured patients presenting per day and multiplying by the number of days in the sample period to estimate the size of the total population. The sensitivity of the surveillance system was obtained by expressing the number of injuries identified by ACTISPP as a percentage of the estimated total number ‘of injuries in the target population. The 95% CIS of this sensitivity estimate were obtained by dividing the number of injuries recorded in ACTISPP by the 95% limits of the population estimate.

The representativeness of ACT- ISPP was assessed by identifying

~~

532 ~ ~~

ACADEMIC EMERGENCY MEDICINE JUN 1995 VOL 2/NO 6

351

I A, K’ /

0 1 I

5 4 5-14 15-24 25-34 35-44 45-54 &+

Yetndage I FIGURE 1. The age distributions of the Australian Capital Territory Injury Surveillance and Prevention Project (ACTISPP) database and criterion sample.

those injuries in the ACTISPP database that had occurred to ACT residents between January 20 and September 19,1992, and comparing them with those in the study sample for age, sex, postal code of residence, nature of injury, body part injured, and whether the patient had been hospitalized. The study sample was considered to represent the true distribution of the variables in the population, and where the ACT- ISPP database differed from this criterion sample it was considered to be nonrepresentative. Percentage differences for each of the categories of the study variables were identified and the significance for each of these differences was assessed using chi-square statistics with an o! of 0.05. With our large criterion sample set, there was sufficient power in the study to identify significance for 2% differences from ACTISPP with a power of 80%.

were not available in 5% of cases, and the remainder provided a satisfactory account of the required variables.

Sensitivity There were 2,013 subjects in the

study sample. Thus, an estimated

first-presentation injuries were seen at EDs in the ACT during the study period. The ACTISPP registered 4,971 people during this same time, resulting in a sensitivity for the surveillance system of 31% (95% CI =

16,100 (95% CI = 15,500-16,700)

30-32 %).

Representativeness Distribution by sex. The ratios

of men to women in the ACTISPP database and the criterion sample

16 T

were 64:36 and 66:34, respectively (p = 0.234).

Distribution by age. The age distributions of the two samples were similar, except that the 5-14-year age group was overrepresented in the ACTISPP database (p < 0.001) (Fig. 1).

Distribution by postal code. There was no statistically significant difference between the ACTISPP database and the criterion standard by postal code of residence (Fig. 2). As far as postal code of residence is a valid measure of socioeconomic status, there was no apparent socioeconomic status bias in the surveillance process.

Distribution by nature of injury. There was some difference in the distributions of the injury types in the ACTISPP data and the target population (p < 0.001) (Fig. 3). This difference was less than 5% in any category and did not markedly affect the surveillance data profile for this variable.

Distribution by body part injured. The ACTISPP sample differed from the criterion standard for body part injured, with lower limb injuries’ being 7% more common in the A n I S P P database (p C 0.001). Again, however, the overall pattern

RESULTS

Data Quality The search of the ED register

was found to be an effective method of identifying injury presentations 1

3 8

6

to the ED. On the retrospective comprehensive check of 10% of days,

2

0 fewer than 1% of injury presentations were found to have been Porlrod.d- missed. The medical records of patients identified from the register

I FIGURE 2. Distribution of postal i d e s of residence for the Australian Capital Territory Injury Surveillance and Prevention Project (ACTSIPP) database and criterion sample.

ACTISP Project, McClure, Burnside 533

of the ACTISPP data followed the expected profile (Fig. 4).

Distribution by admission to hospital. About 2% of the ACTISPP sample had been admitted to hospital compared with an estimated 4% in the target population. This difference was significant (p < 0.001).

DISCUSSION This study has examined the representativeness and sensitivity of the ACTISPP database by comparing it with the estimated frequency and distribution of injury in the population. The anticipated low sensitivity of the ACTISPP data collection process was confirmed by the study and the statisticaliy significant differences between the resulting database and the estimated population distributions of several variables were noted.

The importance of these differences in distribution must be considered in the light of the requirements of satisfactory surveillance. In general terms, disease surveillance is the process of collecting health-related data and analyzing and interpreting the gathered information to direct courses of action for disease control. The methods used in the ongoing scrutiny are chosen for their practicability and rapidity, rather than for their complete ac- curacy,"' and though completeness of data collection is desirable, it is of secondary importance to the maintenance of regular and systematic reporting. Furthermore, the characteristics of surveillance systems are interrelated. In passive surveillance systems, the collection center waits for the service provider to report the information of interest. Passive systems are a practical and inexpensive means of amassing detailed information but generally have a low sensitivity. In active surveillance systems, the investigators con- h c t special searches, which im- prove data quality. The thoroughness

m LomralbK e r e olher

WedLlluy

I FIGURE 3. Distribution of natures of injury for the Australian Capital Territory Injury Surveillance and Prevention Project (ACTISPP) database and criterion sample.

upperm Lowerlimb Foce TWnk ohel

Bodvscrt~rJuod I FIGURE 4. Distribution of body parts injured for the Australian Capital Territory Injury Surveillance and Prevention Project (ACTISPP) database and criterion sample.

of this approach is often at the expense of data detail or prohibits routine use.

The basic criteria used for judg- ing the adequacy of a surveillance system is determined by the system's goals. The purpose of a prevention- orientated injury surveillance system such as ACTISPP is to identify clusters of similar injuries and the hazards that increase incidence and severity," and not primarily to de- termine injury incidence and distribution. The small differences described here between the distribution of injuries in the ACTISPP database and their distribution in the target population should not affect the ability of ACTISPP to achieve its objectives.

A low inclusion rate, in itself, does not compromise the usefulness of a surveillance system. In the case

of commonly occurring injuries, a small random sample provides sufficient information to generate an appropriate response. While it is true that to identify clusters of a particular injury that occurs only rarely may require a complete record of all injuries, there is no public health need to do this for rare injuries of the trivial type. Patients who have rare serious injuries, who should rightfully be the focus of injury prevention, are generally admitted to hospital and so fall within the scope of a more complete information- gathering process than the outpatient system described here.

LIMITATIONS AND FUTURE QUESTIONS The primary limitation of this study concerns the assumption that simi-

534 ACADEMIC EMERGENCY MEDICINE JUN 1995 VOL 2/NO 6

larities between the criterion sample and the ACTISPP database with re- gard to the five study variables re- flect a general similarity for all of the variables on the ACTISPP surveillance instrument. In particular, this assumes that the causes of those injuries that are included on the database are similar to the etiologic factors involved in those injuries not included, provided the distributions of the two groups of injury types and demographic details are the same. This, in fact, may not be the case, and further surveys of surveillance nonresponders will be needed to es- tablish the validity of these assump- tions.

Clearly, while ACTISPP is re- stricted to ED surveillance, it can- not attempt to describe the whole of the injury problem in the ACT community. There is the potential in the current system, however, to extend the coverage to the equally large number of injuries being managed by general practitioners.

The evaluation of the representativeness of the ACTISPP samples assumes the criterion sample accu- rately estimated the population pa- rameters. Four possible sources of error could have distorted the results. These include sampling vari- ation, unavailable medical records,

incomplete documentation, and missing an injury case in the initial screening of the ED register. In this study it would appear that these errors have been kept to a minimum. The sampling strategy and the large sample size ensured that the sampling errors be of little consequence. The small number of core variables meant that, where medical records were available, complete data were obtained. That as many as 5% of the medical records were missing, however, was clearly a problem, and a relevant bias may have been- intro- duced at this point.

CONCLUSION Approximately one-third of all incident injuries managed in the EDs of the A m are included in the ACTISPP database. Although some sampling bias is introduced in the surveillance process, the characteristics of the ACTISPP database resemble those of the scrutinized target population. There is little jus- tification for replacing the current surveillance methods with a system that ensures complete injury case identification at the expense of detailed information regarding etiologic events.

REFERENCES

I. Graitcer PL. The development of state and local injury surveillance systems. J Safety Res. 1987; 18:191, 193.

2. Vimpani G. Injury surveillance: a key to effective control of childhood injuries. Aust Paediatr J. 1989; 25:lO-3.

3. Harrison J, Tyson D. Injury surveillance in Australia. Acta Paediatr Jpn. 1993;

4. World Health Organization. Manual of the International Classification of Dis- eases, Injuries and Causes of Death- Ninth Revision. Geneva: WHO, 1977.

5. McLennan W. Socioeconomic indexes for areas. Canberra: Australian Bureau Sta- tistics catalogue no. 1356.0, 1990.

6. Gallagher SS, Finison K, Guyer B, Goodenough S. The incidence of injuries among 87,000 Massachusetts children and adolescents: resufts of the 1980-81 State- wide Childhood Inlury Prevention Pro- gram Surveillance System. Am J Public Health. 1984; 74:1340-7.

7. Kalsbeek WD, Bowling JM. Morris PD. Choosing criteria to screen for specific medical conditions from nonspecific sources: an application to injury surveillance. Am J Epidemiol. 1992; 135:553- 63.

8. National Injury Surveillance Unit. ISIS Injury Research Information System. Adelaide: NISU, 1992.

9. SPSS Inc. SPSS Reference Guide. Chi- cago: SPSS, 1990.

10. Last JM. A Dictionary of Epidemiology (2nd ed). Oxford, UK: Oxford Univer- sity Press, 1988, p 125.

1 I . Robertson LS. Injury Epidemiology. Oxford, UK: Oxford University Press, 1992. p 58.

35:171-8.

the australian capital territory injury surveillance and prevention project

Documents