food safety risk assessment tool

7/30/2019 Food Safety Risk Assessment Tool

1/15

A simple, spreadsheet-based, food safety risk assessment tool

Thomas Ross*, John Sumner

Centre for Food Safety and Quality, School of Agricultural Science, University of Tasmania, GPO Box 252-54, Hobart,

Tasmania 7001, Australia

Received 21 June 2001; received in revised form 13 November 2001; accepted 18 January 2002

Abstract

The development and use of a simple tool for food safety risk assessment is described. The tool is in spreadsheet software

format and embodies established principles of food safety risk assessment, i.e., the combination of probability of exposure to a

food-borne hazard, the magnitude of hazard in a food when present, and the probability and severity of outcomes that might

arise from that level and frequency of exposure. The tool requires the user to select from qualitative statements and/or to provide

quantitative data concerning factors that that will affect the food safety risk to a specific population, arising from a specific food

product and specific hazard, during the steps from harvest to consumption. The spreadsheet converts the qualitative inputs into

numerical values and combines them with the quantitative inputs in a series of mathematical and logical steps using standard

spreadsheet functions. Those calculations are used to generate indices of the public health risk. Shortcomings of the approach

are discussed, including the simplifications and assumptions inherent in the mathematical model, the inadequacy of data

currently available, and the lack of consideration of variability and uncertainty in the inputs and outputs of the model. Possibleimprovements are suggested. The model underpinning the tool is a simplification of the harvest to consumption pathway, but

the tool offers a quick and simple means of comparing food-borne risks from diverse products, and has utility for ranking and

prioritising risks from diverse sources. It can be used to screen food-borne risks and identify those requiring more rigorous

assessment. It also serves as an aid to structured problem solving and can help to focus attention on those factors in food

production, processing, distribution and meal preparation that most affect food safety risk, and that may be the most appropriate

targets for risk management strategies. D 2002 Elsevier Science B.V. All rights reserved.

Keywords: Food safety; Hazard analysis; Qualitative risk assessment; Relative risk; Spreadsheet

1. Introduction

Formal risk assessment techniques have been devel-

oped and exploited in many areas of human enterprise

and activity for decades (NRC, 1983, 1994, 1996;

Morgan, 1993; Bernstein, 1996). The application of

risk assessment techniques to food safety issues is

being strongly promoted by national and international

organisations (CAST, 1994; Kindred, 1996; ILSI,

1996; WHO/FAO, 1999) and several authors have

reviewed the application of risk assessment methods

to food safety (Jaykus, 1996; Kindred, 1996; Lam-

merding, 1997; ICMSF, 1998; Voysey and Brown,

2000).

0168-1605/02/$ - see front matterD 2002 Elsevier Science B.V. All rights reserved.P I I : S 0 1 6 8 - 1 6 0 5 ( 0 2 ) 0 0 0 6 1 - 2

* Corresponding author. Tel.: +61-3-6226-1831; fax: +61-3-

6226-2642.

E-mail address: [email protected] (T. Ross).

www.elsevier.com/locate/ijfoodmicro

International Journal of Food Microbiology 77 (2002) 3953


2/15

Risk assessment involves the identification of a

hazard and the methodical description of a system,

and its failures, which could give rise to that hazard,

including all possible routes whereby that hazard canarise. This qualitative description can be made quanti-

tative by expressing in mathematical terms, the system,

and the relationships between those elements that

contribute to the risk. Full quantitative assessment of

the risk can be achieved if the distributions of values of

the factors in the system that contribute to the risk are

known. Approaches that use all of this information, the

so-called stochastic, or probabilistic treatments,

are the preferred option for risk assessment (Vose,

1996;Cassin et al., 1998a). Methods for microbial food

safety risk assessment are being developed by various

organisations (FAO, 1995; CAC, 1999; Kindred, 1996;

ILSI, 1996; Buchanan, 1997; PCCRARM, 1997) and,

since the mid-1990s, a number of microbiological risk

assessments have been presented. These were sum-

marised by Schlundt (2000). Others have since been

released for public comment/peer review (WHO/FAO,

2000a,b,c, 2001; FDA/FSIS, 2001a,b).

The effort expended to assess a specified risk

should be commensurate with the magnitude of that

risk and, in general, there is a large difference in effort

and rigour between qualitative and quantitative risk

assessment. The latter are typically expensive, labourintensive and technically demanding processes taking,

in some cases, many person-years to complete (FSIS,

1998; FDA/FSIS, 2001a). Despite this, many food

safety risk assessments have concluded that there are

insufficient data to enable a reliable numerical esti-

mate of risk within narrow confidence limits. Pre-

screening of the risk by simpler methods can aid

decisions about the value of investing resources in

fully quantitative risk assessments.

Risk managers may have difficulty comparing risks

from different sources for risk management prioritisa-tion. The fundamental objective of risk assessment is

to provide support for decisions, and there are a

number of decision-support tools to assist in deter-

mining whether a pathogen is, or could be, an impor-

tant hazard in a given food/food process combination.

These include various semi-quantitative scoring sys-

tems, decision trees,etc. (see, e.g. Notermans andMead,

1996; Todd and Harwig, 1996; ICMSF, 1996; Van

Schothorst, 1997). van Gerwen et al. (2000) presented

a step-wise approach and developed a computerised

expert system, named SIEFE, for quantitative micro-

biological risk assessment of food products and pro-

cesses that begins to address this problem. Schemes to

assist qualitative risk assessments have also been devel-oped (Corlett and Pierson, 1992; Huss et al., 2000).

While the approaches of Corlett and Pierson (1992)

and Huss et al. (2000) are valuable in categorising risk

and in directing broad mitigation strategies, neither

provides good discrimination of risk (e.g. neither could

be used to assess an as-yet undocumented risk), nor of

the effect of contributions to risk of individual risk-

affecting factors. Consequently, those schemes do not

focus attention on steps where control could most

effectively be applied.

In this paper, we describe a simple and accessible

food safety risk calculation tool intended as an aid to

determining relative risks from different product/

pathogen/processing combinations, that addresses

some of the shortcomings identified above.

2. Methods and materials

2.1. Development of the decision support tool

The decision-support tool was developed to assist

in translating an academic understanding of the riskassessment approach and philosophy into a useful tool

for ranking the risk from different food/hazard combi-

nations. In particular, the tool was intended to make

the techniques of food safety risk assessment more

accessible to non-expert users, both as a decision-aid

and an educational tool.

It was recognised that the tool would have to

incorporate all factors that affect the risk from a

hazard in a particular commodity including:

(1) Severity of the hazard.(2) Likelihood of a disease causing dose of the

hazard being present in a meal.

(3) Probability of exposure to the hazard in a

defined period of time.

In turn, it was recognised that a number of factors

affect each of the above. Disease severity is affected by:

(a) the intrinsic features of the pathogen/toxin, and

(b) the susceptibility of the consumer.

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 395340


3/15

Exposure to the hazard will depend on how much

is consumed per meal by the population of interest,

how frequently they consume the food, and the size of

the population exposed.Probability of exposure to an infectious dose will

depend on:

(a) serving size,

(b) probability of contamination in the raw pro-

duct,

(c) initial level of contamination,

(d) probability of contamination at subsequent

stages in the farm-to-fork chain, and

(e) changes in the level of the hazard during the

journey from farm to fork, including, e.g.

simple concentration and dilution, growth or

inactivation of pathogens.

The tool was developed to assist users to describe

the product/pathogen/pathway of interest to them. In

prototypes, the user was prompted to choose from a

list of qualitative answers in response to each of a

series of simple questions, so that risk could be

estimated or compared without recourse to numerical

data. After trials, it was realised that relying on a

small and finite range of qualitative answers greatly

limited the ability of the tool to discriminate levelsof risk. Consequently, the capacity for users to pro-

vide numerical answers to some questions was in-

cluded.

2.2. User interface

The user interface represents a generic conceptual

model of the factors that contribute to food safety

risk.

The model was developed in MicrosoftR Excel

spreadsheet software, using standard mathematicaland logical functions. The List Box macro tool,

an inbuilt MicrosoftR Excel function available on the

Forms toolbar, was used to automate the conver-

sion from qualitative inputs to quantities for use in

calculations. The list box tool allows users to select

from a range of options by simply mouse-clicking

on their choice. The software converts that selection

into a numerical value.

The user is required to answer 11 questions,

related to the three main factors identified in Section

2.1. The underlying mathematical model equates

each descriptor with a numerical value or weight-

ing. The weightings currently used in the model are

detailed in Table 1. Some of the weightings arearbitrarily defined, while others are based on known

mathematical relationships, e.g. from days to weeks,

or years. To assist users to make objective and

reproducible responses, and to maintain transparency

of the model, examples of the subjective descriptors

are provided, or the weighting factors applied to

the descriptors are shown in the user interface

(see Fig. 1). Alternatively, where the options pro-

vided do not accurately reflect the situation being

modelled, users can enter a numerical value that is

appropriate.

Different iterations of the spreadsheet model were

tested by food safety managers. Through that process,

ambiguities in the structure of the questions were

revealed. Thus, the questions were modified to make

their intent clearer.

Questions 1 and 2 consider the susceptibility of the

population of interest and the severity of the illness.

The hazard severity is arbitrarily weighted by factors

of 10 for increasing levels of severity. The weighting

factors for susceptibility of various population sub-

groups include values for relative risk of infection/

intoxication for a variety of hazards. That weighting isloosely based on epidemiological data for relative

susceptibility to listeriosis calculated by Jurado et al.

(1993), Jones et al. (1994) and Nolla-Salas et al.

(1993). Absolute risk is based on the population size,

the proportion of the population consuming the food

and how frequently people eat the food. This infor-

mation is selected in Questions 35. The selection of

a sub-population from the general population (Ques-

tion 2) automatically reduces, by the proportions

indicated in Table 1, the total population estimated

to be exposed.Using Australian population age structure data

(ABS, 2000) and 1998 data from the US Center

for Disease Prevention and Control (cited in FDA/

FSIS, 2001a) for the proportion of listeriosis cases

in defined age categories, we also estimated the

relative susceptibility by age. The proportion of the

population in these categories in Australia was

estimated from ABS (2000), and also by Paoli

(1999, pers. comm.) for North American populations.

Both estimates were in the range of 1520%, consis-

T. Ross, J. Sumner / International Journal of Food Microbiology 77 (2002) 3953 41


4/15

Table 1

Weighting values used in the current model (V.1)

Comment

1. Hazard severitySEVERE hazardcauses death to most victims 1 arbitrary weighting factors

MODERATE hazardrequires medical intervention

in most cases

0.1

MILD hazardsometimes requires medical attention 0.01

MINOR hazardpatient rarely seeks medical attention 0.001

2. How susceptible is the consumer?

GENERALall members of the population 1 100% of population

SLIGHTe.g., infants, aged 5 20% of population

VERYe.g., old, very young, diabetes, alcoholic etc. 30 3% of population

EXTREMEe.g. AIDS, transplants recipients,

cancer patients, etc.

200 0.1% of population

arbitrary weightings, but based on relative

susceptibility to listeriosis, populationestimates based on Australian health statistics

3. Frequency of consumption

daily 365 simple algebra

weekly 52

monthly 12

a few times per year 3

once every few years 0.3

4. Proportion of population consuming

all (100%) 1 arbitrary weighting factors

most (75%) 0.75

some (25%) 0.25

very few (5%) 0.05

5. Size of population of interest User selected or specified

6. Proportion of product contaminated?

Rare (1 in a 1000) 0.001 0.01% of samples

Infrequent (1%) 0.01 1% of samples

Sometimes (10%) 0.1 10% of samples

Common (50%) 0.5 50% of samples

All (100%) 1 all samples

OTHER user input

7. Effect of process

The process RELIABLY ELIMINATES hazards 0 arbitrary weighting factorsThe process USUALLY (99% of cases) ELIMINATES hazards 0.01

The process SLIGHTLY (50% of cases) REDUCES hazards 0.5

The process has NO EFFECT on the hazards 1

The process INCREASES (10 ) the hazards 10The process GREATLY INCREASES (1000 ) the hazards 1000

8. Is there a potential for recontamination?

NO 0 arbitrary weighting factors

YESminor (1% frequency) 0.01

YESmajor (50% frequency) 0.50

OTHER user input



5/15

tent with that of Lindqvist and Westoo (2000), Hitch-

ins (1996) and Buchanan et al. (1997). The propor-

tions in different susceptibility classes were used to

modify the number of cases predicted, as describedbelow.

The frequency of contamination of food and the

implications of subsequent processing and handling

are considered in Questions 6 9 and Question 11.

Some factors, such as processing or cooking, may

completely eliminate the risk. The model includes,

however, the possibility that re-contamination may

occur subsequently, and re-introduce risk. Subsequent

pathways of cross-contamination are not explicitly

considered in the model.

Neither the concentration of the hazard nor thesize of the serving is considered explicitly in the

model. Both factors are included indirectly in the

response to Question 10, which requests an estimate

of the increase required for the initial contamination

level to reach ID50.1 In the calculation of relative risk,

for pathogens believed to have a high infectious

doses, the distribution of pathogen loads in the food

system has little effect (WHO/FAO, 2001). Rather, it

is the total load of such pathogens in the foodsupply that determines the overall population health

risk.

The model multiplies the factors to calculate var-

ious measures of risk, described below. Some esti-

mates consider only the probability of illness, while

others also consider the severity to produce an esti-

mate of the risk of the illness and the numbers of

consumers affected. As a descriptor is selected or

changed, the risk estimates are automatically recalcu-

lated.

2.3. Structure of the tool and mathematical bases

Full details of the logic and equations leading to

the risk estimates are detailed below.

Four measures of risk are calculated.

To simplify the description of the calculation of

these values, it is useful to describe some intermediate

calculations. These are the following.

PDD: Probability of a disease-causing dose being

present in a portion of the product of interest. This is

1 The dose expected to result in 50% of the exposed population

becoming ill.

Table 1 (continued)

Comment

9. How much increase from level at processing is required to reach an infectious or toxic dose for the average consumer?

none 1 arbitrary weighting factorsslight (10-fold increase) 0.1

moderate (100-fold increase) 0.01

significant (10,000-fold increase) 0.0001

OTHER user input

10. How effective is the post-processing control system?

WELL CONTROLLEDsystems in place, audited, well-trained staff 1 arbitrary weighting factors

CONTROLLEDsystems in place, audited, well-trained staff 3

NOT CONTROLLEDno systems in place, untrained staff 10

GROSS ABUSE OCCURS 1000

NOT RELEVANTlevel of risk agent does not change 1

11. Effect of preparation for meal

Meal preparation RELIABLY ELIMINATES hazards 0 arbitrary weighting factorsMeal preparation USUALLY ELIMINATES (99%) hazards 0.01

Meal preparation SLIGHTLY REDUCES (50%) hazards 0.50

Meal preparation has NO EFFECT on the hazards 1.00

user-input

OTHER value



6/15

defined as whichever is the larger value of the

product of

the proportion of product contaminated

value of Question 6 the effects of

processing on the probability of contamination

value of Question 7 the effect of

post processing handling=storage

value of Question 9 the increase in the

initial level of the factor required to reach an

infectious dose value of Question 10 the

effect of preparation prior to eating

value of Question 11

or

the proportion of product re contaminated

value of Question 8 the effect of

post processing handling=storage

value of Question 9 the increase in the initial

level of the factor required to reach an infectious

dose value of Question 10 the effect of

preparation prior to eating value of Question 11

The probability of a portion of food being con-taminated with a toxic dose cannot exceed 1. Accord-

ingly, if the value of the above calculations exceeds 1,

it is set equal to 1.

Pexp: Probability of exposure to the product per

person per day, given by:

the frequency of consumption

value of Question 3 proportion of the

population that consumes the product

value of Question 4

Exposure: Total number of portions of the product

of interest eaten per day in the general population,

given by:

the frequency of consumption

value of Question 3 proportion of the

population that consumes the product

value of Question 4 the total population

considered value of Question 5

The first measure: Probability of illness per con-

sumer per day is calculated as:

PDD Pexp

This value is not strictly a measure of risk, because

it does not include the severity of the illness resulting

from exposure to the hazard.

The second measure Total predicted illnesses/

annum in population of interest does not differentiate

severity either, but provides another measure that

might be more readily understood. Total predicted

illnesses/annum in population of interest is calcu-

lated as:

365 i:e: days per year Probability of illness

per consumer per day as described above

fraction of population considered in at risk

class part of Question 2 the total population

value of Question 10

The Comparative Risk in the population of

interest is a measure of relative risk and is independ-

ent of the size of the population, but does consider the

proportion of the population consuming. It is calcu-

lated as:

Probability of illness per day per consumer of

interest as described above Hazard severity

Question 1 Proportion of population

consuming Question 4 Proportion of total

population in population of interest

part of Question 2

and can be used to rank the relative risk of the

pathogen/product/process combination and consump-

tion patterns, independent of population size. Whenspecific sub-populations are selected at Question 2, the

estimate of the absolute number of cases among the

total population is amended by the weighting factors

shown in Table 1 for relative susceptibility to infection,

and also the proportion of the total population in that

sub-group. The model is constructed, and the weight-

ing factors selected, so that the Comparative Risk

can never exceed 1. A Comparative Risk of 1

represents the situation where every person in the

population consumes the product of interest daily,



7/15

and that each portion of the product contains a lethal

dose of the hazard.

The Comparative Risk measure is cumbersome,

and its numerical value is not readily understood as ameasure of risk. Relatively small changes in one of

the answers can lead to alarming changes in the

predicted number of cases. Furthermore, the specifi-

cation of the numerical value of risk is misleading, as

it provides no information regarding the confidence

one should place in that numerical estimate. To

provide a more user-friendly and robust index of

relative risk, the Risk Ranking measure was devel-

oped based on the Comparative Risk estimate.

The Risk Ranking value is scaled logarithmically

between 0 and 100, where 0 represents no risk, and 100

represents the opposite extreme where every member

of the population eats a meal that contains a lethal dose

of the hazard every day. To set the Risk Ranking

scale, we chose a probability of mild food-borne illness

of less than or equal to one case per 10 billion people

(greater than current global population) per 100 years

as a negligible risk. The Comparative Risk estimate

that corresponds to this value is 2.75 10 17. Weequated the Risk Ranking corresponding to this

level to zero. Analogously, we set the upper limit of

Risk Ranking at 100, corresponding to a Compa-

rative Risk of 1. All the estimates generated by themodel are based on the multiplication of factors, many

set at factor of 10 differences. The end-points of the

Risk Ranking scale lead to an increment of six Risk

Ranking units, corresponding approximately to a

factor of 10 difference in the absolute risk estimate.

Thus, Risk Ranking is defined as:

If

Comparative Risk Q 2.75 10 17 thenRisk Ranking= 0 or else Risk Rank-

ing=(100/17.56) (17.56 + LOG10(Compara-tive Risk)).

In the spreadsheet, the result is rounded to the

nearest integer value, reflecting the level of discrimina-

tion we believe appropriate given the bases of the tool.

2.4. Evaluating the tool

To relate the Risk Ranking scale to practical ex-

perience, we use predicted rates of food-borne illness

in Australia, estimated by ANZFA (1999), and the

estimates of Mead et al. (1999) for food-borne illness

in the USA, to generate Risk Ranking values.

To evaluate the performance of the tool, severalscenarios were modelled and compared to actual data

or other risk assessments. Specifically, conditions

leading to an outbreak of hepatitis A from consump-

tion of oysters in Australia in 1997 were simulated

using the tool, and compared to the epidemiological

data reported by Conaty et al. (2000).

Secondly, the data and assumptions of the quanti-

tative risk assessment of Cassin et al. (1998b), for the

risk of illness from enterohaemorrhagic E. coli in

hamburgers in north American culture, were used to

derive the answers to the questions of the risk assess-

ment spreadsheet. The results of both assessments

were compared.

3. Results

The model interface is shown in Fig. 1.

3.1. Risk ranking

ANZFA (1999) calculated the incidence of food-

borne disease in Australia as f 4,000,000 cases perannum. The vast majority of these cases were consid-

ered to pass unreported. Thus, we set Hazard Severity

(Question 1) to minor hazard. The ANZFA (1999)

estimates are for the total population: we set Question 2

to general. We manipulated other inputs so that the

Total Predicted Illnesses per annum in the Population

of Interest equalled f 4,000,000. Australia was

selected in Question 5. Under these, and all other

conditions leading to a total predicted 4,000,000 minor

food-borne illnesses among the Australian population

off

20 million, the Risk Ranking was 64.Mead et al. (1999) estimated that there were

76,000,000 cases of food-borne disease per year in

the USA, of which 325,000 resulted in hospitalisation

and 5000 caused deaths. Thus, we performed three

assessments. In the first, the Hazard Severity (Question

1) was set to minor hazard and the other questions

adjusted to yield an estimate of 76,000,000 illnesses. In

the second assessment, Hazard Severity (Question 1)

was set to moderate hazardrequiring medical inter-

vention in most cases, and the other questions manip-



8/15

Fig. 1. User interface. The risk model user interface, using Australian populations as an example. Users mouse-click on their choice in each lis

required. As choices are made and values entered, the risk estimates are automatically recalculated. The values shown are those used in Case


9/15

ulated to yield an estimate of 325,000 illnesses. In the

third, the Hazard Severity was set to severe haz-

ardcauses death to most victims, and the other

questions manipulated to yield an estimate of 5000cases. In all cases General population was selected

in Question 2, and other selected under Question 5,

with the USA population estimated at 270,000,000.

The Risk Ranking estimates for these three scenar-

ios were 65, 63 and 58, respectively.

3.2. Case study 1: viruses in oysters

An outbreak of hepatitis A involving consumption

of oysters occurred in Australia in 1997. The outbreak

is discussed in Conaty et al. (2000) from which it can

be estimated that production from the affected area

was approximately 280 bags of oysters per week. A

bag of oysters contains approximately 200 serves of

six oysters. The first positive samples were identified

from oysters harvested on 24 December and the area

was closed to further harvest on 14 February, indicat-

ing that contaminated oysters were harvested for up to

7 weeks. Thus, we estimate that the population was

exposed to 390,000 servings of potentially contami-

nated oysters. If spread over the entire Australian

population over an entire year, this would correspond

to 0.02 serves per person per annum. To represent thislevel of exposure in the model we selected Once

every few years at Question 3, and Very Few at

Question 4. (Note that, even though the exposure

occurred only over a 7-week period, we assume that

it was spread over an entire year, and that even though

the exposure was restricted to a geographic region,

that all Australian consumers were potentially

exposed, consistent with the above estimate of expo-

sure level.)

Conaty et al. (2000) report that of 63 samples of

one dozen oysters, 6 tested positive for hepatitis Avirus using a PCR method. From this, we assumed

that 5% of servings of six oysters were HAV-positive

(Question 5).

The level of contamination was not reported.

Where detection of enteric viruses in shellfish has

occurred the levels of contamination are low, ranging

from 0.3 to 200 plaque forming units (PFU) per 100 g

of shellfish (Jaykus et al., 1994; Rose and Sobsey,

1993; CAST, 1994), a typical serving size. Rose and

Sobsey (1993) presented a dose response model

relating the probability of infection with HAV to the

amount ingested. It suggests that the ID50 for HAV isf 500 PFU. Assuming an exponential doseresponse

relationship, 1 PFU would be expected to lead toinfection in 1 in 500 consumers. Thus, it appears

likely that not all contaminated serves would have a

high probability of causing infection. To implement

this relationship at Question 7, we selected OTHER

and entered 65 (the ID50 divided by the geometric

mean of the contamination per serving) at Question

10. The values used are summarised in Table 2.

Australia-wide during the outbreak period (20

January to 4 April), there were 444 cases of hepatitis

A associated with consumption of oysters, the vast

majority of which were believed to be due to oysters

from the contaminated area (Conaty et al., 2000).

Under the assumptions given above, the number of

cases predicted by the spreadsheet model was 225,

and the Risk Ranking was 52.

3.3. Case study 2: comparison to risk estimate of

Cassin et al. (1998b)

Cassin et al. (1998b) developed a process risk

model from which to determine the effectiveness of

a range of strategies to reduce the risk and incidence

Table 2

Values used in the assessment of risk from viral contamination of

oysters in Australia in an outbreak

Risk criteria Input appropriate

to outbreak

Dose and severity

1. Hazard severity Moderateoften requires

medical attention

2. Susceptibility Generalall population

Probability of exposure

3. Frequency of consumption once every few years

4. Proportion consuming very few (5%)

5. Size of population Australia (19,500,000)

Probability of infective dose

6. Probability of contamination Other (5% of servings)

7. Effect of Process Has no effect on the hazard

8. Possibility of recontamination None

9. Post-process control Not relevant

10. Increase to infective dose other (65)

11. Effect of treatment

before heating

Not effective in

reducing hazard



10/15

of E. coli O157:H7 infections from hamburgers in the

North American cuisine.

It was difficult to make a direct comparison with

the results of Cassin et al. (1998b). Many of the valuesthat were required to be entered in the spreadsheet

were not given explicitly by those authors, but were

intermediate calculations in their model. However,

using data and statistics for the general population

presented in that report, we entered the values shown

in Table 3 into the spreadsheet.

We considered the total population, for whom

infection with E. coli O157:H7 will generally cause

mild disease (Questions 1 and 2). For more susceptible

individuals, medical attention will be required. The

population of the USA is approximately 270,000,000

(Question 5). Walls and Scott (1996) reported that on

any day in the USA, 9% of the population consume a

hamburger meal or, equivalently, that in any week 63%

of the population eat a hamburger meal. We entered this

as most people consume a hamburger weekly

(Question 4). It was difficult to determine the predicted

level of contamination during processing. We estimate

a contamination rate on carcass meat of < 1%. Cassin et

al. (1998b) discussed various factors that affect the

contamination rate during the processing of meat, and

concluded that overall, a reduction in the initial con-

tamination of between 10 and 300 could be expected.

We chose The process usually eliminates. . . at

Question 7. However, in their calculations, Cassin et

al. (1998b) predicted that 2.9% of the packages of retailground beef of size 3001000 g are contaminated. We

implemented this directly at Question 8 which over-

rides the contamination changes predicted from the

answers to Questions 6 and 7.

The geometric mean of the contamination levels

estimated by Cassin et al. (1998b) is f 20 CFU/

package. The average serving size is 83 g for adults.

Thus, based on the average package size, average

serving size and average contamination level, we

estimate the average CFU/meal serving as 3 at retail.

We have assumed that the USA has good temperature

control and handling systems for raw meat and have

selected Controlled for Question 9. By analogy

with shigellosis, the ID50 for O157:H7 was estimated

by Cassin et al. (1998b) as f 2000 CFU. Thus, we

assumed that a f 1000-fold increase in dose would

be required to cause infection in the average case.

Cassin et al. (1998b) cited the results of MacIntosh

et al. (1994) for hamburger cooking preference. Nine-

teen percent of consumers were reported to prefer

rare or medium rare cooked meat products. We

assume that all other degrees of cooking result in the

elimination of the pathogen, and that of the remaining20% preferring less thoroughly cooked meat, the

cooking eliminates 75% of the pathogens present.

We implement this as cooking eliminates 95% of the

pathogens present in all hamburgers consumed (Ques-

tion 11). The values used are summarised in Table 3.

The model predicts a per-meal risk of 6.2 10 7,and predicts 45,800 cases per year in the USA. Cassin

et al. (1998b) estimated the risk per meal to be 3.7 and

5.5 10 5 for children and adults, respectively, fromtheir model. On the assumption that half of the

10,000 20,000 cases annually of E. coli O157:H7illness in the USA are due to consumption of ham-

burgers, Cassin et al. (1998b) estimated the per-meal

risk at between 5.7 10 7 and 1.2 10 6. TheRisk Ranking estimate is 58.

4. Discussion

The spreadsheet tool was originally developed as a

means of quickly assessing the performance of various

Table 3

Values used in the assessment of risk from enterohaemorrhagic E.

coli in hamburgers in north America

Risk criteria Input

Dose and severity

1. Hazard severity Moderateoften requires

medical attention

2. Susceptibility Generalall population

Probability of exposure

3. Frequency of consumption weekly

4. Proportion consuming most

5. Size of population Other (270,000,000)

Probability of infective dose

6. Probability of contamination Other (1% of servings)

7. Effect of process Usually eliminates

the hazard

8. Possibility of recontamination Other (2.9%)

9. Post-process control Controlled

10. Increase to infective dose Other (1000)

11. Effect of treatment

before heating

Other (0.05)



11/15

conceptual models for food safety risk assessment. It

was quickly realised that the spreadsheet itself was a

valuable risk assessment and risk communication tool,

and the conceptual model and the spreadsheet userinterface then were developed in parallel. Refinement

of the conceptual model was based largely on exper-

imentation with the model. This involved trying to

recreate scenarios for which data to describe the model

inputs, and epidemiological data by which to evaluate

the corresponding model outputs, were available. For

example, early iterations of the model required a

number of correction factors so that numerical

predictions of cases of illness matched those reported.

Other risk assessment models have used such factors to

make the predicted number of cases better match the

observed rates of illness (Farber et al., 1996; FDA/

FSIS, 2001a). Experience with the model enabled the

refinement of the questions posed and data required so

that the correction factors were eliminated from the

model. Elimination of correction factors is impor-

tant because a major tenet of the risk assessment

approach is that all assessments should be trans-

parent, i.e. the basis of all calculation should be made

explicit (CAC, 1999).

The spreadsheet interface has also been improved

through feedback from a diverse range of users. We

emphasise, however, that while the tool is presentedas an example of how food safety risk assessments

can be simplified and its benefits made more acces-

sible to risk managers, the tool is not definitive. It can

still be improved, and cannot be expected to be

appropriate to all food safety risk assessment prob-

lems. We discuss some of the shortcomings and

tangential benefits of the model below.

4.1. Evaluation of performance

We compared the predictions of the model toindependently obtained epidemiological data and esti-

mates for food-borne illness in Australia and USA to

calibrate the Risk Ranking value. The estimates

obtained suggest that Risk Ranking in the range

6065 describes the status quo for all microbial food-

borne disease in Australia and USA. We consider

those to be representative of many developed nations.

This gives a reference point from which to evaluate

Risk Ranking values for other product/hazard/path-

way combinations. It should be noted that the Risk

Ranking is independent of population size, but

reflects the relative risk to an individual within the

selected population. Thus, the Risk Ranking can be

used potentially to compare the risks across diversefoods, hazards and cultures.

The USA data enabled the Risk Ranking to be

estimated from different disease end-points (e.g. esti-

mated total illness, estimated hospitalisations, esti-

mated deaths) and revealed that the Risk Ranking

value depended on the end-point chosen. Perhaps

surprisingly, then, the Comparative Risk estimated

from the USA fatality estimates was 10-fold lower

than the Comparative Risk estimates based on total

estimated cases or total cases requiring hospital treat-

ment. In the conceptual model underpinning the tool,

the weighting applied for disease severity (arbitrarily)

assumed death to be 1000 times more serious than

a mild case of illness not requiring medical attention.

It is clearly difficult to deduce an objective, quantita-

tive, measure to compare the severity of death to that

of mild food-borne illness. The Risk Ranking

values based on USA data but using different disease

end-points suggest, however, that the weighting fac-

tors for illness severity used in the model are inappro-

priate (see discussion further on).

The prediction of the model for a scenario based on

a food-borne disease outbreak in Australia was 225cases, within a factor of two of the observed number

of cases (f 440). The inputs to the model were as

consistent with the events surrounding the outbreak as

was possible given the available data. Where no data

was available, reasonable assumptions or estimates

based on analogous data or experience were used so

that, in the scenario modelled, there was little oppor-

tunity to manipulate inputs to achieve the specific

results.

Similarly, using as inputs data taken from Cassin et

al. (1998b) yielded results that were consistent withthe results of that stochastic risk assessment. The per-

meal risk predicted by the spreadsheet model was 1 in

16 million, within the range predicted by Cassin et al.

(1998b) of 1 in 17 million to 1 in 830,000 meals

consumed. Clearly, these two examples do not prove

that the model is reliable. In our experience, however,

the spreadsheet model predictions are usually within

an order of magnitude of independent estimates of the

number of cases of food-borne illness for specific

product/hazard/pathway combinations. This level of



12/15

accuracy is expected on the basis of the models re-

liance on multiplication of a series of weighting fac-

tors, many of which are in 10-fold increments. Further

examples of the models performance and utility as arisk management aid are presented in Sumner and

Ross (this issue), and provide perspective on the rela-

tion of risk ranking values with recognised hazards.

4.2. Limitations/weaknesses

The creation of the model was largely a reactive

process, i.e., during testing against available epide-

miological data, when the model failed, the source of

the failure was analysed and the model modified to

overcome that deficiency. Despite the apparent utility

of the model, we have not been able to systematically

and objectively evaluate the models performance,

because there are few detailed data sets describing

exposure and food-borne disease incidence.

There are other limitations and weaknesses. Some

are general problems associated with risk assessment

modelling, while others are specific to the tool pre-

sented here.

Even though we have attempted to make the

questions unambiguous, the intent of the question

can still be misinterpreted. For example, Question 4

refers to the proportion of any population that con-sumes the product. It does not need to be adjusted by

the user when a sub-population is selected in Question

2, because the spreadsheet model automatically modi-

fies the size of the population exposed when a sub-

population is selected at Question 2. Similarly, the

answer to Question 10 is intended to be based on the

ID50 for a healthy member of the normal population,

irrespective of whether a susceptible population is

selected at Question 2. Again, as described in Meth-

ods and materials, the calculations in the spreadsheet

make adjustment for the selection in Question 2.In modelling any complex and variable system, it

is necessary to balance the need to make simplifying

assumptions against the loss of detail that ensues. In

general, the Australian and USA statistics infer a risk

of mild food-borne illness of one case per person

every 5 to 10 years, roughly equivalent to a risk of 1

in 500011,000 meals. While the risk of outbreaks is

much less, food safety managers are often more

interested in understanding the sets of specific circum-

stances that lead to these relatively rare events of

food-borne illness outbreaks. Using a small number of

descriptors of those conditions hinders discrimination

of small, but potentially critical differences, so that

important information can be lost in the averagingprocess that results.

Another problem associated with these low levels

of discrimination is that many choices automatically

lead to at least a factor of 10 difference in the estimated

risk. The option within some questions for the user to

enter a specific value other than those offered arose

from the realisation that the model could not make

accurate predictions, unless a wider range of values, or

narrower intervals between levels, were available.

Following from the above, it must be emphasised

that some of the weighting factors employed in the

model are arbitrarily derived. Other weights may be

more appropriate. For example, the weighting of

relative susceptibility to illness of consumers with

known predisposing conditions (Question 2) is cur-

rently based on the relative risk of listeriosis. While

those factors may be broadly appropriate to suscept-

ibility to infections, they may be irrelevant to the risk

of intoxications from microbial, or other toxins.

Earlier, we referred to the weights applied to the

disease severity descriptors. One way to make these

weights more objective is to express the severity of

diseases in terms of days of quality or life lost, a non-specific approach to measuring the health burden of

illness that is increasingly advocated in the domain of

public health (HCP, 2000). One such measure is

disability adjusted life years (DALY), which enables

the integration of different disease end-points.

Using this approach, the difference in weights

given to life-threatening food-borne disease compared

to mild gastrointestinal forms was suggested to be

too small (calculations not shown). As discussed

earlier, the Risk Ranking estimates based on differ-

ent disease end-points for the USA data similarlyraised the question whether the weights applied to

disease severity were appropriate. Weight factors

based on DALYs would also simplify the comparison

of illness from diverse sources, e.g. the acute effects

of food-borne infections compared to the chronic

effects of intoxications from chemical residues,

increasing the applicability and universality of the

proposed model. The weights and values used in the

spreadsheet for these, and other variables, can be

easily changed as necessary or appropriate. Care



13/15

should be exercised, however, that such changes do

not lead to unrealistic values in some of the inter-

mediate calculations in the model.

Stochastic approaches to risk modelling are pre-ferred because risk involves the element of probability

(Cassin et al., 1998a). A limitation of the tool is that

while it provides an estimate of the most probable

outcome, it does not provide information about the

level of confidence we have in that estimate, or more

importantly, the probable range of illnesses for different

scenarios. A possible refinement of the model would be

to allow users to enter a range of values, or distribution

of values that would offer some of the benefits of sto-

chastic modelling, but still in a relatively simple tool.

4.3. Peripheral benefits of the tool

Apart from its use for ranking perceived risks, the

spreadsheet tool helps to focus the attention of the users

on the interplay of factors that contribute to food-borne

disease. The model can be used easily to explore the

effect of different risk-reduction strategies, or the

extent of change required to bring about a desired

reduction in risk. Users must remember, however, that

some of the weighting factors are arbitrarily derived.

Consequently, the predicted effect may not reflect

reality but only the assumptions on which the modelis based, and users should ensure that the model is

appropriate to their risk assessment question.

Whether the mathematical model underlying the

tool is correct or not, we found the spreadsheet tool to

be a powerful aid for teaching the principles of risk

assessment. The model forces users to think about

factors affecting food safety and can help train food

safety managers to think in terms of risk, and the

interaction among factors that contribute to risk, rather

than in absolute terms such as zero tolerance of

hazards. Using the model to recreate scenarios quicklyreveals where data critical to estimating risk are lack-

ing, and so can be used to prioritise research needs.

5. Conclusion

The motivation for the development of the risk

assessment spreadsheet was to facilitate risk manage-

ment prioritisation. Its application, thus, is similar to

the Level 1 risk assessment proposed by van Gerwen

et al. (2000). The model is intended to be generic but

robust, and to include all elements that affect food

safety risks. We propose that the tool can be used by

risk managers and others without extensive experi-ence in risk modelling and as a simple and quick

means to develop a first estimate of relative risk. It can

also be used as a training and risk communication aid

to help determine data needs.

The tool is preliminary, and should be seen as a

prototype, not a definitive model. The tool also

requires that users understand the models limitations.

Despite those limitations, the model includes all

elements required to estimate the risk of illness from

foods. It can be modified to suit the specific question

of the risk assessor or risk manager, and we have

indicated possible developments and refinements to

improve the utility of the tool.

Tools such as these can help managers to think

about how risks arise and change and, thus, to help to

decide where interventions might be applied with

success. We consider the tool as a useful and con-

venient aid to help risk managers reach food safety

decisions more objectively. The spreadsheet can be

downloaded from: http://www.agsci.utas.edu.au/

downloads/ratool.zip.

Acknowledgements

The authors wish to acknowledge the helpful and

constructive comments of Dr. D. Jordan, New South

Wales Agriculture, Dr. D. Schaffner, Rutgers Univer-

sity; Dr. E. Todd of Michigan State University and Mr.

A. Fazil of Health Canada that led to improvements in

the model structure and interface. The spreadsheet tool

had its inception in food safety risk assessments

conducted for Australias Dairy Research and Develop-

ment Corporation, SafeFood NSW and Seafood

Services Australia. TR also thanks Dr. R. Chandlerand Mr. C. Chan for the impetus and encouragement

they provided to develop early prototypes of the tool.

We are also indebted to Meat and Livestock Australia

for ongoing support for microbial food safety research.

References

ABS (Australian Bureau of Statistics), 2000. Download (14 July

2000) from: http://www.abs.gov.au/Ausstats/.



14/15

ANZFA, 1999. Food Safety Standards Costs and Benefits. AGPS,

Canberra.

Bernstein, P.L., 1996. Against the Gods: The Remarkable Story of

Risk. Wiley, New York, 383 pp.

Buchanan, R.L., 1997. National advisory committee on microbio-logical criteria for foods Principles of risk assessment for ill-

nesses caused by foodborne biological agents. J. Food Prot. 60,

14171419.

Buchanan, R.L., Damert, W.G., Whiting, R.C., van Schothorst, M.,

1997. Use of epidemiologic and food survey data to estimate a

purposefully conservative dose response relationship for Liste-

ria monocytogenes and incidence of listeriosis. J. Food Prot. 60,

918922.

CAC (Codex Alimentarius Commission), 1999. Proposed Draft

Principles and Guidelines for the Conduct of Microbiological

Risk Assessment (At step 5 of the Procedure). ALINORM, 99/

13, Appendix IV, pp. 5864.

Cassin, M.H., Paoli, G.M., Lammerding, A.M., 1998a. Simulation

modeling for microbial risk assessment. J. Food Prot. 61, 1560

1566.

Cassin, M.H., Lammerding, A.M., Todd, E.C.D., Ross, W., McColl,

R.S., 1998b. Quantitative risk assessment for Escherichia coli

O157:H7 in ground beef hamburgers. Int. J. Food Microbiol. 41,

2144.

CAST, 1994. Foodborne pathogens: risk and consequences. Council

for Agricultural Science and Technology, USA. Task Force Re-

port No. 122.

Conaty, S., Bird, P., Bell, G., Kraa, E., Grohmann, G., McAnulty, J.,

2000. Hepatitis A in New South Wales, Australia from con-

sumption of oysters: the first reported outbreak. Epidemiol. In-

fect. 124, 121 130.

Corlett, D.A., Pierson, M.D., 1992. Hazard analysis and assignmentof risk categories. In: Pierson, M.D., Corlett, D.A. (Eds.),

HACCP: Principles and Applications. Van Nostrand-Reinhold,

New York, pp. 29 38.

FAO (Food and Agriculture Organisation of the United Nations),

1995. Application of risk analysis to food standards issues. Re-

port of the Joint FAO/WHO Expert Consultation. Geneva, 13

17 March. WHO, Geneva.

Farber, J.M., Ross, W.H., Harwig, J., 1996. Health risk assessment

of Listeria monocytogenes in Canada. Int. J. Food Microbiol.

30, 145156.

FDA/FSIS, 2001a. Draft assessment of the relative risk to public

health from foodborne Listeria monocytogenes among selected

categories of ready-to-eat foods. Download from: http://www.

foodsafety.gov/~dms/lmrisk.html.FDA/FSIS, 2001b. Draft risk assessment on the public health im-

pact of Vibrio parahaemolyticus in raw molluscan shellfish.

Download (1 July 2001) from: http://www.foodsafety.gov/.

FSIS, 1998. Risk assessment for Salmonella enteritidis in eggs.

Food Safety Inspection Service. USA, WWW site: http://www.

fsis.usda.gov/ophs/risk/contents.htm.

HCP (European Commission, Health and Consumer Protection Di-

rectorate-General), 2000. First report on the harmonisation of

risk assessment procedures. The Report of the Scientific Steer-

ing Committees Working Group on Harmonisation of Risk

Assessment Procedures in the Scientific Committees Advising

the European Commission in the Area of Human and Environ-

mental Health, 26 27 October. Published on the internet 20/12/

2000. Download from http://europa.eu.int/comm/food/fs/sc/ssc/

out83_en.pdf.

Hitchins, A.D., 1996. Assessment of alimentary exposure to Listeriamonocytogenes. Int. J. Food Microbiol. 30, 7185.

Huss, H.H., Reilly, A., Ben Embarek, P.K., 2000. Prevention and

control of hazards in seafoods. Food Control 11, 149156.

ICMSF (International Commission for the Microbiological Specifi-

cations for Foods), 1996. In: Roberts, T.A. (Ed.), Micro-organ-

isms in Foods: 5. Microbiological Specifications of Food

Pathogens. Blackie Academic and Professional, London, 508

pp.

ICMSF (International Commission on Microbiological Specifica-

tions for Foods), 1998. Potential application of risk assessment

techniques to microbiological issues related to international

trade in food and food products. J. Food Prot. 61, 10751086.

ILSI (International Life Science Institute), North AmericaRisk

Science Institute Pathogen Risk Assessment Working Group,

1996. A conceptual framework to assess the risks of human

disease following exposure to pathogens. Risk Anal. 16, 841

848.

Jaykus, L.-A., Hemard, M., Sobsey, M.D., 1994. Human enteric

viruses. In: Hackey, C.R., Pierson, M.D. (Eds.), Environmental

Indicators and Shellfish Safety. Chapman & Hall, New York, pp.

92153.

Jaykus, L.-A., 1996. The application of quantitative risk assessment

to microbial food safety risks. Crit. Rev. Microbiol. 22, 279

293.

Jones, E.M., McCulloch, S.Y., Reevest, D.S., MacGowan, A.P.,

1994. A 10 year survey of the epidemiology and clinical aspects

of listeriosis in a provincial English city. J. Infect. 29, 91 103.Jurado, R.L., Farley, M.M., Pereira, E., Harvey, R.C., Schuchat, A.,

Wenger, J.D., Stephens, D.S., 1993. Increased risk of meningitis

and bacteremia due to Listeria monocytogenes in patients with

human immuno-deficiency virus infection. Clin. Infect. Dis. 17,

224227.

Kindred, T.P., 1996. Risk analysis and its application in FSIS. J.

Food Prot., 24 30, Supplement.

Lammerding, A.M., 1997. An overview of microbial food safety

risk assessment. J. Food Prot. 60, 14201425.

Lindqvist, R., Westoo, A., 2000. Quantitative risk assessment for

Listeria monocytogenes in smoked or gravad salmon and rain-

bow trout in Sweden. Int. J. Food Microbiol. 58, 181 196.

Macintosh, W.A., Christensen, L.B., Acuff, G.R., 1994. Perceptions

of risks of eating undercooked meat and willingness to changecooking practices. Appetite 22, 8396.

Mead, P.S., Slutsker, L., Dietz, V., McCaig, L.F., Bresee, J.S., Sha-

piro, C., Griffin, P.M., Tauxe, R.V., 1999. Food-related illness

and death in the United States. Emerging Infect. Dis. 5, 607

625.

Morgan, M.G., 1993. Risk analysis and management. Sci. Am.,

3241 July.

Nolla-Salas, J., Anto, J.M., Almela, M., Col, P., Gasser, I ., Plasen-

cia, A., the collaborative study group of listeriosis of Barcelona,

1993. Incidence of listeriosis in Barcelona, Spain, in 1990. Eur.

J. Clin. Microbiol. Infect. Dis. 12, 157161.



15/15

Notermans, S., Mead, G.C., 1996. Incorporation of elements of

quantitative risk analysis in the HACCP system. Int. J. Food

Microbiol. 30, 157173.

NRC (National Research Council), 1983. Risk Assessment in the

Federal Government: Managing the Process. National AcademyPress, Washington, DC.

NRC (National Research Council), 1994. Science and Judgement in

Risk Assessment. National Academy Press, Washington, DC.

NRC (National Research Council), 1996. Understanding Risk: In-

forming Decisions in a Democratic Society. National Academy

Press, Washington, DC.

PCCRARM (Presidential/Congressional Commission on Risk As-

sessment and Risk Management), 1997. Framework for Envi-

ronmental Health Risk Management. The Presidential/Congres-

sional Commission on Risk Assessment and Risk Management.

213 pp. Available online at http://www.riskworld.com.

Rose, J.B., Sobsey, M.D., 1993. Quantitative risk assessment for

viral contamination of shellfish and coastal waters. J. Food Prot.

56, 1043 1050.

Schlundt, J., 2000. Comparison of microbiological risk assessment

studies published. Int. J. Food Microbiol. 58, 197202.

Sumner, J.L., Ross, T., 2002. A semi-quantitative seafood safety

risk assessment. Int. J. Food Microbiol., this issue.

Todd, E.C.D., Harwig, J., 1996. Microbial risk analysis of food in

Canada. J. Food Prot., 1018, Supplement.

van Gerwen, S.J.C., te Giffel, M.C., vant Reit, K., Beumer, R.R.,

Zwietering, M.H., 2000. Stepwise quantitative risk assessment

as a tool for characterization of microbiological food safety. J.

Appl. Microbiol. 88, 938951.

Van Schothorst, M., 1997. Practical approaches to risk assessment.

J. Food Prot. 60, 14391443.

Vose, D., 1996. Quantitative risk analysis: a guide to Monte Carlo

simulation modelling. John Wiley and Sons, N.Y.

Voysey, P.A, Brown, M., 2000. Microbiological risk assessment: a

new approach to food safety control. Int. J. Food Microbiol. 58,

173180.Walls, I., Scott, V.N., 1996. Validation of predictive mathematical

models describing the growth of Escherichia coli 0157:H7 in

raw ground beef. J. Food Prot. 59, 13311335.

WHO/FAO (World Health Organisation/Food and Drug Organisa-

tion), 1999. Risk assessment of microbiological hazards in

foods. Report of a Joint FAO/WHO Expert Consultation. Gen-

eva, Switzerland. 15 19 March, 1999. World Health Organisa-

tion, Geneva, 24 pp.


tion), 2000a. Exposure assessment of Listeria monocytogenes in

ready-to-eat foods. Download (28 October 2001) from: http://

www.who.int/fsf/mbriskassess/index.htm.


tion), 2000b. Exposure assessment of Salmonella Enteritidis in

eggs. Download (28 October 2001) from: http://www.who.int/

fsf/mbriskassess/index.htm.


tion), 2000c. Exposure assessment of Salmonella spp. in broil-

ers. Download (28 October 2001) from: http://www.who.int/fsf/

mbriskassess/index.htm.


tion), 2001. Risk characterization of Salmonella spp. in eggs and

broiler chickens and Listeria monocytogenes in ready-to-eat

foods. Download (28 October 2001) from: http://www.who.int/

fsf/mbriskassess/index.htm.


food safety risk assessment tool

Documents