strategies for addressing the problems of subjectivity and

gxpand jv t .com Journal of Validation technology [Autumn 2010] 85

For more Author

information,

go to

gxpandjvt.com/bios[

P E E R - R E V I E W E D

Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Management Exercises

Part II: Risk Communication and Perception Issues

ABSTRACTRisk communication is a key component of the qual-ity risk management process for which problems of subjectivity and uncertainty may arise. This is a result of the fact that stakeholders form judgements about risks based on their own perceptions of those risks. Individual perceptions may lead to subjective differ-ences in opinion and uncertainty about the outcomes of quality risk management exercises. It is important to ensure that risks are communicated to decision makers, stakeholders, and other interested parties in a way that minimizes misperception.

This paper reviews some of the factors relating to risk perception and problems of subjectivity and uncer-tainty during quality risk management work. The good manufacturing practice (GMP) environment may benefit from adopting a more rigorous approach to risk

communication than is currently applied. Applying a more rigorous approach may help reduce the extent of subjectivity and uncertainty that currently affect quality risk management activities. The literature shows that there has been little, if any, formal work done in the GMP environment on how the problems of risk perception may arise.

A number of practical strategies are presented for how improvements may be realized at a practical level in this area. These are based on research work that has been carried out in various fields such as experimental psychology and uncertainty analysis. The paper also provides a discussion on the selection of stakeholders for risk communication, the selection of employees for delivery of risk communication within a pharma-ceutical company, and the types of information that should generally be covered in risk communication.

Kevin O’Donnell

ABOUT THE AUTHORKevin O’Donnell, Ph.D., is a senior GMP inspector and market compliance manager at the Irish Medicines Board in Dublin, Ireland. He can be reached by e-mail at [email protected].

86 Journal of Validation technology [Autumn 2010] i v t home.com


INTRODUCTIONThis paper is the second of two papers that further develops some of the key points made in an article published in the Journal of Validation Technology in Feb-ruary 2007 (1). That 2007 paper, titled “Simple Strate-gies for Improving Qualitative Quality Risk Manage-ment Exercises during Qualification, Validation, and Change Control Activities,” addresses how problems of subjectivity and uncertainty associated with the outputs of quality risk management exercises may be addressed. These include the use of more rigorous approaches to the identification of potential failure modes and their related good manufacturing practice (GMP) controls.

This set of papers focuses on some of the other fac-tors that may give rise to subjectivity and uncertainty during quality risk management work. These include potential adverse influences of what are known as human heuristics (2) and issues relating to risk com-munication and perception, which are addressed in this paper.

As discussed in part one of this paper (2), many existing quality risk management tools do not formally deal with the problems of subjectivity and uncertainty that can arise during quality risk management exer-cises. In this regard, most tools do not provide any formal strategies for addressing such problems during the application of quality risk management in GMP or other environments. This is despite the fact that one of the core principles underpinning effective risk management is the principle that risk management explicitly addresses uncertainty. Risk management explicitly takes account of uncertainty, the nature of that uncertainty, and how it can be addressed (3).

Risk communication is perhaps the most important component of the quality risk management process for which problems of risk perception can play a role. Risk perception issues can lead to subjectivity and uncertainty in the outcomes of quality risk manage-ment exercises. International Conference on Har-monisation (ICH) Q9 states, “each stakeholder might perceive different potential harms, place a different probability on each harm occurring, and attribute different severities to each harm” (4).

Stakeholders form judgements about risks based on their own perceptions of those risks, and such judge-ments may be attributed to differences in values, needs, assumptions, concepts, and concerns between the rel-evant parties (3). It is important, therefore, to ensure that the risks that are identified for the process or item under study are communicated to decision makers,

stakeholders, and other interested parties, in a way that minimizes misperception.

This paper presents a discussion on issues relating to risk communication, risk perception, and human heuristics, and on how the GMP environment might benefit from a more rigorous approach to risk com-munication than is currently applied. It is the author’s conclusion that doing so may reduce the extent of sub-jectivity and uncertainty that currently affect quality risk management activities.

WHAT IS RISK COMMUNICATION?ICH Q9 defines risk communication as the sharing of information about risk and risk management between decision makers and others (4). Risk communication can be regarded as a two-way process in which properly informed decisions can be made about the level of risk and the need for risk treatment against properly-established and comprehensive risk criteria (3).

Ronteltap et al. discuss how communication and information become especially important when people have to rely on their judgement rather than on cer-tainty (5). This has implications for quality risk man-agement activities, which by definition have a degree of uncertainty associated with the assessment of risks and which require judgements to be made when decid-ing on the acceptability of risks. They explain how uncertainty in one’s risk perceptions occurs in situa-tions that are “ambiguous, complex, unpredictable, probabilistic” and when “information is unavailable or inconsistent” (5).

Risk communication is an area that has received varying degrees of attention in risk management stan-dards published to date. The Canadian Standards Association’s (CSA) risk management standard gives perhaps one of the highest levels of attention to risk communication activities of any risk management standard available today. CSA defines risk commu-nication as “any two-way communication between stakeholders about the existences, nature, severity, or acceptability of risks” (6).

A number of risk management standards published by national and international official accreditation bodies, such as the 2001 Japanese International Stan-dard titled Guidelines for Development of Risk Manage-ment System (7), as well as those published by pro-fessional organizations such as the Institute of Risk Management in the United Kingdom (8), were subject to a comprehensive review by Raz and Hillson in 2005 (9). This review, which extended to nine published risk management standards (all relatively recent, the


K E V I N O ’ D O N N E L L

oldest going back to 1997), presented a number of interesting findings. Many common features were found between all of the standards reviewed.

For example, Raz and Hillson found that, while there were substantial differences in the scope of the various standards reviewed and in the level of detail covered, all of the standards were relatively similar in the risk management process employed by each. Three main components were identified in the risk management process presented by each standard—risk identification, risk analysis, and risk treatment and reduction. Other common components were also identified, such as planning for the risk management exercise, monitoring and controlling the risk manage-ment process, and the effectiveness of risk treatment actions (9).

One of the differences between the nine standards reviewed by Raz and Hillson was the inclusion in some standards of important activities not considered to be central to risk management activities in some of the other standards. For example, activities such as risk communication and consultation with stakeholders were prominent in the 2004 risk management standard of Standards Australia and Standards New Zealand (10) and in the aforementioned CSA risk management standard (6), but not so much in others.

The International Organization for Standardiza-tion (ISO) standard for medical devices, ISO/IEC 14971:2007, titled Medical Devices–Application of Risk Management to Medical Devices, while comprehen-sive in the risk analysis, assessment, evaluation, and control elements of risk management processes for medical device design and manufacturing, is quite light on requirements relating to risk communica-tion (11). The definition given in the standard for the term “risk management” makes no reference to risk communication activities. The risk management process outlined in the standard does not appear to formally include risk communication activities as an element of that process. The standard does, however, address the factors affecting stakeholder perceptions of risk, citing factors such as the socio-economic and educational background of the society concerned, and the actual and perceived state of health of the patient.

Communication activities are absolutely central to the risk management process outlined in the ISO risk management standard, ISO 31000; it refers to commu-nication within the introductory section. In addition, amongst the attributes of organizations that have a high level of performance with regard to their manage-

ment of risk, continual communications with external and internal stakeholders are considered important in the ISO standard, which views communication with stakeholders as an integral and essential component of risk management (3). In ISO Guide 73:2009, the term stakeholder is defined as a person or organization that can affect, be affected by, or perceive themselves to be affected by a decision or activity.

Communication (and consultation) with external and internal stakeholders can take place during all stages of the risk management process. ICH Q9 does not give much guidance for how risk communica-tions should be executed, but it does state that “par-ties can communicate at any stage of the [quality] risk management process” and that “the output/result of the quality risk management process should be appropriately communicated and documented” (4). It explains that such communications might include those among interested parties (e.g., regulators and industry, industry and the patient; within a company, industry or regulatory authority, etc.) It states that the information communicated might relate to the “exis-tence, nature, form, probability, severity, acceptability, control, treatment, detectability, or other aspects” of risks to quality (4).

ISO 31000 also promotes the use of communica-tion activities at all stages of the risk management process. For example, even in the area of risk analy-sis, which normally occurs very early in the process, it states that confidence in the determination of the level of risk and its sensitivity to preconditions and assumptions should be considered in the analysis and communicated effectively to decision makers and, as appropriate, other stakeholders. The idea here is that any factors that influenced the risk analysis can be highlighted, such as differences of opinion among experts, uncertainty, limitations, etc.

At a high level, risk communication is an exercise worth doing, because it likely helps promote a culture of risk awareness within the organization. Research by Kaplan and Garrick in 1981 demonstrated that risk awareness is an important safeguard that may actu-ally help to reduce risks (12). The following simple example used by those researchers demonstrates this: When people are made aware about “a hole in the road around the corner,” the risk presented by the hole in the road is reduced for those that know about the hole relative to the risk when people do not know about it (12).

The idea that making people aware of a risk can help to reduce that risk is a concept that directly relates



to risk communication activities during quality risk management work. This is one reason why risk com-munication is so important. Any action that serves to help reduce risk should be considered.

Risk communication serves to demonstrate to stake-holders that risks are being effectively managed. In the pharmaceutical industry, the concerns that the public may have about the risks presented by medicines may be offset to some degree by demonstrating to the public that those risks are being effectively managed—hence again a benefit for effective risk communication.

FACTORS THAT MAY INFLUENCE HOW RISKS ARE PERCEIVEDRisk communication is a component of the quality risk management process that can be affected by prob-lems of subjectivity and uncertainty. This may be because different decision makers, stakeholders, and other interested parties may perceive risks differently. ICH Q9 also acknowledges that risk perception is an important issue, stating that each stakeholder might perceive different potential harms, place a different probability on each harm occurring, and attribute different severities to each harm (4).

Risk perception issues may lead to differences in the perceived value of the various risk-mitigating actions that have been proposed by those risks. This was evident during a number of workshops that were run by the author with post-graduate validation students, as well as with regulatory inspectors and industry staff during research into the development of quality risk management approaches for GMP applications (1, 13, 14).

For example, during one workshop in which a qual-ity risk management process was applied to a tablet coating process, there was a perception amongst some team members that it was not possible to ever identify risk-mitigating actions that could reduce the severity of the consequences of failure modes in the coating pro-cess. This was also observed in a workshop in which a proposed change control at an active pharmaceutical ingredient (API)-manufacturing facility was subjected to a quality risk management exercise. (The change control in question involved a proposal to switch from using a cone dryer to dry batches of an API to a new fil-ter dryer at the site.) In both workshops, there was the perception that the severity ratings assigned to failure modes could not be reduced via the implementation of new risk-mitigating controls, whatever the nature of those controls. See Reference 1 for a detailed review of the second of these workshops.

In order to address this perception issue, practi-cal examples of controls that can alter the severity of the consequences of failure modes were developed. These examples proved beneficial when a third work-shop was run using the same team members and the same quality risk management process. The third workshop concerned the application of a quality risk management methodology to the early scale-up steps in a fermentation process used in the manufacture of a monoclonal antibody product.

In relation to biotechnology-related risks, research by Savadori et al. in 2004 found that risk perception and cost/benefit considerations were directly inter-related (15). In relation to the risks presented by nuclear power plants, Costa-Font et al. reported in 2008 how, at least in the UK, people’s attitudes about nuclear power were highly politically motivated. Indi-viduals were found to adjust their own attitudes and perceptions to conform to the “underlying values held by their political position or party affiliation” (16). In relation to food, Ronteltap et al. reported in 2007 how many of the risk characteristics that engender the greatest concerns among consumers are associ-ated with food technologies and related innovations (5). These include characteristics such as when new technologies are perceived to be largely “involuntary, unobservable, out of control of the consumer,” and which may have health effects that are thought to be “unknown, delayed, and potentially fatal” (5).

Understanding the factors that govern how risks are perceived is made complicated by the influence of psychological and cognitive processes. Risk perception has long been an important area of research for psy-chologists in various disciplines. The most prominent of these researchers have probably been Kahneman, Tversky, and Slovic (26-28) who have done a lot of work in the area of human heuristics. See Part 1 of this paper for a more detailed discussion in this regard.

How different groups perceive risk has been found to be important, and there is a large body of research published in this regard (5, 15-25). When there are differing views and opinions among stakeholders on the significance or importance of a particular risk, it can be difficult to reach agreement on the acceptability (or non-acceptability) of that risk, or on the suitability of a course of action proposed to address the risk, if one is required. For example, one stakeholder group might believe that strong mitigation measures are required to address a particular risk, while another may believe that the risk at hand is not one that requires any risk mitigation at all. It is important, therefore, to



give consideration to issues relating to risk perception when determining what risk communication activities are required during quality risk management work.

The literature shows that there has been little, if any, formal work done in the GMP environment on how the problems of risk perception may apply to GMP-related risk communications. It is likely that risk perception is as relevant to the GMP environment as it is to other industries, such as the insurance, nuclear power, and aerospace industries. For example, the pharmaceutical industry is one in which technology plays an important role. In relation to technology, Nivolianitou (29) and others (5, 25, 30) have written extensively about how the general public sometimes fears modern technology and can exhibit a strong aversion to it. These researchers promote the idea that there is a need for the general public to be better trained on how to understand and perceive risks relat-ing to modern technology in order to “rationalise the fear and aversion against modern technology” that they may have (29).

The above is supported by research performed by Darby and Karni in the early 1970s. Darby and Karni identified certain attributes of goods and services that can be the subject to problems of risk perception, because the average consumer cannot readily verify those attributes or qualities (31). These attributes were termed ‘credence qualities,’ and in healthcare settings, as reported by Ford et al. in 1988, an example of a service that has so-called credence attributes is gall bladder surgery (32). Here, the patient is “unlikely to know with absolute certainty whether the diagnosis was correct and is unlikely to have the technical exper-tise to assess the quality of the service provided” (32).

Ronteltap et al. further develop this concept, argu-ing that credence qualities are also associated with new technologies such as those developed by the food industry, the “costs and benefits of which cannot be unambiguously verified by the individual con-sumer from personal experience” (5). Examples of such credence qualities in this area would include the safety, sustainability, and naturalness of the new technology (5).

In the GMP environment, it can be useful to know about such research findings, particularly when apply-ing quality risk management principles and tools to change control activities. Change control is often the process that is used to introduce new technologies and products into a manufacturing site. The EU GMPs require that risk analysis be employed in relation to proposed changes in facilities, systems, and equip-

ment (33). How people perceive the risks that may be associated with such proposed changes during risk analysis and quality risk management activities has the potential to lead to subjectivity and uncertainty in the outcomes of such exercises. Further research on the potential influences of credence qualities dur-ing such activities could be beneficial for the GMP environment.

The research that has been performed on the factors that influence risk perception is likely to be useful when incorporating quality risk management activi-ties into pharmaceutical GMP environments. For example, in relation to the insurance industry, Litai’s work at the Massachusetts Institute of Technology (MIT) in the late 1970s produced a listing of nine so-called quantifiable “risk factors” (23). When applied to different risks, these were used to indicate how risks were perceived by the general public. Litai produced a dichotomous scale that accompanied each risk factor, allowing risks to be classified by selecting one of the two options from the scale for each risk factor. See the Table for details.

Litai (23), Starr (24), and others developed weight-ings (termed “risk conversion factors”) for each risk factor, in an attempt to reflect in numerical terms the relative importance to the public of the various risk characteristics. These risk conversion factors were then used to score different risks, and relative risk perception scores were thus determined.

Litai’s work suggested that the public was willing to accept risks that could be considered voluntary (e.g., the risk from smoking cigarettes) that were up to 100 times the magnitude of risks that were considered to be involuntary (e.g., the risks posed by high volt-age electricity lines located near one’s home.) This was the case even when the risk associated with some voluntary activities was estimated to have been a lot larger than some often cited risks that are involun-tarily assumed.

For example, Keller performed an analysis of death rates in the US per annum for various “risky” activities and situations and classified these as either voluntary or involuntary (25). It was found that the calculated risks associated with voluntary activities (e.g., motorcy-cling, receiving surgical anaesthesia and smoking) were found to be significantly higher than risks that were involuntarily assumed (e.g., rail travel accidents, being struck by lightning, and suffering from influenza) (25).

It should be noted, however, that the risk con-version factors shown in the Table should not be considered to be absolute in magnitude; neither were



they intended to be an accurate model of risk percep-tion in all areas of application. Litai’s risk factors and their associated risk conversion factors were derived from an extensive analysis of the database of a major US life insurance company (23), and other researchers in this area have attributed sometimes quite different risk conversion factor values to the same risk factors. Starr (24), for example, assigned a risk conversion factor value of 1000 to the voluntary:involuntary risk factor, which was ten times higher than that estimated by Litai (23).

Despite such differences, the voluntary versus invol-untary nature of risks is a factor that is generally well accepted. This is illustrated in the aforementioned ISO Standard for Medical Devices, ISO/IEC 14971:2007, which states that the way a risk is perceived is related to whether exposure to the risk seems to be involuntary, avoidable, from a man-made source, due to negligence, or directed at a vulnerable group within society (11).

In addition, in his 1981 review of “The Applica-tion of Probabilistic Risk Assessment Techniques to Energy Technologies,” Rasmussen (34) discusses how ‘societal preferences’ for different types of risks are important for decision makers to consider when making public policy decisions on the basis of the results of risk assessments. This is because public official decision makers should be responsive to public preferences (34).

As discussed by Morgan in a widely quoted Scien-tific American paper from 1993 (22), a great deal of research has been performed by experimental psy-chologists into how risks are perceived, and several research studies identified similar influencing factors to those identified by Litai. Research by Slovic and Fischhoff in the late 1970s, for example, indicated that such factors may be grouped into the following three main categories:

• The degree of “dreadfulness” associated with the risk

• The degree to which the risk was understood• The number of people exposed to the risk in

question (35).

Slovic and his colleagues used these categories to define what was called a “risk space,” and when a hazard came within that space, it was argued that a person’s perception of the risk associated with that hazard tended to be significantly affected (35).

Such findings should prove useful when determin-ing how to communicate risks during, or following, quality risk management exercises performed in a GMP environment. This is because the factors that may relate to risk perception issues may influ-ence how the outputs of quality risk management exercises may be judged and accepted by decision makers and stakeholders following risk communica-tion activities.

The findings from experimental psychology stud-ies also demonstrate the important role that human heuristics (or unconscious rules of thumb) may play in the perception of risk. See Part I of this paper (2) for a detailed discussion in this regard.

Process experts are often employed when qual-ity risk management exercises are being performed. As discussed by Garfield in 1982 (36), the work of Fischhoff et al. (35) demonstrated that an expert’s perception of risk can differ markedly from that of the general public. Fischhoff ’s work showed how lay people tend to regard as risky any technology that is new, imposed on them, unfamiliar, and beyond their control. Such findings are probably important to consider when performing GMP-related quality risk management exercises, given the different groups and stakeholders to whom risk information may be com-

Table: Litai’s risk factors (1980).Risk Factor Dichotomous Scale Risk Conversion Factor

Volition Voluntary : Involuntary 100

Severity Ordinary : Catastrophic 30

Origin Natural : Man-made 20

Manifestation of Effects Delayed : Immediate 30

Exposure Pattern Continuous : Occasional 1

Controllability Controllable : Uncontrollable 5-10

Familiarity Common/Old Hazard : Dread/New Hazard 10

Benefit Clear : Unclear 10

Necessity Necessary : Luxury 1



municated and the highly technical nature of GMP activities in general.

STRATEGIES FOR ACHIEVING EFFECTIVE RISK COMMUNICATIONSThe following simple strategies can be adopted when designing the risk communication component of a quality risk management process to reduce the effects of individual perception that may arise and to enhance the effectiveness of the risk communication.

Strategy OneEnsure that those charged with the task of performing risk communication understand the factors that can influence how risks are perceived and that measures are taken to counteract the effects of those factors.

The factors that can influence how risks are per-ceived are discussed previously in some detail. A practical understanding of those factors will prove helpful for personnel in GMP environments who are responsible for communicating the outcomes of quality risk management exercises.

For example, consider the risks that may be asso-ciated with glass or other particulates in injectable medicinal products. Such risks might fall into Fis-chhoff and Slovic’s high “dreadfulness” group of risks (35), and in accordance with Litai’s risk factors (23), such risks may also be characterized as involuntary risks, with perceived catastrophic consequences. These types of risks are likely going to be subject to problems of misperception. It would be important, therefore, during any risk communication activity relating to those risks that time is spent on clearly describing how such risks are controlled at a practical, detailed level, using clear and definite language. This may help ensure that stakeholders know and understand the measures that are in place for addressing such risks and the effectiveness of those measures.

Strategy TwoConsider allowing stakeholders an opportunity to review, to learn about, or to be involved in the quality risk manage-ment process or the item under study at an early time-point.

Research has shown that problems of misperception can be reduced during risk communication activities when stakeholders have had an opportunity to partici-pate in or review the quality risk management process or the item under study at an early time-point (30, 37). Early communication about a quality risk management exercise is beneficial in addressing potential problems of risk misperception that may arise later on, and more

value may be realized. It may not always be useful to wait until the end of the exercise to carry out risk communication activities.

ISO 31000 actually goes further than just promoting risk communication in this regard; it indicates that a consultative approach with stakeholders should be considered. Consultation is regarded as a two-way pro-cess of informed communication, occurring between an organization and its stakeholders on an issue prior to making a decision or prior to determining a direc-tion to take on a particular issue. It is a process that impacts on a decision through influence rather than through power. While it is not joint decision making, it produces inputs to decision making (3).

There are advantages to be gained with such an approach. For example, the interests of stakeholders can be better understood and considered; risks can be more adequately identified and different views can be appropriately taken into account when defining risk acceptance criteria and when evaluating risks. In addi-tion, a consultative approach may secure endorsement and support for the risk control or treatment plan that is proposed (3).

ICH Q9 indicates that risk communications may occur between regulators and industry, between indus-try and the patient; within a company, industry or regulatory authority, etc. (4). In GMP environments, it can be difficult to see how risk communications (never mind risk consultations) with certain stakeholder groups (such as patient groups) might be achieved in practice. The highly confidential and technical nature of active substance and medicinal product manufactur-ing are factors that probably influence which stake-holders companies may communicate with and the nature and content of those communications. While such issues may not easily be addressed, there are some opportunities to be realized in this area.

A relatively simple example of this might be found during a regulatory GMP inspection. The company might choose to proactively invite the inspector to review and discuss early parts of a quality risk man-agement exercise. This might simply involve com-municating the list of potential failure modes or nega-tive events that have been identified to date to the inspector, or it might extend to sharing information about which of those failure modes or negative events are to be studied in greater detail during subsequent steps of the quality risk management process. Making early communications in this way may help ensure that, at the next regulatory inspection at the site, the inspector is not surprised [when reviewing the final



report on the same quality risk management exercise] by the nature of the failure modes or negative events and their associated risks that were identified. This is a simple example of how a company may put in practice the ideas relating to communication and consultation that are put forward by ISO 31000. It is important to note, however, that GMP inspectors from competent authorities are regulators. They are not there to act as consultants to the pharmaceutical industry.

Strategy ThreeEnsure that the key elements outlined as follows are designed into the risk communication process.

These were presented by Vesper in his text titled “Risk Assessment and Risk Management in the Phar-maceutical Industry” (38), and they reflect the work of Morgan (30) and others in this area. Vesper proposed that the following elements make up any good risk communication strategy:

• Identify the stakeholders of the quality risk man-agement exercise and determine those stakehold-ers to whom risk communications should be made following the exercise.

Note: This helps determine the audiences for the risk communication activity, the actual content of the risk communication, and the technical level at which to pitch the risk communication message.

• Ensure that the persons designated to communi-cate the outcomes of the quality risk management exercise are credible, trustworthy, honest, and have the necessary level of expertise to field questions and explain the quality risk management process and its outcomes.

Note: It is tempting to only appoint persons who are considered experts in the item studied in the quality risk management exercise to be the risk com-municators. This may not always be beneficial, however, and others may be better communicators for the audience at hand. The truthfulness of any risk communications should also be assured, and thus the individuals that are assigned responsibility for making risk communicators should be carefully selected.

• Ensure that the content of the risk communica-tion is balanced and open, and not manipulated by bias or ulterior motives.

Note: There will usually be some uncertainty asso-ciated with the outcomes of the quality risk man-agement exercise. It is thus important to openly communicate any significant sources of uncertainty and any pertinent assumptions that relate to the quality risk management process and its outcomes.

• Before risk communications are carried out, study the risks generated by the risk management exer-cise to determine whether particular risks might be misperceived by the people to whom the risk communication is targeted.

Note: See the example given above relating to glass or other particulates in injectable medicinal products for how this element might be applied in practice.

OPTIMIZING RISK COMMUNICATIONThe following are some ideas and recommendations for how these elements of a good risk communi-cation strategy as suggested by Vesper and others may be applied at a practical level within the GMP environment.

Selection of Stakeholders for Risk Communication Stakeholder considerations are important in any qual-ity risk management process. ISO 31000 highlights this and promotes the concept of evaluating and under-standing the external context in which the organiza-tion operates. This includes evaluating the organiza-tion’s relationships with external stakeholders, as well as the perceptions and values of those stakeholders (3). In a pharmaceutical and GMP environment, the list of stakeholders can be very large, but the main groups to consider for risk communication include the following:

• Employees within the company who should be aware of a risk that has been identified via a qual-ity risk management exercise relating to their own work area

• Company management, including technical, administrative, and financial managers, where relevant

• Suppliers and customers, when considered necessary

• Healthcare professionals, when considered necessary

• Patient and patient representative organizations, when considered necessary



• GMP inspectors and other regulators, when con-sidered necessary.

The above list indicates that many of these groups are internal rather than external. ISO 31000 also addresses such communications and it promotes the use of internal communication and reporting mecha-nisms in order to support and encourage account-ability and ownership of risk. It indicates that these mechanisms should ensure that “key components of the risk management framework, and any subsequent modifications, are communicated appropriately” and that there are processes in place for consultation with internal stakeholders (3).

It is important also to determine the needs of the particular audience to whom risk communication will be made and to consider the level of information the audience may already have on the item under study. This allows the risk communication message to be tailored accordingly.

For example, GMP inspectors may be technically versed in the process or item studied in the quality risk management exercise. They may be highly interested in the detail of how the risks were arrived at, in the risk mitigation measures proposed, and in how compliance with the applicable GMPs was achieved. On the other hand, senior management staff at the company, while they may also be concerned with those same things, may be less technically familiar with the details of the item studied in the exercise. Management may be more interested in the cost and resource implications of the recommendation arising from the quality risk management exercise. Therefore, the following two things should be considered:

• The information that the different stakeholder groups will likely need to receive

• How interested in the technical details the dif-ferent stakeholder groups may be in the area of concern and in the quality risk management pro-cess that was used.

Some quality risk management exercises performed in GMP environments are carried out to support quali-fication, validation, and change control activities. In the EU, Annex 15 to the EC guide to GMP requires that a “risk assessment approach should be used to determine the scope and extent of validation,” and that the likely impact of changes “should be evaluated, including risk analysis” (33).

A question that sometimes arose during the author’s research into the use of quality risk management

approaches for qualification, validation, and change control activities was whether all stakeholders needed to receive direct risk communication following a qual-ity risk management exercise of relevance to them. For example, while patients are clear stakeholders of any process that produces, controls, or regulates medici-nal products, should they receive risk communica-tions from a pharmaceutical company describing the outcomes of quality risk management exercises that are focussed on qualification, validation, and change control activities?

In the author’s opinion, this is not usually neces-sary. The application of a quality risk management tool in GMP environments was specifically to support qualification, validation, and change control activi-ties, its outcomes are likely to be highly technical in nature and concerned with the specifics of GMP. While patients are of course clear stakeholders of any such manufacturing activity, the outcomes of such qual-ity risk management exercises may not be of direct value to patients or their representative organizations and may be of more value to those regulating such activities.

Having said that, and recognizing the fact that there is no official guidance available in this regard (that the author knows about), it seems reasonable to con-sider each situation on a case-by-case basis. This is because there may be a specific reason to carry out such risk communication with patients or with their representative organizations, even when the quality risk management exercise was performed specifically for qualification, validation, and change control pur-poses. It most likely depends on the purpose of the exercise that was performed. For example, if the exer-cise was performed as a result of a complaint from a patient, or if the manufacturing process or item under study in the exercise was the subject of adverse media coverage, it may be useful (and indeed necessary) to communicate the results of the exercise with patients or their representative organizations.

Risk Communication PerformanceTruthfulness and trust are important in risk communi-cations. This is particularly the case from the viewpoint of external stakeholders, as ensuring this can lead to improved stakeholder confidence (3). Costa-Font et al. found that, in relation to the risks presented by nuclear power plants and radioactive waste, the level of trust that people have in those providing risk-related information can be a significant determinant of how people view nuclear power generation (16).



Ronteltap et al. highlight the need for risk regulators and managers to be considered trustworthy in the eyes of the public, as the lack of such trust has been shown to give rise to problems of risk perception (5). This is especially the case in situations where the item of concern is associated with so-called credence qualities.

The quality risk management team leader should ensure that the person or persons designated to com-municate the outcomes of the quality risk management exercise are credible, trustworthy, honest, and have the necessary level of expertise to field questions and to explain the quality risk management process and its outcomes.

It can be tempting to only appoint persons who are considered experts in the manufacturing process or item studied to be the risk communicators, but this may not always be beneficial; others may be better communicators with the audience at hand. Vesper points out that experts sometimes “use terminology that is technically correct but which overwhelms” the lay stakeholder (38). Even in GMP applications, when the audience for the risk communication may comprise technical people such as engineers, chem-ists, and regulatory GMP inspectors, it is important to recognize that these people will likely not all have the same degree of knowledge in the technology underpin-ning the item under study as the risk communicator may have.

Risk Communication Content ICH Q9 states, “the included information [in risk com-munication] might relate to the existence, nature, form, probability, severity, acceptability, control, treatment, detectability, or other aspects of risks to quality” (4). This is useful guidance, but as there will usually be some uncertainty associated with the outcomes of the quality risk management exercise, it is also impor-tant to openly communicate any significant sources of uncertainty and any pertinent assumptions that relate to the quality risk management process and to its outcomes.

Vesper discusses how the “fair and balanced inclu-sion of other, conflicting, points of view” can “shape how recipients feel” about the [risk management] process (38). It is important also to truthfully report the findings of the exercise, without manipulation of the results or data to suit the desired outcomes. How people feel about particular risks has also been shown to be an important factor that can affect how risks are perceived, as illustrated by the work of Slovic and Peters in 2006 (28).

When communicating about a particular risk, the following items should be considered for inclusion:

• The likely sources or causes of the risk• The likely consequences of the risk• The estimated magnitude of that risk• Any important uncertainties and assumptions

associated with the risk estimation • The level of acceptability of the risk before and

after risk control measures• The risk control measures adopted, or those that

will be adopted• For risks that may still be regarded as unacceptably

high following risk control measures, informa-tion on what actions will be or have been taken as a result

• The procedure in place governing risk acceptance decisions.

Communication and consultation activities may help ensure that those accountable for implement-ing the risk management process and its stakeholders understand the basis on which decisions are made and the reasons why particular actions are required (3). But another question that can arise is whether all of the risks that may be identified should be addressed in risk communications with stakeholders.

In the author’s experience of performing and review-ing quality risk management exercises, this would seem an unrealistic requirement if it were in place, as the value of doing this is questionable. Indeed, ICH Q9 recognises this, stating that communications “need not be carried out for each and every risk accep-tance” (4). The team that performed the quality risk management exercise should use their experience and technical judgement to decide which risks or issues need communicating and which do not.

In practice, it might be useful to adopt the policy that, for all risks identified during a quality risk man-agement exercise, the team must formally review each risk and decide whether it, or a related issue, needs to be communicated, and with whom. A good rule of thumb might be that a risk probably needs to be com-municated to those who either could be affected by that risk or to those who are important in mitigating or controlling that risk.

Finally, with regard to the content of risk communi-cations being balanced and open, this may be one of the highest areas of concern for external stakeholders. Truthfully reporting the findings of the quality risk management exercise without manipulation of the results or data to suit the desired outcome can help



ensure this happens. Significant sources of uncer-tainty and pertinent assumptions made during the quality risk management exercise should also be addressed during such communication. It is also useful if conflicting and alternate points of view are openly communicated, if applicable, as the balanced inclusion of other, conflicting, points of view can “shape how recipients feel about the [risk manage-ment] process” (38).

CONCLUSIONSThe area of risk communication is a key component of the quality risk management process to which prob-lems of subjectivity and uncertainty may apply. This is a result of the fact that stakeholders form judge-ments about risks based on their own perceptions of those risks. Individual perceptions may lead to sub-jective differences in opinion and uncertainty about the outcomes of quality risk management exercises. Therefore, it is important to ensure that when risks are identified in a process or item under study, they are communicated to decision makers, stakeholders, and other interested parties in a way that minimizes misperception.

This paper has reviewed some of the factors relat-ing to risk perception that can give rise to problems of subjectivity and uncertainty during quality risk management work. It is argued that the GMP envi-ronment may benefit from adopting a more rigorous approach to risk communication than is currently applied, because this may help reduce the extent of subjectivity and uncertainty that currently affect qual-ity risk management activities. The literature shows that there has been little, if any, formal work done in the GMP-environment on how the problems of risk perception may arise.

Research performed by experimental psycholo-gists into how risks are perceived has found that the factors that influence risk perception may be grouped into three main categories—the degree of “dreadfulness” associated with the risk, the degree to which the risk was understood, and the number of people exposed to the risk in question. Such findings can be useful when designing risk communication strategies in order to reduce the problems of risk misperception.

The intent of parts one and two of this paper is to deliver useful and practical information on some of the factors that can give rise to problems of subjectiv-ity and uncertainty during quality risk management work. These factors relate to the potential adverse

influences of what are known as human heuristics and issues relating to risk communication and percep-tion. The strategies outlined in these two papers for addressing some of those problems are relatively easy to implement and inexpensive in nature. It is hoped that this work may lead to more research work being carried out within the GMP environment in an effort to improve the outcomes of quality risk management exercises with respect to the problems of subjectivity and uncertainty.

REFERENCES1. O’Donnell, K., Greene, A., “Failure Modes-Simple Strat-

egies for Improving Qualitative Quality Risk Manage-ment Exercises During Qualification, Validation, and Change Control Activities,” Journal of Validation Technol-ogy, Vol. 13, No. 2, February 2007.

2. O’Donnell, K., “Strategies for Addressing the Problems of Subjectivity and Uncertainty in Quality Risk Manage-ment Exercises: Part I—The Role of Human Heuristics,” Journal of Validation Technology, Vol. 16, No. 3, Summer 2010.

3. International Standard ISO 31000, Risk Management–Principles and Guidelines, First Edition November 15, 2009, (ISO Reference number ISO 31000:2009(E)). Published in Ireland by the National Standards Author-ity of Ireland (NSAI) on October 15, 2009.

4. ICH, Quality Risk Management (ICH Q9), Interna-tional Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, November 9th, 2005, available at www.ich.org.

5. Ronteltap, A., van Trijp, J.C.M., Renes, R.J., Frewe, L.J., “Consumer Acceptance of Technology-based Food In-novations,” Appetite, Vol. 49, pp. 1–17, 2007.

6. CSA, Risk Management: Guidelines for Decision-makers, Standard No. CAN/CSA-Q850-97, Canadian Standards Association, Misisauga, Ontario, Canada, 2002.

7. Japanese Standards Association, No. JIS Q2001:2001, Guidelines for Development of Risk Management System, Tokyo, JSA, 2001.

8. Institute of Risk Management/National Forum for Risk Management in the Public Sector/Association of Insur-ance and Risk Managers: A Risk Management Standard, No. IRM/ALARM/AIRMIC, 2002.

9. Raz, T., Hillson, D., “A Comparative Review of Risk Management Standards,” Risk Management: An Interna-tional Journal, 2005, 7(4), 53-66.

10. Risk Management, Standard No. AS/NZS 4360:2004, Standards Australia/Standards New Zealand, Home-bush, New South Wales, Australia, 2004.



11. ISO 14971:2007, Medical Devices–Application of Risk Management to Medical Devices, International Electronic Commission, Geneva, Switzerland, March 2007.

12. Kaplan, S., Garrick, B. J., “On the Quantitative Defini-tion of Risk,” Risk Analysis, Vol. 1, No. 1, pp 11-27, 1981.

13. Workshop held in Cork, Ireland, in November 2005 run by the author with the Validation Sub-Group of Phar-maChemical Ireland. This workshop involved 29 staff members from validation or quality-related roles at 21 pharmaceutical manufacturing companies located in Ireland.

14. PIC/S Risk Management Workshop, Düsseldorf, May 31st-June 2nd, 2006, run by the author with GMP In-spectors from 20 PIC/S member countries.

15. Savadori, L., Savio, S., Nicotra, E., Rumiati, R., Finu-cane, M., & Slovic, P., “Expert and Public Perception of Risk from Biotechnology,” Risk Analysis, 24(5), pp. 1289–1299, 2004.

16. Costa-Font, J., Rudisill, C., Mossialos, E., “Attitudes as an Expression of Knowledge and ‘Political Anchoring,” Risk Analysis, Vol. 28, No. 5, pp.1273-1287, 2008.

17. Kunreuther, H., Novemsky, N., Kahneman, K., “Making Low Probabilities Useful,” Journal of Risk and Uncertainty, Vol. 23, No. 2, pp. 103–120, 2001.

18. Satterfield, T., et al., “Anticipating the Perceived Risk of Nanotechnologies,” Nature Nanotechnology, Vol. 4, pp. 752-758, 2009.

19. MacDonald, J.A., Small, M.J., Morgan, M.J., “Explosion Probability of Unexploded Ordnance,” Risk Analysis, Vol. 28, No. 4, pp. 825-841, 2008.

20. Willis, H.H., DeKay, M.L., Morgan, M.G., Florig, H.K, Fischbeck, P.S., “Ecological Risk Ranking,” Risk Analysis, Vol. 24, No. 2, pp. 363-378, 2004.

21. Fischhoff, B., Bostrom, A., Quadrel, M., “Risk Perception and Communication,” Annual Review of Public Health, Vol. 14, pp.183–203, 1993.

22. Morgan, M. G., “Risk Analysis and Management,” Sci-entific American, pp 32-41, July 1993.

23. Litai, D., “A risk comparison methodology for the assess-ment of acceptable risk,” Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, Mass., 1980.

24. Starr, C., “Social Benefit vs. Technological Risk,” Science, 165:1232-38.

25. Keller, A. Z., Perception and Quantification of Risk, ISPRA courses, Reliability and Data, JRC ISPRA, 21020, Italy, 1984.

26. Kahneman, D., Tversky, A., “Subjective Probability: A Judgement of Representativeness,” Cognitive Psychology, 3:430-354, 1972.

27. Kahneman, D., Tversky, A., “On the Psychology of Pre-diction,” Psychological Review, 80, No 4, 237 – 251, 1973.

28. Slovic, P., Peters, E., “Risk Perception and Affect,” Cur-rent Directions in Psychological Science, Vol. 15, pp. 322–325, 2006.

29. Nivolianitou, Z., “Risk Analysis and Risk Management: A European Insight,” Law, Probability and Risk, 2002, 1(2): 161-174.

30. Morgan, M. G., Morris, S. C., Henrion, M., Amaral, D. A. L., Rish, W. R., “Technical Uncertainty in Quantitative Policy Analysis: A Sulphur Air Pollution Example,” Risk Analysis, 201-216, September 4, 1984.

31. Darby, M. R. and Karni, E., “Free Competition and the Optimal Amount of Fraud,” Journal of Law and Economics, 16 (April), 67-88, 1973.

32. Ford, G.T., Smith, D.B., Swasy, J.L., “An Empirical Test of the Search, Experience and Credence Attributes Frame-work,” Advances in Consumer Research, Volume 15, (eds. Houston, M.J. & Provo, UT, Association for Consumer Research, Pages: 239-244.), 1988.

33. Annex 15 to the Rules Governing Medicinal Products in the European Community, Volume IV, published by the European Commission, and available at http://ec.europa.eu/enterprise/sectors/pharmaceuticals/docu-ments/eudralex/index_en.htm.

34. Rasmussen, N. C., “The Application of Probabilistic Risk Assessment Techniques to Energy Technologies,” Ann. Rev. Energy, 6:123-38, 1981.

35. Fischhoff, B., Slovic, P., Lichtenstein, S., “Fault Trees: Sensitivity of Estimated Failure Probabilities to Problem Representation,” J. Exp. Psychol.–Human Perception Perf. 4:330-44, 1978.

36. Garfield, E., “Risk Analysis, Part 1, How we Rate the Risks of New Technology,” Essays of an Information Sci-entist, Vol. 5, p643-651, August 16, 1982, available at www.garfield.library.upenn.edu.

37. Morgan, M. G., Henrion, M., Uncertainty–A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, 1990.

38. Vesper, J. L., Risk Assessment and Risk Management in the Pharmaceutical Industry, Davis Healthcare International Publishing, 2006.JVT

ARTICLE ACRONYM LISTINGAPI Active Pharmaceutical IngredientCSA Canadian Standards AssociationGMP Good Manufacturing PracticeICH International Conference on HarmonisationISO International Organization for Standardization

strategies for addressing the problems of subjectivity and

Documents