dr.laila professional responsibilty
TRANSCRIPT
-
8/7/2019 Dr.laila Professional Responsibilty
1/133
CPIS 428Professional Computing Ethics
Dr. Laila Nassef
5/1/2011 1
-
8/7/2019 Dr.laila Professional Responsibilty
2/133
Student Responsibilities
Getting up, getting dressed, and eating
breakfast in order to leave for college on time being dependable in carrying out
obligations and duties
Being committed to activeinvolvement in your community
5/1/2011 2
-
8/7/2019 Dr.laila Professional Responsibilty
3/133
Professional Responsibility
-
5/1/2011 3
-
8/7/2019 Dr.laila Professional Responsibilty
4/133
Professional Responsibility
5/1/2011 4
-
8/7/2019 Dr.laila Professional Responsibilty
5/133
-
8/7/2019 Dr.laila Professional Responsibilty
6/133
5/1/2011 6
-
8/7/2019 Dr.laila Professional Responsibilty
7/133
5/1/2011 7
-
8/7/2019 Dr.laila Professional Responsibilty
8/133
-
8/7/2019 Dr.laila Professional Responsibilty
9/133
Recall Responsibilities from
IEEE - Code of Ethics
5/1/2011 9
-
8/7/2019 Dr.laila Professional Responsibilty
10/133
Engineers Responsibility
Engineers are expected to exhibit the higheststandards of
Honesty
Integrity Impartiality
Fairness
Equity Must be dedicated to the protection of the
public health, safety , and welfare
5/1/2011 10
-
8/7/2019 Dr.laila Professional Responsibilty
11/133
IEEE - Code of Ethics Engineers are committed to conduct the highest ethical and
professional manner and agree to: accept responsibility in making decisions consistent with
safety, health, and welfare of the public
avoid real or perceived conflicts of interest
be honest and realistic in stating claims or estimates
reject bribery in all forms
improve understanding of technology, its application, and
potential consequences maintain and improve our technical competence and
undertake technological tasks for others only if qualified
5/1/2011 11
-
8/7/2019 Dr.laila Professional Responsibilty
12/133
IEEE - Code of Ethics
seek, accept, and offer honest criticism oftechnical work
acknowledge and correct errors
credit properly the contributions of others
treat all persons fairly regardless of race, religion,gender, disability, age, or national origin
avoid injuring others, their property, reputation,
or employment by false or malicious action assist colleagues and co-workers in their
professional development and to support them infollowing this code of ethics
5/1/2011 12
-
8/7/2019 Dr.laila Professional Responsibilty
13/133
Professional Responsibility
5/1/2011 13
-
8/7/2019 Dr.laila Professional Responsibilty
14/133
Employer to employee Based on contract
Employee agree to carry out assigned jobs Employer agree to pay compensation
5/1/2011 14
-
8/7/2019 Dr.laila Professional Responsibilty
15/133
Employer to employee
Employer provides appropriate tools and safework environment
Employer avoids asking the employee to do
anything illegal
5/1/2011 15
-
8/7/2019 Dr.laila Professional Responsibilty
16/133
Employer to employee Employee must be honest about his
qualification and experience
Employee should be loyal to employer Instructions are followed conscientiously
Work done diligently and cooperatively Company trade secretes are not revealed
5/1/2011 16
-
8/7/2019 Dr.laila Professional Responsibilty
17/133
Professional to Professional Team member must cooperate with each other
to provide helpful advice and assistance
5/1/2011 17
-
8/7/2019 Dr.laila Professional Responsibilty
18/133
Professional to Client Fiduciary
5/1/2011 18
-
8/7/2019 Dr.laila Professional Responsibilty
19/133
Professional to user
Accepting jobs only if one is competent toperform them, exercising proper care and
diligence ,through testing the finalproduct before delivering it to consumer
5/1/2011 19
-
8/7/2019 Dr.laila Professional Responsibilty
20/133
Professional to Society Take responsibility for the impact of their
products and services not just upon humanbeings but actually upon the whole earth
5/1/2011 20
-
8/7/2019 Dr.laila Professional Responsibilty
21/133
Employee Loyalty
5/1/2011 21
-
8/7/2019 Dr.laila Professional Responsibilty
22/133
Do Employees Have a Special
Obligation to Employers? Some believe we have a prima facie
obligation ofloyalty in employment contexts.
In other words, all things being equal, anemployee should be loyal to his or her employerand visa versa.
225/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
23/133
Does employee loyalty still make sense in thecontext of a large computer corporation?
Duska (1991) argues that in employmentcontexts, loyalty only arises in specialrelationships based on a notion that he calls"mutual enrichment ."
So in relationships in which parties are
pursuing their self-interests, the notion of loyalty would not be applicable.
235/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
24/133
DuskasA
rgument Duska believes that employer-employee relationships
at least where corporations are concerned are
based on self-interest and not on mutual enrichment. He concludes that employees should not necessarilyfeel any sense of obligation of loyalty to corporateemployers.
Corporations like employees to believe that they have
an obligation of loyalty to their employers becausebelieving that serves the corporations interests.
245/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
25/133
Ladds Criticism of Employee Loyalty
Ladd also believes that in the context of corporations,loyalty can only be in one direction.
He argues that a corporation cannot be loyal to anemployee in the same sense that employees aresupposed to be loyal to it.
A corporation's goals are competitively linked to the
benefits employees bring to the corporaion. A corporation can be good to employees only because
it is good for business, i.e., it is in the company's ownself interest.
255/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
26/133
Ladds and Duskas Criticisms Both Duska and Ladd cite corporate self-interest as
an obstacle for a balanced employer-employee
relationship that is required for mutual loyalty. Consider that corporations often go through
downsizing phases in which loyalemployees who have served a company faithfully forseveral years are dismissed as part of restructuring plans.
265/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
27/133
Sometimes Employers Have Been Loyal
Consider a case in which an employercontinues tokeep an employee on the payroll even though that
employee has a chronic illness, which causes her tomiss several months of work.
Also consider a case in which several employees arekept on by a company despite the fact that their
medical conditions have caused the corporation'shealth insurance costs to increase significantly,thereby reducing the company's overall earnings.
275/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
28/133
Employer Loyalty Consider a recent case involving the owner of
Malden Mills, whose physical plant in
Massachusetts was destroyed by fire. The mill's proprietor, Aaron Feurestein, could have
chosen to rebuild his facility in a different state orcountry where employees would work for lower
wages. Instead, Feurestein continued to pay and provide
benefits for his employees while a new facility wasbeing built in Mass.
285/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
29/133
-
8/7/2019 Dr.laila Professional Responsibilty
30/133
-
8/7/2019 Dr.laila Professional Responsibilty
31/133
Whistle-blowing
5/1/2011 31
-
8/7/2019 Dr.laila Professional Responsibilty
32/133
5/1/2011 32
-
8/7/2019 Dr.laila Professional Responsibilty
33/133
:
5/1/2011 33
-
8/7/2019 Dr.laila Professional Responsibilty
34/133
Corruption :
5/1/2011 34
-
8/7/2019 Dr.laila Professional Responsibilty
35/133
(Transparency International)
1995
5/1/2011 35
-
8/7/2019 Dr.laila Professional Responsibilty
36/133
2004133 43 4.6
6.32.2
6.12.3
5.62.6
5.33
3
3.3
3.35/1/2011 36
-
8/7/2019 Dr.laila Professional Responsibilty
37/133
13-4-1432
!
5/1/2011 37
-
8/7/2019 Dr.laila Professional Responsibilty
38/133
:
5/1/2011 38
-
8/7/2019 Dr.laila Professional Responsibilty
39/133
:
5/1/2011 39
-
8/7/2019 Dr.laila Professional Responsibilty
40/133
:
5/1/2011 40
-
8/7/2019 Dr.laila Professional Responsibilty
41/133
:
5/1/2011 41
-
8/7/2019 Dr.laila Professional Responsibilty
42/133
5/1/2011 42
-
8/7/2019 Dr.laila Professional Responsibilty
43/133
:
:
:
:
5/1/2011 43
-
8/7/2019 Dr.laila Professional Responsibilty
44/133
Six types of sanctions
for misconduct Blame Social and professional ostracism and
boycott
Public or private reprimands from professionalsocieties
Exclusion form membership in a professional
society Lawsuit
Suspension - or revocation oflicense to practice 5/1/2011 44
-
8/7/2019 Dr.laila Professional Responsibilty
45/133
-
8/7/2019 Dr.laila Professional Responsibilty
46/133
Whistle-blowing Bowie (1982) defines whistle-blowing as "the act of
an employee informing the public on the immoral or
illegal behavior of an employee or supervisor." Bok (1997) defines whistle blowing as an act in
which one "makes revelations meant to callattention to negligence , abuses , or dangers that threaten the public interest."
465/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
47/133
Whistle-blowing Whistle-blowing situations can arise in cases
of overt wrongdoing (i.e., involvingspecific acts that are either illegal or immoral).
They can also arise in instances of
negligence where one or moreindividuals have failed to act.
475/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
48/133
whistle blowing
A call for public attention, including
and especially that of a higherauthority such as a government
5/1/2011 48
-
8/7/2019 Dr.laila Professional Responsibilty
49/133
8-49
Motives of Whistleblowers People become whistleblowers for different
reasons
Morality of action may depend on motives Good motive
Desire to help the public
Questionable motives Retaliation Avoiding punishment
5/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
50/133
-
8/7/2019 Dr.laila Professional Responsibilty
51/133
-
8/7/2019 Dr.laila Professional Responsibilty
52/133
-
8/7/2019 Dr.laila Professional Responsibilty
53/133
Be prepared to live with the results
5/1/2011 53
-
8/7/2019 Dr.laila Professional Responsibilty
54/133
If you believe that knowledge of unethical practices
would cause a change in the practices: Reality check (make sure you are right)
The goal is to get management to recognize andremedy problem with minimal conflict.
Take problem outside the organization as last resortand act as an individual, not an employee.
Be prepared to live with the results.
Document everything.
Be on your best behavior.
5/1/2011 54
-
8/7/2019 Dr.laila Professional Responsibilty
55/133
8-55
Corporate Response to Whistleblowing
Whistleblowing has many harms
Bad publicity
Disruption of organizations social fabric Makes it hard for people to work as team
5/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
56/133
-
8/7/2019 Dr.laila Professional Responsibilty
57/133
DeGeorges Whistleblowing Richard DeGeorges questions for whistleblowing
1. Is serious harm to the public at stake ?2. Have you told your manager?
3. Have you tried every possible inside channel?
4. Do you have persuasive documented evidence ?
5. Are you sure whistleblowing will work?
Under what conditions must you blow the whistle?
DeGeorge: If all five conditions are met
Others: If conditions 1-3 are met
Still others: Whistleblowing is nevermorally required
5/1/2011 57
-
8/7/2019 Dr.laila Professional Responsibilty
58/133
-
8/7/2019 Dr.laila Professional Responsibilty
59/133
When an Engineer is Permittedto
Blow the Whistle 1) The harm that will be done by the product to
the public is serious and considerable .
2) The engineers (or employees) have made theirconcerns known to their superiors.
3) The engineers (or employees) have received nosatisfaction from their immediate supervisors andthey have exhausted the channels availablewithin the corporation, includinggoing to the board of directors .
595/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
60/133
When an Engineer is Requiredto
Blow the Whistle De George claims that two additional criteria are
needed for requiring an engineer to blow the
whistle. 4) The engineer has documented evidence that
would convince a reasonable, impartial observer that his/her view of the situation is
correct and the company policy wrong.
5) There is strong evidence that making theinformation public will in fact prevent thethreatened serious harm.
605/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
61/133
Evaluating De Georges Criteria James (1991) believes that De George's conditions
are too lenient .
An individual has a moral obligation to blow thewhistle when the first three conditions are met, aswell.
We have a prima facie obligation to "disclose
organizational wrongdoing" that we are unable toprevent, which could also occur when De George'sfirst three conditions are satisfied.
615/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
62/133
James Critique of De Georges Criteria For James, the degree of the obligation depends
on the extent to which we are capable of
foreseeing the severity and the consequences of the wrongdoing. He worries that De George's model leaves us with
no guidance when we are confronted with cases
involving sexual harassment , violations ofprivacy, industrial espionage , and soforth.
Also there is a problem with the word harm.
625/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
63/133
Alperns Criticism of De Georges Criteria Alpern (1991) argues that De George's model lets
engineers off too easily from their whistle-
blowing responsibilities. Alpern believes that engineers must be willing to
make greater sacrifices than othersbecause they are in a greater position to do
certain kinds of social harm.
He believes that these obligations come from afundamental principle of "ordinary morality" viz., we must do no harm.
635/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
64/133
-
8/7/2019 Dr.laila Professional Responsibilty
65/133
An Alternative Strategy De George and Ladd seem correct in
claiming that engineers should not berequired to be moral heroes or saints.
James and Alpern also seem to be correctin noting that engineers, because of the
positions of responsibility they hold, shouldbe expected to make greater sacrifices.
655/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
66/133
A Compromise View McFarland (1991) argues that, collectively,
engineers might be held to a higher standard of
social responsibility than ordinary individuals. However, the onus of responsibility
should not fall directly on engineers as individualengineers.
Rather, it should be shouldered byengineers as members of the engineeringprofession.
665/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
67/133
-
8/7/2019 Dr.laila Professional Responsibilty
68/133
McFarlands Argument The analogy for engineers, McFarland draws from
the Genovese case is that when no other sources of
help are available, engineers should takeresponsibility by banning together. If engineers act as individuals, they might not
always have the ability to help.
If they act collectively , however, they mightbe able to accomplish goals that would otherwisenot be possible.
685/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
69/133
McFarlands Argument McFarland believes that an engineer's work must be
seen in a wider social context , i.e., in
its relation to society. Without that context, an adequate account of moral
responsibility for engineers cant be given.
Unless engineers work collaboratively on ethicalmatters , they will not beable to meet all of their responsibilities.
695/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
70/133
McFarlands Argument McFarland's model encourages engineers
to shift their thinking about responsibility
issues from:
the level of individual responsibility, toresponsibility at the broader level of the
profession itself
705/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
71/133
Professional Responsibility
Involve all the things for which an individual isconsidered to be accountable
Role Responsibility (duties of anindividual) The individual obligations to behave in a proper
manner
Produce computer systems according tointernational standards
5/1/2011 71
-
8/7/2019 Dr.laila Professional Responsibilty
72/133
Professional Responsibility
Casual Responsibility Casual chain (a sequence of factors leading to
a final effect)
Difficult to determine which factor is realcause of an event
Database error may be due user entry error orsoftware developer inadequate verifications ofdata entry procedures
5/1/2011 72
-
8/7/2019 Dr.laila Professional Responsibilty
73/133
Professional Responsibility Blameworthiness
A person may be the cause of an event but notblameworthy for the situation
A doctor who uses a medical computer system
approved by the hospital is not blameworthy if treatment suggested by the system is wrong.
5/1/2011 73
-
8/7/2019 Dr.laila Professional Responsibilty
74/133
Professional Responsibility A person could be held responsible even if he or
she did not intend the outcome.
Robert Morris, who launched the "Internet worm"in 1988, claimed that he did not intend for theInternet to be brought to a standstill .
Morris was held responsible for the outcomecausedby his act of unleashing thecomputer worm.
745/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
75/133
Professional Responsibility Persons can also be held responsible when they intend
for something to happen, even if they ultimately fail to
cause (or bring about) the intended outcome. Suppose a disgruntled student intends to blow up a
computer lab, but is discovered at the last minute andprevented from doing so.
Even though the student failed to carry out his objective,we hold the student morally culpable because ofhis intentions .
755/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
76/133
Liability Liability is the legal obligation of an entity
that extends beyond criminal or contract
law to include the legal obligation to makerestitution, or to compensate, for wrongfulacts .
5/1/2011 76
-
8/7/2019 Dr.laila Professional Responsibilty
77/133
Liability Liability refer to accountability
of an individual for an action or event
Computer engineer who releases a robot tothe market is strictly liable for the productand responsible for any problems caused to
the end user including physical harms or property damage
5/1/2011 77
-
8/7/2019 Dr.laila Professional Responsibilty
78/133
If an employee, acting with or without the
authorization of the organization, performs anillegal or unethical act that causes some degreeof harm, the organization can be heldfinancially liable for that action.
An organization increases its liability if itrefuses to take strong measures, known as duecare, to make sure that every employee knows
what is acceptable or unacceptable behavior,and knows the consequences of illegal orunethical actions.
5/1/2011 78
-
8/7/2019 Dr.laila Professional Responsibilty
79/133
Due diligence requires that anorganization make a valid effort to protect
others and continually maintain this levelof effort.
With the global impact of the Internet,
those who could be potentially injured orwronged by an organizations memberscould be anywhere, in any state or any
country, around the world.5/1/2011 79
-
8/7/2019 Dr.laila Professional Responsibilty
80/133
-
8/7/2019 Dr.laila Professional Responsibilty
81/133
Liability vs. Responsibility
Liability is a legal concept It is sometimes used in the narrow sense of
"strict liability."
To be strictly liable for harm is to be liable tocompensate for it even though one did notnecessarily bring it about through faulty action(e.g., when a someone is injured on a persons
property). The moral notion of "blame" may be left out.
815/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
82/133
Accountability (vs. Liability and Responsibility
Responsibility is only part of what is covered by thenotion ofaccountability . (Nissenbaum)
Accountability means that someone, or some group ofindividuals, or perhaps even an entire organization isanswerable.
there will be someone, or several people to answernot only for malfunctions in life-critical systemsthat cause or risk grave injuries and causeinfrastructure and large monetary losses, but evenfor the malfunctions that cause individual losses oftime, convenience, and contentment.
825/1/2011
R ibilit Li bilit d A t bilit
-
8/7/2019 Dr.laila Professional Responsibilty
83/133
Responsibility, Liability, and Accountability
Moral Responsibility Legal Liability Accountability
Attributes of blame (orpraise) to individuals.
________________________Usua
lly attributed to individualsrather than "collectivities"or groups.
___________________
Notions of guilt and shame
apply, but no legalpunishment orcompensation need result.
Does not attribute blame orfault to those held liable.
___________________
Typically applies in the caseof corporations andproperty owners.
___________________
Compensation can berequired even when
responsibility in a formalsense is not admitted.
Does not necessarilyattribute blame (in a moralsense).
___________________
Can apply to individuals,groups of individuals, andcorporations.
_____________________________
Someone or some group isanswerable (I.e., it goes
beyond mere liability).
835/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
84/133
The Problem of Many Hands in a computing Context
Computer systems are the products of engineeringteams or of corporations, as opposed to the products
of a single programmer working in isolation. So "many hands" are involved in their development.
It is difficult to determine who exactly is accountablewhenever one of these safety-critical systems results
in personal injury or harm to individuals.
845/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
85/133
The Problem of Many Hands Two problems for assigning accountability
(e.g., Therac 25 Case):
(a) we tend to think of responsibility assomething that applies to individuals butnot to groups (or collectivities);
(b) we tend to think of responsibility inexclusionary terms: If X is responsible,then Y is not, and vice versa.
855/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
86/133
Two forms of responsibilities
regarding computer reliability Responsibilities concerning the computer
professionals including engineers, developers ,
designers, manufactures, and vendors(hardware malfunction, software defects orcommunication failure)
Responsibilities concerning individuals whouse the computer system including consumersand end users (malpractice, misuse or negligence)
5/1/2011 86
Computer Professional Responsibilities
-
8/7/2019 Dr.laila Professional Responsibilty
87/133
Computer Professional Responsibilities computer professionals responsibilities towards their
employers and their clients; proper documentation of software by designers and
developers;
propertesting of software by designers and developers;
handover of systems to clients; providing maximum security of software to clients;
computer professionals honouring the proprietary issuesrelating to the algorithms, procedures and data;
computer professionals working towards accurate systemswith the aim of complete data integrity;
5/1/2011 87
Inadequacies in system life cycle
-
8/7/2019 Dr.laila Professional Responsibilty
88/133
Inadequacies in system life cycle
Inadequate system analysis
Inadequate system design
Inadequate system development
Inadequate system testing
Wrong system installation Mismanagement
Lack of service
Malpractice Misuse
Negligence
5/1/2011 88
-
8/7/2019 Dr.laila Professional Responsibilty
89/133
Computer Reliability
Ability of a computer to perform its
required functions for a given periodof time.
5/1/2011 89
ComputerReliability
-
8/7/2019 Dr.laila Professional Responsibilty
90/133
Co pute e ab ty
It involves
Hardware Reliability, ability to perform its mechanicaloperations without errors
Software Reliability, ability to deliver its usableservices according to design when those services aredemanded
Data Reliability, include the following:
Data security; confidential, safe, private
Data privacy; only authorized personsData consistency; correctness of data duringprocessing
Data Integrity; accuracy of data5/1/2011 90
-
8/7/2019 Dr.laila Professional Responsibilty
91/133
Computer Reliability Depends on both computer professional and user
5/1/2011 91
-
8/7/2019 Dr.laila Professional Responsibilty
92/133
Solution to the computer reliability problem
Computer professional should provide faulttolerant computer systems capable of
providing
fail-safe (full functionality of a computer systemdespite the occurrence of a single fault or
fail-soft (reduced functionality of a computersystem despite the occurrence of a single fault)
5/1/2011 92
-
8/7/2019 Dr.laila Professional Responsibilty
93/133
Solution to the computer reliability problem
Well designed liability laws that can enhancecomputer systems consistency and safety.
Responsibility of computer manufactured hasto be underscored by particular standards(rules, regulations, warranties)
5/1/2011 93
-
8/7/2019 Dr.laila Professional Responsibilty
94/133
Failures and Errors in Computer Systems
Most computer applications are so complex itis virtually impossible to produce programswith no errors
The cause of failure is often more than onefactor
Computer professionals must study failures tolearn how to avoid them
Computer professionals must study failures tounderstand the impacts of poor work
5/1/2011 94
-
8/7/2019 Dr.laila Professional Responsibilty
95/133
Example
Inaccurate and misinterpreted data in databases
Large population where people may share
names Automated processing may not be able to
recognize special cases
Overconfidence in the accuracy of data Errors in data entry
Lack of accountability for errors
5/1/2011 95
High-level Causes of Computer-System Failures
-
8/7/2019 Dr.laila Professional Responsibilty
96/133
g p y
Lack of clear, well thought out goals andspecifications Poor management and poor communication
among customers, designers, programmers, etc.
Pressures that encourage unrealistically lowbids, low budget requests, and underestimates oftime requirements
Use of very new technology, with unknown
reliability and problems Refusal to recognize or admit a project is in
trouble
5/1/2011 96
-
8/7/2019 Dr.laila Professional Responsibilty
97/133
Safety-Critical Applications
"fly-by-the-wire" airplanes (many systems arecontrolled by computers and not directly by thepilots) Between 1988-1992 four planes crashed
Air traffic control is extremely complex, andincludes computers on the ground at airports,devices in thousands of airplanes, radar,databases, communications, and so on - all of
which must work in real time, tracking airplanesthat move very fast In spite of problems, computers and other
technologies have made air travel safer
5/1/2011 97
-
8/7/2019 Dr.laila Professional Responsibilty
98/133
Whistle Blowing
Case Studies
5/1/2011 98
-
8/7/2019 Dr.laila Professional Responsibilty
99/133
Cases Where Whistle-blowing
Could Have Saved Lives
995/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
100/133
Case 1: September 11 Colleen Rowley, an FBI employee, came forth to
describe the way in which critical messages hadfailed to be sent up theFederal Bureau's chain of command in the daysimmediately preceding the tragic events of September11, 2001.
Was it appropriate for this individual to blow thewhistle on her supervisor?
Was she also possibly being disloyal to her supervisorand fellow employees in doing so?
1005/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
101/133
Case 2: retirement savings Should individuals in positions of authority in
corporations such as Enron and WorldCom have
blown the corporate whistle about the illegalaccounting practices in thosefirms?
One could argue that failing to blow the whistle in the
Enron case resulted in thousands of individuals losingtheir retirement savings )( ,and in some cases their entire life savings.
1015/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
102/133
Case 3The David LaMacchia Case
5/1/2011 102
-
8/7/2019 Dr.laila Professional Responsibilty
103/133
-
8/7/2019 Dr.laila Professional Responsibilty
104/133
The main thrust of digital technology is tomake it easy to copy and manipulate
information. Thats what computers are for. But it turns out that this benefit doesnt suit the
owners of information. They dont want a free
flow of information.
5/1/2011 104
-
8/7/2019 Dr.laila Professional Responsibilty
105/133
-
8/7/2019 Dr.laila Professional Responsibilty
106/133
Case 5:
-
8/7/2019 Dr.laila Professional Responsibilty
107/133
Case 5:
The Therac-25 Accidents
5/1/2011 107
-
8/7/2019 Dr.laila Professional Responsibilty
108/133
Therac-25: a computer-controlled radiationtherapy machine, build by Atomic Energy ofCanada Ltd (AECL) used in US and Canadian
hospitals & clinics during the 1980's. The Therac-25 was the successor to the Therac-6
and Therac-20 models.
Unlike its predecessors the Therac-25 reliedmore on software control mechanisms
1085/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
109/133
5/1/2011 109
-
8/7/2019 Dr.laila Professional Responsibilty
110/133
Therac-25 delivers two kinds ofelectron beams:low energy and high energy.
A raw high energy beam is dangerous to living
tissue so magnets are used to spread the beamenergy so as to produce a safe therapeuticconcentration.
1105/1/2011
Among the parameters a Therac-25 operatorwas
-
8/7/2019 Dr.laila Professional Responsibilty
111/133
g p pable to set are the beam energy levels & beam
modes. The latter effects the setting of themagnets.
Operators have two ways of setting the system
parameters: data entry procedure
screen based editing
Aproblem arose when the values established viathe data entry procedure are edited during themagnet set-up phase, i.e. screen display did notreflect actual settings.
1115/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
112/133
This problem resulted in high-powered electron
beams striking patients with 100 times (approx)the intended dose of radiation
Several patients showed the symptoms of
radiation poisoning, 3 patients died later fromradiation poisoning
Aside: (Therac-25 excluded the possibility of
software defects since extensive testing hadbeen undertaken!)
1125/1/2011
The Therac 25 a computerized radiation therapy
-
8/7/2019 Dr.laila Professional Responsibilty
113/133
The Therac-25, a computerized radiation therapymachine, massively overdosedpatients at least six
time between June 1985 and January 1987.
Each overdose was several times the normaltherapeutic dose and resulted in the patients
severe injury or even death.
Overdoses primarily occurred because of errorsin the Therac-25s software and because the
manufacturer did not follow proper softwareengineering practices.
5/1/2011 113
-
8/7/2019 Dr.laila Professional Responsibilty
114/133
Poor, unsafe software design.
Overconfidence in the softwares abilities.
Unrealistic risk assessments.
Unacceptable follow-through onaccident reports.
Misconceptions in themanufacturers attitude
5/1/2011 114
-
8/7/2019 Dr.laila Professional Responsibilty
115/133
For safety-critical software design,rigorous testing and failure analyses
are essential and trained softwareengineers, not simply any reasonablyexperienced engineers, should
implement the software design.
5/1/2011 115
-
8/7/2019 Dr.laila Professional Responsibilty
116/133
Massive overdoses of radiation were given; themachine said no dose had been administered at
all
Caused severe and painful injuries and the
death of three patients Important to study to avoid repeating errors
Manufacturer, computer programmer, and
hospitals/clinics all have some responsibility
5/1/2011 116
-
8/7/2019 Dr.laila Professional Responsibilty
117/133
Software and Design problems:
Re-used software from older systems, unawareof bugs in previous software Weaknesses in design ofoperator interface Inadequate test plan Bugs in software
Allowed beam to deploy when table not inproper position
Ignored changes and corrections operatorsmade at console
5/1/2011 117
-
8/7/2019 Dr.laila Professional Responsibilty
118/133
Why So Many Incidents?
Hospitals had never seen such massive overdosesbefore, were unsure of the cause
Manufacturer said the machine could not havecaused the overdoses and no other incidents had
been reported (which was untrue) The manufacturer made changes to the turntable
and claimed they had improved safety after the
second accident. The changes did not correct anyof the causes identified later
5/1/2011 118
-
8/7/2019 Dr.laila Professional Responsibilty
119/133
Observations and Perspective:
-
8/7/2019 Dr.laila Professional Responsibilty
120/133
Observations and Perspective: Minor design and implementation errors usually
occur in complex systems; they are to beexpected
The problems in the Therac-25 case were notminor and suggest irresponsibility
Accidents occurred on other radiation treatmentequipment without computer controls when thetechnicians: Left a patient after treatment started to
attend a party Did not properly measure the radioactive
drugs
5/1/2011 120
Case 6:
-
8/7/2019 Dr.laila Professional Responsibilty
121/133
Case 6:Hughes Aircraft Whistle blowing
5/1/2011 121
-
8/7/2019 Dr.laila Professional Responsibilty
122/133
Between 1985 and 1987, theMicroelectronic Circuits Division of Hughes
Aircraft shipped hybrid microelectronics to
every branch of the U.S. military withoutcompleting various environmental chiptesting processes required by contract.
This is a whistle-blower case where the
allegations against Hughes Aircraftresulted in a criminal case and a civil case.
5/1/2011 122
-
8/7/2019 Dr.laila Professional Responsibilty
123/133
-
8/7/2019 Dr.laila Professional Responsibilty
124/133
No matter how secure the technicalpart of a system is, there is no
security if the personnel cannot betrusted.
5/1/2011 124
Case 6: Hughes Aircraft Factory for military-grade hybrid chips
-
8/7/2019 Dr.laila Professional Responsibilty
125/133
8-125
Factory for military grade hybrid chips
Some defective chips being approved
Ruth Goodearl reported incidents to uppermanagement
Consequences for Goodearl
Harassed Fired
Unemployment
Bankruptcy Divorce
Goodearl and Ruth Aldred sued Hughes AircraftunderFalse Claims Act and won
5/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
126/133
Case 7System Failure of DenverAirport
5/1/2011 126
-
8/7/2019 Dr.laila Professional Responsibilty
127/133
DenverAirport:
Baggage system failed due to real world problems,
problems in other systems and software errors Main causes:
Time allowed for development was insufficient
Denver made significant changes in specifications
after the project began
5/1/2011 127
-
8/7/2019 Dr.laila Professional Responsibilty
128/133
Case 8 Ariane 5 Ariane 5: European Expendable Launch
System - designed to deliver payloads into orbitaround the Earth
Manufactured by the European SpaceAgency
SupersedesAriane 4, and developed overa 10year period at a cost $7 billion
1285/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
129/133
5/1/2011 129
-
8/7/2019 Dr.laila Professional Responsibilty
130/133
In 1996 onthemaidenflight of Ariane 5, just 39seconds into itsmaidenflight Ariane 5 exploded
1305/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
131/133
Ariane 5 was running Ariane 4 software,however, underlying hardware architectureswere different
Ariane 5 guidance system tried to convert a64-bit number (velocity data) into a 16-bitformat - resulting in an overflow error
Ariane 5 interpreted the result ofthe overflow
as evidence that it was out of control andinitiated a self-destruction operation!
1315/1/2011
-
8/7/2019 Dr.laila Professional Responsibilty
132/133
Questions?5/1/2011 132
-
8/7/2019 Dr.laila Professional Responsibilty
133/133
Thank you!