philip stack

39
CAUBO Annual Meeting Winnipeg, Manitoba June 16, 2008 Concurrent Session Business Continuity and IT Disaster Recovery: Ensuring an Integrated Approach

Upload: nostrad

Post on 14-Jun-2015

301 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Philip Stack

CAUBO Annual Meeting Winnipeg, Manitoba

June 16, 2008 Concurrent Session

Business Continuity and IT Disaster Recovery:

Ensuring an Integrated Approach

Page 2: Philip Stack

Overview of Presenters

Gerry Miller

University of Manitoba

Philip Stack

Associate Vice President Risk Management Services

University of Alberta

Page 3: Philip Stack

Presentation Outline

Part 1

Overview of Integrated Emergency Management

Part 2

IT Disaster Recovery

Page 4: Philip Stack

“An emergency will occur at some point in the history of the university. Never assume it only happens to someone else.”

(1999 Harrell, G. North Carolina Hurricane)___________________________________________

“The Whole Place is Underwater!”

Teaching, research completely halted by rising floodwaters

Another Campus ShootingUniversity mourns. President under fire for lack of preparation

Radiation Leak Stuns Administrators

University authorities didn’t even know the dangers, says prof

Page 5: Philip Stack

•Unexpected•Unscheduled•Unplanned•Unprecedented•Definitely Unpleasant

“It’s not a matter of whether a disaster or emergency scenario will confront a campus but when. I have confronted numerous emergency situations requiring rapid decisions, such as several campus evacuations and extended closures that threatened the institution’s academic program. Dealing with the long-term trauma people faced was a humbling and daunting experience.“Our decision to create comprehensive plans and to continually monitor and update these plans has proved to be one of the best uses of our time and resources.” John Cavanaugh, President University of West Florida

An Emergency at the University/College

5

Page 6: Philip Stack

Why Worry about Emergency Management? 1/2

•Society’s Tolerance - more informed, wiser society not willing to accept uncertainty as in the past.•Institutional Accountability – to the Community, the Board, Government, to Us. New legislation closes gaps for corporate immunity e.g. the directing mind.•Legal Risk - an act or lack of an act could land the University in court and someone potentially with a record. The trend to hold the University responsible for failing to take reasonable steps to prevent a crisis. Or, for failing to be adequately prepared to manage a crisis situation.

Making emergency preparedness a priority may require building crisis management into job descriptions, personnel evaluations and audits.- Poland (1994)

6

Page 7: Philip Stack

Why Worry About Emergency Management? 2/2

•Reputation - Potential damage to the University’s reputation, and, just as important, damage to your own reputation.•Fragile - The systems may be overloaded and the infrastructure easily broken. Large interdependencies can result in disastrous failures e.g. power outage in eastern Canada and USA, failure of the IT system, failure of communications.•Educational institutions - are not exempt from regulations e.g. WH&S/OH&S and the need to provide a safe environment. They may be different in inherent risks and operational risks – but they are still accountable.

“The key to risk management is delivering risk information, in a timely and succinct fashion, while assuring that key decision makers have the time, the tools, and the incentive to act upon it…it follows that the biggest single responsibility of the risk management function is intelligent communication”.Kloman, Felix. (Risk Management Reports, 2001)

7

Page 8: Philip Stack

What are we trying to achieve?

1. Integrated Emergency Management Program

2. Involvement of Faculties, Departments and Planning

3. Business Continuity including Pandemic readiness

4. Enhancing Emergency Preparedness and Management components

Page 9: Philip Stack

Preparedness

ResponseRecovery

Prevention-Mitigation

The Goal

• Increase readiness

•Building capacity and reliability

•University wide approach

•Systems, adaptable and flexible

•Emergency management principles

•Strengthen practices and decision making

•Protect the core businesses

Page 10: Philip Stack

Level 1

Initial Emergency

Response

Faculty/Department

Action

Disaster/ Major

Emergency/

Outage

Level 2 or 3

EOC Activation

CMT Activation

Faculty/Department Unit Action Plan

Assessment

Recovery

Restoration

Resumption

Continuity

Internal and External Stakeholders

Normal

Operations

PreventionPlansPreparednessTraining

When The Wheels Come Off !

IEMP

Page 11: Philip Stack

University of Alberta Crisis Communications

Plan

University of Alberta Emergency Master Plan

Faculty/DepartmentAction Plan

Department/Unit Action Plan

University’s Integrated Emergency Management Program

University’s Integrated Emergency Management Program

Health AuthoritiesEmergency ResponseDepartmentsGovernment Agencies

Layered Planning and Interoperability

Page 12: Philip Stack

12

Administration andMaintenance

Risk, Prevention,Preparedness

Action Plans: Response, Recovery, Res.

Roles, Responsibilities, Checklists

Incident Command System and SOPs

Incident Command System

Appendix

Post Incident Measures

Resources and Forms

Emergency Contacts - In/Ex

Activation and Notification, Operation

U of AIntegrated

Emergency Management

Program

General, Introduction, Policy, Overview

Loss of Criticalvendor

Loss of IT,Communications

Loss of Utilities

Loss of PeopleCapacity

Loss of Equipment/Vehicles

Loss of Facility/Office/Workspace

Business Continuity -Action Plans

Emergency Master Plan &Faculty/Department Action Plans.

Contingency P

lans,

Altern

ative M

easures,

Mitigatio

n and Pro

tection

Crisis Communication Plan and TeamsSupporting: Preparedness, Response, Recovery and Resumption -University wide

Business Contin

uity Planning

Page 13: Philip Stack

Business Continuity to Action PlansPhased Development:

1. Analysis2. Alternate Measures, Solutions and

Strategies 3. Implementation

(Faculty/Department: Emergency Operations Plan/Action Plan)

4. Maintenance

How do you get there?

Page 14: Philip Stack

14

Business Impact Analysis

• Critical business services • Work flows • Maximum acceptable

downtime• Vital records and documents • Priorities for recovery and

resumption • Interdependencies

Planning For A Catastrophe Is Positive Thinking. Not Thinking Is A Disaster!

Caring, Protecting, Responsible

Page 15: Philip Stack

15

Scenario Planning

• Loss of access

• Loss of utility

• Loss of facility

• Loss of people

• Loss of IT and or Telecommunications

• Loss of critical vendor

How to Recover Lost Business Services and Functions

Caring, Protecting, Responsible

Page 16: Philip Stack

16

University and Risks• Risk of fire, flood, tornado: Water, structural damage

• Risk of crime, disorder, terrorism: Theft, bomb threat, work place violence, civil disturbance, hostage, shooter, fraud

• Public Health Emergency: avian pandemic, meningitis

• Risk to utilities: High temperatures, High or low humidity

• Risk to environment: Mold and mildew, pests, asbestos

• Risk of hazards on roads• Human error • IT risks• Financial Risks• Regulatory Risks• Reputation Risk

You are in the Risk Management Business!

Page 17: Philip Stack

17

Potential Consequences

• Health, safety and security• Injuries or loss of life• Animal care• Specimens, data, vital records• Legal• Regulatory• Financial• Infrastructure• Reputation• Loss of students• Loss of Faculty and Staff• Loss of collections• Loss of valuable documents• Morale

Risk Does Not Respect Boundaries!

Page 18: Philip Stack

18

Risk Analysis Tool

Natural Disaster/Man-Made Emergency

Probability Severity Risk Level Priority

Fire Remote Catastrophic Medium 3

Flood Occasional Catastrophic High 2

Major Power Outage Probable Critical High 1

Bomb Threat Improbable Critical Low 4

Risk:What can go wrong?How likely is it?What are the consequences?

Source:NaturalTechnicalMan-Made

Caring, Protecting, Responsible

Page 19: Philip Stack

19Response

Staff

U of A PHR

Strategy Crisis Communications

Plan

U of A IntegratedEmergency Management

Program

U of A EmergencyMaster Plan

Faculties Research AdministrationFacilities andOperations

EssentialServices

Animal care

Labs

Teaching

IT and Records

Campus Security

EH&S

PowerHuman Resources

Water

Planning

Residence Services

Communications

Heat

Staff

Sponsors

Finance

Payroll

Redeployment

Grounds

BuildingsOperations

CommunicationsPerishables

IT

Analysis and Action Plans

Page 20: Philip Stack

Integrated Emergency Management Program - Model

Leadership and CommitmentLeadership and Commitment

Risk Management CultureRisk Management Culture

Functions, Services, Functions, Services, Systems and ProcessesSystems and Processes

Ready, Resilient and Ready, Resilient and Robust UniversityRobust University

Page 21: Philip Stack

Incident Command System – The Building Blocks

P u b licIn fo rm ation

L ia isonO ffice r

S a fe tyO ffice r

O p e ra tio ns P la nn ing L o g is tics F in an ce /A d m in is tra tion

In c ide n tC o m m a n d er Command

Command Staff

General Staff

Doers Thinkers Getters Payers

21

First Responders

Page 22: Philip Stack

Sample Emergency Operations Centre

EOC DirectorUniversity President University Emergency Policy Group:VPs and General Counsel

Finance &AdministrationSection Chief

OperationsSection Chief

Liaison OfficerFaculty and Deans

Liaison Officer: Internal/External

Public Information Officer

Registrar

Public Safety

HR

Facilities Management

Student/ResidentsServices

Financial Services

Risk Mgnt &Insurance

Contracts

EOC Coordinator

Planning andIntelligence

Section Chief

DocumentationUnit leader

SituationStatus

Demobilization

LogisticsSection Chief

Facilities Management

IT &Telecomm

SupplyManagement

Capital Projects

Resource Tracking

Deputy EOC Director

Financial Services

22

Page 23: Philip Stack

•Emergencies prompt a change in management style•From Consultative to Command and Control

“You’ve got to take stock of the damage and how you’ll recover from it. You’ve also got to take stock of your human resources, who’s available and what’s their work capacity. Remember that damage isn’t just physical. Take stock of outside resources. Who can help? The big thing: Take control. As president, as a CIO, you’re in the best position to look out for your own institution. Don’t rely upon FEMA (Emergency Management Alberta, Public Safety Canada ). Don’t rely upon the government. Don’t rely upon the state (province). Take control of the situation.”John Lawson, VP Information Technology and CIO, Tulane

Management Style During an Emergency at a University

23

Page 24: Philip Stack

In Summary

• Leadership commitment

• Integrated approach

• Build a risk culture

• Train and exercise

Page 25: Philip Stack

Here‘s why we need to be ready for emergencies...

Here‘s why we need to be ready for emergencies...

Page 26: Philip Stack

Seventh place...

Page 27: Philip Stack
Page 28: Philip Stack

6th placeSixth place...

Page 29: Philip Stack
Page 30: Philip Stack

5th placeFifth place...

Page 31: Philip Stack
Page 32: Philip Stack

4th placeFourth place...

Page 33: Philip Stack
Page 34: Philip Stack

3rd placeThird place...

Page 35: Philip Stack
Page 36: Philip Stack

2nd placeSecond place...

Page 37: Philip Stack
Page 38: Philip Stack

And the WINNER is...

Page 39: Philip Stack