major incident draft template

Major Incident Draft (MID) Template

Methodology

There is a close relationship between problem management and the major incident process.

An incident is any event that is not part of the standard operation of a service and that causes an interruption or a reduction in the quality of that service. Incidents are recorded in a standardized system which is used for documenting and tracking outages and disruptions. A Major Incident is an unplanned or temporary interruption of service with severe negative consequences. Examples are outages involving core infrastructure equipment/services that affect a significant customer base, such as isolation of a company site, which is considered a Major Incident. Any equipment or service outage that does not meet the criteria necessary to qualify as a Major Incident is either a Moderare, Minor or Normal Incident. Major incident reports are escalated to the problem manager for quality assurance.

Incident Pyramid

The scale of incidents follows an Incident Pyramid where the most incidents are normal, escalating up to a singular Major Incident.

Thinking problem management! * Default MID Template– Page: 1

http://thinkingproblemmanagement.blogspot.com/

http://bp0.blogger.com/_AVODjjM-COk/RwU2pK5pPPI/AAAAAAAABJM/FtO4wOrAhEM/s1600-h/davinci.gif

The major incident process consists of following components:

Impact assessment: Classification, outage analysis and risk management. Measurement using the Incident User Metric (IUM). Grading the resources involved in a major incident.

Major incidents can be seen in the context of a quadrant diagram. In this quad major incidents are either failures or misses and result from undesired outcomes. The difference between a miss and a failure is that the former is associated with desired activity while the latter is not. Success is measured as having both desired activity and outcomes. In the context of problem management a lucky break, desired outcome associated with undesired activity, is not optimal and should be investigated.

Iceberg

Incidents are a portion of activity in problem management that forms the tip of an iceberg. The major incident process deals with the visible portion of the iceberg, while in the greater field of

problem management a large number of non-visible issues are lurking.



Notification (issued within 6 working hours of trigger) <X>

Preliminary (issued within 6 working hours of workaround)

<X>

Final (issued within 6 working hours of normal business operations resuming)

<X>

Details

DescriptionService desk / Risk logging <References>

Trigger (who requested the report/notification) <Job title of person>

Service affected <Name in service catalogue>

Data networks <X> AD <X>Messaging <X> Security <X>Payments <X> Operations <X>Voice <X> Service desk <X>Hosting <X> Monitoring <X>Intranet <X> Printing <X>Documents <X> Third party <X>Ecommerce <X> Extranet <X>Backups <X> <X>Storage <X> <X>Identification (please clearly describe the incident and its symptoms – immediate and visual causes)

<Description of the incident or outage and including the symptoms displayed or experienced>

Business impact (please describe clearly the undesired outcome)

<Describe how the business was impacted by stating the undesired outcome>

Conditions (please describe the environment – business or IT – conditions that caused or were present during the incident)



<The business and IT conditions present when the incident or outage occurred>

ResolutionInitial (describe the workaround)

<Initial actions and any possible workaround>

Final (describe the steps taken to resume normal operations)

<Steps undertaken to resume normal operations. Include any root cause analysis>

Proximate cause analysisChange <X> Component Failure <X>Capacity <X> Configuration <X>Carrier <X> Service Provider <X>Environmental <X> Bug <X>Hardware <X> Vendor <X>Process <X> <X>

<X> <X><X> <X><X> <X>

Identification

AssetOwner (who does the asset belong to) <Owner>

Physical component or system (CI name in CMDB)

<CI in CMDB>

Value (in $) <$x>

Resources Headcount affected (application or service)

<Count>

Please supply the names of the following contacts involved in the incident1st line escalation <First name, last name>,<Vendor>

2nd line escalation <First name, last name>,<Vendor>

3rd line escalation) <First name, last name>,<Vendor>

Region including head count (Mark with a X)Country <Country>,<Headcount> Location <Location, site>,

<Headcount>

Business Unit including head count (Mark with a X)Please try and supply an accurate count of the number of users involved.Business unit <Name of business unit>, <Headcount>

Execution

Timelines (date and times) the expanded incident lifecycle

Time when incident started (actual – something has happened to a CI or a risk event has occurred)

<dd/mm/yy> <hh:mm>

Time when incident was detected (incident is detected either by monitoring tools, IT

<dd/mm/yy> <hh:mm>



personnel or, worse case, the user/customer)

Time of diagnosis (underlying cause – we know what happened?)

<dd/mm/yy> <hh:mm>

Time of repair (process to fix failure started or corrective action initiated)

<dd/mm/yy> <hh:mm>

Time of recovery (component recovered – the CI is back in production – business ready to be resumed)

<dd/mm/yy> <hh:mm>

Time of restoration (normal operations resume – the service is back in production)

<dd/mm/yy> <hh:mm>

Time of workaround (Service is back in production with workaround)

<dd/mm/yy> <hh:mm>

Time of escalation (to problem management team)

<dd/mm/yy> <hh:mm>

Time period service was unavailable (SLA measure) <minutes>

Time period service was degraded (SLA measure) <minutes>

Measurement

Function Please select the most appropriate

<Data networks, Messaging, Voice, Payments, Hosting, Intranet, Security, Document management, AD, Storage, Service desk, Backups, Operations, Third party, Printing, Monitoring, Ecommerce>Cause Please select the most appropriate

<Availability, Configuration, Carrier, Service provider, Environmental, Bug, Hardware, Vendor, Process, Capacity, Change>Type (mark with a X) To calculate the IUM please select a single type which best describes the incident

Outage Usage Multiplier

Scrutiny by management

Profit 3 Yes <X> No* <X>

Effect on productivity*

Staff 10 Yes*

<X> No <X>

Impact on company’s image

Share price 5 Yes <X> No* <X>

Direct financial impact

Assets <Actual value (no multiplier)>

Yes <X> No* <X>

Liability or vulnerability

Nominal 1 Yes <X> No* <X>

Limited to internal IT process

Budget 2 Yes <X> No* <X>

Incident User Metric Cost of Downtime Analysis <X>Metric is based on Outage type multiplier * outage time in minutes * percentage of users effected. Calculation is also weighted based on degradation at 60% and non-business hours at 50%. Default outage type is “Effect on productivity.”

Classification

Scope (Mark with a X) Dashboard designation = S

(4) More than 50% of customers affected <X>

(3) More than 25% of customers affected <X>

(2) Less than 25% of customers affected* <X>

(1) Less than 1% of users affected <X>



(0) Single IT customer affected <X>

Credibility (Mark with a X) Dashboard designation = CR

(4) Areas outside the company will be affected negatively <X>

(3) Company affected negatively <X>

(2) Multiple business units affected negatively <X>

(1) Single business units affected negatively <X>

(0) No credibility issue* <X>

Operations (Mark with a X) Dashboard designation = OP

(4) Interferes with core business functions <X>

(3) Interferes with business activities* <X>

(2) Significant interference with completion of work <X>

(1) Some interference with normal completion of work <X>

(0) No work interference <X>

Urgency (Mark with a X) Dashboard designation = U

(4) Underway and could not be stopped <X>

(3) Caused by unscheduled change or maintenance <X>

(2) Incident caused by a change <X>

(1) Incident caused by scheduled maintenance <X>

(0) Completion time not important* <X>

Prioritization (Mark with a X) Dashboard designation = P

Reviewing the scope , credibility, operations and urgency please classify the priority of the incident(4) Critical - An immediate and sustained effort using all available resources until resolved. On-call procedures activated, vendor support invoked.

<X>

(3) High - Technicians respond immediately, assess the situation, and may interrupt other staff working low or medium priority jobs for assistance.

<X>

(2) Medium - Respond using standard procedures and operating within normal supervisory management structures.

<X>

(1) Low - Respond using standard operating procedures as time allows. *

<X>

(0) No prioritization <X>

ROC calculationS CR OP U P TOTAL / 20 %

<X> <X> <X> <X> <X> <X> <X%>

Outage analysis

Service period outage classification (Mark with a X) Dashboard designation = P

(4) Critical - App, server, link (network or voice) unavailable for greater than 4 hours or degraded for greater than 1 day – negative business delivery for more than 1 month

<X>

(3) Major - App, server, link (network or voice) unavailable for greater than 1 hour or degraded for greater than 4 hours - negative business delivery for more than 1 week

<X>

(2) Moderate - App, server, link (network or voice) <X>



unavailable for greater than 30 minutes or degraded for greater than 1 hour - negative business delivery for more than 1 day (1) Minor - App, server, link (network or voice) unavailable greater than 5 minutes or degraded for greater than 30 minutes - negative business delivery for more than 1 hour

<X>

(0) Low* - App, server, link (network or voice) unavailable for less than 5 minutes or degraded for less than 30 minutes - negative business delivery for less than 1 hour

<X>

Service consequence outage classification (Mark with a X) Dashboard designation = C

(4) Critical - Financial loss, which puts a business unit in a critical position - greater than $10m or substantial loss of credibility or litigation or prosecution or fatality or disability.

<X>

(3) Major - Financial loss which severely impacts the profitability of a business unit - greater than $1m or serious loss of credibility or sanction or impairment

<X>

(2) Moderate - Financial loss which impacts the profitability of the business unit, greater than $100k or embarrassment or reported to regulator or hospitalization.

<X>

(1) Minor -Financial loss with a visible impact on profitability but no real effect, greater than $10k or some embarrassment or rule or process breaches or medical treatment

<X>

(0) Low* - Financial loss with no real effect, less than R50k or irritating or no legal or regulatory issue or no medical treatment.

<X>

ROC analysisP C Total / 8 %

<X> <X> <X> <X%>

Risk management

Risk impact (Mark with a X) Dashboard designation = I

Evaluate the data and information that is directly effected by the incident taking into account the involvement of the people, process, products and partners.“At Risk” issues<People, process, products and partners>Confidentiality (Information is accessible only to those authorized)

Secure <X>

Confidential <X>

Restricted* <X>

Public <X>

Integrity (Safeguarding the accuracy and completeness of information)

Very high <X>

High <X>

Moderate* <X>

Low <X>

Availability (Authorised users have access to information when required.)

Mandatory <X>

Very high <X>



High <X>

Moderate* <X>

Low <X>

Rating Taking into account the above please rate the Risk impact

(4)Critical (3)Major (2)Moderate (1)Low (0)None<X> <X> <X> <X> <X>

Risk vulnerability (Rate as either low, moderate, high or major) Dashboard designation = V

Rate the vulnerability in the following categories of the information or data that is affected by the incidentLoss <low, moderate, high, major>Error <low, moderate, high, major>Failure* <low, moderate, high, major>Rating Taking into account the above please rate the Risk vulnerability


Countermeasures Dashboard designation = CM

What measures are in place to mitigate any risks identified with the information or data affected by the incident<Due diligence>

Rating Taking into account the above please rate the Risk Countermeasures


ROC analysisI V CM TOTAL / 12 %

<X> <X> <X> <X> <X%>

Closure

Escalations (Mark with a X)

Further Root Cause analysis required Yes No*

Escalated to Operational Risk Yes No*

Escalated to CTO/CIO (logged as a problem at Service Desk)

Yes No*

Escalated to Infrastructure Manager Yes No*

No Escalation Yes* No



major incident draft template

Technology