incident response planning, session 1 introduction of the
TRANSCRIPT
Incident Response Planning, Session 1
Introduction of the cyber incident
response plan template
Greg Bugbee, CISSP, Chief Technology Officer
Agenda & Topics
• Introductions
• Incident response overview
o Difference between an event and an incident
o Preparation:
▪ Asset management
▪ Regulatory requirements
▪ Roles and responsibilities
• External • Internal
▪ Communications
▪ Documentation
▪ Detection and limiting exposure
▪ Training
• Case study of incident response gone bad
Introductions
Presenter – Greg Bugbee, Novus Insight
About Novus Insight:
• Based in East Hartford, Connecticut
• Began as IT department of a nonprofit (CT Center for Advanced Technology)
• Technology services provider & consultant
o Security, compliance, and business process consulting
o IT managed support
o Private cloud hosting
o Application development & data consulting
• Created the Connecticut Municipal Cloud – a hybrid cloud for municipal, education, and nonprofit use
About You
We’d like to get to know you! Please…
1. Introduce yourself and municipality
2. Do you have an incident response plan? Has it ever been tested- simulation or real life?
Why does my municipality need an Incident Response Plan?
• An incident is going to happen. It’s not a matter of if, but when. Your municipality needs a plan to ensure operations can quickly be resumed.
• Without a plan, response will be chaotic and ineffective. Citizens lose trust and confidence in the town when services are disrupted.
• Regulations require notification of incidents within a designated time period – without a plan, those requirements may not be met.
• Engaging cyber insurance companies requires a plan. If you don’t have some type of incident response plan, your insurance response will be severely limited.
• Just as you plan for natural disasters, cyber incidents require the same amount of preparation. Cyber incidents are increasingly becoming life safety events.
Incident vs. EventIt is critical to know the difference
• Cyber incidents must have some level of confirmed and negative impact to the confidentiality, integrity, or availability of the information system, business processes, and/or data.
o Data breach
o Information theft
o Unauthorized changes to information
o Exposure of regulated data
• Cyber events are an observable occurrence
o A phishing email is received
o Someone downloads software
o A virus/malware is quarantined
Events happen every day. Incidents are less frequent and require verification.
All incidents are events, but not all events are incidents.
Incident Response Planning 101Incident response process
1. Assets – What are the critical systems? Where is the sensitive data?
2. Regulations – What are the breach notification regulations? Who needs to be notified?
3. Responsible parties – Who are the data owners, data custodians, and data subjects for each of the
identified systems and/or sensitive data types?
4. Communications plan – Who is responsible for communications and with whom do we communicate?
5. Incident response team – Who will respond when an incident happens?
6. Incident response system – Where will we put evidence?
7. Recovery plan – How will we get things back online?
Incident Response Planning 101Preparation
Asset Map:
• Inventory of all applications/systems with personally identifiable information
o Think beyond your firewall - internal and external systems, 3rd parties, etc., are all “in scope”. If there’s a breach, even if it is with a 3rd party, you need to inform the impacted employees or citizens.
• Inventory and ranking of all applications and systems you need to conduct business. Think, “If this system was unavailable, how would that impact operations?”
• Dependency map - For critical applications, what systems are necessary for service delivery? Internet, single-sign-on, etc.
• Data owner and custodians for each system
It all starts with knowing your assets…
• Create a table based on the regulations and governing bodies and individuals that need to be notified in the event of an incident.
• Examples may include:
o State AG Office and Office of Consumer Affairs and Business Regulation for PII related incidents
o FBI for CJIS related incidents
o Homeland Security – energy and critical infrastructure incidents
o Health and Human Services – HIPAA covered entity incidents
o Individual(s) impacted by breach
Table of Regulations: Who do we need to notify?
Roles & ResponsibilitiesRole Individual or Group Responsibility
Cyber Security Incident Response Manager (CSIRM)
Chief Information Security Officer (CISO)
Provides overall direction to the CSIRT.Establishes communications regarding an incident per the incident response plan.Provides information to senior management during an incident.
Data Owner SME Manager of Department Provides department level coordination of incident response. Provides process related information to technical SMEs.
Physical Security SME Office & Facilities Manager Provides for support for all actions regarding physical security and safety operations.Serves as CSIRT member for responding to cyber security incidents.
Technical SME IT Department/ Managed Services Provider
Provides technical management support functions including evaluation and remediation of risk exposure in response to cyber security incidents.Serves as CSIRT member for responding to cyber security incidents.
Application SME Application provider or technical expert
Provides for support and troubleshooting of specific line of business applications.Serves as CSIRT member for responding to cyber security incidents.
Communications SME Chief Financial Officer or Chief Executive Officer
Provides media communication of cyber security incident involving Confidential or PII data breach.Serves as CSIRT member for responding to incidents when such a breach is confirmed.
Forensic SME Cyber Insurance Firm Provides forensic analysis and can review the environment to ensure the threat has been eradicated.
Legal SME General Counsel Responsible for all legal actions to ensure meeting all legal requirements for incident response. Provides legal support to investigate all incidents to determine legal ramifications and Memorandum of Understanding (MOU) requirements.Serves as CSIRT member for responding to cyber security incidents.
Internally, your incident response plan must include a list of internal and external parties who are part of the incident response process.
These might include:
Roles & Responsibilities – 3rd Parties
• Understanding of responsibilities related to these systems. What is the hosting or software provider responsible for? What am I responsible for as a customer? Example – AWS shared responsibility model
• Most cloud providers will only alert you if their system is breached, not your instancerunning on their system.
• Contact information for 3rd parties and internal support. Need a point of contact to reach out to when things go wrong. Map the contact information to the asset.
https://docs.aws.amazon.com/whitepapers/latest/aws-security-incident-response-guide/shared-responsibility.html
• Based on the previous tables, create a communications plan
• Your tables should include:
o How to contact individuals
o When to contact specific parties
o What to say – pre-prepared emails and talking points will help save time
o Designated party for communication – Only one person should be communicating to external
parties, and it should not be the people directly responsible for investigating the incident.
Communications Are Critical
Quote from an actual incident response plan:
"The central storage location for all artifacts is located here: (To be established when first needed)”
• Don’t wait until an incident has taken place to develop a method of documentation.
• Incidents need to be reported, documented, and tracked throughout the incident lifecycle.
• Repositories for documentation may include:
o An online form and document repository
o A ticketing system
o An email box used exclusively for incidents
• Make sure your incident system is “out of band” from your production system.
• You will need secure storage to house documents and potentially artifacts such as screen shots, photos, and/or log files. Make sure this repository is secured – encrypted, limited access rights, multifactor authentication.
Process & Documentation
Your end users are on the first line and need to be educated as to how to report an incident:
1. Provide users with at least 2 forms of notifying IT or designee that there is an issue
• Primary- ticket or email
• Secondary- phone call if information system is unavailable
2. Instruct users to not tamper with the computer or try to troubleshoot the issue; actions by end users can compromise evidence or make the situation worse.
3. Make sure end users know what to report – Receiving a phishing email is an event; giving up your password is an incident.
4. Make sure end user requests are handled in a timely manner and IT staff are not dismissive- you want users to report incidents. End users must feel supported by IT.
Reporting Procedure
Documentation & Forms
• Keeping an audit trail of incident/event activities is essential.
• Even if there isn’t a reportable incident, keeping track of major events that require investigation is important.
• If there is an impact down the road and the event becomes an incident, showing that there was due care and due diligence will go a long way.
Once your plan and procedures have been created, you must train your team members on incident response.
1. Communicate – Ensure stakeholders are aware of their roles and responsibilities; everyone from end users to legal counsel must be aware of what to do when something goes wrong.
2. Validate – Do the stakeholders know what to do? Use simple exercises like phishing simulations to understand if people know what to do. During a phishing simulation, are end users reporting when they click a link? If not, there’s a breakdown in the process.
3. Simulate – Run a tabletop exercise to simulate a variety of incidents. Start small, i.e. PII breach, then work up to a system-wide outage simulation. Tabletops are designed to walk through the entire process without impacting business.
Training Program
• For each of the assets, a detection strategy must be created.
• Detection methods can include:
o Security operations center
o End user notification
o SIEM – Security Incident and Event Management Platform
o Event logs
o Antivirus/Endpoint Detect and Respond alerts
o Threat hunting – looking for indicators of compromise
o Third party notifications
How will we know something happened?
• One of the most beneficial tools in an incident response portfolio is an Endpoint Detect and Respond (EDR) Tool.
• EDR goes beyond regular antivirus by detecting behavior-based anomalies and acting on them.
• Antivirus typically looks at files and compares them against a known list of malicious files, while EDR looks at suspicious activities and will take actions like:
o Firewalling off the computer so it can’t reach the network
o Removing the computer from the domain
o Disabling the user’s account
• EDR helps prevent events from becoming incidents by reducing exposure and disrupting the attack.
• It is better to generate a help desk ticket for a single impacted user vs. a large-scale incident.
• EDR platforms, when deployed properly, can link together a variety of systems to help paint a more comprehensive picture of an incident.
Limiting Exposure
• Balancing recovery with investigation is one of the biggest challenges of an incident response program.
• The response team must be aware that, although getting things back up and running quickly is a top priority, care must be taken not to destroy evidence. Data custodians must be equipped with tools that allow for recovery and preservation. Talk with your IT providers to make sure they have what they need to be successful. This may include:
o External hard drives for copying images onto*
o Spare hardware for restoration*
o Access to cloud services for temporary recovery*
▪ * Insurance may cover certain aspects of the recovery program
• A good incident response plan requires a sound recovery plan that allows business to quickly resume operations. Incident recovery should not be a scramble.
Recovery Plan
• Backups are the number one target of attackers. Without good backups, the recovery process will not be successful.
• Municipalities must prepare by adopting best practices for backing up systems.
• “Air-Gap” backup systems:
o Use different credentials for backup systems from production systems. Often, backup boxes are joined to the production domain and can easily be attacked.
o Use a firewalled network for backup hardware.
• Follow the 3-2-1 backup rule:
o 3 copies of data, on 2 different media*, 1 copy offsite
o Production data
o Backup copy 1 (onsite, but air-gapped)
o Backup copy 2 (offsite, air-gapped as well)
▪ * different media can mean cloud or different types of backup to disk systems
Hardening Recovery Systems
Incident Response Case Study:
Lack of preparation leads to lack of trust
• 9/17/2021- Ransomware attack hits Westmoreland, Kansas
• 9/28/2021- Westmoreland public relations officer goes public with a story about a $1 million ransomware demand against the city. County posts a message to the website about systems being offline.
Westmoreland, Kansas pop. 763 – Ransomware Attack
• 9/28/2021- Reporter finds out about outage and attack and begins asking around town. Departments gave inconsistent stories about what was happening. Most referred to public relations, but others were transparent and talked to the reporter.
• Town’s people were kept in the dark and rumors started swirling.
• 10/1/2021- Emails start being returned as undeliverable.
• 10/4/2021- Commission meeting, ransomware attack discussed at executive session with commissioners deferring to a press release. Treasurer had communicated to the press the previous week, indicated that taxes could not be collected.
• 10/7/2021- Systems begin coming back online, but public is not informed of estimated time of restoration.
• 10/14/2021- Town discloses that ransom was negotiated and they had paid the ransom for systems to be restored.
• People complained that the county was “hush-hush” about the attack, or that it was trying to “sweep it under the rug.”
• Direct all questions to a central point of contact. Westmoreland originally did this, but the communications protocol quickly fell apart.
• Provide citizens with a clear outline of what is working and what is not. Give them clear instructions on what to do.
• Steady and clear communications will ensure that citizens don’t perceive a cover-up.
• Check in with departments and make sure everyone is consistent with referring media and others to the central communications person.
• Certain operations could not function, resulting in citizens not being able to receive services. Have a contingency plan and implement it.
• Keep elected officials informed but don’t allow them to be the spokespeople. All communications must go through the designated point person.
Lessons Learned from Westmoreland
Key Takeaways• Municipalities need an incident response plan – it’s not a matter of if an incident will happen, but when.
• Cyber incidents and events are two different things. Incidents have some confirmed, negative impact to the information environment. Events are an observable occurrence where information systems are not impacted. All incidents are events, but not all events are incidents.
• Incident response cycle: preparation, detection & analysis, containment, eradication & recovery, post-incident activity.
• Incident response planning starts with knowing your assets. Develop an inventory of all information systems that contain PII.
• Your incident response plan must include a list of internal and external parties (i.e., 3rd parties) who are part of the incident response process.
• Creating a communications plan is critical – be sure to have one designated party for communication.
• You will need secure storage to house documents and potentially artifacts such as screen shots, photos, and/or log files. Make sure this repository is secured, including encryption, limited access rights, & multifactor authentication.
• Create detection strategies for each asset and limit your exposure. Cybersecurity training for staff is key, as is training on incident response for all incident response team members.
• A good incident response plan requires a sound recovery plan that allows business to quickly resume operations. Incident recovery should not be a scramble.
Next workshop will focus on:
• Incident response in detail
• Role of cyber insurance in your incident response program
Next Steps
• NIST Risk Management Framework- https://csrc.nist.gov/projects/risk-management
• MS-ISAC- Cyber resources for local government- https://www.cisecurity.org/ms-isac/
• CISA- Cybersecurity & Infrastructure Security Agency- https://www.cisa.gov/
• Mass.gov Enterprise Security Office- https://www.mass.gov/cybersecurity Good resource for model policies and
standards
• MA Breach Notification Act overview- https://www.mass.gov/info-details/requirements-for-data-breach-notifications
• DOL Cyber guidelines- Cyber Recommendations
• Minimum Cyber Baselines- Minimum Baseline of Cybersecurity for Municipalities | Mass Cyber Center
Resources