copyright statement ©william c. dougherty, 2008. this work is the intellectual property of the...

Copyright StatementCopyright Statement

©William C. Dougherty, 2008. This work is ©William C. Dougherty, 2008. This work is the intellectual property of the author. the intellectual property of the author. Permission is granted for this material to be Permission is granted for this material to be shared for non-commercial, educational shared for non-commercial, educational purposes, provided that this copyright purposes, provided that this copyright statement appears on the reproduced statement appears on the reproduced materials and notice is given that the materials and notice is given that the copying is by permission of the author. To copying is by permission of the author. To disseminate otherwise or to republish disseminate otherwise or to republish requires written permission from the author.requires written permission from the author.

Data Collection after Data Collection after a Tragedya Tragedy

Preparing for litigation after Preparing for litigation after the events of April 16the events of April 16thth, 2007 , 2007

at Virginia Techat Virginia Tech

AgendaAgenda

TimelineTimeline– What happened and What happened and

when, related to when, related to data preservation data preservation and collectionand collection

ProceduresProcedures– What was done, What was done,

how and whyhow and why

StatisticsStatistics– How much, how How much, how

many, and how longmany, and how long

Issues encountered Issues encountered during the processduring the process

Lessons LearnedLessons Learned

But first, a few definitionsBut first, a few definitions Cyberforensics:Cyberforensics: a specialized form of e-discovery in which a specialized form of e-discovery in which

an investigation is carried out on the contents of the hard an investigation is carried out on the contents of the hard drive of a specific computer. drive of a specific computer.

E-discovery:E-discovery: refers to any process in which electronic data refers to any process in which electronic data is sought, located, collected, secured, and ultimately is sought, located, collected, secured, and ultimately searched with the intent of using it as evidence in a civil or searched with the intent of using it as evidence in a civil or criminal legal case. criminal legal case.

ESIESI (Electronically Stored Information):(Electronically Stored Information): As data is As data is requested during the discovery portion of a hearing or court requested during the discovery portion of a hearing or court case, ESI increasingly represents the bulk of what is case, ESI increasingly represents the bulk of what is requested, particularly in civil cases. With the recent requested, particularly in civil cases. With the recent (December 2006) update to the U.S. Federal Rules of Civil (December 2006) update to the U.S. Federal Rules of Civil Procedure (FRCP), ESI received the same legal status as the Procedure (FRCP), ESI received the same legal status as the more traditional “paper” files. more traditional “paper” files.

Metadata: Metadata: Generally defined as “data about data” or Generally defined as “data about data” or information within the electronic version of a document that information within the electronic version of a document that travels with its file, but is usually not visible or otherwise travels with its file, but is usually not visible or otherwise apparent in printed format. apparent in printed format.

Data Preservation and Data Preservation and CollectionCollection

Timeline:Timeline:– April 16April 16thth; meeting with central IT ; meeting with central IT

Support staff—Systems Support (System Support staff—Systems Support (System Administrators), Database Management Administrators), Database Management Systems (DB Admins), Web Hosting (for Systems (DB Admins), Web Hosting (for both data preservation and load both data preservation and load balancing of hosts to handle ever balancing of hosts to handle ever increasing traffic)increasing traffic)

– April 18April 18thth-27-27thth; Direct Interaction with law ; Direct Interaction with law enforcement (FBI, State Bureau of enforcement (FBI, State Bureau of Investigation, local police, and VT PD)Investigation, local police, and VT PD)

– April 23April 23rdrd; First preservation memo ; First preservation memo issued by University Legal Counselissued by University Legal Counsel

Actual verbiage from “Hold Actual verbiage from “Hold Memo”Memo”

In accordance with state and federal law, you are required to In accordance with state and federal law, you are required to preserve any and all documents relating to the events, the preserve any and all documents relating to the events, the suspect, and the victims regardless of whether the suspect, and the victims regardless of whether the documents and information was created before or after documents and information was created before or after event.event.

In an abundance of caution, you should consider the phrase In an abundance of caution, you should consider the phrase “documents and information” to be defined broadly. By way “documents and information” to be defined broadly. By way of illustration, not limitation, it includes all writings of any kind of illustration, not limitation, it includes all writings of any kind (handwritten, printed, electronic) including the originals, (handwritten, printed, electronic) including the originals, drafts, and all non-identical copies, regardless of their origin drafts, and all non-identical copies, regardless of their origin or location including, without limitation, correspondence, or location including, without limitation, correspondence, memoranda, notes, calendars, letters, minutes, contracts, memoranda, notes, calendars, letters, minutes, contracts, reports, studies, statements, receipts, summaries, interoffice reports, studies, statements, receipts, summaries, interoffice and intra-office communications, notes of any conversations and intra-office communications, notes of any conversations or meetings, bulletins, computer printouts, facsimiles, or meetings, bulletins, computer printouts, facsimiles, drawings, sketches, worksheets, spreadsheets, photographs, drawings, sketches, worksheets, spreadsheets, photographs, and electronic recordings of any kind (including tapes, disks, and electronic recordings of any kind (including tapes, disks, hard drives, and thumb drives). Documents and information hard drives, and thumb drives). Documents and information specifically include electronic data (including “metadata”). specifically include electronic data (including “metadata”).

Actual verbiage from “Hold Actual verbiage from “Hold Memo”Memo”

The following specific items referencing or regarding the The following specific items referencing or regarding the event, the suspect and/or the victims must be event, the suspect and/or the victims must be preserved:preserved:– All electronic mail and information about e-mail All electronic mail and information about e-mail

(including message contents, header information and (including message contents, header information and logs of e-mail system usage) sent or received; logs of e-mail system usage) sent or received; databases; activity logs; word processing files and file databases; activity logs; word processing files and file fragments; electronic calendar and scheduling fragments; electronic calendar and scheduling program files or file fragments; spreadsheet files.program files or file fragments; spreadsheet files.

To further minimize the risk of loss and/or destruction of To further minimize the risk of loss and/or destruction of relevant information:relevant information:– All modification or deletion of any on-line electronic All modification or deletion of any on-line electronic

data files should cease; all activity that may result in data files should cease; all activity that may result in the loss of any off-line data, such as the rotation, the loss of any off-line data, such as the rotation, overwriting, or destruction of such media—including overwriting, or destruction of such media—including disk defragmentation or data compression—should disk defragmentation or data compression—should cease.cease.


Timeline (continued)Timeline (continued)– May 9May 9thth; First meeting with consultant; First meeting with consultant– May 10May 10thth; First meeting with ; First meeting with

departmental I.T. representativesdepartmental I.T. representatives– June 7June 7thth; First image taken; First image taken– Bulk of images (99%) completed late Bulk of images (99%) completed late

November 2007; last image taken November 2007; last image taken January 8January 8thth, 2008; but there are “re-dos”, 2008; but there are “re-dos”

– Now beginning process to restore and Now beginning process to restore and search data for e-discoverysearch data for e-discovery


Procedures:Procedures: – Collection procedures could not be fully Collection procedures could not be fully

initiated until criminal investigation was initiated until criminal investigation was concluded. concluded.

– Members of ITSO, colleagues at Cornell, Members of ITSO, colleagues at Cornell, and consultants hired reviewed plans and consultants hired reviewed plans prior to implementation; collection prior to implementation; collection procedures were developed and tested procedures were developed and tested by GIAC certified engineers from VT.by GIAC certified engineers from VT.


Procedures (continued):Procedures (continued):– Meetings and interviews were conducted Meetings and interviews were conducted

to determine who were likely data to determine who were likely data custodians, what type of data was custodians, what type of data was relevant, what types of equipment were relevant, what types of equipment were in use, and where the data was housed. in use, and where the data was housed.


Procedures (continued):Procedures (continued): – E-mail & personal web site content was E-mail & personal web site content was

extracted for storage, and transmission extracted for storage, and transmission toto Law enforcement and families of victimsLaw enforcement and families of victims

– Initial imaging attempt used network for Initial imaging attempt used network for transfer direct to storage with encryption transfer direct to storage with encryption and compression; network speed and compression; network speed presented an issue. (Hoped to avoid presented an issue. (Hoped to avoid second step of copying data from USB second step of copying data from USB drives to the NAS.) drives to the NAS.)


Procedures (continued)Procedures (continued)– Moved to local USB drives using “dd” and Moved to local USB drives using “dd” and

“lzop.”“lzop.”– MD5 checksum performed on way out MD5 checksum performed on way out

and while loading to NAS.and while loading to NAS.– Some data types did not lend themselves Some data types did not lend themselves

to compression (audio and video files). to compression (audio and video files). – Once copied to the NAS, files were Once copied to the NAS, files were

archived to tape backup and media archived to tape backup and media removed to off-site facility. removed to off-site facility.


Procedures (continued):Procedures (continued): – GPG Encryption (2K key size) used to store on GPG Encryption (2K key size) used to store on

NAS. NAS. – Keys passed to University Legal and stored in Keys passed to University Legal and stored in

sealed envelope in records preservation vault.sealed envelope in records preservation vault. A few laptops had encrypted data as well A few laptops had encrypted data as well

(BitLocker); keys for those were obtained (BitLocker); keys for those were obtained and provided to University Legal as well. and provided to University Legal as well.

– Custodians signed and returned documents Custodians signed and returned documents and survey forms.and survey forms.


Statistics:Statistics:– 27 departments interviewed (including 27 departments interviewed (including

entire College of Engineering)entire College of Engineering)– 150 individual custodians (over 200 total 150 individual custodians (over 200 total

images)images)– 7TB stored for imaging7TB stored for imaging– 10,000+ tapes set aside from backup 10,000+ tapes set aside from backup

systems; no rotation of tapes for 14 over systems; no rotation of tapes for 14 over weeks; over 900TB storedweeks; over 900TB stored

– 5TB of log files stored5TB of log files stored


Statistics (continued):Statistics (continued):– Avg size of hard disk imaged= 80GBAvg size of hard disk imaged= 80GB

Largest disk imaged= 500GB; Largest disk imaged= 500GB; smallest= 20GBsmallest= 20GB

– Avg image process duration= 1.75 hrsAvg image process duration= 1.75 hrs Longest= 27.5 hours (250GB iMac);Longest= 27.5 hours (250GB iMac); Shortest= 20 minutes (40GB Dell Shortest= 20 minutes (40GB Dell

D410)D410)– Approx. 1600 person-hours spent on Approx. 1600 person-hours spent on

collection process so far, and counting.collection process so far, and counting.


Issues: Issues: – PrivacyPrivacy– Academic FreedomAcademic Freedom– Research Projects: Pros and Cons Research Projects: Pros and Cons

(Surveys, & funded research)(Surveys, & funded research)– Storage space (online and in vault)Storage space (online and in vault)– Scheduling; length of time required Scheduling; length of time required

(MACs vs Intel products)(MACs vs Intel products)


Issues (continued):Issues (continued): – Equipment in homes.Equipment in homes.– Impact on operations, both staff that Impact on operations, both staff that

performed imaging and those who had performed imaging and those who had to give up access to their computers to give up access to their computers during the process.during the process.

– Assisting departments with resources Assisting departments with resources such as additional tapes, desktops, such as additional tapes, desktops, servers.servers.


Issues (continued):Issues (continued):– Assuming control of resources purchased Assuming control of resources purchased

by/owned by other departments.by/owned by other departments.– ““Chain of evidence”; always 2 people on Chain of evidence”; always 2 people on

site; documenting various elements site; documenting various elements including—Owner of equipment (used including—Owner of equipment (used PID); size of device; unique identifier for PID); size of device; unique identifier for image file (especially when multiple image file (especially when multiple hosts were in use by individual); time to hosts were in use by individual); time to image; cheksum value; type of machine image; cheksum value; type of machine (MAC vs. Intel; no LINUX based (MAC vs. Intel; no LINUX based workstations in group). workstations in group).


Lessons LearnedLessons Learned– Take time now to meet with your Take time now to meet with your

Security Officer and University Legal Security Officer and University Legal Counsel.Counsel.

– Review your existing data retention Review your existing data retention policies; update or modify after policies; update or modify after consultation with ISTO and counsel.consultation with ISTO and counsel.

– Document where your data is/are.Document where your data is/are.– Review existing privacy policies and Review existing privacy policies and

regulations, Is a “Freedom of regulations, Is a “Freedom of Information Act” part of your purview?Information Act” part of your purview?


Lessons Learned (continued)Lessons Learned (continued)– Consider funding “extra” storage and Consider funding “extra” storage and

media for data preservation; potential media for data preservation; potential for huge amounts is likely.for huge amounts is likely.

– Open dialogues with peers; many have Open dialogues with peers; many have been through this already.been through this already.

– Provide training to key staff in IT.Provide training to key staff in IT.– Forewarn community of processes that Forewarn community of processes that

will unfold if and when necessary. Make will unfold if and when necessary. Make sure preservation memos make it to the sure preservation memos make it to the right people.right people.


Lessons Learned (continued):Lessons Learned (continued):– Ensure space is available in secure, off-site Ensure space is available in secure, off-site

location to store media and equipment. location to store media and equipment. Usage of such space at VT grew by 350% Usage of such space at VT grew by 350% over normal. over normal.

– If you haven’t already purchased or If you haven’t already purchased or investigated e-mail archiving products, you investigated e-mail archiving products, you may wish to begin now. may wish to begin now.

– Update or prepare your Standard Update or prepare your Standard Operating Procedures (SOP) document. Operating Procedures (SOP) document. Include references to applicable policies Include references to applicable policies

and information about centrally provided and information about centrally provided services.services.

Contact info:Contact info:

William DoughertyWilliam DoughertyAssistant DirectorAssistant DirectorNI&S-Systems Support Dept.NI&S-Systems Support Dept.Virginia TechVirginia Tech1700 Pratt Drive1700 Pratt DriveBlacksburg, VA 24060Blacksburg, VA 24060(540) 231-9239(540) [email protected]@vt.edu

copyright statement ©william c. dougherty, 2008. this work is the intellectual property of the...

Documents

electronic data

data preservation

data collection

stored information

electronic version

phrase documents

discovery portion

electronic recordings