university of colorado colorado...
TRANSCRIPT
SECURE ASYMMETRIC ISCSI SYSTEM FOR ONLINE STORAGE
by
SARAH A. SUMMERS
B.Eng., Materials Engineering, Swansea University, 1993Eng.D., Materials Engineering, Swansea University, 1999
A project submitted to the Graduate Faculty of the
University of Colorado at Colorado Springs
in partial fulfillment of the
requirements for the degree of
Master of Science
Department of Computer Science
2008
© Sarah A. Summers 2008
All Rights Reserved
This project for the Master of Science in Computer Science degree by
Sarah A. Summers
has been approved for the
Department of Computer Science by
C. Edward Chow, Chair
Terrance Boult
Joe Zhou
Rick Bauer
Date
Summers, Sarah A. (M.S., Computer Science)
Secure Asymmetric iSCSI System for Online Storage
Project directed by Professor C. Edward Chow
Security and availability of data stored on computer systems has become increasingly important
over recent years. The recently developed Efficient Asymmetric Secure iSCSI provides
availability of data through the use of the iSCSI application level protocol to enable remote
network attached storage; security of data is provided by an enhanced IPsec implementation
which secures data not only in transit but also when stored on the remote storage medium.
The Secure Asymmetric iSCSI System for Online Storage proposed in this project, enhances the
Efficient Asymmetric Secure iSCSI scheme. The enhancements include the simulation of the
transfer of files of arbitrary size and the transfer of files to two targets. This was achieved through
modifications to the sg_dd utility, form the sg3_utils package, which is used to write and read
files to and from target storage media.
A VMware Server virtual machine test bed, for development and testing, was created by cloning
the Efficient Asymmetric Secure iSCSI physical test bed using UltimateP2V. Testing of the
proposed scheme on this test bed revealed that the changes to the sg_dd utility do not adversely
affect overall system performance.
A graphical user interface was developed to simplify the usage of the scheme when setting up
security keys, associations and databases required for IPsec, and writing and reading files.
Additional research was carried out to determine how the Efficient Asymmetric Secure iSCSI
scheme could be modified to use a mounted target storage device and the cp utility for writing
and reading files.
Dedication
To my parents and Sandy, without whose support and, encouragement it would not have been
possible to undertake the Master of Computer Science program.
Acknowledgements
I would like to express my sincere appreciation to Dr. C. Edward Chow for his continual support
and encouragement during this project and also during the many classes that I have taken with
him. His enthusiasm for Computer Science is contagious.
Appreciation and thanks also to my committee members Dr. Terrance Boult, Dr. Joe Zhou and
Rick Bauer.
I would also like to thank the School of Engineering and Applied Science and the Department of
Computer Science, for the use of the computing facilities without which it would not have been
possible to complete this project.
Thanks go to Dr. Katherine Flesia and her daughter Cillian for sharing their home with me during
my last few months in Colorado Springs and making that time an enjoyable experience. Thanks
also to Trish Marquez and Suphannee Sae Chai for their encouragement and assistance during the
early stages of this project.
CONTENTS
Chapter
Dedication.........................................................................................................................................v
Acknowledgements..........................................................................................................................vi
1. Introduction...............................................................................................................1
1.1 Data Storage Explosion.............................................................................................1
1.2 Legal Implications for Secure Storage......................................................................1
1.2.1 HIPAA......................................................................................................................2
1.2.2 SOX...........................................................................................................................2
1.3 Motivation for Secure Online Storage......................................................................2
1.4 Internet Protocol (IP) Storage Options.....................................................................3
1.5 iSCSI.........................................................................................................................4
2. SCSI and iSCSI.........................................................................................................5
2.1 SCSI (Small Computer Systems Interface)...............................................................5
2.2 SCSI Architecture Model..........................................................................................5
2.3 SCSI Protocol............................................................................................................7
2.3.1 Command Descriptor Blocks....................................................................................7
2.3.1.1 Command Descriptor Block Fields...................................................................8
2.3.2 Logical Unit Numbers.............................................................................................10
2.3.3 Phases of an I/O Operation.....................................................................................10
2.4 SCSI Architecture in the Linux Kernel...................................................................12
2.4.1 SCSI Subsystem......................................................................................................13
2.4.1.1 Upper Layer....................................................................................................14
2.4.1.2 Mid Layer.......................................................................................................15
2.4.1.3 Lower Layer....................................................................................................15
2.5 Limitations of SCSI................................................................................................16
2.6 iSCSI (internet SCSI)..............................................................................................16
2.6.1 iSCSI Protocol........................................................................................................16
2.6.1.1 Naming............................................................................................................17
2.6.1.2 Establishing Connections between Initiator and Target(s).............................18
2.6.1.2.1 iSCSI Protocol Data Units...........................................................................18
2.6.1.2.2 Sessions.......................................................................................................19
2.6.1.2.2.1 Discovery Session..................................................................................20
2.6.1.2.2.2 Normal Session......................................................................................24
2.6.2 Data Transfer between Initiator and Target(s)........................................................25
2.6.2.1 Progression of a SCSI Command through the Initiator..................................26
2.6.2.2 Progression of a SCSI Command through the Target.....................................28
2.7 Implementation of iSCSI........................................................................................29
2.7.1.1 iSCSI Host Adapter Implementations.............................................................29
2.7.1.1.1 Software iSCSI............................................................................................29
2.7.1.1.2 Software iSCSI with TCP Offload..............................................................30
2.7.1.1.3 Hardware iSCSI with TCP Offload.............................................................31
2.7.1.2 Software iSCSI Implementations....................................................................31
2.7.1.2.1 Open-iSCSI.................................................................................................31
2.7.1.2.1.1 Open-iscsi-0.4-434.................................................................................32
2.7.1.2.2 iSCSI Enterprise Target (IET).....................................................................32
2.7.1.2.2.1 iscsitarget-0.4.11....................................................................................32
2.7.2 sg3_utils and sg_dd.................................................................................................33
2.7.3 iSCSI and Data Security.........................................................................................34
3. IPsec........................................................................................................................35
3.1 IPsec Security Associations and Security Policies.................................................36
3.1.1 Security Policies and Security Policy Databases....................................................36
3.1.2 Security Associations and Security Association Databases....................................37
3.2 Transport and Tunnel Modes..................................................................................38
3.3 Authentication Header............................................................................................39
3.4 Encapsulating Security Payload..............................................................................41
3.4.1 ESP Header Calculation and Placement.................................................................41
3.4.2 ESP Trailer Calculation and Placement..................................................................42
3.4.3 ESP Authentication Field Calculation and Placement............................................43
3.5 Internet Key Exchange............................................................................................43
4. Efficient Asymmetric Secure iSCSI.......................................................................45
4.1 Review of Efficient Asymmetric Secure iSCSI (EASI).........................................45
4.1.1 EASI Enhancements to IPsec..................................................................................45
4.1.1.1 Sending side code on the initiator...................................................................47
4.1.1.2 Receiving side code on the target...................................................................50
4.1.1.3 Sending side code on the target......................................................................51
4.1.1.4 Receiving side code on the initiator................................................................52
4.2 Motivation for Enhancing EASI.............................................................................53
4.2.1 Goals for the Secure Asymmetric iSCSI System for Online Storage.....................53
5. Testing of EASI Implementation............................................................................55
5.1 Test Bed..................................................................................................................55
5.2 Limitations of using a File as a Target Storage Device..........................................56
5.3 Copying User Files using the sg_dd and cp utilities...............................................57
5.3.1 Writing/Reading Files to a Target Storage File using sg_dd..................................57
5.3.2 Writing/Reading Files to a Target Storage File using cp........................................60
5.3.3 Writing/Reading Files to a Mounted Target Storage File System..........................62
5.3.3.1 Writing/Reading Files to Mounted Target Storage without Encryption.........62
5.3.3.1.1 Writing/Reading with the sg_dd Utility......................................................63
5.3.3.1.2 Writing/Reading with the cp Utility............................................................63
5.3.3.2 Writing/Reading Files to/from Mounted Target Storage using EASI............64
5.4 Summary of the test results and their impact on enhancements.............................64
6. Proposed Enhancements.........................................................................................65
6.1 Writing/Reading Files of Arbitrary Size.................................................................65
6.2 Transferring a File to Two Targets.........................................................................69
6.2.1 Encryption of Payload.............................................................................................69
6.2.2 Native IPsec Keys...................................................................................................70
6.2.3 Steps to Transfer a File to Two Targets..................................................................70
6.3 User Interface..........................................................................................................70
6.3.1 Functionality of User Interface...............................................................................71
6.3.1.1 Initiator User Interface Tasks.........................................................................72
6.3.1.2 Target User Interface Tasks............................................................................72
7. Performance Results and Analysis..........................................................................75
7.1 Testing Regime.......................................................................................................75
7.2 Anticipated Results.................................................................................................76
7.2.1 Base Line Tests using the Original sg_dd Code.....................................................76
7.2.2 Tests using the sg_dd code modified for arbitrary sized files................................77
7.2.3 Testing using sg_dd code modified for two target storage devices........................78
7.3 Actual Results.........................................................................................................78
7.3.1 Analysis of the Results from the Base Line Tests..................................................85
7.3.2 Analysis of the Results from the Tests using the sg_dd Arbitrary Code................87
7.3.2.1 sg_dd Original vs sg_dd arbitrary...................................................................89
7.3.3 Analysis of the Results from Tests using the sg_dd Two Targets Code................92
7.3.3.1 sg_dd Original vs sg_dd Two Targets............................................................94
7.4 Summary of Results................................................................................................96
8. Additional Research................................................................................................97
8.1 New Virtual Machine Test Bed..............................................................................97
8.2 Network Protocol Analyzer Logs...........................................................................98
8.2.1 Discovery Process...................................................................................................98
8.2.2 Login Process..........................................................................................................99
8.2.3 Mounting the Target Storage Device File System on the Initiator.......................101
8.3 Writing to Mounted Target Storage using cp.......................................................103
8.4 Reading from Mounted Target Storage using cp..................................................106
8.5 Summary...............................................................................................................108
9. Disaster Recovery.................................................................................................109
9.1 Disaster Recovery Plan.........................................................................................109
9.1.1 Restart Solutions...................................................................................................110
9.2 Secure Asymmetric iSCSI for Disaster Recovery................................................111
10. Lessons Learnt and Observations.........................................................................113
11. Future Directions..................................................................................................115
12. Conclusions...........................................................................................................117
Bibliography.................................................................................................................................119
Appendices....................................................................................................................................122
A. Wireshark..............................................................................................................122
B. UltimateP2V and VMware Server Virtual Machine Creation..............................124
B.1 Creation of a virtual machine on the VMware Server..........................................125
B.2 Booting the Virtual Machine with the UlitimateP2V CD.....................................135
B.3 Cloning the Hard Disk from the Physical Machine..............................................137
C. Creating a Mounted Target Storage Device..........................................................140
C.1 Creation of an Additional Hard Disk....................................................................140
C.2 Configuring the Added Hard Disk as an iSCSI Target Device............................144
D. Typical run through for Transferring Arbitrary Sized Files.................................148
D.1 Starting the Target and Initiator Software............................................................148
D.2 Write User Data to the Target...............................................................................149
D.3 Reading User Data from the Target......................................................................149
E. Typical run through for Transferring to Two Targets..........................................150
E.1 Starting the Target and Initiator Software............................................................150
E.2 Write User Data to the Target(s)...........................................................................151
E.3 Reading User Data from the Target......................................................................151
F. Python and Tkinter................................................................................................152
G. User Interface Operating Instructions...................................................................153
G.1 Starting the Target Graphical User Interface........................................................153
G.2 Starting the Target Software.................................................................................154
G.3 Starting the Initiator Graphical User Interface......................................................156
G.4 Starting Initiator Software.....................................................................................157
G.5 Discovering Available Targets.............................................................................158
G.6 Logging in to Target Storage................................................................................159
G.7 Writing to Target Storage.....................................................................................160
G.8 Reading from Target Storage................................................................................161
G.9 Disconnecting from Target Storage......................................................................162
G.10 Stopping Initiator Software...................................................................................164
G.11 Stopping the Target Software...............................................................................164
G.12 Additional Functionality of the Initiator and Target GUI’s..................................165
G.12.1 Additional Target Functionality............................................................................165
G.12.2 Additional Initiator Functionality.........................................................................168
H. Open-iscsi-2.0-869.2.............................................................................................172
H.1 Installing the initiator software.............................................................................172
H.2 Running the iscsi initiator.....................................................................................173
H.2.1 Starting the intiator...............................................................................................173
H.2.2 Discovering Available Targets.............................................................................173
H.2.3 Login to Target.....................................................................................................173
H.2.4 To logout of the Target.........................................................................................174
H.2.5 Stopping the Initiator............................................................................................174
I. iscsitarget-0.4.16...................................................................................................175
I.1 Installing the target software.................................................................................175
I.2 Running the target.................................................................................................176
I.2.1 Starting the target..................................................................................................176
I.2.2 Stopping the Target...............................................................................................176
TABLES
Table 2-1: sg_dd Options.........................................................................................................34
Table 3-1: Descriptions of AH Header Fields..........................................................................40
Table 7-1: Testing Regime.......................................................................................................76
Table 7-2: Sizes of various files used during testing................................................................78
Table 7-3: Combined Write/Read Times for 1K and 750 byte files........................................79
Table 7-4: Combined Write/Read Times for 10K and 9966 byte files....................................80
Table 7-5: Combined Write/Read Times for 100K and 102126 byte files..............................81
Table 7-6: Combined Write/Read Times for 1MB and 1048302 byte files.............................82
Table 7-7: Combined Write/Read Times for 10MB and 10485486 byte files.........................83
Table 7-8: Combined Write/Read Times for 100MB and 104857326 byte files.....................84
Table 8-1: Packets for copying file to mounted storage disk.................................................105
FIGURES
Figure 2-1: SCSI Standards Architecture Model [2-9]...................................................................6
Figure 2-2: Example of a Basic SCSI Architecture.....................................................................7
Figure 2-3: Typical CDB for 10 byte Commands [2-11].................................................................8
Figure 2-4: Minimum SCSI I/O Operation................................................................................11
Figure 2-5: SCSI Data I/O Operation........................................................................................12
Figure 2-6: SCSI Subsystem in the Linux Kernel [2-13]...............................................................13
Figure 2-7: Three Layered Architecture of the SCSI Subsystem [2-14].......................................14
Figure 2-8: General Structure of an iSCSI PDU........................................................................19
Figure 2-9: Basic Header Segment Layout................................................................................19
Figure 2-10: Wireshark log for the Login Command for the Discovery Session................21
Figure 2-11: Login Response for Discovery Session..........................................................22
Figure 2-12: Request to SendTargets in Full Feature Phase of Discovery Session.............23
Figure 2-13: Response to SendTargets request...................................................................24
Figure 2-14: Login Command for Normal Session.............................................................25
Figure 2-15: iSCSI Protocol Layer Model [4 -2]....................................................................26
Figure 2-16: Data Encapsulation into Network Packets......................................................28
Figure 2-17: Options for iSCSI Adapter Implementation [2-22].............................................30
Figure 3-1: Example of Security Policy Database Entries........................................................37
Figure 3-2: Example of Security Association Database Entries................................................38
Figure 3-3: Packets for IPSec Tunnel and Transport Mode [3-5].................................................39
Figure 3-4: Format of AH Header.............................................................................................40
Figure 3-5: Encapsulating Security Payload Header Format.....................................................42
Figure 4-1: Efficient Asymmetric Secure iSCSI Scheme [1-11]...................................................46
Figure 4-2: Packet modification under proposed scheme [1-11]...................................................49
Figure 5-1: Schematic of Test Bed............................................................................................56
Figure 5-2: Error Message displayed when writing a User File from Initiator to Target without
specifying a ‘count’ value.......................................................................................59
Figure 5-3: Error Message displayed when writing a User File from Target to Initiator without
specifying a ‘count’ value.......................................................................................59
Figure 5-4: Output of file read back to the Initiator using the sg_dd utility..............................62
Figure 6-1: Result of using stat utility on target storage file.....................................................68
Figure 7-1: sg_dd Original No Encryption................................................................................86
Figure 7-2: sg_dd Original IPsec...............................................................................................86
Figure 7-3: sg_dd Original EASI...............................................................................................87
Figure 7-4: sg_dd Arbitrary No Encryption..............................................................................88
Figure 7-5: sg_dd Arbitrary IPsec.............................................................................................88
Figure 7-6: sg_dd Arbitrary EASI.............................................................................................89
Figure 7-7: Comparison of time values for the original and arbitrary sg_dd code without
encryption...............................................................................................................90
Figure 7-8: Comparison of time values for the original and arbitrary sg_dd code with IPsec. .91
Figure 7-9: Comparison of time values for the original and arbitrary sg_dd code with EASI..91
Figure 7-10: sg_dd Two Targets No Encryption.................................................................92
Figure 7-11: sg_dd Two Targets IPsec................................................................................93
Figure 7-12: sg_dd Two Targets EASI................................................................................93
Figure 7-13: Comparison of times values for the original and two target sg_dd codes with
no encryption..................................................................................................95
Figure 7-14: Comparison of times values for the original and two target sg_dd code with
IPsec................................................................................................................95
Figure 7-15: Comparison of times values for the original and two target sg_dd code with
EASI................................................................................................................96
Figure 8-1: Wireshark Log obtained during the Discovery Session..........................................99
Figure 8-2: Wireshark Log obtained during the Login Session...............................................100
Figure 8-3: Wireshark Log obtained during Mounting the Target Storage Device File System
on the Initiator.......................................................................................................103
Figure 8-4: Writing to mounted target storage using cp..........................................................104
Figure 8-5: Data field in SCSI payload contained within a Data-out PDU.............................105
Figure 8-6: Contents of TCP packet following the first iSCSI/SCSI ‘Read’..........................107
Figure 8-7: Reading from mounted target storage using cp....................................................108
Figure B-1: VMWare Server Console......................................................................................125
Figure B-2: Typical or Custom Virtual Machine Configuration..............................................126
Figure B-3: Selecting Guest Operating System........................................................................126
Figure B-4: Naming of the Virtual Machine............................................................................127
Figure B-5: Setting the Access Rights......................................................................................128
Figure B-6: Startup/Shutdown Options....................................................................................128
Figure B-7: Processor Configuration........................................................................................129
Figure B-8: Virtual Machine Memory.....................................................................................129
Figure B-9: Network Connection.............................................................................................130
Figure B-10: I/O Adapter Types........................................................................................131
Figure B-12: Virtual Disk...................................................................................................131
Figure B-13: Disk Type Selection......................................................................................132
Figure B-14: Disk Capacity................................................................................................132
Figure B-15: Disk File........................................................................................................133
Figure B-16: Virtual Machine Tab.....................................................................................134
Figure B-17: Editing the machine settings to boot from the CD........................................134
Figure B-18: Window for Network support.......................................................................135
Figure B-19: Network Configuration Profile.....................................................................136
Figure B-20: IP Address Configuration.............................................................................137
Figure B-21: Selecting Peer to Peer transfer via TCP/IP in Slave Mode...........................138
Figure B-22: GRUB Menu of Newly Cloned Virtual Machine.........................................139
Figure C-1: Graphical user interface for editing the virtual machine settings.........................141
Figure C-2: Hardware Wizard display.....................................................................................141
Figure C-3: Hardware addition selector...................................................................................142
Figure C-4: Select a Disk.........................................................................................................142
Figure C-5: Select a Disk Type................................................................................................143
Figure C-6: Specify disk capacity and allocation.....................................................................143
Figure C-7: Disk file name.......................................................................................................144
Figure G-1: Initial Window of Target Processing GUI............................................................154
Figure G-2: Start/Stop Target Tab of Target Processing GUI..................................................155
Figure G-3: Message indicating that the Target Software has been Started.............................155
Figure G-4: Initial Window of Initiator Processing GUI..........................................................156
Figure G-5: Start Initiator Tab of Initiator Processing GUI.....................................................157
Figure G-6: Message indicating that the Initiator Software has been Started..........................158
Figure G-7: Target Storage Discovery.....................................................................................159
Figure G-8: Login to Target Storage........................................................................................160
Figure G-9: Writing a File to Target Storage...........................................................................161
Figure G-10: Reading a File from Target Storage..............................................................163
Figure G-11: Logging Out from Target Storage................................................................163
Figure G-12: Stopping the Initiator Software.....................................................................164
Figure G-13: Stopping the Target Software.......................................................................165
Figure G-14: Checking the Status of the Target Software.................................................166
Figure G-15: Creation of a New Target File......................................................................167
Figure G-16: Configuration of the ietd.conf File...............................................................168
Figure G-17: Checking the Status of the Initiator Software...............................................169
Figure G-18: Generating Keys for use with IPsec..............................................................170
Figure G-19: Generating SAD and SPD Entries................................................................171
1
Chapter 1
1. Introduction
The following are the main objectives of this project.
Enhance the existing Efficient Asymmetric Secure iSCSI scheme to enable the transfer of
files of arbitrary size.
Improve the existing scheme to enable the transfer of files to two targets.
Develop and implement a user interface to simplify usage of the scheme.
1.1 Data Storage Explosion
The last few years has seen an explosion in data growth and with it the requirement for increased
data storage capabilities. The driving force for this rapid data growth has been the increased usage
of database applications and email for online business-to-business transactions, business-to-
consumer transactions, and the Radio Frequency Identification Systems (RFID), such as those
used by stores for tracking goods inventory [1-1].
1.2 Legal Implications for Secure Storage
In addition to the need for increased storage requirements, businesses are now faced with the need
to comply with government regulations related to data storage. The Health Insurance Portability
and Accountability Act (HIPAA) [1-2] of 1996, and the Sarbanes-Oxley Act (SOX) [1-3] of 2002
both have implications with respect to security, privacy, and accountability of stored data.
2
1.2.1 HIPAA
Title II of HIPAA, the Administrative Simplification (AS) provisions have implications for
Health Care providers in terms of the security and privacy of health data. In addition, the act has a
number of implications in terms of data storage. For example, data must be stored in a manner
that it can be accessed and disclosed within a period of thirty days. Consequently, data must be
backed up and there must be a Disaster Recovery Plan in place within the organization that
includes the recovery of data in the event of a disaster. If in a worst case scenario the disaster
resulted in total destruction, inclusion of a remote storage solution in the plan would enable the
data to be recovered. In addition, there should be physical controls in place to protect against
inappropriate access of the data. Also, the Information Systems storing patient information must
be protected from intrusion and unauthorized modification.
1.2.2 SOX
SOX was produced in response to a number of corporate and accounting scandals. The act is wide
ranging, however, the area of interest for Information Technology relates to the security,
accuracy, and reliability of the systems that manage and report the financial data. SOX, like
HIPAA, requires that data be readily available for disclosure and should be protected from
inappropriate access, intrusion, and unauthorized modifications.
1.3 Motivation for Secure Online Storage
The motivation for secure online storage is not limited to the legal implications associated with
HIPAA and SOX. Computing services are vital to an ever increasing number of businesses and
organizations. In many cases, loss of computer services will result in the business or organization
not being able to carry out their day to day activities. Hiles [1-4] has stated that with loss of
3
computer services, “some financial applications can ‘go critical’ within hours if not minutes” and
that loss of real time control systems can result in “immediate injury or loss of life”.
From the preceding discussion, it should be apparent that secure storage of data is essential, not
only from the legal standpoint, but also in terms of ethical and good business practice. Provisions
must be in place to recover data in the event of a data destruction catastrophe at the primary
storage location. One possible solution to address these needs is online data storage.
1.4 Internet Protocol (IP) Storage Options
Recent developments in data storage have focused on IP storage methods. These implementations
use Transmission Control Protocol/Internet Protocol (TCP/IP) as storage interconnect to transfer
block level data. The use of TCP/IP for transporting user data for storage has led to the
development of a number of protocols. These include iSCSI (internet Small Computer System
Interface) and the Fibre Channel [1-5, 1-6, 1-7, 1-8, 1-9] protocols FCIP (Fibre Channel over IP), iFCP
(internet Fibre Channel Protocol), all of which make use of the established IP.
IP has a number of advantages over other mechanisms. These include [1-10]:
reduced costs in terms of training and staffing through the use of familiar network
technology and management
increased reliability through the use of a proven transport infrastructure
provisions for remote data replication and disaster recovery as a result of long distance
scalability
lower total cost of ownership resulting from use of the Ethernet for storage
4
1.5 iSCSI
Of the three online storage options mentioned above, iSCSI has advantages which make it the
most suitable option for many enterprises. It uses TCP/IP to transport data and does not require
special hardware for its operation. Rather, SCSI commands are encapsulated in TCP/IP packets.
In order to carry out the work proposed for this project, it is necessary to gain an in-depth
understanding of the protocols and technologies which may be involved. These include SCSI,
iSCSI, Open-iscsi, iscsitarget and Internet Protocol Security (IPsec). They will be discussed in the
following chapters. In addition, it is also necessary to gain a detailed understanding of the
existing Efficient Asymmetric Secure iSCSI [1-11] (EASI) scheme. Consequently, a review of this
scheme is presented along with testing to identify factors which may affect the proposed
enhancements.
5
Chapter 2
2. SCSI and iSCSI
2.1 SCSI (Small Computer Systems Interface)
The SCSI protocol, which is an application layer storage protocol, was originally developed as a
technology for providing a standard device interface bus enabling host computer systems to
perform block data input/output (I/O) operations to a wide variety of peripheral devices [1-8, 2-2, 2-3, 2-
4, 2-5, 2-6, 2-7] .
The primary motivation for SCSI was to provide a way to logically address blocks and eliminate
the need to physically address data blocks in terms of cylinder, head, and sector. The advantage
of logical addressing is that it frees the host from having to know the exact physical organization
of a drive [2-1].
Since its initial conception, the SCSI protocol has undergone a number of developments from
SCSI-1 [2-7] through SCSI-2 [2-8] with the current protocol being SCSI-3. During this development
process, the protocol has expanded to be a collection of related standards organized into three
general categories. The categories are: commands, protocols, and interconnects. They are defined
by the SCSI-3 Architecture Model (SAM), Figure 2-1.
2.2 SCSI Architecture Model
The SCSI architecture is based upon a client/server model, with the host computer being the
client and the server being a resource such as a disk device, printer, scanner etc. In terms of
storage, the client is also known as the initiator since it is from the client that commands to write
CommandSets
TransportProtocols
PhysicalInterfaces
6
or read to/from the storage device are issued. The server or storage device is also known as the
target, it responds to the commands issued by the client. A basic SCSI architecture is shown in
Figure 2-2.
Figure 2-1: SCSI Standards Architecture Model [2-9]
7
Figure 2-2: Example of a Basic SCSI Architecture
2.3 SCSI Protocol
In order for successful transfer of data between the initiator and target, certain information about
the SCSI device must be known. Examples of such information are how the device may be
addressed, commanded to perform operations, and how to read or write data to/from the initiator.
This information is defined in the SCSI protocol.
2.3.1 Command Descriptor Blocks
The SCSI protocol uses a format known as a Protocol Data Unit (PDU) to allow sending and
receiving nodes to agree upon what is being communicated. The PDU for SCSI is known as a
Command Descriptor Block (CDB). If the initiator wishes to write to or read from a target, it
creates a CDB. The CDB contains information as to the type of command to be performed (write
or read) and information about where the data to be written or read can be located.
8
Not all SCSI commands are the same length and consequently the CDB’s do not have a
prescribed consistent length [2-2]. The length of the CDB can be between 6 and 16 bytes [2-1]. An
example of a 10 byte CDB is shown in Figure 2-3.
2.3.1.1 Command Descriptor Block Fields
As can be seen from Figure 2-3, the CDB is comprised of a number of fields, some are described
in detail below.
Operation Code
The operation code (opcode) identifies the operation being requested by the CDB and also how
long the CDB will be. For the purposes of this project, the main opcodes of interest are those for
read and write operations, since it is these that are used for transfer of user data. The read and
write opcodes for 10 byte CDBs are 28h and 2AH respectively.
BitByte
7 6 5 4 3 2 1 0
0 Operation Code1 Reserved (formerly LUN) Service Action 2 (MSB)
Logical Block Address 345 (LSB)6 Reserved7 (MSB) Transfer Length (if required)
Parameter List Length Allocation Length 8 (LSB)
9 ControlFigure 2-3: Typical CDB for 10 byte Commands [2-11]
9
Logical Block Address
Logical block address is used to identify the location of the data on the physical medium. The
logical block address on logical units or within a partition on SCSI device volumes begin with
block zero and are contiguous up to the last logical block on the unit or partition. Blocks, which
are measured in bytes, are the smallest unit of measurement on a device. A 10 byte CDB contains
a 32 bit logical block address field.
Transfer Length
The transfer length specified in the CDB indicates to the target the amount of data to be
transferred, which is usually the number of blocks. However, some commands transfer no data at
all. For this reason this field is marked ‘if required’ in Figure 2-3.
Parameter List Length
This field identifies the number of bytes sent from the Data-out buffer. It is typically used for
parameters that are sent to a device server, for example mode parameters, diagnostic parameter,
log parameters, etc. If the parameter length is zero no data is transferred.
Allocation Length
The allocation length field specifies the maximum number of bytes that an application client
wishes to transfer. If the value is zero, then no data will be transferred. This field is used to limit
the maximum amount of data returned to an application client. The data that may be returned is
sense data, mode data, log data, diagnostic data, etc.
Control
The control byte field is used for special operations such as command linking.
10
2.3.2 Logical Unit Numbers
The SCSI protocol defines how to address the various units to which the CDB is to be delivered.
Each SCSI device (target) can be subdivided into one or more logical units (LUs). A logical unit
is a virtual controller that handles SCSI communications on behalf of storage devices in the
target. Each logical unit has an address associated with it which is referred to as the logical unit
number. Each target must have at least one LUN. If only one LUN is present, it is designated as
LUN0 [2-2].
2.3.3 Phases of an I/O Operation
There are three main phases of an I/O operation; ‘Command’, ‘Data’, and ‘Status’. In the first
phase, ‘Command’, the initiator sends the required command and parameters to the target in a
CDB. In the second phase, ‘Data’, data is transferred in accordance with the command issued in
the CDB. The final phase, ‘Status’, provides confirmation that the command executed is received.
As a bare minimum, a SCSI operation consists of the initiator issuing a SCSI command with the
target returning a completion status, but no data is transferred. This type of I/O operation is
shown in Figure 2-4, and occurs when commands such as ‘Test Unit Ready’, ‘Start/Stop Unit’,
or ‘Rewind’ are issued.
SCSI Command
SCSI Status
Initiator Target
Interconnecting subsystem
11
Figure 2-4: Minimum SCSI I/O Operation
An example of a SCSI I/O operation in which data is transferred between the initiator and target
is shown in Figure 2-5. In this case, if data is being transferred (read) from the target to the
initiator, a Data-in PDU is utilized. While in the case of data transferred (written) from the
initiator to the target, a Data-out PDU is utilized. It should be noted that data can be transmitted
all at once or may take a number of data phases to complete the transfer. The types of commands
that involve data transfer are ‘read’, ‘write’, and ‘inquiry’.
SCSI Command
SCSI Status
Initiator Target
Interconnecting subsystem
‘read’ or ‘write’ Data
12
Figure 2-5: SCSI Data I/O Operation
2.4 SCSI Architecture in the Linux Kernel
Figure 2-6 [2-13] shows the location of the SCSI subsystem in the Linux kernel. As can be seen, the
kernel space is layered, the top layer being the ‘System Call Interface’. It is this top layer that
controls the routing of calls from the user space to the appropriate destination in the kernel. The
next layer is the ‘Virtual File System’. It is this layer that routes the requests to the correct file
system within the ‘File System’ layer below. The majority of the file systems, communicate via a
buffer cache which lies between the file system and block device drivers layers. As can be seen in
Figure 2-6, the SCSI subsystem is one such block device driver.
13
Figure 2-6: SCSI Subsystem in the Linux Kernel [2-13]
2.4.1 SCSI Subsystem
The SCSI subsystem is comprised of a three layered architecture as can be seen in Figure 2-7 [2-
14]. The ‘upper layer’ is the main entry point to the SCSI subsystem. It is comprised of drivers for
both block and character devices. The ‘mid layer’ is the unifying layer; it provides common
services for the upper and lower layers. The ‘lower layer’ contains the “actual drivers for the
physical interfaces that are applicable to SCSI” [2-13].
SDdisks
block device{sd_mod.o}
SRcdroms/dvdsblock device{st_mod.o}
STtapes
char device{st.o}
SGpass throughchar device
{sg.o}UpperLayer
MidLayerl
LowerLayer
SCSIUnifying Layer{scsi_mod.o}
scsi.[hc], hosts.[hc], constants.c
Host BusAdapterdrivers
(eg aic7xxx)
Pseudo drivers for non SCSI
buses(eg ide-scsi)
14
Figure 2-7: Three Layered Architecture of the SCSI Subsystem [2-14]
2.4.1.1 Upper Layer
As already stated, the upper layer is the main entry point into the SCSI subsystem. It accepts
requests from outside the SCSI subsystem and converts them into actual SCSI requests. These
requests are then passed to the mid layer. When processing of the request has been completed, a
status call is sent from the mid layer to the upper layer. The upper layer then passes the status to
the external layer from which the request originated [2-15].
As can be seen in Figure 2-7, there are four drivers in the upper layer: disk, tape, CDROM, and a
generic driver. For the purposes of the current project there are only two drivers of interest, the
disk and the generic drivers. As such, the following discussion is limited to these.
15
Disk Driver (sd)
There are two types of SCSI device that are accessible via the disk driver (sd); direct access
devices which are generally magnetic disks and have a SCSI peripheral code of 0, and optical
memory devices which have a SCSI peripheral code of 7. The SCSI disk driver is implemented in
./linux/drivers/scsi/sd.c. The function sd_init_command in this code processes the requests
entering the upper layer converting it into a SCSI read or write command.
Generic Driver (sg)
In comparison to the disk driver, the generic driver (sg) is much more versatile since all types of
devices are accessible via this driver [2-14]. An advantage of the sg driver is that it allows user
applications to send SCSI commands to devices, this is achieved by using commands in the
sg3_utils package.
2.4.1.2 Mid Layer
The mid layer is unifying layer, it defines not only the internal interfaces but also provides
common services to the upper and lower layer drivers. It provides queuing of SCSI commands
between the upper and lower levels. Additionally, it provides error and timeout handling.
2.4.1.3 Lower Layer
The last of the three layers of the SCSI subsystem is the lower layer. It is comprised of a
collection of SCSI low level drivers which are the specific drivers that interface with the physical
devices. Of the many low level drivers, the one of interest in this project is the iSCSI driver.
The iSCSI driver acts as an iSCSI protocol initiator to transport SCSI requests and responses over
an IP network between the client and target storage device. It combines with the TCP/IP stack of
16
the client, the network drivers and Network Controller Interfaces (NICs), thus providing the same
functions as a SCSI adapter driver with an HBA.
2.5 Limitations of SCSI
Although SCSI has been successfully used for many years, it has limited capabilities in terms of
the realization of storage networks due the SCSI bus. The length of the bus limits the distance
over which SCSI may operate (maximum of around 25 meters). In addition, the bus also has
limitations on the number of devices that may be connected to it. Despite these limitations, the
SCSI protocol is still of importance since it can be used with other protocols simply by replacing
the SCSI bus with a different interconnection type, for example fibre channel, IP networks etc.
2.6 iSCSI (internet SCSI)
The limitations of the SCSI bus, identified above, and the increased desire for IP storage led to
the development of a number of new protocols, one of which was iSCSI. iSCSI was developed as
an end-to-end protocol to enable transportation of storage I/O block data over IP networks [2-16, 2-17]
thus dispensing with the physical bus implementation as the transport mechanism. iSCSI works
by mapping SCSI functionality to the TCP/IP protocol. By utilizing; TCP flow control,
congestion control, segmentation mechanisms, IP addressing, and discovery mechanisms, iSCSI
facilitates remote backup, storage, and data mirroring [2-16, 2-18].
2.6.1 iSCSI Protocol
The iSCSI protocol standard [2-20] defines amongst other things, the way SCSI commands can be
carried over the TCP/IP protocol. It defines a naming scheme to provide both initiators and
targets with unique names and also how to establish connections between them. In addition, it
provides the methods by which initiators and targets can authenticate themselves to each other.
17
Hufferd [2-19] provides detailed explanations of all aspects of iSCSI. For completeness, brief
overviews of the main points of interest to this project are provided here.
2.6.1.1 Naming
Each initiator and target node is required to have a single name. This name is used as a part of all
sessions established between them. There are two possible formats for these node names; the
iSCSI qualified name (iqn) format and the enterprise unique identifier (eui) format.
The iSCSI qualified name is presented in a human readable form, an example is shown below:
iqn.1998-03.com.xyc.ajax.wonder.jump
It is comprised of four components. From left to right, in the example, they are the type
designator, iqn, a date (1998-03), a DNS (Domain Name Server) name assigned by an internet
naming authority, and finally additional unique identifiers. The DNS name and unique identifiers
are reversed in the iSCSI qualified name. In the example given above, the original DNS name
would have been ajax.xyc.com and the additional unique identifier would have been
jump.wonder.
Unlike an iqn name, an eui name is written in hexadecimal format. An example of a eui name
could be:
eui.acde48234567abcd
The eui name is constructed from the identity formed using the IEEE EUI format, EUI-64. Every
EUI-64 identity is unique. It is formed by appending the combination of a company ID value and
a value chosen by the company after eui. The combination of these values result in 16
hexadecimal digits (64 bits). The most significant 24 bits are the company ID. This value is
18
assigned to the company by the IEEE Registration Authority. The least significant 40 bits are
chosen by the company. In the example given above, acde48 is the IEEE assigned company ID
and 234567abcd is the extension chosen by the company.
2.6.1.2 Establishing Connections between Initiator and Target(s)
Data is transferred between an initiator and target(s) via an iSCSI session. An iSCSI session is a
physical or logical link which carries TCP/IP protocols and iSCSI PDUs, between an initiator and
target. The PDUs in turn carry SCSI commands and data in the form of SCSI CDBs. There are
two types of iSCSI session; discovery and normal. Before going into specific details of session
establishment and data transfer it is first necessary to understand the iSCSI PDU which is used in
all communications between the initiator and target.
2.6.1.2.1 iSCSI Protocol Data Units
The iSCSI PDU is effectively the equivalent of the SCSI CDB. It is used to encapsulate the SCSI
CDB and any associated data. The general format of a PDU is shown in Figure 2-8. It is
comprised of a number of segments, although in many cases only the basic header segment
(BHS) is used.
The BHS segment layout is shown in greater detail in Figure 2-9; it has a fixed length of 48
bytes. The ‘Opcode’, ‘TotalAHSLength’, and ‘DataSegmentLength’ fields in the BHS are
mandatory in all iSCSI PDUs. Where values are given for the Initiator Task Tag, Logical Unit
Number and Flag fields, they always appear in the same location on the header [2-17].
The Additional Header Segment (AHS) begins with 4-byte Type-Length-Value (TLV)
information. This specifies the length of the actual AHS following the TLV. The Header and Data
19
digests are optional values. Their purpose is to protect the integrity the authenticity of the header
and data respectively. The digest types are negotiated during the login phase.
Byte 0 1 2 3Bit 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
00 --------Basic Header Segment (BHS)
-------- 4748 --------
Additional Header Segment (AHS) (Optional)-------- nnnn + 1 --------
More AHS's (Optional)-------- mmmm + 1 --------
Header Digest (Optional)-------- mm + 4mm + 5 --------
Data Segment (Optional)-------- pppp + 1 --------
Data Digest (Optional)-------- pp + 4
Figure 2-8: General Structure of an iSCSI PDU
Byte 0 1 2 3Bit 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
0 to 3 .| I | Opcode Opcode-Specific Fields
4 to 7Total AHS
LengthData Segment Length
8 t 15 Logical Unit Number (LUN) or Opcode-Specific Fields16 to 19 Initiator Task Tag (ITT) or Opcode-Specific Fields20 to 31 Opcode-Specific Fields32 to 47 Command Descriptor Block (CDB) or Opcode-Specific Fields
Figure 2-9: Basic Header Segment Layout
2.6.1.2.2 Sessions
As stated previously, there are two types of iSCSI session, these are discovery and normal. In
both cases, the establishment of the session normally occurs in three phases, security negotiation
phase, login operational negotiation phase, and full-feature phase.
20
The security negotiation phase is optional. In order to use it, the Current Stage (CSG) field must
be set for security negotiation in the initial login PDU. Since the security negotiation phase is not
used in this project it will not be discussed further.
When in the login operational negotiation phase, requests and responses are exchanged between
the initiator and target. It is only when the target sends its final response that the full feature phase
be entered - assuming that the final response accepted the login. SCSI commands (CDBs) can
only be sent or received in the iSCSI PDUs once the full feature phase has been entered. The
information in the requests and responses varies dependent upon the type of session that is
established.
2.6.1.2.2.1 Discovery Session
The goal of the discovery session is to discover the names and addresses of the target nodes to
which the initiator can connect.
When attempting to login to a discovery session, there are several pieces of information that the
initiator must supply. Firstly, the session to be established must be identified as a discovery
session. This is achieved by declaring key=value and SessionType=Discovery pairs, in the first
login PDU [2-2]. The name of the initiator must also be included in the login request so that the
target entity can identify it, and apply security filters to responses that are sent [2-21]. Finally, the IP
address and TCP port must be specified for the unidentified iSCSI target entity that is listening.
All of this information can be seen in the lower part of Figure 2-10 which shows the Wireshark
network protocol analyzer log of the ‘Login Command’ for discovery. The installation of
Wireshark is discussed in Appendix A.
21
Figure 2-10: Wireshark log for the Login Command for the Discovery Session
Once a successful response is issued by the target, the full feature phase of the discovery session
is entered. An example of a successful login response can be seen in the Wireshark log in Figure
2-11. During this phase the only iSCSI command that can be issued is the ‘SendTargets’
command shown in the log in Figure 2-12. On receipt of this command, the target responds by
sending a ‘SendTargets’ response. This response contains the iSCSI node names of the targets
accessible to the initiator via the IP address and TCP port where the ‘SendTargets’ command was
received. This log is shown in Figure 2-13.
22
Figure 2-11: Login Response for Discovery Session
23
Figure 2-12: Request to SendTargets in Full Feature Phase of Discovery Session
24
Figure 2-13: Response to SendTargets request
2.6.1.2.2.2 Normal Session
The purpose of the normal session is to allow data to be transferred from the initiator to the target
and vice versa. When attempting to login to a normal session, the initiator must specify the names
of the target node and initiator. This information is included in text format in the ‘DataSegment’
field of the PDU; it can be seen in the log shown in Figure 2-14.
25
Figure 2-14: Login Command for Normal Session
2.6.2 Data Transfer between Initiator and Target(s)
Once the full feature phase of the normal session has been established, data can be exchanged
between the initiator and the target(s), and vice versa, using SCSI commands contained within
CDBs sent or received in the iSCSI PDUs.
26
Consider that an application on the initiator wishes to perform storage I/O to/from the target. This
can be broken down into two stages: progression of the SCSI command through the initiator, and
progression of the SCSI command through the target. Details of both are provided below.
To assist in understanding the progression of the commands, the iSCSI protocol layering model is
shown in Figure 2-15 [4-2].
Figure 2-15: iSCSI Protocol Layer Model [4 -2]
2.6.2.1 Progression of a SCSI Command through the Initiator
1. The user or kernel application on the initiator issues a system call for an I/O
operation. This system call is then sent to the SCSI layer (SCSI class driver).
Initiator Target
27
2. On receipt at the SCSI layer, the system call is converted into a SCSI command
such as ‘read’ or ‘write’, and a CDB containing this information is constructed.
The SCSI CDB is then passed to the iSCSI initiator protocol layer.
3. On arrival at the iSCSI protocol layer, the SCSI CDB and any SCSI data are
encapsulated into a PDU. It is then forwarded to the TCP/IP layer.
4. In the TCP layer, the iSCSI PDU is divided into segments, and a TCP header is
added. Next the IP layer encapsulates the TCP segment by adding an IP header
before the TCP header.
5. The IP datagram is then passed to the Ethernet Data Link Layer where it is framed
with Ethernet headers and trailers. The resulting datagram is then placed on the
network.
In the case where native IPsec is utilized, an additional step is required. Immediately after step 4,
described above, the IP packet is passed to the IPsec layer where the TCP payload, headers and
trailers are encrypted and authenticated before being passed to the Ethernet Data Link Layer.
Figure 2-16 shows the iSCSI/TCP/IP/Ethernet transport encapsulation that is placed on the
network in the last step above, when IPsec is not used.
TCP Segment
IP Datagram
Ethernet Frame
EthernetHeader
IP TCP iSCSI SCSI Commands
Optional Data
28
Figure 2-16: Data Encapsulation into Network Packets
2.6.2.2 Progression of a SCSI Command through the Target
1. On receipt at the target, the Ethernet frame is stripped off at the Data Link Layer and the
IP datagram is passed up to the TCP/IP layer.
2. The IP and TCP layers each check and strip off their respective headers leaving the iSCSI
PDU which is passed up to the iSCSI layer.
3. At the iSCSI layer, the SCSI CDB is extracted from the iSCSI PDU and passed along
with its related parameters and data to the SCSI layer.
4. Finally, the SCSI layer sends the SCSI ‘write’ request and data to the LU.
If native IPsec has been used on the initiator, then on receipt at the target the Ethernet Frame is
stripped off at the Data Link Layer as described above. However, the packet is sent to the IPsec
layer where re-authentication will take place. The IPsec headers and trailers are removed before
the packet proceeds to the second step described above.
29
2.7 Implementation of iSCSI
The discussion so far has focused on the theory of iSCSI. It is now necessary to examine the
physical application of iSCSI.
2.7.1.1 iSCSI Host Adapter Implementations
An iSCSI host adapter is used to connect the initiator to the target device on which the data (files)
are to be stored. It is analogous to the SCSI host bus adapter used for direct attached SCSI.
The iSCSI host adapter can be implemented in a number of ways including software only
implementations, software with embedded TCP off-load engines (TOE), and combined TCP/IP
offload, and iSCSI in silicon. Figure 2-17 shows these options. The implementation chosen
depends upon such factors as price, sensitivity, bandwidth requirements, and host (initiator) CPU
overhead requirements [2-22]. In this project and the Efficient Asymmetric Secure iSCSI [1-11]
(EASI) scheme which it aims to enhance, a software only implementation was utilized. However
for completeness, brief discussions of all of the implementations are provided.
2.7.1.1.1 Software iSCSI
Software iSCSI utilizes software applications to enable the initiator and target to communicate
with the iSCSI protocol. The iSCSI drivers can run on off-the-shelf Ethernet NIC or interface. In
this way, block I/O over IP networks can be performed by a server or workstation running iSCSI
in software using an Ethernet adapter. As a result, software iSCSI provides a significant
advantage in terms of cost. However, it has disadvantages in terms of performance and CPU
overhead. In this type of implementation, the CPU must not only carry out the TCP processing
but must also process the iSCSI protocol that is carried by TCP/IP [2-22].
30
Figure 2-17: Options for iSCSI Adapter Implementation [2-22]
2.7.1.1.2 Software iSCSI with TCP Offload
In certain situations, the performance and CPU overhead issues that can arise with software iSCSI
may be unacceptable. In this case, TCP offload can provide a better option. A TOE is basically an
integrated component which assembles TCP packets before they are sent out and disassembles
the TCP packets when they are received. By utilizing TOEs, significant processing overhead that
would otherwise be incurred by the host CPU is off-loaded to a host network interface card. Thus,
rather than having to carry out both the TCP and iSCSI processing, the CPU only has to carry out
the iSCSI processing [2-22].
31
2.7.1.1.3 Hardware iSCSI with TCP Offload
This type of implementation provides high speed iSCSI transport with minimal CPU overhead.
This is achieved by offloading processing of the iSCSI in the TCP protocol to a specialized card.
It does however have the disadvantage of a significant cost burden due to the requirement of a
specialized interface adapter card [2-22].
2.7.1.2 Software iSCSI Implementations
During this project, software iSCSI was utilized. A variety of different software implementations
are available, however, only the initiator and target implementations used in this project are
discussed.
2.7.1.2.1 Open-iSCSI
Open-iSCSI is a “high performance, transport independent, multi-platform implementation of
RFC3720 iSCSI” [2-20] for initiators. The implementation is partitioned into two sections: user and
kernel. The kernel portion is comprised of three loadable modules: scsi_transport_iscsi.ko,
libiscsi.ko and iscsi_tcp.ko. It is through these modules that the kernel portion implements the
iSCSI data path (iSCSI ‘read’ and iSCSI ‘write’). The iSCSI daemon (iscsid) is part of user
space. It implements the control path of the iSCSI protocol. In addition, it can also be used to
implement certain management facilities.
Like the iSCSI daemon, the iSCSI configuration utility (iscsiadm) is part of the user space. It is
used to manage the persistent database by allowing the user to perform operations, via the
command line, on the iSCSI nodes, sessions, connections, and discovery records which are stored
in the database.
32
2.7.1.2.1.1 Open-iscsi-0.4-434
The Open-iSCSI project is under continual development and as such, numerous versions of the
software are available. At the time of writing, the latest semi-stable release is open-iscsi-2.0-
869.2. This version requires a Linux kernel version of 2.6.16 or greater. However, the EASI [1-11]
scheme utilized an earlier version, specifically open-iscsi-0.4-434. It is this version that was used
predominantly in this project. That being said, the current version (open-iscsi-2.0-869.2) was
utilized with Linux kernel 2.6.23.17 for additional research carried out during this project.
2.7.1.2.2 iSCSI Enterprise Target (IET)
iSCSI Enterprise Target is open source software, it enables the building of a target iSCSI storage
system on Linux. IET can provide regular files, block devices, and virtual block devices to the
initiator as storage media. It is comprised of kernel modules, a daemon (ietd), and control tool
(ietadm). The latter is used for managing the IET software dynamically.
2.7.1.2.2.1 iscsitarget-0.4.11
The iSCSI target software, like the initiator software, is under constant development. At the time
of writing, the latest stable version is 0.4.16. This version requires a Linux kernel version of
2.6.14 or greater. The version used in the EASI [1-11] scheme was 0.4.11 and it is this version that
is used predominantly in this project. However, the 0.4.16 version was utilized with Linux kernel
2.6.23.17 for additional research carried out during this project. Details of the installation and
configuration of open-iscsi version 0.4-434 and iscsitarget version 0.4.11 are available in the
report on EASI [1-11] and are not reproduced here.
33
2.7.2 sg3_utils and sg_dd
The EASI [1-11] scheme utilized the sg_dd utility from the sg3_utils package for reading and
writing between the initiator and target. In order to develop the enhancements proposed for this
project, it was necessary to gain an understanding of not only the sg_dd utility but also the
sg3_utils package.
The sg3_utils package contains low level utilities for devices using the SCSI command sets; it
utilizes the sg interface described earlier. The utilities can be categorized as; variants of the Unix
dd command, scanning and mapping utilities and, SCSI support and timing and testing. Of the
multiple utilities in the package, only one is of specific interest to this project - the sg_dd utility
which is described below.
The sg_dd utility [2-23] from the sg3_utils package [2-24] is a specialized variant of the standard Unix
dd command for copying files. It is specialized for block oriented devices that use the SCSI
command set in the Linux operating system [2-23]. Although the basic syntax of sg_dd is the same
as that of the dd command, additional syntax is supported. The syntax as used in this project is
presented below. Full syntax information can be found in the sg3_utils README files [4-8].
A typical example of the sg_dd command syntax used in this project is:
sg_dd if=test.txt of=/dev/sdb bs=1024 bpt=1 odir=1 count=1024 skip=0 seek=0
The above command copies 1024 x 1024 byte blocks from the file test.txt to the device associated
with /dev/sdb. Descriptions of the various options are given in the Table 2-1.
34
sg_dd Option Default Description
if = IFILE stdin File (or device) to read from.
of = OFILE stdout File (or device) to write to
bs = BS 512 Number of bytes in each block
bpt = BPT 128 or 32BPT is the blocks per transfer.The default is 128 with BS < 2048 bytes otherwise the default is 32
odir = 0|1 0 O_DIRECT flag on open() when set
count = COUNTThe blocks in
The number of blocks to copy. the IFILE
skip = SKIP 0The block number (origin 0) in the IFILE to commence reading
seek = SEEK 0The block number (origin 0) in the IFILE to commence writing.
Table 2-1: sg_dd Options
2.7.3 iSCSI and Data Security
iSCSI itself does not provide security to data, however, it was developed to allow the
implementation of a variety of security measures. Since it utilizes IP for transport, IPsec would
seem to be an obvious solution for providing security, especially as it requires no special
negotiations between iSCSI end devices. Therefore, IPsec is described in detail in the next
chapter.
35
Chapter 3
3. IPsec
In its standard form, IP does not provide any security during the transmission of data. However,
security can be implemented using the IPsec protocol, which is an extension of the IP protocol.
It can provide security (authentication, integrity, and confidentiality) to IP and the upper layer
OSI (Open Systems Interconnection) protocols [3-1] during transmission.
IPsec is comprised of three sub-protocols [3-1]; Authentication Header (AH), Encapsulating
Security Payload (ESP), and the Internet Key Exchange (IKE). The AH and ESP protocols can be
used separately or in combination [3-2, 3-3]. In addition, two possible modes exist for exchanging the
secured data; these are tunnel and transport modes.
In the EASI [1-11] implementation, ESP and IKE were utilized with the transport mode selected.
The native IPsec implementations of ESP and IKE were utilized to transmit data securely
between the initiator and target. However, to enable the data to be stored encrypted on the target,
modifications were made to the ESP code.
Since the goals of this project are to provide enhancements to the EASI implementation, only
IKE and ESP in transport mode will be discussed in any great detail. However, for completeness,
a brief overview will be provided of AH and tunnel mode.
36
3.1 IPsec Security Associations and Security Policies
IPsec Security Associations (SAs), Security Policies (SPs) and their respective databases play an
important role in the application of the IPsec protocol. They are used to determine how datagrams
entering or leaving IPsec capable devices are handled.
3.1.1 Security Policies and Security Policy Databases
SPs are rules programmed into IPsec which are used to identify which network traffic should be
protected. The parameters defined by the SP are: source and destination addresses of the packets
to be protected, the protocol and port number requiring protection, and the SAs used to protect
the packets. These parameters are stored in a Security Policy Database (SPD). A typical example
of an SPD entry used in this project is shown in Figure 3-1. The command to add an SP is as
follows:
spdadd 192.168.2.69 192.168.2.64 any –P in ipsec esp/transport//require;
When a packet is received by a device, the device checks the SPD to determine how it should be
processed. It may be that the packet does not need to be processed by IPsec, alternatively it may
require to be processed with AH and/or ESP, in which case reference must be made to the
associated SA.
37
Figure 3-1: Example of Security Policy Database Entries
3.1.2 Security Associations and Security Association Databases
The SA specifies how the packet should be protected by IPsec by describing the relationship
between two or more devices in terms of the security services that the devices will use to
communicate. An SA consists of the following fields: source and destination addresses of the
devices protecting the packets, the IPsec protocol to be used (ESP or AH), the algorithm and
secret key to be used by the protocol, and a Security Parameter Index (SPI). The SPI is used to
identify the particular SA. Additional fields that may be specified include the IPsec mode
(transport or tunnel), the lifetime of the SA, and the size of the sliding window if the packet is to
be protected against replay attacks. All of these fields are stored in the device’s Security
Association Database (SAD). The command to add a SA is as follows:
add 192.168.2.64 192.168.2.69 esp 0x201
-E aes-cbc 0x7aeaca3f87d060a12f4a4487d5a5c3355920fae69a96c831
-A hmac-md5 "authentication!!" ;
Although SAs define the source and destination addresses of the devices, they are unidirectional
and can only protect one direction of traffic [3-4]. Therefore, in order to protect both directions in
38
two-way communications, there must be an SA for each direction. An example of an SAD used
on the initiator in this project is shown in Figure 3-2.
Figure 3-2: Example of Security Association Database Entries
3.2 Transport and Tunnel Modes
It has already been stated that IPsec can be used to protect traffic in one of two modes – transport
or tunnel. The IPsec transport mode is used to protect transmitted data end-to-end between hosts
[5-3]. Only the data (payload) contained in the IP packet is encrypted and/or authenticated. This is
achieved by processing the packet with the relevant IPsec protocol and inserting the IPsec header
between the IP header and the upper layer protocol header [3-4].
In comparison, in tunnel mode the entire IP packet is protected during transportation between
hosts. This is achieved by completely encapsulating the original IP packet with a new packet [3-1, 3-
4]. When the packet is processed by the relevant IPsec protocol, the IPsec header is inserted in
front of the original IP header and a new IP header is added in front of the IPsec header. The
IP AH ESP TCP Data
IP TCP
DataDataDataAH ESP IP TCP
Data
IP
Transport Mode
Original Packet
TunnelMode
New IPHeader
39
packets created for transport and tunnel modes using the IPsec AH protocol are shown in Figure
3-3.
Figure 3-3: Packets for IPSec Tunnel and Transport Mode [3-5]
3.3 Authentication Header
The AH protocol provides authentication and data integrity but not confidentiality. Since this
project focuses on encryption, AH will be described in general terms only. Its use with transport
and tunnel modes will not be discussed.
Authentication of the packet can be provided to all or part of the contents of the packet [3-1]
dependent upon the mode in which AH is used, tunnel or transport. In order to provide
authentication, IPsec computes a cryptographic hash-based message authentication code
(HMAC). The HMAC is created using a hash algorithm such as MD5 or SHA, with the hash
Original IPHeader
40
calculated based on a secret key and contents of the IP packet [3-4]. The HMAC is then stored with
other data in the IPsec AH header. The general format of the AH header is shown in Figure 3-4.
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
Next Header AH Length Reserved
Security Parameters Index (SPI)
Sequence Number (Replay Defense)
Hash Message Authentication Code (variable)Figure 3-4: Format of AH Header
It has already been stated that the HMAC is stored in the AH. However, for completeness, the
other fields in the AH are described in Table 3-1.
AH Header Field Description
Next Header Protocol type of the following payload
AH Length Total length of AH header in 32-bit words
Reserved 2 bytes reserved for future use
Security Parameter Index Security association to be used when packet is
decapsulated
Sequence Number (Replay
Defense)
Used to protect the packet against replay attacks
Table 3-1: Descriptions of AH Header Fields
41
3.4 Encapsulating Security Payload
The ESP protocol can provide not only confidentiality of a packet, but can also optionally provide
authentication. Authentication with ESP is achieved in a similar manner to that of AH. First, the
packet is encrypted and then the HMAC is computed. Unlike AH, the authentication does not
cover the entire IP packet, rather it covers the ESP header and encrypted payload [3-3].
In simplistic terms, ESP provides packet confidentiality by encrypting the IP packet. The data
contained within the packet is transformed into an encrypted form by combining it with a key
using an encryption algorithm. The encrypted packet is then repackaged using a special format
and transmitted to its destination, where it is decrypted with the same algorithm and key. The
format of the ESP header is shown in Figure 3-5.
There are three basic steps performed by ESP, they are:
Header calculation and placement
Trailer calculation and placement
ESP authentication field calculation and placement
3.4.1 ESP Header Calculation and Placement
The ESP header is comprised of two fields, the SPI and the Sequence Number. These two fields
also appear in AH. The SPI is a 32 bit number used to uniquely identify a particular SA for a
connected device. The Sequence Number is effectively a counter which is initialized when an SA
is formed between two devices. It is used to provide protection against replay attacks, since its
value is incremented for each packet sent using the SA.
The placement of the ESP header depends upon the mode in which ESP is being used; transport
or tunnel. When ESP is operating in transport mode, the header appears after the original header,
42
while in tunnel mode, the ESP header is placed after the IP header of the new datagram thus
encapsulating the original.
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
Security Parameters Index (SPI)
Sequence Number
Initialization Vector
Payload * (variable)
Padding (0-255 bytes)
Pad Length Next Header
Hash Message Authentication Code (variable)Figure 3-5: Encapsulating Security Payload Header Format
3.4.2 ESP Trailer Calculation and Placement
The ESP trailer is appended to the data to be encrypted. It is comprised of three fields; Padding,
Pad Length, and Next Header. The main purpose of the trailer is to provide any padding required
to align the data for encryption. The actual padding is taken care of in the Padding field, while the
Pad Length field specifies the number of bytes in the Padding field. The Next Header field
contains the protocol number of the next header in the datagram. This protocol number specifies
which header should be examined next; this could be the destination options extension header, the
TCP/UDP header, or the IP header. The header to which it refers depends on which mode is being
43
used. When operating using ESP in transport mode the Next Header references the TCP/UDP
header, while when using ESP in tunnel mode the Next Header references the IP header.
Once the trailer is in place, and if ESP is operating in transport mode, the payload (TCP/UDP
message) and trailer are encrypted. If operating in tunnel mode, the encapsulated IP datagram and
trailer are encrypted.
3.4.3 ESP Authentication Field Calculation and Placement
The authentication field is optional and is used when the ESP authentication feature is used. The
field contains an Integrity Check Value (ICV). It is computed over the entire ESP datagram.
3.5 Internet Key Exchange
In order to work, IPsec relies on the fact that the devices exchanging information share a common
piece of information - a secret that is used by the security protocols. This highlights one of the
problems associated with secure communications – how to securely exchange this secret piece of
information. The IKE protocol of IPsec addresses this issue. IKE is used to exchange SA
proposals and negotiates these associations based on the Internet Security Association and Key
Management (ISAKMP) protocol.
The IKE protocol functions in two phases. During the first phase, agreement is made between the
two devices as to how they will communicate future information in a secure manner. The
authentication of the two devices is usually based on pre-shared keys (PSK), RSA keys, and
x.509 certificates [3-4]. The negotiation between the two devices results in the creation of a SA for
ISAKMP. This SA is then used to securely exchange information in the second phase [3-1]. The
attributes negotiated in this phase include; the encryption algorithm to be used, a hash algorithm,
44
an authentication method, and a Diffie Hellman group. During the second phase, the IPsec SAs
are negotiated and setup using the ISAKMP SAs.
45
Chapter 4
4. Efficient Asymmetric Secure iSCSI
This project focuses on the use of iSCSI as a secure online storage solution. It builds on the
Efficient Asymmetric Secure iSCSI (EASI) [1-11] scheme developed and implemented by Mr.
Andukuri. Therefore, it is necessary to review his work in order to develop enhancements for the
current project.
4.1 Review of Efficient Asymmetric Secure iSCSI (EASI)
As has been stated earlier, iSCSI does not provide security to user data when it is transmitted.
However, it does allow for the provision of security of the data in transit through the use of
standard IPsec. That being said, neither iSCSI nor IPsec provide security of the data once it is
saved on the storage device. Thus, the main goal in the development of the EASI scheme
developed and implemented by Mr. Andukuri was to address this limitation. Since IPsec can
provide security for data in transit, he chose to enhance standard IPsec to additionally provide
security of stored data.
4.1.1 EASI Enhancements to IPsec
In his EASI [1-11] scheme, Mr. Andukuri proposed that security of stored data could be achieved
via a dual-key cryptographic enhancement to IPSec. In brief, the user data to be stored would be
encrypted at the IPsec layer using a custom key. The TCP packet then used to transmit the user
data would then be encrypted using a different key. On receipt at the storage device, only the
encryption on the TCP packet would be removed. An overview of the EASI implementation is
shown in Figure 4-1.
Only headers Encrypted here
scsi
iscsi
tcp
ip
ipsec
Encryptedpayload
Encryptedpayload
Target
To iscsitarget
To iscsiinitiator
Payload decrypted herewithCustom key
PayloadEncrypted withCustom key
scsi
iscsi
tcp
ip
ipsec
Unencryptedpayload
Initiator
Only headers Decrypted here
46
Figure 4-1: Efficient Asymmetric Secure iSCSI Scheme [1-11]
The EASI enhancements to IPSec can be considered to be comprised of four sections:
1. Sending side code for the initiator.
2. Receiving side code for the target.
3. Sending side code for the target.
4. Receiving side code for the initiator.
47
The iSCSI protocol does not have a protocol ID associated with it. However, an iSCSI target uses
a default port of 3260. Consequently, since TCP headers contain fields for both source and
destination ports, the information contained in these field can be used to identify iSCSI traffic.
4.1.1.1 Sending side code on the initiator
When writing user data (payload) from the initiator to the target, the steps carried out in the
sending side of the code for the initiator are as follows:
1. Identify all iSCSI traffic.
This is achieved by examining the destination port field in the TCP header. If the
destination port is 3260, then iSCSI traffic is being sent to the target.
2. Determine if the iSCSI traffic contains user data.
User data is carried to the target in a Data-out PDU. Therefore, it is possible to determine
if there is likely to be user data by examining the ‘opcode’ in the iSCSI header. If the
‘opcode’ value is ,5 then there is a Data-out PDU which may contain user data. In order
to confirm whether or not user data is present, the length of the packet must be
determined. If the length of the packet exceeds the length of the headers plus the TCP
header offset from IP, then the packet contains user data.
3. Encrypt packets not containing user data.
iSCSI packets not containing user data are routed through native IPsec. The TCP and
iSCSI header combination is treated as a unit. The unit is checked to see if it is the correct
size for the encryption algorithm. If necessary, padding is appended at the end of the
TCP/iSCSI unit. The padding is computed based on the algorithm block size. The
TCP/iSCSI unit is then encrypted using IPsec keys generated by IKE.
48
4. Encrypt packets containing user data.
The encryption of packets containing user data is split into four sections.
a. Verifying Headers.
It is first necessary to check to see if the combination of the TCP header and iSCSI
header is an integer multiple of block size. This is done because the encryption algorithm
requires that each packet is a certain bock size. Ordinarily the block size is achieved by
padding the TCP packet by adding a trailer containing the padding at the end of the
packet. However, the presence of the user data (payload) following the TCP and iSCSI
headers precludes padding being added at the end of the packet. In addition, the BHS of
the iSCSI header is a fixed length of 48 bytes. Therefore, there is no room for padding.
Consequently, any padding required has to be added to the end of the TCP header such
that it is between the TCP and iSCSI headers. In order to achieve this, the TCP header
has to be moved to create a gap between it and the iSCSI header into which padding can
be inserted. The gap created is the same size (in bytes) as the padding required. This
packet modification is shown in Figure 4-2.
b. Encrypt the user data.
The next stage of the process is to isolate the user data and encrypt it using the custom
key. This key is generated independently of the IKE mechanism and is not shared with
the target. It should be noted that one of the limitations of the iSCSI scheme is that the
user data must be block size, or an integer multiple of block size. In the EASI scheme the
block size is 1024 bytes.
49
c. Re-compute the TCP checksum and update the TCP header with this value.
Once the user data has been encrypted, the TCP checksum is recomputed to reflect the
new length of the packet produced by the addition of padding. The TCP header is then
updated with the value.
d. Encrypt the TCP and iSCSI headers.
Finally, the TCP and iSCSI headers are encrypted using IPsec keys generated by IKE.
Figure 4-2: Packet modification under proposed scheme [1-11]
50
4.1.1.2 Receiving side code on the target
When writing user data from the initiator to the target, the steps carried out in the receiving side
of the code for the target are as follows:
1. Identify iSCSI traffic.
The packets arrive at the IPsec layer of the target in an encrypted form. Therefore, to
identify iSCSI traffic the packet must be decrypted using the IPsec keys generated by
IKE, to reveal at least the first 32 bytes of the TCP header. This allows the source and
destination port fields to be examined. If the destination port field value is 3260, then
there is iSCSI traffic coming from the initiator to the target.
2. Determine if the iSCSI traffic contains user data.
The next step is to decrypt the packet to the end of the iSCSI header. This is achiveved
using the IPsec keys generated by IKE. If the ‘opcode’ in the iSCSI header is 5, then
there is a Data-out PDU which may contain user data. Confirmation that user data is in
fact present is achieved by checking the length.
3. Decrypt packets not containing user data.
If the packet does not contain user data, then the remainder of the packet is decrypted
using the IPsec keys generated by IKE.
4. Process packets containing user data.
If the packet contains user data, it should not be decrypted since it is to be stored on the
target storage in an encrypted form. However, the ESP header must be removed and the
‘next’ header must be set to TCP. The packet is then passed to the upper layers for
processing and finally stored in an encrypted form.
51
4.1.1.3 Sending side code on the target
When reading user data from the target to the initiator, the steps carried out in the sending side of
the code for the target are as follows:
1. Identify all iSCSI traffic.
This is achieved by examining the source port field in the TCP header. If the source port
is 3260, then iSCSI traffic is being sent from the target to the initiator.
2. Determine if the iSCSI packet contains user data.
User data is carried to the initiator in two packets: a Data-in PDU which contains the
length of the data to be sent in its header. This packet is followed by a plain vanilla TCP
packet with the ‘push’ bit set in the header. It is in this TCP packet that the user data is
sent.
Therefore, the presence of user data can be determined by examining the ‘opcode’ in the
iSCSI header and the ‘push’ bit in the TCP packet. If the ‘opcode’ is 37, then there is a
Data-in PDU and the ‘push’ bit is set in the TCP packet then there is user data.
3. Encrypt packets not containing user data.
This is achieved using the same method described in step 3 of section 4.1.1.1.
4. Encrypt packets containing user data.
The encryption of packets containing user data is split into three sections.
a. Verifying Headers.
This is achieved using the same method described in step 4a of section 4.1.1.1.
52
b. Re-compute the TCP checksum and update the TCP header with this value.
This is achieved using the same method described in step 4c of section 4.1.1.1.
c. Encrypt the TCP header and iSCSI header.
This is achieved using the same method described in step 4d of section 4.1.1.1.
It should be noted that it is not necessary to encrypt the user data when it is being read
from the target since it is already in an encrypted form.
4.1.1.4 Receiving side code on the initiator
When reading user data from the target to the initiator, the steps carried out in the receiving side
of the code for the initiator are as follows:
1. Identify iSCSI traffic.
The packets arrive at the IPsec layer of the initiator in an encrypted form. Therefore, to
identify iSCSI traffic, the packet must be decrypted using the IPsec keys generated by
IKE, to reveal at least the first 32 bytes of the TCP header. If the source port field value is
3260, then there is iSCSI traffic coming from the target to the initiator.
2. Determine if the iSCSI traffic contains user data.
When user data is read from the target to the initiator, the data is carried in two packets: a
Data-in PDU, which contains the length of the data to be sent in its header. This is
followed by a plain vanilla TCP packet with the ‘push’ bit set in the header. It is in this
TCP packet that the user data is sent.
Therefore the presence of user data can be determined by examining the packet to see if
there is an iSCSI header. If there is no iSCSI header, then the packet is a TCP packet
53
which may contain user data. If the ‘push’ flag is set, and the length of the packet exceeds
that of just the TCP header, then there is user data.
3. Decrypt packets containing user data.
The decryption of packets containing user data is split into two sections.
a. Decrypt the user payload.
If the packet contains user data, the user data is isolated and decrypted using the custom
key.
b. Re-compute the TCP checksum and update the TCP header with this value.
Once the user data has been decrypted, the TCP checksum is recomputed and the TCP
header is updated with the value.
4. Decrypt packets not containing user data.
If the packet does not contain user data, then the remainder of the packet is decrypted
using the IPsec keys generated by IKE.
4.2 Motivation for Enhancing EASI
The EASI scheme goes a long way to addressing the issues of online data storage. However, in
order to bring the implementation closer to a complete and usable secure data transfer/storage
system, a number of areas have been identified that need further consideration. The areas selected
for further investigation in this project are discussed in the following section.
4.2.1 Goals for the Secure Asymmetric iSCSI System for Online Storage
The existing EASI implementation permits only the transfer of block size data or integer
multiples of block size. Any attempts to transfer non-block size (arbitrary size) files results in the
54
encryption scheme breaking. Obviously, it is highly desirable to be able to transfer files of
arbitrary size. Thus, this is one of the areas that are addressed in this project.
A second area which needs addressing is the enhancement of the EASI scheme to allow for
duplicate transfer of files to a second target storage device. This would potentially provide
additional disaster recovery support.
Another limiting area identified in the EASI scheme is the complexity in terms of the
configuration that needs to be undertaken by the user. In order to make the implementation more
complete, and in order for it to be more widely accepted, simplification of the configuration is
desirable. The obvious way to achieve this is through the development and implementation of a
graphical user interface (GUI) or an application program interface (API).
In addition to the above goals, a further goal is to investigate the potential for using the
implementation in a disaster recovery goal.
Having gained an understanding of the existing EASI scheme and goals for the current project,
the next stage was to test the existing EASI scheme. This testing is described in the next chapter.
55
Chapter 5
5. Testing of EASI Implementation
Prior to developing detailed proposed enhancements for this project, it was first necessary to test
the EASI implementation. In this way, the problems associated with transferring files of arbitrary
size could be more clearly understood. In addition, it would enable other limitations, which may
affect potential enhancements, to be identified and understood.
5.1 Test Bed
Although a physical test bed with the EASI implementation was available, a decision was made
to preserve it. It was decided to recreate the EASI implementation using VMware Server, a server
virtualization software.
In order to gain a full understanding of the EASI scheme, attempts were made to recreate both the
initiator and target machines. Two virtual machines were created using Fedora Core 4 as the
operating system. Since the EASI scheme utilizes the Linux 2.6.12.1 kernel, the source code for
this kernel was installed on the virtual machines. Attempts were then made to incorporate the
changes to the IPsec code which had been made in the EASI scheme. However, these attempts
were unsuccessful.
In view of the problems encountered, a decision was made to clone the physical test bed for use
as a virtual machine. This was achieved using the UltimateP2V software plug-in; details of the
procedure and the subsequent installation of the clones on the VMware Server are provided in
Appendix B. A schematic of the resulting virtual machine test bed is shown in Figure 5-1. The
test bed was comprised of a laptop running Windows XP Pro, on which VMware Server 1.04 was
installed. The virtual machine images were stored on an external hard drive.
VMware Server 1.04
Physical Host IP: 192.168.2.100
iSCSI Initiator
VMware instance running linux-2.6.12.1 and
open-iscsi-0.4-434
VMware specifications:
Physical Memory: 256MB
IP: 192.168.2.64
iSCSI Target 1
VMware instance running linux-2.6.12.1 and
iscsitarget-0.4.11
VMware specifications:
Physical Memory: 256MB
IP: 192.168.2.69
iSCSI Target 2
VMware instance running linux-2.6.12.1 and
iscsitarget-0.4.11
VMware specifications:
Physical Memory: 256MB
IP: 192.168.2.25
Host: Laptop running Windows XP Service Pack 2Physical Memory: 2GB
Processor: AMD Turion 64 Mobile 2 GHzHard Disk: 80 GB
External Hard Disk: 500GB
56
Figure 5-1: Schematic of Test Bed
5.2 Limitations of using a File as a Target Storage Device
The EASI implementation utilizes a file as a target storage device. Testing revealed that, although
suitable as a proof-of-concept, using a file as target storage has severe limitations. For example, a
user is limited to storing only one file at a time. Any attempt to store another user file results in
overwriting of the user file currently residing on the target storage. The overwrite may be partial
or total, dependent on the size of the user file already stored on the target and the size of the new
file being written to the target storage. In addition, the pre-set size of the target storage file limits
the size of the user file which can be stored upon it. These limitations preclude using a file as a
57
target storage device in a real world environment. As a result of these findings, it was decided to
test the EASI scheme not only with a file as a target storage device, but also using a mounted file
system as target storage.
5.3 Copying User Files using the sg_dd and cp utilities
The EASI scheme utilizes the sg_dd utility on the initiator to write/read user files to/from the
target storage. The files which can be written/read using the scheme must be block size (1024
bytes), or integer multiples of block size. In order to write/read files or arbitrary size, Mr.
Andukuri [1-11] stated that it would be necessary to utilize a different utility such as the cp utility.
Therefore, to fully test the EASI implementation, files were written/read by the initiator to/from
the target using both the sg_dd and cp utilities. For completeness, the tests were repeated using
the EASI scheme and also without using any encryption scheme.
5.3.1 Writing/Reading Files to a Target Storage File using sg_dd
As stated in Chapter 2, the command utilized to write a user file to target storage is:
sg_dd if=test.txt of=/dev/sdb bs=1024 bpt=1 odir=1 count=1024 skip=0 seek=0
and the command to read a file from the target storage is:
sg_dd if=/dev/sda of=test_ret.txt bs=1024 bpt=1 odir=1 count=1024 skip=0 seek=0
The various options were discussed in Chapter 2. However, in view of their relevance to the
following discussion, the ‘bs’ and ‘count’ values are described again here.
The ‘bs’ option is used to specify the size of the blocks to be written/read, which in the
commands shown above is 1024 bytes. If no ‘bs’ value is specified, a default value of 512 bytes
is used. The ‘count’ option is used to specify the number of blocks to be written/read. If the
58
‘count’ value is omitted, attempts are made to determine the value to be used by examining the
input and output devices, which are represented by ‘if’ and ‘of’ respectively.
As expected, the sg_dd utility could be successfully used to write/read block size files, or integer
multiples of block size files, to/from the target storage file. This was possible both with and
without the EASI scheme in place. However, it was noted that it was necessary to specify the ‘bs’
(block size) and ‘count’ (number of blocks to be written/read) values in the sg_dd command in
order for some writes/reads to be successful, this is discussed below.
Without the EASI scheme, it was found that if both the ‘bs’ and ‘count’ values were omitted, the
file was successfully written to the target, although a message was displayed indicating that the
block size was being assumed to be 512 bytes, the default value. However, when reading from the
target, if both the ‘bs’ and ‘count’ values were omitted, the complete target storage file was read
irrespective of its size and whether or not it had any data on it.
In the case where the EASI scheme was used, omission of the ‘bs’ value was problematic since
its default value was 512 bytes and because the EASI scheme requires a block size of 1024 bytes.
If a ‘bs’ value was specified but the ‘count’ value was omitted, no write/read occurred and an
error message was displayed. The reason for this was that with this particular test bed, the target
was seen as a block device with a block size of 512 bytes while the specified ‘bs’ was 1204 bytes.
Consequently, the ‘count’ value could not be determined. The error messages for the write and
read are shown in Figures 5-2 and 5-3 respectively.
59
Figure 5-2: Error Message displayed when writing a User File from Initiator to Target without specifying a ‘count’ value
Figure 5-3: Error Message displayed when writing a User File from Target to Initiator without specifying a ‘count’ value
60
5.3.2 Writing/Reading Files to a Target Storage File using cp
The cp utility was used to write/read user files to/from the target storage file using both the EASI
scheme and without encryption. When writing block size or integer multiples of block size files
using EASI, it was found that the write was successful and the user file was stored in an
encrypted form. However, when attempting to read the user file back to the initiator two issues
were identified.
1. The data read to the initiator was found to be the same size as the entire target storage
file.
2. Although the file should have been decrypted when it was read back to the initiator, it
was found to still be in an encrypted form.
The fact that the file had not been decrypted was not altogether unexpected, since the EASI
scheme was designed for use with block size or integer multiples of block size files. Since the cp
utility was not specifically designed for block transfers, it does not allow for the specification of
the number of blocks to be transferred, or the size of the blocks.
What was of interest, but in hindsight not entirely unexpected, was the size of the file read back
to the initiator. The reason that the entire target storage file was read back can be easily
explained. The cp utility was designed to copy files and since the target storage is in fact a file, it
would be expected that it would be read to the initiator in its entirety.
If the sg_dd utility was used to read from the target, when the write had been carried out with the
cp utility, it was found that the file would be in its decrypted form on the initiator. However, it
was necessary to specify the appropriate ‘count’ value for the file. This is of interest since it
suggests that if the cp utility could be modified to allow the specification of a ‘count’ value, then
it could be used to read the user file from the target storage file. However, due to the block size
61
limitation of the EASI scheme, the file being read would still have to be block size or an integer
multiple of block size.
For completeness, attempts were made to write/read arbitrary sized files using the cp utility and
the EASI scheme. Examination of the target storage file after the arbitrary size file had been
written revealed that the contents of the storage file were encrypted. However, it was not known
if they had been successfully encrypted. In an attempt to determine if the encryption had indeed
been successful, the sg_dd utility was used to read the user file back to the initiator. Since the file
that had originally been written was not block size, it was necessary to specify a ‘count’ value of
the next block size above the arbitrary size so that the read with sg_dd would be successful.
Examination of the file read back to the initiator revealed that it had been successfully decrypted.
This indicated that it had originally been encrypted correctly when the file was written with the cp
utility. However, since it was necessary to read slightly more of the target file using the sg_dd
utility; it was found that the additional bytes read contained what appeared to be garbage. Further
examination of these bytes suggested that they had in fact been white space on the target storage
file, therefore when the file had been read with sg_dd, the EASI scheme had decrypted them. A
screen shot of the output is shown in Figure 5-4.
When an attempt was made to read the user file from the target storage file using the cp utility, it
was found that the decryption failed, and the entire target storage file was read back to the
initiator.
62
Figure 5-4: Output of file read back to the Initiator using the sg_dd utility
5.3.3 Writing/Reading Files to a Mounted Target Storage File System
As has already been stated in this report, there are limitations when using a file as a target storage
device. Therefore, it was decided to add a second virtual hard disk to the target virtual machine,
and use this as the storage device. Details of adding the second hard disk, the procedure for
creating a file system upon it, and mounting it on the initiator are provided in Appendix C.
5.3.3.1 Writing/Reading Files to Mounted Target Storage without Encryption
When attempting to write/read files to a mounted target file system without using any encryption,
it was found that files could be successfully written/read using both the sg_dd and cp utilities.
63
5.3.3.1.1 Writing/Reading with the sg_dd Utility
When using the sg_dd utility to write/read to/from mounted storage, it was found that it was still
necessary to specify the ‘count’ value. This was to be expected since the block size of 1024 bytes
used in the sg_dd command was at odds with the block size of the storage. In addition, it was
found that it was necessary to specify the mount point of the storage along with a file name which
would be used to store the user file. Examples of the commands to write/read using the sg_dd
utility are shown below:
To write files to the target use:
sg_dd if=test_file_1K.txt of=/mnt/iscsi_disk/test1K.txt bs=1024 bpt=1 odir=1 count=1 skip=0
seek=0
To read files from the target use:
sg_dd if=/mnt/iscsi_disk/test1K.txt of=test1K_ret.txt bs=1024 bpt=1 odir=1 count=1 skip=0
seek=0
5.3.3.1.2 Writing/Reading with the cp Utility
When writing/reading files to a mounted storage device using the cp utility, it was found that
unlike the case of the un-mounted target storage file, the reading of a file from the mounted
storage device resulted in only the relevant file data being read. Examples of the commands to
write and read files to mounted storage using the cp utility are shown below.
To write files to the target use:
cp test_file_1K.txt /mnt/iscsi_disk/
To read files from the target use:
cp /mnt/iscsi_disk/test_file_1K.txt test1K_ret.txt
64
Having ascertained that both cp and sg_dd could be used to write/read files from mounted storage
without encryption or with native IPsec encryption, the next stage was to determine if writes and
reads would be successful with the EASI encryption scheme in place.
5.3.3.2 Writing/Reading Files to/from Mounted Target Storage using EASI
On attempting to mount the target storage device on the initiator when using the EASI scheme, it
was found that mounting failed, and the following error message was displayed.
[root@siscsi test_files]# mount /dev/sda /mnt/iscsi_disk/
mount: wrong fs type, bad option, bad superblock on dev/sda, missing codepage or other error
Obviously since the mount failed, it was not possible to write/read files to the storage. Since this
problem had not been encountered when no encryption or native IPsec was used, it seemed likely
that the problem arose from commands issued during mounting being interpreted as user
(payload) data by the EASI scheme. In an attempt to determine whether or not this idea was
correct, it was decided that some additional research should be carried out to determine the
packets passing between the initiator and target when mounting a target storage disk using the
Wireshark network protocol analyzer. This testing, and the results obtained are presented in
chapter 8.
5.4 Summary of the test results and their impact on enhancements
In summary, the testing of the existing implementation revealed the following:
1. Ideally a mounted target device should be utilized in the implementation.
2. Mounting the target device with the existing implementation is not possible.
3. When using the sg_dd utility the user must know the size of the file to be transferred and
‘count’ and ‘bs’ values must be specified.
65
Chapter 6
6. Proposed Enhancements
It should be apparent from the discussion of the results from the testing of the EASI scheme that
there are a number of factors that significantly affect the way in which enhancements to the
scheme can be realized.
6.1 Writing/Reading Files of Arbitrary Size
One of the goals of this project was to be able to transfer and store files of arbitrary size, in an
encrypted form onto a target storage device. To determine which enhancements would be
attempted, the first factor to decide was whether files would be written/read to an un-mounted
target, as in the EASI scheme, or to a mounted target storage.
It was clear from testing the EASI scheme that writing/reading files to a mounted target would
require not only changes to the encryption code to enable arbitrary sized files to be transferred,
but would also require changes that would allow the target storage to be mounted. In order to
achieve the latter, it would first be necessary to undertake a detailed study of the commands
utilized for mounting of the target on the initiator. The next step would be to determine whether
the existing iSCSI/SCSI commands would need modification in order to allow differentiation
between commands required to set up the target device and those to transfer user files. This in
itself was considered to be a major task, and although ultimately desirable, laid outside the scope
of the goals originally intended for this project.
66
Having decided not to use a mounted target, the determining factor was modification of the
encryption code to decipher the different packets and commands transferred when using the cp
utility. The main problem requiring resolution in this case was the inability to successfully
recreate the existing implementation. Various attempts had been made to reproduce the
implementation. These included using a more recent Linux kernel and making changes to the
encryption code which had been modified in the EASI scheme. Despite considerable efforts to
achieve this, none proved successful. It became apparent that a more simplistic approach would
have to be taken that would utilize the existing encryption scheme without the need to modify the
kernel code.
Files/Data are stored on disks in blocks. The size of these blocks can be set when the file system
is created. A typical example of block size currently used is 4096 bytes. If a file of size 750 bytes
is created and the block size is 4096 bytes, the file will still consume 4096 bytes on the disk. The
remaining 3346 bytes of the block will be empty.
The sg_dd utility copies blocks of data, and since files are stored in blocks it can be used to
transfer files irrespective of their size. However, when a file is copied, the entire block is copied,
including any empty space within the block, unlike the cp utility which copies only the file data
within the block(s).
It should be apparent from the above that the sg_dd can be used to copy files of arbitrary size but
with the limitation that the entire block containing the file will be copied. Thus, when the sg_dd
utility is used to copy files of arbitrary size, it is only simulating the copy of an arbitrary sized
file.
As stated in Chapter 5, there a number of limitations when transferring a file/user data to a file
that is acting as a target storage device. The target file appears as a special block device. If the
67
‘bs’ value conflicts with that found for the target file, the ‘count’ value cannot be determined and
therefore a value must be entered and consequently, the user must know the size of the file.
In order to simulate the transfer of a file of arbitrary size, it is desirable that the user does not
have to specify the ‘bs’ value or the ‘count’ value. This can be achieved when writing a file to
target storage by:
Changing the default ‘bs’ value in the sg_dd code to 1024
Adding a function to determine the size of the file in bytes. If the returned value is not
block size (1024 bytes) or an integer multiple of block size, the count value must be set to
the next integer multiple of the number of blocks to be written. This value is then used for
constructing the command descriptor blocks, reading the data from the initiator, and
writing to the target.
Determining the size of the file can be achieved in a number of ways, for the purposes of this
project the ‘stat’ utility was used. Testing of the implementation revealed that although using the
‘stat’ utility worked when writing user data to the target storage, a problem was encountered
when reading user data from the target back to the initiator. The problem that arises is that when
the ‘stat’ utility is used on the target storage, it does not return the size of the target storage. This
can be seen in Figure 6-1.
Alternative methods were investigated for determining the size of the target storage. However, it
was found that they returned the size of the entire target storage file rather than the amount of
user data.
68
Figure 6-1: Result of using stat utility on target storage file
Attempts were also made to write a function to read the contents of the target file and determine
the amount of trailing white space and in this way also determine the amount of user data.
However, since the data is stored in an encrypted form on the target storage this proved to be
unsuccessful. In view of the fact that the current form of the target storage has limitations that
would preclude its use in a real world environment, it was decided that to pursue this further
would not be constructive. Consequently, the sg_dd code was modified such that the count value
would not be determined when reading from the target and that the user would be required to
specify a value, as is the case in the existing EASI scheme.
A typical run through on the VMware test bed for writing/reading files to the target using the
code enhanced for transferring files of arbitrary size is presented in Appendix D.
69
6.2 Transferring a File to Two Targets
The existing EASI implementation considered only the transfer of a file to a single target.
However, the ability to transfer to an additional target is increasingly advantageous especially
with the increasing desire/requirement to store/backup data at multiple remote locations. The
transfer of a file to two or more targets is perfectly possible with the existing implementation. It
just requires the user to issue multiple sg_dd commands, for example:
sg_dd if=test_file.txt of=target1_device bs=1024 bpt=1 odir=1 count=1 seek=0 skip=0
sg_dd if=test_file.txt of=target2_device bs=1024 bpt=1 count=1 odir=1 seek=0 skip=0
However, when writing data to multiple targets there are a number of points that need to be
considered, such as:
Is the same key to be used to encrypt the user data?
Are the same keys to be used for the native IPsec communications between the initiator
and targets?
These points are considered in detail below.
6.2.1 Encryption of Payload
For additional security, it may be desirable to utilize different keys to encrypt the user data that is
being transferred to different targets. This would require modifications to the existing EASI
scheme and also the IPsec implementation, as at the present time a single key is hard coded into
the IPsec encryption code. However, since at this stage the project remains a proof-of-concept,
and because of the difficulties encountered in recreating the EASI scheme and thus making
changes to the IPsec implementation, a decision was made to use the same key to encrypt the
payload for all targets.
70
6.2.2 Native IPsec Keys
In view of the fact that the same key is to be used to encrypt the user data for all targets, it was
decided that different IPsec keys should be used for secure communications between the initiator
and targets. This can be achieved manually or automatically in a relatively straightforward
manner. However, since one of the goals of this project is to simplify the implementation, it was
decided to create a user interface that would simplify the creation of the keys and database
entries.
6.2.3 Steps to Transfer a File to Two Targets
The following changes were required to the sg_dd code in order to enable the transfer of a file to
two targets:
Add another command line argument to allow the user to specify a second target device.
Process the command line options to determine whether one or two target devices are
specified.
If a second target is specified, determine the type of device it is.
Determine the number of blocks and the block size of the second target.
Create a second command descriptor block for the second target.
A typical run through on the VMware test bed for writing/reading files to the target using the
code enhanced for transferring files to two targets is presented in Appendix E.
6.3 User Interface
One of the limitations of the existing implementation is that configuring and starting the open-
iscsi and iscsitarget implementations and transferring and retrieving files requires the user to
71
interact with the command line. Although, the target users of the implementation, such as systems
administrators, are likely to have significant computing experience, it is conceivable that they
have little or no experience of the open-iscsi and iscsi-target implementations. Due to the fact that
the commands utilized are not particularly intuitive, it was decided that a user interface should be
developed to simplify the interactions thus making the implementation more user friendly.
As stated above the main users of the Secure ISCSI implementation are likely to have significant
computing experience and as such the user interface should be functional rather than be
ascetically pleasing.
It was decided to develop the user interface in a graphical format using Python. Python is a
powerful open source dynamic programming language. It is relatively easy to learn and is
available for most major operating systems. In addition, it is easy to interact with the operating
system through the use of shell commands. The installation of Python and Tkinter, used to create
the graphical user interface widgets, is presented in Appendix F.
6.3.1 Functionality of User Interface
In order to develop a suitable user interface, it was first necessary to determine what tasks it
should be able to carry out, and its appearance. It was decided that two user interfaces would be
developed: one for the initiator and one for the target. Although it would be possible to develop a
user interface for the basic configuration of the initiator and target, it was decided that since this
would be undertaken only once that for the current project, it would be better to focus on other
tasks. Consequently, the tasks that the user interfaces should be able to carry out are limited to
those listed below.
72
6.3.1.1 Initiator User Interface Tasks
Generate IPsec keys to be used to provide a secure connection between the initiator and
target.
Generate Security Association Database and Security Policy Database entries.
Start the iSCSI initiator program.
Login to target(s).
Transfer user data to the target(s).
Retrieve files from the target.
6.3.1.2 Target User Interface Tasks
Create additional target files.
Configure the ietd.conf file with new targets.
Generate Security Association Database and Security Policy Database entries.
Start/stop the iSCSI target program.
Generating IPsec Keys
In order to transfer files in a secure manner, it is necessary to utilize IPSec. The EASI scheme
utilizes a modified version of IPSec for the secure transfer and storage of files. It incorporates a
hard coded key which is used to encrypt the user payload. However, it is still necessary to
generate IPSec keys that will be used in the Security Association Database (SAD) and the
Security Policy Database (SPD), and also create the SAD and SPD entries. This is carried out
manually in the EASI scheme.
Since Security Associations (SA’s) are unidirectional, keys need to be generated for each
direction. These keys need to be shared with both the initiator and the target. The secure sharing
of these keys is not covered in this project.
73
Generate Security Association Database and Security Policy Database entries
Requires that IPSec keys have already been generated and are available for retrieval and
use.
Requires the user to specify both the initiator and target IP addresses.
Requires interaction with the OS, this is achieved through system commands module of
Python.
Start the open-iscsi initiator program
Requires interaction with OS.
Ideally display to the user the targets discovered, so that user can enter the target id to
login to in the next stage. Also acts as a check that the connection has been made.
Login to the target
Requires the user to specify how many target(s) to login to.
Requires the user to specify the targets to which to connect.
Requires interaction with the OS.
Logout from Target
Requires the user to specify the target(s) to logout from.
Requires interaction with the OS.
Transfer files to the target(s)
User must specify whether they want to transfer a file to one or two targets.
User must specify where they are transferring from and where they are transferring to.
Ideally show confirmation that the transfer(s) has/have been successful.
Requires interaction with the OS.
Retrieve files from the target(s)
74
User must specify where they are transferring from and where they are transferring to.
Ideally show confirmation that the transfer(s) has/have been successful.
Requires interaction with the OS.
Create additional target files
Requires user to specify location, name, and size of target file to be created.
Requires interaction with the OS.
Configure the ietd.conf file with new targets
Requires the user to enter the directory containing the target file, and the name of the
target file.
Requires interaction with the OS.
Start/stop the iscsi target program
Requires user to start/stop the target.
Ideally should have a display to show that target has been started or stopped successfully.
A user manual for the user interface is provided in Appendix G.
Chapter 7
75
7. Performance Results and Analysis
In order to determine the impact on performance of the changes proposed in this project, a variety
of tests were carried out. The results are discussed here. The testing was carried out on the virtual
machine test bed which was created from the images of the physical test bed utilized for the EASI
scheme.
7.1 Testing Regime
The tests were divided into three sections:
1. Writing/Reading files to a target storage device using the original sg_dd code. These tests
were carried out to provide a base line against which the other results could be compared.
2. Writing/Reading files to a target storage device using the modified sg_dd code, during
this project, to allow the transfer of files of arbitrary size.
3. Writing/Reading files using the sg_dd code modified, during this project, to allow the
transfer of files to two target storage devices.
All the tests were repeated for writing/reading using no encryption, IPsec encryption, and finally
EASI encryption. The overall testing regime is shown in Table 7-1. The files and results are
available on the DVD accompanying this report.
76
Transfer Type Without encryption With Native IPSec With EASIBlock size file transfer using original sg_dd code
fileorigNoEncry.txt fileorigIPsec.txt fileorigEASI.txt
Arbitrary size file transfer using modified sg_dd code
filearbNoEncry.txt filearbIPsec.txt filearbEASI.txt
Transfer of files to targets using modified sg_dd code
file2targsNoEncry.txt file2targsIPsec.txt file2targsEASI.txt
Table 7-1: Testing Regime
In these tests, performance was determined based on the time to write/read the files to/from the
target storage device. The times were determined using the standard time utility available in the
Linux distribution. This time utility determines three separate time values:
Real time: The elapsed time from the time of invocation of the command, in this case the
sg_dd command to write/read a file, to the time of termination of the command.
User time: The time used by the program/utility (sg_dd in this case) itself and any library
subroutines and calls.
System time: The time used by the system call invoked by the program (directly or indirectly).
7.2 Anticipated Results
Before examining the actual results obtained from the test, the anticipated results are discussed.
These are then compared to the results actually obtained.
7.2.1 Base Line Tests using the Original sg_dd Code
Base Line No Encryption
In this case, it would be anticipated that the time to write/read would increase as the size of the
file increases. The reason for this is that it would be expected for it to take longer to write/read a
77
larger file and would also require that larger files would require the writes/reads to occur across
multiple packets.
Base Line IPsec
In this case, it would be expected that similar behavior to that described above would be seen.
However, because the payloads in each packet used for writing/reading will be encrypted when
being sent out and decrypted when received, it is likely that the transfer times would be higher
than those seen when no encryption is used.
Base Line EASI
In this case, it would again be anticipated that similar behavior to the non-encrypted transfer
would be seen. However, since the EASI scheme encrypts not only the packets containing the file
which is to be transferred but also encrypts the user file within the packet. Thus, the transfer times
would be expected to be higher than not only the non encrypted transfers, but also higher than the
IPsec transfers.
7.2.2 Tests using the sg_dd code modified for arbitrary sized files
In these tests, files of arbitrary size are transferred. It was decided that the files should be less
than their block size equivalents. The changes to the sg_dd code result in size of the file to be
written to the target storage being calculated. The value obtained is then used to calculate and set
the appropriate ‘count’ value. In view of this extra processing that is carried out, it is expected
that in all cases (no encryption, IPsec and EASI) the transfer times may be slightly higher than
the base line values. However, there should still be a trend of increasing transfer times from no
encryption to IPsec to EASI.
78
7.2.3 Testing using sg_dd code modified for two target storage devices
In these tests, block size files are written to two target storage devices. When reading back to the
initiator, the file is read from only one target. Since the file is written to two targets, this involves
additional processing. As a result, it is anticipated that in all cases (no encryption, IPsec and
EASI) the transfer times will be approximately double the time for the base line values. It is
expected that the general trend of increasing transfer times from no encryption to IPsec to EASI
will hold.
7.3 Actual Results
As stated above, when tests were carried out to transfer files of arbitrary size, it was decided that
the files should be less than their block size equivalents. The sizes of the block sized and arbitrary
sized files used are shown in Table 7-2. The speed of the connection used for the transfers was
100Mbps.
Size of File in KB Size of Block File in bytes Size of Arbitrary File in bytes
1K 1024 75010K 10240 9966100K 102400 1021261MB 1048576 104830210MB 10485760 10485486100MB 104857600 104857326Table 7-2: Sizes of various files used during testing
The complete results, for the combined times (write + read) obtained for all of the tests are shown
in Tables 7-3 to 7-8. In the tables, ‘Write’ indicates the transfer of the file from the initiator to the
target and ‘Read’ represents the transfer of a file from the target to the initiator.
79
Write + Read Times with No Encryption sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.03 0 0 0.01 0 0 0.06 0 0.012 0.01 0 0 0.01 0 0 0.06 0 0.023 0.04 0 0 0.03 0 0.01 0.05 0 0.024 0.03 0 0.01 0.02 0 0 0.05 0 0.025 0.02 0 0.01 0.01 0 0 0.04 0 0.026 0.03 0 0 0.02 0 0.01 0.05 0 0.027 0.02 0 0.01 0.01 0 0.01 0.02 0 08 0.05 0 0 0 0 0 0.05 0 0.029 0.01 0 0 0.02 0 0 0.05 0 0.01
10 0.01 0 0 0.03 0 0.02 0.04 0 0.02 Averages 0.025 0 0.003 0.016 0 0.005 0.047 0.000 0.016 Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.03 0 0 0.02 0 0 0.05 0 0.022 0.02 0 0 0.02 0 0 0.03 0 03 0.02 0 0 0.01 0 0 0.03 0 0.014 0.03 0 0 0.01 0 0 0.03 0 0.015 0.02 0 0 0.02 0 0.02 0.05 0 0.016 0.02 0 0 0.02 0 0 0.05 0 07 0.02 0 0 0.02 0 0 0.04 0 0.018 0.02 0 0 0.02 0 0 0.03 0 09 0.02 0 0 0.01 0 0 0.04 0 0.01
10 0.02 0 0 0.02 0 0 0.04 0 0.02 Averages 0.022 0 0 0.017 0 0.002 0.039 0 0.009 Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.05 0 0 0.03 0 0.01 0.04 0 0.022 0.02 0 0 0.02 0 0.01 0.02 0 0.013 0.02 0 0 0.02 0 0.01 0.04 0 0.024 0.02 0 0 0.02 0 0.01 0.02 0 05 0.03 0 0.01 0.03 0 0.02 0.05 0 0.026 0.02 0 0 0.03 0 0.01 0.02 0 07 0.03 0 0 0.02 0 0.02 0.04 0 0.018 0.02 0 0 0.03 0 0.02 0.03 0 0.019 0.02 0 0 0.02 0 0 0.02 0 0.01
10 0.04 0 0.01 0.03 0 0 0.03 0 0 Averages 0.027 0.000 0.002 0.025 0 0.011 0.031 0 0.01
Table 7-3: Combined Write/Read Times for 1K and 750 byte files
80
Write + Read Times with No Encryption sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.12 0 0.02 0.01 0 0.01 0.27 0 0.032 0.06 0 0.01 0.01 0 0.02 0.25 0 0.043 0.1 0 0.02 0 0 0.02 0.22 0 0.044 0.06 0 0.01 0.01 0 0.02 0.21 0 0.035 0.05 0 0.02 0 0 0.02 0.26 0 0.036 0.07 0 0.02 0 0 0.02 0.28 0 0.037 0.07 0 0.01 0 0 0.02 0.25 0 0.038 0.1 0 0.03 0 0 0.02 0.3 0 0.039 0.11 0 0.02 0.003 0 0.02 0.18 0 0.02
10 0.14 0 0.03 0 0 0.01 0.21 0 0.04 Averages 0.088 0 0.019 0.0033 0 0.018 0.243 0 0.032 Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.13 0 0.02 0.1 0 0.02 0.23 0 0.052 0.06 0 0.02 0.09 0 0.02 0.25 0 0.043 0.08 0 0.02 0.15 0 0.02 0.22 0 0.034 0.15 0 0.02 0.09 0 0.01 0.19 0 0.045 0.08 0 0.01 0.09 0 0.02 0.22 0 0.036 0.17 0 0.03 0.13 0 0.02 0.32 0 0.047 0.08 0 0.02 0.1 0 0.01 0.21 0 0.038 0.08 0 0.01 0.09 0 0.01 0.26 0 0.049 0.05 0 0.03 0.11 0 0.03 0.12 0 0.03
10 0.1 0 0.01 0.07 0 0.01 0.31 0 0.04 Averages 0.098 0 0.019 0.102 0 0.017 0.233 0 0.037 Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.08 0 0.02 0.16 0 0.04 0.21 0 0.042 0.07 0 0.03 0.13 0 0.02 0.16 0 0.033 0.13 0 0.03 0.13 0 0.03 0.24 0 0.044 0.13 0 0.04 0.23 0 0.03 0.2 0 0.025 0.08 0 0.04 0.14 0 0.03 0.15 0 0.036 0.08 0 0.04 0.14 0 0.04 0.24 0 0.037 0.12 0 0.03 0.08 0 0.04 0.14 0 0.038 0.13 0 0.02 0.06 0 0.04 0.2 0 0.049 0.07 0 0.02 0.1 0 0.03 0.25 0 0.04
10 0.11 0 0.03 0.15 0 0.04 0.19 0 0.04 Averages 0.1 0 0.03 0.132 0 0.034 0.198 0 0.034
Table 7-4: Combined Write/Read Times for 10K and 9966 byte files
81
Write + Read Times with No Encryption sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.68 0 0.14 0.69 0 0.15 1.75 0 0.282 0.87 0 0.2 0.74 0 0.18 1.74 0 0.253 0.75 0 0.12 0.52 0 0.19 1.51 0 0.234 0.57 0 0.12 0.59 0 0.15 1.69 0 0.215 0.67 0 0.2 0.57 0 0.11 1.47 0 0.236 0.59 0 0.18 0.61 0 0.14 1.72 0 0.237 0.68 0 0.14 0.61 0 0.11 1.75 0 0.218 0.56 0 0.2 0.51 0 0.08 1.63 0 0.249 0.63 0 0.12 0.72 0 0.13 1.51 0 0.27
10 0.64 0 0.17 0.61 0 0.12 1.55 0 0.27 Averages 0.664 0 0.159 0.617 0 0.136 1.632 0 0.242 Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.88 0 0.15 1.11 0 0.21 1.49 0 0.312 0.89 0 0.19 0.75 0 0.11 1.4 0 0.223 0.66 0 0.14 0.73 0 0.14 1.49 0 0.254 0.71 0 0.12 0.78 0 0.2 1.54 0 0.215 0.77 0 0.22 0.78 0 0.21 0.98 0 0.186 0.8 0 0.13 0.71 0 0.26 1.27 0 0.217 0.72 0 0.21 0.69 0 0.18 1.47 0 0.238 0.83 0 0.16 0.73 0 0.19 1.53 0 0.229 0.83 0 0.27 0.9 0 0.25 1.28 0 0.2
10 0.99 0 0.2 0.88 0 0.16 1.31 0 0.26 Averages 0.808 0 0.179 0.806 0 0.191 1.376 0 0.229 Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 0.78 0 0.15 0.85 0 0.27 1.18 0 0.232 1.01 0 0.28 1 0 0.26 1.08 0 0.223 0.93 0 0.15 0.96 0 0.22 1 0 0.234 0.73 0 0.17 0.86 0 0.25 1.05 0 0.25 0.69 0 0.14 0.86 0 0.21 1.01 0 0.236 0.74 0 0.11 0.87 0 0.26 0.93 0 0.247 0.74 0 0.29 0.8 0 0.27 1.21 0 0.28 0.6 0 0.23 0.86 0 0.25 1.01 0 0.179 0.69 0 0.13 0.9 0 0.26 1.38 0 0.22
10 0.75 0 0.22 1.15 0 0.21 1.06 0 0.15 Averages 0.766 0.000 0.187 0.911 0 0.246 1.091 0 0.209
Table 7-5: Combined Write/Read Times for 100K and 102126 byte files
82
Write + Read Times with No Encryption sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 4.66 0 0.81 4.17 0 0.82 16.14 0 2.782 4.75 0 0.8 3.96 0 0.79 15.79 0 2.83 4.51 0 0.69 4.03 0 0.7 15.68 0 2.94 4.32 0 0.75 4.17 0 0.69 12.5 0 1.685 4.49 0 0.74 3.85 0 0.6 9.04 0 1.216 4.57 0 0.77 3.94 0 0.68 9.43 0 1.147 4.52 0 0.72 4.05 0 0.72 9.12 0 1.38 4.59 0 0.73 3.96 0 0.5 9.26 0 1.429 4.43 0 0.83 4.25 0 0.73 9.34 0 1.19
10 4.6 0 0.78 3.95 0 0.62 9.48 0 1.31 Averages 4.544 0 0.762 4.033 0 0.685 11.578 0 1.773 Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 5.5 0 0.95 5.4 0 1.03 11.27 0 2.142 5.23 0 0.96 4.88 0 0.94 10.38 0 1.763 5.22 0 0.85 4.89 0 1.14 10.59 0 1.764 5.27 0 1.05 4.72 0 0.99 10.93 0 1.845 5.61 0 1.02 4.91 0 1.03 10.72 0 1.886 5.38 0 1.04 5.15 0 1.13 10.22 0 1.837 5.29 0 0.85 4.91 0 0.83 11.05 0 1.778 5.26 0 0.97 4.77 0 0.96 10.77 0 1.929 5.26 0 0.96 4.76 0 1.05 10.86 0 1.76
10 5.79 0 0.96 4.86 0 0.95 11.31 0 1.62 Averages 5.381 0 0.961 4.925 0 1.005 10.81 0 1.828 Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 5.08 0 1.43 5.82 0 1.18 8.71 0 1.392 5.17 0 1.21 4.31 0 1.01 8.38 0 1.63 5 0 1.37 4.19 0 0.93 8.69 0 1.484 5.29 0 1.55 4.32 0 0.81 8.54 0 1.565 5 0 1.22 4.68 0 1.06 8.51 0 1.386 5.21 0 1.34 4.27 0 0.76 9 0 1.657 4.99 0 1.28 4.33 0 0.93 8.47 0 1.578 5.32 0 1.3 4.27 0 0.92 8.48 0 1.369 5.22 0 1.55 4.27 0 0.86 8.81 0 1.47
10 5.16 0 1.32 4.5 0 1.12 8.49 0 1.54 Averages 5.144 0.000 1.357 4.496 0 0.958 8.608 0 1.5
Table 7-6: Combined Write/Read Times for 1MB and 1048302 byte files
83
Write + Read Times with No Encryption sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 44.91 0 7.29 43.93 0.01 6.58 92.4 0.03 12.542 45.65 0 7.47 40.04 0.01 6.36 91.48 0.02 133 46.11 0.01 7.92 40.05 0 6.36 89.65 0.02 12.714 46.56 0 7.45 40.43 0 6.32 87.04 0.02 12.965 47.23 0.01 7.65 40.82 0 6.5 77.15 0.02 12.356 47.37 0.01 7.75 41.97 0.01 6.65 72.73 0.03 12.767 48.61 0.01 7.67 42.12 0.01 7.14 72.57 0.01 12.628 48.35 0 7.43 42.33 0.01 7.04 73.4 0.01 12.539 48.93 0.01 7.71 43.33 0.01 6.45 74.14 0.02 12.56
10 49.2 0 7.62 44.01 0.01 6.54 74.86 0.02 12.15 Averages 47.292 0.005 7.596 41.903 0.007 6.594 80.542 0.02 12.618 Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 53.9 0 9.61 55.02 0.02 10.87 117.74 0.03 20.712 54.36 0.01 10.36 49.48 0.01 9.71 108.97 0.02 18.233 53.97 0 9.73 50.16 0.01 9.55 110.02 0.03 18.574 54.44 0 9.74 50.16 0.01 9.97 110.73 0.03 18.625 53.22 0.01 10.45 50.93 0.01 9.73 110.56 0.03 18.76 52.54 0.01 9.77 51.23 0 9.87 109.18 0.03 18.697 51.23 0.01 9.94 52.72 0 10.2 104.18 0.01 18.048 47.94 0 9.96 52.25 0.01 9.47 91.02 0.01 18.119 42.65 0.01 9.75 53.19 0.01 10.63 85.57 0.02 18.04
10 42.26 0.01 10.28 52.91 0 9.54 86.39 0.01 17.48 Averages 50.651 0.006 9.959 51.805 0.008 9.954 103.436 0.022 18.519 Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 TargetsRun Real User System Real User System Real User System
1 55.93 0 10.25 46.83 0.01 9.36 89.78 0.02 16.932 56.07 0.01 10.84 43.12 0.01 8.86 87.39 0.02 16.063 55.53 0 11.35 43.44 0.01 9.44 89.08 0.02 16.044 54.3 0.01 11.01 43.91 0.01 9.15 91.46 0.02 16.095 52.87 0.01 10.52 43.8 0.01 9.34 94.23 0.01 16.16 49.41 0.01 10.19 44.46 0 8.84 97.19 0.03 16.367 44.07 0.01 10.38 45.26 0 9.49 98.56 0.02 16.178 43.21 0 10.09 45.22 0.01 8.98 101.12 0.02 16.079 43.48 0.01 10.59 46.19 0.01 8.97 103.68 0.02 16.46
10 43.53 0 10.52 45.58 0 8.96 104.16 0.03 16.14 Averages 49.840 0.006 10.574 44.781 0.007 9.139 95.665 0.021 16.242
Table 7-7: Combined Write/Read Times for 10MB and 10485486 byte files
84
Write + Read Times with No Encryption
85
sg_dd Original sg_dd arbitrary sg_dd 2 Targets
Run Real User System Real UserSystem Real User System
1 477.52 0.08 76.28 510.53 0.18 71.28 829.99 0.28 126.152 386.25 0.09 73.99 479.09 0.14 69.69 843.02 0.24 127.823 407.47 0.08 74.58 385.75 0.1 67.86 882.75 0.31 1354 460.95 0.09 76.98 399.97 0.13 67.45 1209.94 0.4 158.315 488.02 0.09 76.09 442.19 0.09 66.98 874.01 0.28 128.496 397.61 0.1 75.64 481.7 0.12 70.79 856.18 0.25 127.817 400.63 0.09 76.49 453.24 0.14 69.86 839.05 0.25 127.938 445.9 0.1 73.99 387.59 0.14 68.61 881.88 0.23 128.029 489.64 0.11 77.93 416.8 0.13 67.75 815.08 0.26 127.16
10 413.99 0.11 75.67 489.61 0.14 69.53 909.21 0.26 128.72 Averages 436.798
0.094 75.764 444.647
0.131 68.98 894.111
0.276 131.541
Write + Read Times with IPsec sg_dd Original sg_dd arbitrary sg_dd 2 Targets
Run Real User System Real UserSystem Real User System
1 427.07 0.1 100.95 531.17 0.16 99.18 1063.47 0.33 193.762 465.13 0.13 100.23 423.24 0.15 94.94 1006.67 0.29 179.233 529.47 0.11 103.48 451.64 0.14 96.98 1025.16 0.31 179.614 470.19 0.09 101.43 509.78 0.15 98.7 927.48 0.31 175.015 431.2 0.11 97.89 523.76 0.12 96.68 1028.46 0.3 176.16 501.96 0.12 96.92 425.41 0.15 94.37 904.8 0.28 168.037 533.07 0.11 100.8 446.88 0.14 96.45 1013.99 0.29 166.158 437.74 0.1 100.19 503.88 0.17 97.02 865.3 0.3 163.089 431.36 0.09 91.42 528.01 0.12 98.16 1012 0.27 162.84
10 485.83 0.1 91.57 432.96 0.14 96.88 878.44 0.31 165.45 Averages 471.302
0.106 98.488 477.673
0.144 96.936 972.577
0.299 172.926
Write + Read Times with EASI sg_dd Original sg_dd arbitrary sg_dd 2 Targets
Run Real User System Real UserSystem Real User System
1 440.48 0.12 104.16 555.75 0.17 99.12 937.83 0.27 167.792 491.03 0.12 106.5 540.69 0.16 94.04 973.37 0.27 163.183 564.72 0.13 113.42 435.33 0.12 92.09 915.01 0.26 1614 452.04 0.11 102.46 445.59 0.13 91.94 989.46 0.29 161.735 452.19 0.09 104.87 503.39 0.11 93.5 879.71 0.25 158.16 518.46 0.11 105.5 541.35 0.14 95.35 979.43 0.28 160.277 535.34 0.11 106.37 438.33 0.12 91.05 879.26 0.24 158.828 432.81 0.11 103.05 442.62 0.11 90.76 994.46 0.27 162.299 466.23 0.13 102.41 498.66 0.13 91.1 876.37 0.28 157.17
10 536.65 0.12 104.36 537.96 0.13 91.45 985.29 0.25 159.65 Averages 488.995
0.115 105.310 493.967
0.132 93.04 941.019
0.266 161
Table 7-8: Combined Write/Read Times for 100MB and 104857326 byte files
86
7.3.1 Analysis of the Results from the Base Line Tests
Figures 7-1 through 7-3 show the average time results obtained for the base line tests described
previously in this chapter.
In Figure 7-1, the average values obtained for the times when writing/reading files using the
original sg_dd code and no encryption are presented. For both the real and system times, an
increase in the values is observed as the size of the file is increased. Similar behavior is seen with
the user time for the 10MB and 100MB files, while for the other files zero time values were
recorded.
Figure 7-2 shows the average time values obtained when writing/reading files using the original
sg_dd code with IPsec encryption used. Examination of the results revealed similar behavior to
that described when no encryption is used. However, when comparing Figures 7-1 and 7-2, it can
be seen that in all cases the time values are higher than those seen when no encryption is used.
In Figure 7-3, the average time values obtained when writing/reading files using the original
sg_dd code with EASI encryption are presented. Again, similar behavior to that observed in the
previous two tests was observed, with times increasing as the file size increased. When compared
to Figures 7-1 and 7-2, it can be seen that the values for the times are higher.
87
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-1: sg_dd Original No Encryption
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-2: sg_dd Original IPsec
88
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-3: sg_dd Original EASI
7.3.2 Analysis of the Results from the Tests using the sg_dd Arbitrary Code
Figures 7-4 to 7-6 show the average time results obtained for files of arbitrary size. Comparison
of the data in these graphs reveals similar trends to those seen with the base line results. As the
file sizes increase, so do the times for writing/reading.
When comparing the values obtained when no encryption was used to those when native IPsec
was used, it was found that the times for writing/reading were slightly higher when IPsec was
employed, except for the 1K- and 10K- files.
Comparing the times when using native IPsec and EASI, it was found that the times were slightly
higher when using EASI, except for the 1MB-, 10MB- and 100MB- files.
89
1K- 10K- 100K- 1MB- 10MB- 100MB-1
10
100
1000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-4: sg_dd Arbitrary No Encryption
1K- 10K- 100K- 1MB- 10MB- 100MB-1.000
10.000
100.000
1000.000
RealUserSystem
Figure 7-5: sg_dd Arbitrary IPsec
90
1K- 10K- 100K- 1MB- 10MB- 100MB-1
10
100
1000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-6: sg_dd Arbitrary EASI
7.3.2.1 sg_dd Original vs sg_dd arbitrary
Figures 7-7 to 7-9 show the real, user, and system times for writes/reads using the original sg_dd
code and arbitrary sg_dd without encryption, using IPsec and EASI respectively.
When comparing the real time values obtained in Figures 7-7 to 7-9, it was found that in all cases
the time values recorded were higher when using the code which had been modified to simulate
the transfer of arbitrary sized files. No significant differences between the values were noted in
when using IPsec or EASI.
Examination of the system values revealed that when no encryption was used the times recorded
for the original code were slightly higher than for the code modified to simulate the transfer of
arbitrary sized files, with the exception of the 1K file. In the cases where IPsec and EASI were
used, the times recorded were greater than when using no encryption, but a definitive trend could
be identified between IPsec and EASI.
91
In the case of the user times, in all cases the values recorded were higher when using the code
modified for simulating the transfer of arbitrary files. Since only two values were available, it was
not possible to determine a definitive trend when using no encryption, IPsec, or EASI.
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.005
0.018
0.136
0.685
6.594
68.980
0.003
0.019
0.159
0.762
7.596
75.764
0.00700000000000001
0.131
0.00500000000000001
0.094
0.016
0.087
0.617000000000007
4.033
41.903
444.647000000002
0.025
0.088
0.664
4.544
47.292
436.798
orig real arb real orig user arb user orig system arb system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-7: Comparison of time values for the original and arbitrary sg_dd code without encryption
92
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.00200000000000001
0.017
0.191
1.00499999999998
9.954
96.936
0.019
0.179
0.961
9.959
98.488
0.00800000000000002
0.144
0.00600000000000001
0.106
0.017
0.102
0.806
4.925
51.805
477.673
0.022
0.098
0.808
5.381
50.651
471.302
orig real arb real orig user arb user orig system arb system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-8: Comparison of time values for the original and arbitrary sg_dd code with IPsec
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.0110.034
0.246
0.958
9.139
93.040
0.002
0.030
0.187
1.357
10.574
105.310
0.00700000000000001
0.132
0.00600000000000001
0.115
0.025
0.132
0.911
4.496
44.781
493.966999999997
0.027
0.100
0.766
5.144
49.840
488.995
orig real arb Real orig user arb user orig system arb system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-9: Comparison of time values for the original and arbitrary sg_dd code with EASI
93
7.3.3 Analysis of the Results from Tests using the sg_dd Two Targets Code
Figures 7-10 to 712 show the average time results obtained when writing/reading files to/from
two targets. Comparison of the data in these graphs revealed similar trends to those seen with the
base line results. As the file sizes increased, so did the times for writing/reading. The write/read
times also increased with the use of IPsec encryption and EASI encryption. It was noted that in
comparison to the original sg_dd code, the times taken to write/read the files approximately
doubled. This was expected since the base line test were for writing/reading to/from one target.
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-10: sg_dd Two Targets No Encryption
94
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-11: sg_dd Two Targets IPsec
1K 10K 100K 1MB 10MB 100MB1.000
10.000
100.000
1000.000
RealUserSystem
File Size
Tim
e (s
econ
ds)
Figure 7-12: sg_dd Two Targets EASI
95
7.3.3.1 sg_dd Original vs sg_dd Two Targets
Figures 7-13 to 7-15 compare the real, user, and system times for writes/reads using the sg_dd
code modified for transfer to two targets, and the original sg_dd code without encryption, using
IPsec and EASI respectively. It should be noted that the values obtained from the base line tests
using the sg_dd original code were doubled in order to provide a true comparison.
When comparing the real time values obtained in Figures 7-13 to 7-15, it was found that in the
majority of cases, the times recorded were slightly greater when using the code modified for
transferring to two targets.
In the case of the system times, the contrary was found to be true. The majority of the time values
were slightly lower when using the code modified for transferring to two targets. In the cases of
the real and system times, the differences between the times resulting from the use of the two
different sg_dd codes, did not appear to be very significant. However, when examining the user
times, all values recorded for the modified code were higher than the original code. In addition,
there appeared to be a general trend of decreasing time differences between the values as
encryption was applied, with the least differences noted with the times using the EASI scheme.
That being said, in each case only two pairs of non-zero values were obtained, so this trend
cannot be confirmed or refuted.
96
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.0160.032
0.242
1.773
12.618
131.541
0.006
0.038
0.318
1.524
15.192
151.528
0.02
0.276
0.01
0.188
0.047
0.243
1.632
11.578
80.542
894.111
0.0500.176
1.328
9.088
94.584
873.596
orig real 2 targets real orig user2 targets user orig system 2 targets system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-13: Comparison of times values for the original and two target sg_dd codes with no encryption
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.009
0.037
0.229
1.828
18.519
172.926
0.038
0.358
1.922
19.918
196.976
0.022
0.299
0.012
0.212
0.039
0.233
1.376
10.810
103.436
972.577
0.044
0.196
1.616
10.762
101.302
942.604
orig real 2 targets real orig user 2 targets user orig system 2 targets system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-14: Comparison of times values for the original and two target sg_dd code with IPsec
97
1E+02. 1E+03. 1E+04. 1E+05. 1E+06. 1E+07. 1E+08. 1E+09.0.001
0.010
0.100
1.000
10.000
100.000
1000.000
0.004
0.062
0.374
2.204
21.148
210.620
0.0100.034
0.209
1.500
16.242
161.000
0.021
0.266
0.012
0.23
0.031
0.198
1.091
8.608
95.665
941.019
0.054
0.284
1.532
11.360
99.680
977.990
orig real 2 targets real orig user2 targets user 2 targets system orig system
File Size (bytes)
Tim
e (s
econ
ds)
Figure 7-15: Comparison of times values for the original and two target sg_dd code with EASI
7.4 Summary of Results
The actual results obtained in general confirm the anticipated results with a few
exceptions. These exceptions may be the result of the relatively small sample size used
for the tests. However, it does not appear that the changes to the code for either the files
of arbitrary size or for the transfer of files to two targets have a significantly detrimental
effect on the overall performance.
98
Chapter 8
8. Additional Research
The results obtained from testing the EASI implementation (chapter 5) revealed a number of
limitations. One of the most significant was the inability to use the implementation with a
mounted target storage disk. In his thesis, Mr. Andukuri briefly mentioned that during the early
stages of his work he “attempted make the target disk accessible to the initiator by using the
‘mount’ command to mount the target iSCSI disk at a mount point on the initiator” [1-11].
However, he stated that the mount command resulted in “a counter-intuitive sequence of iSCSI
packets” [1-11]. As a result, he chose to use a regular file on the target machine as a storage
medium.
The ability to mount a target iSCSI disk on an initiator is highly desirable in a real-world
environment. It would also assist in using the cp command for files of arbitrary size. In order to
more fully understand the problems and proposed potential solutions, it was decided to examine
the network protocol analyzer log obtained when the mount command is issued. Since it is known
that both the iSCSI ‘Discovery’ and ‘Login’ processes are successful with the EASI scheme, it
was decided to obtain network protocol analyzer logs for these tasks for comparison purposes.
8.1 New Virtual Machine Test Bed
One of the things noted when testing the EASI implementation was that significantly updated
versions of the Linux kernel, open-iscsi and iscsi-target software were available. Since any future
developments on this project should take advantage of such updates, it was decided to create a
new test bed with more recent versions of the Linux kernel, initiator and target software. This test
bed was then used for producing the various network protocol analyzer logs.
99
The new test bed was created on a VMware Server using Fedora 8 with the Linux kernel
2.6.23.17. The initiator and target were implemented using the Open-iscsi-2.0.869.2 and
iscsitarget-0.4.16. The installation and operation of Open-iscsi-2.0.869.2 and iscsitarget-0.4.16
software are presented in Appendix H and I respectively. An additional hard disk was added to
the target machine to act as the target storage.
Once the test bed had been created, attempts were made to update the EASI scheme for use with
the new kernel code. However, since the kernel code had undergone significant changes, this was
not a straight forward task. In view of the fact that this had not been intended as the main focus of
the project, attempts at the modification ceased and it is proposed as future work.
8.2 Network Protocol Analyzer Logs
Figures 8-1 to 8-5 show the various logs obtained during the testing. The full log files are
available on the DVD accompanying this report. Examination of the logs was simplified by the
fact that the EASI scheme only encrypts/decrypts a packet if it is identified as an iSCSI packet
and carries a user payload. If these two criteria are not met then the packet is processed using the
native IPsec scheme. Thus, when attempting to determine processes that may be influenced by
the EASI scheme, it is only necessary to look for packets containing iSCSI ‘Writes’ and ‘Reads’
with associated user payload.
8.2.1 Discovery Process
Figure 8-1 shows the log obtained during the iSCSI ‘Discovery’ process. Examination of this log
revealed that only iSCSI commands were issued. The commands were ‘Login’, ‘Login
Response’, ‘Text Command’ and ‘Text Response’. Therefore, none of these commands would be
affected by the EASI scheme. This was anticipated, since when using the EASI scheme, no
difficulties were encountered when carrying out the Discovery process.
100
Figure 8-1: Wireshark Log obtained during the Discovery Session
8.2.2 Login Process
The log obtained during the iSCSI ‘Login’ session is shown in Figure 8-2. When examining this
log, it was noted that considerably more packets were transferred than when carrying out the
Discovery process. Examination of the log revealed that along with iSCSI ‘Login’ and ‘Login
Response’ commands, a variety of iSCSI/SCSI commands were issued. These included ‘Inquiry’,
‘Test Unit Ready’, ‘Response’, ‘Read Capacity’, ‘Mode Sense’, ‘Report Lun’, Data-in PDU and
‘Read’.
101
The presence of the iSCSI/SCSI ‘Read’ command containing packets followed by plain vanilla
TCP packets, followed by iSCSI/SCSI Data-in PDUs would on initial examination suggest that
the packets may be processed by the EASI implementation. However, examination of the TCP
header in the plain vanilla TCP packets revealed that the ‘push’ (PSH) flag was not set. As has
already been stated in this report, the EASI scheme utilizes the setting of PSH flag to identify
TCP packets containing user data following iSCSI/SCSI ‘Read’ commands. In view of this, it
would appear that although an iSCSI/SCSI ‘Read’ occurs during the login process, the non-
setting of the PSH flag would suggest that the packet does not undergo processing with the EASI
scheme. Hence, login is successful with the EASI scheme.
Figure 8-2: Wireshark Log obtained during the Login Session
102
8.2.3 Mounting the Target Storage Device File System on the Initiator
Figure 8-3 shows the log obtained when mounting the target storage device on the initiator. The
packets transmitted were iSCSI/SCSI ‘Reads’, plain vanilla TCP packets and their associated
iSCSI/SCSI Data-in PDUs. These were followed by an iSCSI/SCSI ‘Write’, ‘Ready to Transfer’
command, iSCSI/SCSI Data-out PDUs and an iSCSI/SCSI ‘Response’. The final packets in the
log are an iSCSI/SCSI ‘Read’, plain vanilla TCP packets and an iSCSI/SCSI Data-in PDU.
Examination of the TCP header in the TCP packets following the iSCSI/SCSI ‘Read’ packets
revealed that the ‘push’ (PSH) flag was not set. Some of the TCP packets contained what
appeared to be spurious data as did some of the iSCSI/SCSI Data-in PDUs.
One packet of particular interest is packet number 137. The data segment of this packet contained
the names of the files and directories on the target storage device. This can be seen in Figure 8-3.
It should be noted that even with this packet the PSH flag was not set in the TCP header. In view
of the fact that the PSH flag is not set in any of the TCP packets associated with iSCSI/SCSI
‘Reads’, Therefore, it would seem that these packets would not undergo processing by the EASI
scheme.
When examining the Data-out PDUs following the iSCSI/SCSI ‘Write’ packets, it was found that
they contained spurious data. It could be expected that they would be processed by the EASI
scheme leading to encryption of the data when this is not what is required when mounting the
target storage media.
In order to understand why iSCSI/SCSI ‘Writes’ and ‘Reads’ occur, it is necessary to consider
what happens when a mount command is issued. The ‘mount’ command used to mount a target
storage disk to the initiator is a file system level operation which is handled by the Virtual File
System (VFS) in kernel space. When the user issues a ‘mount’ command, amongst other things,
103
the kernel reads the file system information from the partition and sets up file system descriptors.
In the case of mounting a storage device resident on the target machine, read and write commands
may be called by functions invoked by the mount command. Since the iSCSI connection is the
only connection between the initiator and target, these commands can result in iSCSI/SCSI
packets being constructed and exchanged between initiator and target. Hence the appearance of
iSCSI/SCSI ‘Writes’ and ‘Reads’ and their associated Data-in and Data-out PDUs in the mount
log file.
More information about the system calls issued when using the mount command can be obtained
by using ‘strace’. strace was run when mounting the target storage device to the initiator. The file
obtained (mount.txt) is available on the DVD accompanying this report. Examination of the file
revealed that many system calls are made when the ‘mount’ command is issued and that some of
these may well be routed through the iSCSI layer resulting in the production of iSCSI/SCSI
‘Reads’, ‘Writes’ etc.
104
Figure 8-3: Wireshark Log obtained during Mounting the Target Storage Device File System on the Initiator
8.3 Writing to Mounted Target Storage using cp
The log obtained when writing a file to a mounted target storage disk using the cp command is
shown in Figure 8-4. Examination of the log revealed that three iSCSI/SCSI ‘Write’s’ occurred
and with each of these were associated a number of Data-out PDUs. Table 8-1 provides a
breakdown of the contents of the Data-out PDUs for each of the ‘Writes’. Examination of the
various PDUs revealed some interesting information. It was found that on some occasions the
105
SCSI payload contained a ‘Data’ field as can be seen in Figure 8-5. The reason for the
appearance of this is not known and requires further investigation.
As was the case with the mount command, the cp command results in a system call being placed
to the kernel, when this is processed by the kernel a number of other system calls can be invoked.
This was found to be the case when strace was used when executing the cp command to write a
file to the target storage disk.
Figure 8-4: Writing to mounted target storage using cp
106
Packet Numbers from 1st iSCSI/SCSI Write
Data-out PDUs
Packet Numbers from 2nd
iSCSI/SCSI Write Data-out PDUs
Packet Numbers from 3rd
iSCSI/SCSI Write Data-out PDUs
User Data as payload
11 None None
No payload and SCSI Data Field
14, 15, 16 25, 28, 31, 37, 40, 43, 49, 52, 55, 61, 64,
67,73, 76, 79, 85, 88, 91, 94, 97, 100, 103
111, 113, 114
Spurious payload and no SCSI Data
Field
None 22 None
Possible File System Content as Payload
and SCSI Data Field
None 34, 70, 82 None
Spurious payload and SCSI Data Field
None 46, 58 None
No payload and no SCSI Data Field
None None 109
Table 8-1: Packets for copying file to mounted storage disk
Figure 8-5: Data field in SCSI payload contained within a Data-out PDU
107
8.4 Reading from Mounted Target Storage using cp
The log obtained when reading a file from a mounted target storage disk using the cp command is
shown in Figures 8-6 and 8-7. The initial iSCSI/SCSI command was a ‘Read’, the plain vanilla
TCP packet following it contained the names of the directories and files currently on the target
storage disk. This can be seen in Figure 8-6. This iSCSI/SCSI ‘Read’ was followed by three
more ‘Reads’. The user data was found in a TCP packet associated with the second of these
additional three ‘Reads’. This is shown in Figure 8-7.
In addition to the ‘Reads’, there were two iSCSI/SCSI ‘Writes’ and associated Data-out PDUs. In
some of the Data-out PDUs, the payload was empty while in others there was data. However, the
data could not be identified.
108
Figure 8-6: Contents of TCP packet following the first iSCSI/SCSI ‘Read’
109
Figure 8-7: Reading from mounted target storage using cp
8.5 Summary
From the above discussion, it should be obvious that some way must be found to differentiate
between iSCSI/SCSI ‘Write’s’ and potentially iSCSI/SCSI ‘Read’s’, and their associated Data-in
and Data-out PDUs sent when mounting a target and when writing or reading a file to/from the
target storage using the cp command.
It is believed that examination of the system calls issued in when mounting the target, writing and
reading user data with the cp command may assist in identifying why some of the iSCSI/SCSI
‘Writes’ and ‘Reads’ appear .
110
Chapter 9
9. Disaster Recovery
As a society, we have become increasingly reliant on computers and associated technologies for
both business and pleasure. As such, loss of computer services can be anything from a nuisance
for a home user to a major problem for businesses.
Loss of computer services can result from major disasters, which hit the headlines, such as
hurricanes, floods, and fires. However, disasters can also encompass what appear to be innocuous
events such as planned and unplanned downtime. Planned downtime has the potential to be
considered a disaster if it exceeds the allotted time planned. Unplanned downtime can result from
such things as user errors, application errors, application and hardware failures, network outages,
false alarms, etc. The reality is that these events, which constitute 95% of what are considered
disasters in computer terms, can result in negative impacts on businesses in terms of trading,
profits, and failure to comply with legal requirements.
9.1 Disaster Recovery Plan
Obviously, since the impact of loss of computer services can be wide ranging, it is essential that
every business and organization have a disaster recovery plan. Such a plan will allow a business
to deal with potential disasters and minimize the impact of a disaster should it occur. The success
or failure of disaster recovery depends to a certain extent on the time take between a disaster and
the restoration of services. In the past, backups of data were made to onsite storage facilities.
However, events such as the disaster at the World Trade Center and the devastation caused by
111
Hurricane Katrina have highlighted the fact that in the case of a catastrophic event, onsite back up
will be totally ineffective.
In order to develop a disaster recovery plan, a business or organization must first carry out a risk
assessment. This enables the identification of potential risks, and their significance in terms of the
business or organization. The exact nature of the disaster recovery plan will vary from business to
business, and is beyond the scope of this project. However, it should incorporate a restart solution
to enable the timely restoration of services, and the total replication of all data and other vital
records.
9.1.1 Restart Solutions
The restart solution chosen will depend upon the nature of the business or organization for which
the disaster recovery plan has been developed. A number of restart solutions have been identified
by Hiles [1-4]. These include fortress, cold site, warm site, hot site, fault tolerant systems, and
continuous operation.
Fortress
A fortress restart solution is defined as one in which management seek to limit risk to a level
where it is decided any further recovery plan is unnecessary.
Cold Site
The cold site solution “provides an environment in which a new computing service can be built
from scratch in time” [1-4]. Obviously, for most businesses the cold site solution is not a viable
solution that can be adopted if they are to stay in business. The reason for this is the length of
time that would be required to reestablish the necessary services. For most
businesses/organizations the only truly viable solutions are warm site, hot site, fault tolerances, or
continuous operation.
112
Warm Site
A warm site solution provides both the environment and basic infrastructure that will allow the
necessary “computing services to be reinstated before its absence becomes critical to business
survival” [1-11].
Hot Site
A hot site is effectively a duplication of the primary computing system located at another site.
However, it does not have all the applications installed upon it.
Fault Tolerant System
A fault tolerant system seeks to eliminate hardware failure through replication of components and
potentially data.
Continuous Operation
Continuous operation provides continuous shadowing or mirroring of the system at an alternate
site. For the purposes of this project, interest is limited to the backup or mirroring of data.
9.2 Secure Asymmetric iSCSI for Disaster Recovery
The advent of online storage has somewhat simplified the backing up of computer data through
mirroring, etc at a remote location. However, with the new regulations of HIPPA and SOX, the
duplication of the data may no longer be sufficient. The backed up data needs to be securely
stored – this is where the Secure Asymmetric iSCSI for Online Storage scheme could be utilized.
However, the problem that needs to be resolved in terms of disaster recovery is that of the key to
decrypt the stored data. In the current implementation, the data is stored on the target in an
encrypted form. As a result, if the initiator is destroyed in a disaster, it will no longer be possible
to decrypt the data that is safely stored on the target, which in effect makes the data useless.
113
Potential solutions are:
1. Duplicating the initiator and consequently custom encryption/decryption key at a
secondary location.
2. Storing the key with a trusted third party such as Verisign.
A potential problem with the first solution is that the custom key is now available at two
locations. Thus the security is in effect diluted.
The second solution would appear to be a better option. However, this can also be problematic in
that the third party must be trusted but more significantly, in the event of a disaster it may take
some time to recover the key from the third party. The time taken may not be acceptable in terms
of the delay of restoring availability of data.
Since neither of the above solutions is ideal, this is certainly an area which needs further
consideration.
114
Chapter 10
10. Lessons Learnt and Observations
During the course of this project a number of lessons have been learnt and a number of
observations have been noted. These are presented here.
The sg_dd code uses defaults of standard input stream (stdin) for the input file (if) and
standard output stream (stdout) for the output file (of) if values are not provided on the
command line. In addition, if no data is to be written an output file of /dev/null must be
specified on the command line. This information was utilized in the enhancements to the
code for transferring files to two targets. The enhancements to the code require that two
targets be specified. Thus, when reading from a target back to the initiator the second
output file (of_1) is given a default value of /dev/null.
During the project, both Wireshark and its earlier incarnation Ethereal were used to
collect and examine network traffic. It was noted that when using the different versions to
examine the same file, different packets were displayed. It appears that the different
versions interpret/display packets in a different manner. In order to avoid confusion, and
since Wireshark is the currently available version, it was used to examine and compare
all network traffic logs.
When an iSCSI connection is established between an initiator and target, commands
issued to mount a target storage disk, or write/read user data using the cp utility, result in
additional calls occurring in the kernel. Some of these are subsequently passed to the
iSCSI layer for processing. As a result, additional iSCSI/SCSI ‘Write’ and ‘Read’
commands may be issued along with their associated Data-in and Data-out PDUs, and
TCP packets.
115
During efforts to develop the arbitrary file transfer implementation, the author attempted
to enhance the code such that only the user data present in the target file would be
retrieved. In order to do this, it was necessary to determine the number of bytes of user
data stored in the target file. From this, the count value required by the sg_dd command
could be determined. A number of problems were encountered which made these
attempts unsuccessful. As stated, the target storage used in the EASI implementation is a
file. Therefore, using commands such as ‘stat’ to determine the size, returns the size of
the entire storage file and not just the user data it contains. Attempts to determine the size
of the user data by identifying trailing white space were also unsuccessful. Encryption of
the user payload, results in a false value of the number of bytes being returned when
attempts are made to read the number of characters.
During the course of this project, the author attempted to update the EASI
implementation for use on a more recent version of the Linux kernel and to utilize the
most current versions of open-iscsi and iscsitarget. On examining recent versions of the
Linux kernel (2.6.21 through 2.6.24), it was found that significant changes have been
made to the kernel code, including the IPsec code. As a result, significant rewriting of
Mr. Andukuri’s code would be necessary. Although efforts were made to amend the
code, it was found not to be straightforward. After considerable effort, it was decided that
these changes should be made outside this project. This observation highlights an
important point, any future developments on this project should take into consideration
changes to the kernel code. If the implementation is to ultimately be utilized in a real
world environment, then ideally it should be included in the kernel tree.
116
Chapter 11
11. Future Directions
Although advances have been made in this project, it still remains a proof of concept. The work
undertaken during this project has identified a number of areas for future work. The suggested
future directions are presented here.
Modify the EASI scheme to use the most current Linux kernel version, open-iscsi and
iscsitarget code. This will necessitate examination of the current Linux kernel code and
the kernel code used in the existing EASI scheme to determine the changes made that
affect the encryption process.
Re-implement the EASI scheme in a way that mounted target storage disk can be used. In
view of the findings of the additional research, this will require an in-depth study of the
additional tasks performed by the kernel to execute the mount command. It will also
require a different method to be used to identify iSCSI/SCSI packets related to the mount
command, as well as enhancing the re-implemented scheme to allow the transfer of files
of arbitrary size. It is suggested that in addition to changes to the IPsec code, this may
require changes to the SCSI and iSCSI implementations, and/or redevelopment of the cp
command specifically for use with iSCSI. This may require the code to construct SCSI
CDBs as is the case in the sg_dd code. It is anticipated that when the cp command is
issued, the file size will have to be calculated, and any padding required by the chosen
encryption algorithm will also need to be calculated. This information may then be added
into the SCSI CDB thus preventing the target from rejecting the packet.
117
Improve the simplification of the setup and use of the implementation through the
development of an API.
Enhance the implementation to utilize a dynamic method, such as ‘racoon’, for
establishing security associations between the initiator and target - this should not be used
for the custom key for encryption/decryption of the user data.
Enhance the scheme to use a non hard coded custom key.
If the scheme is to be used for disaster recovery, the custom key must be accessible after
the disaster in order to decrypt data. This means that the custom key must be available at
another location which poses a problem. The more copies of the key that exist, the less
secure it may potentially become. One solution would be to use a third party to securely
store the key. The downside of this strategy is that it could increase the time to recover
from a disaster, since proof may be needed by the third party in order to retrieve the key.
118
Chapter 12
12. Conclusions
The enhancements to the Efficient Asymmetric Secure iSCSI scheme provided in Secure
Asymmetric iSCSI for Online Storage address some of the problems in bringing the
implementation closer to a viable solution for real world usage.
Transfer of arbitrary sized files has been simulated through enhancements to the sg_dd utility.
Results from the performance tests suggest that the enhancements do not significantly degrade
overall performance.
Additional enhancements were made to the sg_dd utility such that user data could be transferred
to two targets. As with the enhancements for files of arbitrary size, performance tests were
undertaken. The results obtained again suggested that the enhancements do not significantly
degrade overall performance.
Despite the enhancements a number of limitations still exist:
When transferring files of arbitrary size, the user must still specify a count value when
retrieving user data from the target storage. Consequently, they must know the size of the
file on the target storage.
When an arbitrary size file is sent to the target storage, additional data beyond that of the
actually data size it sent. This is done in order that the encryption scheme will not be
broken. As a result, when the user data is retrieved from the target it is no longer of
arbitrary size, rather it is rounded to the next block size (1024 bytes).
119
The enhancements for the transfer of files of arbitrary size were designed specifically for
the existing test bed. Therefore, limitations may exist on other test beds dependent on
how the target storage is identified.
When retrieving user data using the sg_dd utility enhanced to transfer files to two targets,
the file is only retrieved to a single initiator. As a result, the second ‘of’ value in the
command line is set to a default of /dev/null. This means that no data is in fact transferred
to what would be a second initiator. It is not clear whether the specification of /dev/null
as the second ‘of’ results in additional processing being undertaken. This warrants further
investigation.
Additional research has shown that mounting of a target storage disk results in the sending of
iSCSI/SCSI ‘Writes’ and ‘Reads’ which will be interpreted by the Efficient Asymmetric Secure
iSCSI scheme as user data. As a result, mounting of a target storage disk is not possible with the
existing encryption scheme. However, it is believed that usage of a mounted target storage disk
will be possible. This will require changes to be made to the IPsec code to allow packets
transferred during the mount process to be separately identified from those containing user data.
This may also require changes to the SCSI and possibly iSCSI code in order to differently label
such packets.
120
Bibliography
[1-1] Ensuring Data Integrity: Logical Data Protection for Tape Systems,http://www.crossroads.com/Library/WhitePapers/FeaturedWhitePapers.asp
[1-2] HIPAA. Health Insurance Portability and Accountability Act 1996,http://www.legalarchiver.org/hipaa.htm
[1-3] The Sarbanes-Oxley Act 2002, http://www.legalarchiver.ord/soa.htm
[1-4] Andrew Hiles, Surviving a Computer Disaster, Engineering Management Journal, December 1992
[1-5] Fibre Channel – Overview of the Technology, http://www.fibrechannel.org/technology/overview.html
[1-6] http://ceva-dsp.mediaroom.com/index.php?s=glossary
[1-7] http://www.sunrise.uk.com/glossary.html
[1-8] Ulf Troppens, Rainer Erkens and Wolfgang Müller, Storage Networks Explained: Basics and Application of Fibre Channel SAN, NAS, iSCSI and InfiniBand, 2004, Wiley & Sons Ltd, ISBN: 978-0-470-86182-0
[1-9] Jane Shurtleff, IP storage: A review of iSCSI, FCIP, iFCP http://www.iscsistorage.com/ipstorage.htm
[1-10] iSCSI for Storage Networking, http://www.snia.org/forums/ipsf/programs/about/isci/iSCSI_for_Storage_Networking.pdf
[1-11] Murthy S. Andukuri, Efficient Asymmetric Secure iSCSI,http://cs.uccs.edu/~gsc/pub/master/msanduku/doc/report_final.doc
[2-1] Friedhelm Schmidt, SCSI Bus and IDE Interface – Protocols, Applications and Programming, Addison-Wesley, 1995, ISBN: 0-201-17514-2
[2-2] Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices,
Subsystems, Applications, Management, and File Systems, Cisco Press, 2005, ISBN 1-58705-162-1
[2-3] Thomas C. Jepsen, Distributed Storage Networks: Architecture, Protocols and Management, 2003, Wiley & Sons Ltd, ISBN:0-470-85020-5
[2-4] iSCSI Technical White Paper, SNIA IP Storage Forum,http://www.snia.org/forums/ipsf/programs/about/isci/iSCSI_Technical_whitepaper.PDF
[2-5] Integration Scenarios for iSCSI and Fibre Channel. SNIA IP Storage Forum,http://www.snia.org/forums/ipsf/programs/about/isci/iSCSI_FC_Integration_IPS.pdf
121
[2-6] Shuang-Yi Tang, Ying-Pang Lu and David H. C. Du, Performance Study of Software-Based iSCSI Security, Proceedings of the First International IEEE Security in Storage Workshop (SISW ’02)
[2-7] American National Standard for information systems - Small Computer System Interface (SCSI) –Draft, December 16, 1985, http://www.t10.org/ftp/t10/drafts/s1/s1-r17b.txt
[2-8] Information technology - Small Computer System Interface – 2, Working Draft, 7th September 1993, http://www.t10.org/ftp/t10/drafts/s2/s2-r10l.pdf
[2-9] http://www.t10.org/scsi-3.htm
[2-10] http://www.t10.org/ftp/t10/document.00/00-269r1.pdf
[2-11] http://www.t10.org/lists/op-alph.txt
[2-12] Gary Field, Peter Ridge et al., The Book of SCSI, 2nd Edition, No Starch Press, 2000, ISBN: 1-886411-10-7
[2-13] M. Tim Jones, Anatomy of the Linux SCSI subsystem http://www.ibm.com/developerworks/library/l-scsi-subsystem/index.html
[2-14] Douglas Gilbert, The Linux 2.4 SCSI subsystem HOWTO, http://www.ibiblio.org/pub/Linux/docs/HOWTO/other-formats/pdf/SCSI-2.4-HOWTO.pdf
[2-15] http://www.andante.org/scsidoc/SCSI-3.html
[2-16] Yingping Lu and David H. C. Du, Performance Study of iSCSI-Based Storage Subsystems, IEEE Communications Magazine, August 2003, pp 76-82.
[2-17] Integration Scenarios for iSCSI and Fibre Channel.http://www.snia.org/forums/ipsf/programs/about/isci/iSCSI_FC_Integration_IPS.pdf
[2-18] Irina Gerasimov, Alexey Zhuravlev, Mikhail Pershin and Dennis V. Gerasimov, and Implementation of a Block Storage Multi-Protocol Converter, Proceedings of the 20 th
IEEE/11th NASA Goddard Conference of Mass Storage Systems and Technologies (MSS’03)
[2-19] John L. Hufferd, iSCSI The Universal Storage Connection, Addison Wesley, 2003, ISBN: 0-201-78419-X
[2-20] Internet Small Computer Systems Interface (iSCSI), http://www.ietf.org/rfc/rfc3720.txt
[2-21] Storage Networking Protocol Fundamentals: A Comparative Analysis of Ethernet, TCP/IP, and Fibre Channel in the Context of SCSI, James Long, Cisco Press, 2006, ISBN: 1587051605
[2-22] iSCSI Building Blocks for IP Storage Networking, A SNIA IP Storage Forum Whitepaper. http://www.snia.org/forums/ipsf/programs/about/isci/iSCSI_Building_Blocks_01.pdf
122
[2-23] http://sg.torque.net/sg/sg_dd.html
[2-24] http://sg.torque.net/sg/sg3_utils.html
[3-1] Charles Kozierok, The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols Reference, 2005, No Starch Press, ISBN: 15932704X
[3-2] Deb Shinder, Securing Data in Transit with IPSec, July 2004, http://www.windowsecurity.com/articles/Securing_Data_in_Transit_with_IPSec.html
[3-3] Steve Friedl, An Illustrated Guide to IPSec, http://www.unixwiz.net/techtips/iguide-ipsec.html
[3-4] http://www.ipsec-howto.org/x202.html
[3-5] Ralf Spenneberg, IPSec HowTo,http://www.ipsec-howto.org/ipsec-howto.pdf
[B-1] Chris Huss and Mike Laverick, UltimateP2V using Qui Hong’s FixVM-SCSI and NU2’sBartPE,http://www.rtfm-ed.co.uk/docs/vmwdocs/whitepaper-ultimateP2V-QuickStart.pdf
[B-2] Saroj Patil, Study of P2V and V2P Using UltimateP2V, http: // cs.uccs.edu/~cs522/studentproj/projF2006/spatil/doc/report.doc
123
Appendices
A. Wireshark
As stated in the main body of the report, Wireshark is a network protocol analyzer. In order to
install and use Wireshark, a number of other packages must be installed. This appendix provides
a brief overview of the packages and installations required for Wireshark in a Linux environment.
Full documentation of the Wireshark tool can be found at http://www.wireshark.org/docs/.
The packages that must be installed before Wireshark can be installed are GTK+ (The GIMP
Toolkit), Glib, libpcap and libpcap-devel.
GTK+ or The GIMP Toolkit is a cross-platform widget toolkit use for creating graphical user
interfaces. Glib is a cross-platform utility library used by GTK+. It provides data structure
handling for C, portability wrappers, and interfaces for such runtime functionality as an event
loop, threads, dynamic loading, and an object system.
libpcap is a library which provides a packet filtering mechanism based on the BSD packet filter
(BPF). It is the packet capture software used by Wireshark. libpcap-devel provides additional
libraries required to compile programs that use libpcap.
Prior to installing Wireshark, it is first necessary to ensure that the other required packages were
in place. The writer found that the F7 installation being used for development and testing already
had GTK+ and Glib installed. As such, no further discussion of these packages will take place.
Further information can be found at http://www.gtk.org/.
The libpcap and libpcap-devel packages were found to be absent from the installation. These
were installed using the commands below:
124
yum install libpcap
yum install libpcap-devel
Once the above two packages had been installed, the source file for Wireshark was obtained from
http://sourceforge.net/project/showfiles.php?group_id=255. Once downloaded, the wireshark-
0.99.8.tar.gz was extracted to the /usr/src/ directory using the following command:
tar –xvf wireshark-0.99.8.tar.gz
The wireshark-0.99.8 directory was then entered and the following commands were used to build
the application from the source code.
cd wireshark-0.99.8
./compile
make
make install
The application can then be started by typing ‘wireshark’on the command line.
125
B. UltimateP2V and VMware Server Virtual Machine Creation
This appendix provides details for creating a virtual machine on VMware Server, and using
UltimateP2V to creating a clone of a physical machine and installing it on the virtual machine.
UltimateP2V is software (plug-in) that enables the user to clone a physical machine to a virtual
machine and vice versa. For the purposes of this project, only cloning of physical to virtual
machines was undertaken. In addition to cloning the physical machine, the software also allows
necessary system reconfiguration to be carried out in order to make the virtual machine bootable.
UltimateP2V utilizes BartPE (Bart Preinstalled Environment) to boot both the physical and
virtual machines. The clone of the hard disk is created using Symantec Ghost software.
The first stage in creating a clone of the physical machine requires the creation of a BartPE
bootable iso image. Full details of this process can be found in the paper by Huss and Laverick [B-
1] and the report of Patil [B-2] and will not be reproduced here. In this project the BartPE bootable
CD/iso image created by Patil was utilized.
The remaining steps for the P2V transfer are as follows:
1. Creation of a virtual machine on the VMware Server
2. Booting both the physical and virtual machines using the UltimateP2V CD
3. Cloning of the hard disk from the physical machine using Symantec Ghost
4. Booting the virtual machine and configuring the operating system for the hardware
components present.
126
B.1 Creation of a virtual machine on the VMware Server
Figure B-1 shows the virtual machine wizard graphical user interface displayed once the
VMware Server console has been started, and the selection has been made to create a new virtual
machine.
Figure B-1: VMWare Server Console
The next window is the configuration user interface which is shown in Figure B-2. For the
purposes of this project, the custom configuration was selected since this allows more specific
machine parameters to be specified than when using the typical configuration.
The next window presented, shown in Figure B-3, allows the user to select the guest operating
system (OS) and which version of the operating system to be used. Here the Linux was selected
as the OS with the particular version being Red Hat Linux.
127
Figure B-2: Typical or Custom Virtual Machine Configuration
Figure B-3: Selecting Guest Operating System
Having determined the OS and version, the next stage in the creation of the virtual machine is the
naming of the virtual machine and the location to which it is to be saved. The selections made in
this case are shown in Figure B-4.
128
Figure B-4: Naming of the Virtual Machine
The next stages of the configuration were to set the access rights, Figure B-5, the
startup/shutdown options, Figure B-6, the processor configuration, Figure B-7 and the amount of
memory for the virtual machine, Figure B-8.
In terms of access rights, it was decided to make the virtual machine private such that only the
user account used to create the virtual machine would have access to it. This was done in order to
prevent unauthorized users from making changes to the machine that could impact the EASI
implementation installed on the machine. The virtual machine account was selected as the user
powering on the virtual machine and as such, the startup and shutdown options were not
available.
129
Figure B-5: Setting the Access Rights
Figure B-6: Startup/Shutdown Options
In terms of the processor, the physical machine had only one processor and as such this option
was selected for the virtual machine. The memory allocated to the virtual machine was set to be
512MB.
130
Figure B-7: Processor Configuration
Figure B-8: Virtual Machine Memory
The next stages of the configuration involve selecting the type of network connection, Figure A-9
and the I/O adapter type, Figure A-10. The network connection selected for the machines used in
this project was bridge networking. The I/O adapter type selected was a SCSI adapter with LSI
logic.
131
Figure B-9: Network Connection
The last few steps of configuring the virtual machine include the selection of disk for the virtual
machine drive to use, the disk type, disk capacity, and the disk file name. Since the virtual
machine was to utilize a cloned image of the physical machine, a new virtual disk was created as
can be seen in Figure B-11. The virtual disk type selected was SCSI (Figure B-12). The size of
the disk was set to be 40GB as shown in Figure B-13. This size was chosen because the image
itself was approximately 20GB. Finally, the name of the disk file was specified. This is shown in
Figure B-14.
132
Figure B-10: I/O Adapter Types
Figure B-12: Virtual Disk
133
Figure B-13: Disk Type Selection
Figure B-14: Disk Capacity
134
Figure B-15: Disk File
This completes the steps in the initial creation and configuration of the virtual machine. The next
step in creating the virtual machine from the physical machine requires both the virtual machine
and physical machine to be booted from the UltimateP2V CD. However, before starting the
virtual machine, it was first necessary to change the boot in order to boot from the CD instead of
hard disk. This was achieved by clicking on the virtual machine tab (Figure B-16) selecting ‘Edit
virtual machine settings’ and then choosing the CD ROM and setting the path to it as shown in
Figure B-17.
135
Figure B-16: Virtual Machine Tab
Figure B-17: Editing the machine settings to boot from the CD
136
B.2 Booting the Virtual Machine with the UlitimateP2V CD
The next step in creating the virtual machine from the physical machine requires both the virtual
machine and physical machine to be booted from the UltimateP2V CD. Since it was not possible
to take screen shots from the physical machine, only the virtual machine information is discussed.
Once the virtual machine booted from the CD, the window shown in Figure B-18 relating to
network support was displayed. Since it was necessary to have network support to enable the disk
transfer from the physical to virtual machine, network support was started.
Figure B-18: Window for Network support
On choosing to start network support, the window shown in Figure B-19 was displayed.
From the list in this window, ‘Static IP Address (Manual) was selected. The IP address details
were then entered in the next window, shown in Figure B-20.
137
Figure B-19: Network Configuration Profile
138
Figure B-20: IP Address Configuration
B.3 Cloning the Hard Disk from the Physical Machine
The next step in the process was to clone the hard disk from the physical machine. This was
achieved by peer-to-peer transfer available using the Ghost software. This requires a number of
steps to be carried out on both the virtual and physical machines.
Operations on the Virtual Machine
On the main window displayed on the virtual machine click on the ‘GO’ button, from the options
displayed, choose ‘Programs’ followed by ‘Symantec Ghost8’ and then ‘Ghost32’. From the next
list displayed, shown in Figure B-21, select ‘Peer to Peer’, ‘TCP/IP’ and then ‘Slave’.
139
Figure B-21: Selecting Peer to Peer transfer via TCP/IP in Slave Mode
Operations on the Physical Machine
On the main window displayed on the virtual machine click on the ‘GO’ button, from the options
displayed choose ‘Programs’ followed by Symantec Ghost8 followed by Ghost32. From the next
list displayed select ‘Peer to Peer’, followed by ‘TCP/IP’ followed by ‘Master’. In ‘Slave TCP/IP
Address to Join to’ window displayed, type the IP address of virtual (slave) machine.
On the Physical machine choose the ‘Disk To Disk’ transfer option in Ghost. Then select the
local source drive by clicking on the drive number. This is the disk where the local operating
system is located. Then click ‘OK’. Repeat this process for the remote destination drive. In this
case, the first disk should be selected. No screen shots are available for these steps since it was
not possible to take screen shots from the physical machine.
140
The cloning of the physical machine takes place. On completion of the cloning process, a ‘Clone
complete dialog box’ is displayed. This is displayed on both the physical and virtual machines.
On the physical machine the UltimateP2V CD should be removed and the ‘Restart Computer’
button should be clicked. In the case of the virtual machine the ‘Continue button should be
clicked and Ghost should be closed. Once this has been done, the GRUB menu of the newly
cloned virtual machine should displayed, as shown in Figure D-22.
Figure B-22: GRUB Menu of Newly Cloned Virtual Machine
Dependent upon whether the devices connected to the virtual machine are different from the
physical machine, it may be necessary to configure them. This was not found to be necessary in
this case.
141
C. Creating a Mounted Target Storage Device
Various devices can be provided by the target to the initiator as storage. These include regular
files, block devices and virtual block devices such as RAID and LVM. For the purposes of this
project, it was decided to add an additional Hard Disk to the target server machine and configure
it as a LVM virtual block device to act as the storage. This appendix provides the details of
creating an additional hard disk and LVM on the VMware image of the University test bed target
machine along with the steps required to make the hard disk useable as a storage target.
C.1 Creation of an Additional Hard Disk
The first step in the creation of the storage device is to add an additional hard disk to the existing
virtual machine. This must be done with the virtual machine powered off. In the VMware target
machine display, select ‘Edit virtual machine settings’. This will display the machine settings
graphical user interface shown in Figure C-1.
Next select ‘Add’ this will result in the hardware disk wizard being displayed, as shown in
Figure C-2. Select ‘Next’ and then ‘Hard Disk’, as shown in Figure C-3, select ‘Next’ and in the
‘Select a Disk’ screen (Figure C-4) choose ‘Create a new virtual disk’ and click ‘Next’.
The next screen prompts the user for a disk type, select ‘IDE’, (Figure C-5), and click ‘Next’.
When prompted for a disk capacity select the desired capacity, in this case 2G was chosen, then
deselect ‘Allocate all disk space now; as shown in Figure C-6 and then click ‘Next’.
In the next window, shown in Figure C-7, accept the default value for the disk file name and
click ‘Finish’. On the last screen click ‘OK’ and this will complete the addition of the extra hard
disk.
142
Figure C-1: Graphical user interface for editing the virtual machine settings.
Figure C-2: Hardware Wizard display
143
Figure C-3: Hardware addition selector
Figure C-4: Select a Disk
144
Figure C-5: Select a Disk Type
Figure C-6: Specify disk capacity and allocation
145
Figure C-7: Disk file name
C.2 Configuring the Added Hard Disk as an iSCSI Target Device
Having added an additional hard disk to act as the storage, the next step is to create the LVM and
configure it for use.
On the Target Machine
1. Create the physical volume
[root@starget ~]# lvm
lvm > pvcreate /dev/hdb1
2. Create a virtual group
lvm > vgcreate vg /dev/hdb1
3. Create logical volume(s)
lvm > lvcreate –L 1.75G –n sasdisk vg
146
4. Exit lvm by typing the command ‘exit’
5. Edit the ietd.conf file
[root@starget ~]# cd /etc
Add the following entry to the ietd.conf file:
Target iqn.2008-04.com.testbed.scsi.disk.vg.sasdisk
Lun 0 Path=/dev/vg/sasdisk,Type=fileio
6. Start the iSCSI target
cd /etc/init.d
./iscsi-target start
On the Initiator Machine
Start the iSCSI initiator
./start_scsi_iscsi start
The above file starts the iSCSI initiator and discovers targets to which it can be connected. At the
end of a lot of output, the discovered targets are displayed. A typical example is shown of the
output is shown below.
[ad7a1f] 192.168.2.69:3260,1 iqn.2008-04.com.testbed:scsi.disk.vg.sasdisk
2. Login to the discovered target
./iscsiadm --m node --record ad7a1f --login
3. Format the LV that appears as the target storage.
To check that the scsi device is visible to the initiator type:
fdisk -l
147
You should see disk /dev/sda with info that it does not contain a valid partition table.
fdisk /dev/sda
At the prompt the following should be selected:
n (to add a new partition)
p (to select primary partition)
1 (partition number)
First cylinder : 1
Last cylinder: accept the default
w (to write the partition table)
4. Create a file system
mkfs –t ext3 /dev/sda or mkfs.ext3 –j c –b 1024 /dev/sda
6. Mount the partition
mkdir /mnt/iscsi_disk
mount /dev/sda /mnt/iscsi_disc
The target storage device is now mounted on the initiator and can be used for transfers. However,
if transfers are undertaken and then the initiator and target are shut down, you will not be able to
use sda after rebooting unless the following changes are made on the initiator.
Edit the iscsid.conf file
cd /etc
148
In the iscsid.conf file, change the entry:
node.startup=manual
to
node.startup=automatic
Once all transfers have been completed, the storage device should be unmounted and the sync
command executed.
umount /mnt/iscsi_disk
sync
The sync command is necessary because the initiator’s kernel thinks that it is the only entity to
ever access the scsi disk and therefore it may have buffered mount state.
149
D. Typical run through for Transferring Arbitrary Sized Files
D.1 Starting the Target and Initiator Software
The following are the commands for starting the target and initiator software. It should be noted
that the target software should be started first.
Start the target software
cd /etc/init.d
./iscsi_target start
Start the initiator software
cd /etc/init.d
./open-iscsi start
Discover target
The following commands are to be carried out on the initiator.
cd /root/open-iscsi-0.4-434/usr/
./iscsiadm -m discovery –t sendtargets –p 192.168.2.69:3260 –d10
Login to Target
The following commands are to be carried out on the initiator.
cd /root/open-iscsi-0.4-434/usr/
./iscsiadm -m node --record df5b51 –login
150
D.2 Write User Data to the Target
The following command writes a file of slightly less than 10MB to the target storage.
sg_dd if=test_file_10MB-.txt of=/dev/sda bpt=1 odir=1 skip=0 seek=0
D.3 Reading User Data from the Target
The following command reads the user data from the target file back to the initiator.
It should be noted that the count value must be specified.
sg_dd if=/dev/sda of=10MB-ret.txt bpt=1 odir=1 skip=0 seek=0 count=10240
151
E. Typical run through for Transferring to Two Targets
E.1 Starting the Target and Initiator Software
The following are the commands for starting the target and initiator software. It should be noted
that the target software should be started first.
Start the target software
cd /etc/init.d
./iscsi_target start
Start the initiator software
cd /etc/init.d
./open-iscsi start
Discover targets
The following commands are to be carried out on the initiator.
cd /root/open-iscsi-0.4-434/usr/
./iscsiadm -m discovery –t sendtargets –p 192.168.2.69:3260 –d10
./iscsiadm -m discovery –t sendtargets –p 192.168.2.107:3260 –d10
Login to Target(s)
The following commands are to be carried out on the initiator.
cd /root/open-iscsi-0.4-434/usr/
./iscsiadm -m node --record df5b51 –login
./iscsiadm -m node --record f2adee –login
152
E.2 Write User Data to the Target(s)
The following command writes a file of 100MB to the two targets.
sg_dd if=test_file_100MB-.txt of=/dev/sda of_1= /dev/sdb bs=1024 count=10240 bpt=1 odir=1
skip=0 seek=0
E.3 Reading User Data from the Target
The following are the commands for starting the target and initiator software. It should be noted
that the target software should be started first.
sg_dd if=/dev/sda of=100MBret.txt bs=1024 bpt=1 odir=1 skip=0 seek=0 count=102400
153
F. Python and Tkinter
Python is a powerful open source dynamic programming language which is relatively easy to
learn and available for most major operating system. Python was used to develop the user
interface to simplify the tasks of generating keys and transferring files. Since these tasks are
likely to be carried out by systems administrators, the main purpose of the user interface is
functionality rather than to be ascetically pleasing. As such, Python is an ideal choice for
developing the user interface.
The Graphical User Interfaces (GUI’s) used for the configuration of the initiator and target and
the transferring of files was created using the Python programming language. Although it is not
necessary to understand or modify the code in order to use the GUI, it is necessary to have Python
and its associated GUI module (Tkinter) installed on both the initiator and target machines if the
GUI is to be used to configure the system and transfer files. The following are instructions for
installing Python and Tkinter on a Linux OS.
The versions of Python and Tkinter used for this project are python-2.4.3-8.FC4 and tkinter-
2.4.3-8.FC4 respectively. They can be installed using the following commands:
yum install python-2.4.3-8.FC4
yum install tkinter-2.4.3-8.FC4
Once installed python programs can be run using the command:
Python filename.py
154
G. User Interface Operating Instructions
As stated in the main body of this report, two user interfaces were developed, one for use on the
initiator and the other for use on the target. This appendix provides details regarding the operation
of these user interfaces.
The main purposes of the GUI’s are to start/stop the target software, start/stop the initiator
software, discover available target storage and login/logout to/from the target storage, and write
and reading files. In addition, both GUI’s provide additional functionality. This is discussed later
in this appendix.
G.1 Starting the Target Graphical User Interface
The first step is to start the target GUI. This is achieved by opening a terminal window on the
target machine(s). In this case, starget1 is being used. In the terminal window the following
command should be executed:
[root@starget1~]# python user_target_final.py
This results in the launching of the target GUI and the display of the main GUI window as shown
in Figure G-1. As can be seen a number of tabs are available in the GUI.
155
Figure G-1: Initial Window of Target Processing GUI
G.2 Starting the Target Software
In order to start the target software, the ‘Start/Stop Target’ tab should be selected. This results in
the screen shown in Figure G-2 being displayed. To start the target software, click on the ‘Start
Target’ button. Confirmation that the target software has started is displayed in the text display
box, as can be seen in Figure G-3.
156
Figure G-2: Start/Stop Target Tab of Target Processing GUI
Figure G-3: Message indicating that the Target Software has been Started
157
G.3 Starting the Initiator Graphical User Interface
The next step is to start the initiator GUI. This requires that the user changes to the initiator
machine (siscsi in this case).
Once on the initiator machine, a terminal window must be opened and the following command
should be executed.
[root@starget1~]# python user_initiator_final.py
This results in the launching of the initiator GUI and the display of the main initiator GUI
window as shown in Figure G-4. As with the target GUI, a number of tabs are available on the
initiator GUI.
Figure G-4: Initial Window of Initiator Processing GUI
158
G.4 Starting Initiator Software
In order to start the initiator software, the ‘Start Initiator’ tab should be selected. This results in
the screen shown in Figure G-5 being displayed.
To start the initiator software, click on the ‘Start Initiator’ button. Confirmation that the initiator
software has started is displayed in the text display box, as can be seen in Figure G-6.
Figure G-5: Start Initiator Tab of Initiator Processing GUI
159
Figure G-6: Message indicating that the Initiator Software has been Started
G.5 Discovering Available Targets
The next step is to discover the available targets to which the initiator can be connected. In order
to do this, the user must select how many targets that they wish to connect to. This is achieved
using the radio selection buttons on the GUI. The IP address(es) for the target(s) must be entered
in the appropriate text boxes. This is shown in Figure G-7. Once this information has been
entered, the user must click the ‘Discover Target(s)’ button. A list of available target storage
nodes is then displayed in the text box, also shown in Figure G-7.
In the example shown in Figure G-7, the user has selected to connect to one target. As can be
seen in the text display, this target machine had two target files to which the initiator can connect
with record numbers of ad7a1f and df5b51.
160
Figure G-7: Target Storage Discovery
G.6 Logging in to Target Storage
Having obtained the target record numbers through the discovery process described in the
previous section, the user can now login to the target(s). This is achieved by entering the
appropriate target record number(s) (df5b51 in this case) into the appropriate text box(es) and
then clicking the ‘Login to Target(s) button (Figure G-8). Once the login has been achieved, a
message is displayed in the text display box as can be seen in Figure G-8.
161
Figure G-8: Login to Target Storage
Having logged into the target storage, it is possible to write/read files to the target storage. In
order to do this, the user must first navigate to the appropriate tab on the initiator GUI, the ‘Write
file to Target(s)’ tab if a file is to be written to the target storage, or the ‘Read file from Target’
tab if a file is to be read from the target storage.
G.7 Writing to Target Storage
The first step in writing a file to target storage is to navigate to the ‘Write file to Target(s)’ tab
(Figure G-9). It is then necessary to specify how many targets the file is to be transferred to. This
is achieved by selecting either the ‘single’ or ‘duplicate’ radio buttons depending on whether the
file is to be transferred to one or two targets.
Next the user must enter the path to the file to be written in the ‘Transfer From’ text box and the
path to the target storage in the ‘Transfer to’ text box. If two targets are being used for storage,
162
the path to the second target storage should be entered in the ‘Secondary Transfer to’ text box.
Once this data has been entered, the ‘Transfer Files’ button should be clicked. The path to the
target storage can be determined by typing ‘fdisk –l’ in a terminal window. In this case, the target
storage path is /dev/sda.
Once the above data has been entered, the write is performed by clicking on the ‘Transfer Files’
button. Once the file has been transferred, a confirmation message is displayed in the text display
area (Figure G-9).
Figure G-9: Writing a File to Target Storage
G.8 Reading from Target Storage
In order to read a file from the target storage, it is first necessary to navigate to the ‘Read file
from Target’ tab (Figure G-10). Then specify the path to the file on the target. This is achieved
163
by entering the path to the file on the target, in this example /dev/sda in the ‘Location of File on
Target’ text box. As with writing a file to a target, this information can be determined by running
the ‘fdisk –l’ command in a terminal window.
It is then necessary to enter the path on the initiator and the file name to which the file is to be
read. This information is entered in the ‘Location on initiator to transfer to’ text box. In this
example the file is being read to /root/test1K_ret.txt.
Once all of the above information has been entered, the ‘Transfer Files’ button should be clicked.
This will execute the read and a message will be displayed in the text display area (Figure G-10)
indicating that the file has been successfully read to the initiator.
G.9 Disconnecting from Target Storage
Once all of the desired writes and/or reads have been carried out, the next step is to logout of the
target. This requires the user to navigate to the ‘Start Initiator’ tab shown in Figure G-8. In order
to logout of the target, the target record number must be entered in the ‘Record number of target
to logout’ text box. Next the user must click the ‘Logout from Target(s) button. A message will
be displayed in the text box indicating that the logout has been successful (Figure G-11). It
should be noted that the GUI only allows you to logout from one target at a time. In order to
logout from two targets this process must be repeated for each target record number.
164
Figure G-10: Reading a File from Target Storage
Figure G-11: Logging Out from Target Storage
165
G.10 Stopping Initiator Software
Once the user has logged out from all targets, the initiator software can be stopped. In order to do
this it is necessary to navigate to the ‘Initiator Status/Stop’ tab. Stopping the initiator software is
then achieved by clicking on the ‘Stop’ button (Figure G-12). A message is displayed in the text
area indicating that the initiator has been successfully stopped.
Figure G-12: Stopping the Initiator Software
G.11 Stopping the Target Software
Once the initiator software has been stopped the target software can also be stopped. In order to
do this it is first necessary to change to the target machine (starget1). The user must then navigate
to the ‘Start/Stop Target’ tab, shown in Figure G-13. Then in order to stop the target, it is
166
necessary to click on the ‘Stop Target’ button. This will stop the target software and a message
confirming this is displayed in the text display box (Figure G-13).
Figure G-13: Stopping the Target Software
G.12 Additional Functionality of the Initiator and Target GUI’s
In addition to the features already described both the initiator and target GUI’s have a number of
additional features.
G.12.1 Additional Target Functionality
There are three additional features available on the target GUI. These are the ability to:
check the current status of the target software,
create a new target storage file,
configure the ietd.conf file for new target storage.
167
Target Status
In order to check the status of the target software, the user must navigate to the ‘Start/Stop
Target’ tab. The current status of the software can be using the ‘Target Status’ button. The status
is then displayed in the text display box. In the case shown in Figure G-14, it can be seen that the
target software is running.
Figure G-14: Checking the Status of the Target Software
Creating a New Target File The first step in creating a new target file requires the user to navigate to the ‘Create Target File’
tab (Figure G-15). In order to create a new target file, the user must provide several pieces of
information. These are the location where the target file should be created, the name of the target
file and also its size in bytes. The desired path (location) to the target file must be entered in the
‘Location for target file’ text box, in the example shown in Figure G-15 this is /tmp/. The name
168
of the new target file must be entered in the ‘Enter name for new target file’ text box and finally
the size of the target file to be created must be entered in the ‘Enter size for new target file in
bytes’ text box.
Once the required information has been entered, the new target file is created by selecting the
‘Create Target’ button. A message indicating the successful creation of the target file is displayed
the text display box, as can be seen in Figure G-15.
Figure G-15: Creation of a New Target File
Configuring the ietd.conf File
Once a new target file has been created it is necessary to add its information to the ietd.conf file
in order that it can be used. In order to do this, the order must navigate to the ‘Configure
169
ietd.conf’ tab (Figure G-16). Then the user must enter the path to the target file in the ‘Directory
containing target file’ text box. The name of the target file must be entered in the ‘Name of target
file’ text box. Once this information has been entered the ‘Add target to configuration file’ button
should be clicked. This results in an entry being added to the ietd.conf file. Confirmation is
displayed in the text display area, as can be seen in Figure G-16.
Figure G-16: Configuration of the ietd.conf File
G.12.2 Additional Initiator Functionality
There are three additional features available with the initiator GUI. These are the ability to:
check the current status of the initiator software,
generating keys for use with the IPsec implementation,
creating database entries using the keys generated.
Initiator Status
170
In order to check the status of the initiator software, the user must navigate to the ‘Initiator
Status/Stop’ tab. The current status of the software can be checked by clicking the ‘Status’ button.
The status is then displayed in the test display box. In the case shown in Figure G-17, it can be
seen that the initiator software is running.
Figure G-17: Checking the Status of the Initiator Software
Generating IPsec Keys
In order to allow secure communications between the initiator and target and vice versa, it is
necessary to use IPsec. This requires that the initiator use shared keys to set up the
communication. The initiator GUI allows these keys to be generated. It is an optional task as the
keys may already have been generated. In order to generate the keys, the user must first navigate
to the ‘Generate Keys’ tab. The user must then select which keys to generate; the options are
‘Source to Target’ and/or ‘Target to Source’. In the example shown in Figure G-18 both ‘Source
171
to Target’ and ‘Target to Source’ were selected. Once the desired options have been selected, the
keys are generated by using the ‘Create Keys’ button. The generated keys are displayed in the
text display box, as can be seen in the figure.
Figure G-18: Generating Keys for use with IPsec
Generating SAD and SPD Entries
Once the IPSec keys have been generated, the SAD and SPD entries can be generated. In order to
create the database entries the user must select the desired check boxes, ‘Create SAD entries’
and/or ‘Create SPD entries’ and enter the source and destination IP addresses. Once this has been
done, the database entries are generated by clicking the ‘Generate Database entries’ key as seen in
Figure G-19. This function creates two files one containing the configuration for the SAD and
SPD entries on the initiator and the other the configuration for the SAD and SPD entries on the
target. In the case of the initiator the configuration file is executed and the database entries are
172
created. In the case of the target configuration file this must be passed to the target and executed
there to create the database entries. The file must be passed in a secure manner. How this is
achieved is not considered in this project.
Figure G-19: Generating SAD and SPD Entries
H. Open-iscsi-2.0-869.2
173
H.1 Installing the initiator software
tar xvf open-iscsi-2.0-869.2.tar.gzcd open-iscsi-2.0-869.2make KSRC=/usr/src/linuxmake KSRC=/usr/src/linux install
The open-iscsi-2.0-869.2 configuration file iscsid.conf used during the testing on the Fedora 8
test bed is included on the DVD accompanying this report. The following entries were changed
in the file:
# Open-iSCSI default configuration.# Could be located at /etc/iscsi/iscsid.conf or ~/.iscsid.conf## Note: To set any of these values for a specific node/session run# the iscsiadm --mode node --op command for the value. See the README# and man page for iscsiadm for details on the --op command.
#*****************# Startup settings#*****************
# To manually startup the session set to "manual". The default is manual.node.startup = manual
#***************# iSCSI settings#***************
# To enable R2T flow control (i.e., the initiator must wait for an R2T# command before sending any data), uncomment the following line:##node.session.iscsi.InitialR2T = Yesnode.session.iscsi.InitialR2T = Yes## To disable R2T flow control (i.e., the initiator has an implied# initial R2T of "FirstBurstLength" at offset 0), uncomment the following line:## The defaults is No.#node.session.iscsi.InitialR2T = No
## To disable immediate data (i.e., the initiator does not send# unsolicited data with the iSCSI command PDU), uncomment the following line:#node.session.iscsi.ImmediateData = Nonode.session.iscsi.ImmediateData = No
174
# To enable immediate data (i.e., the initiator sends unsolicited data# with the iSCSI command packet), uncomment the following line:## The default is Yes#node.session.iscsi.ImmediateData = Yes
H.2 Running the iscsi initiator
In order to run the target software, the following commands should be used. It should be noted
that the target software should be started before the initiator software.
H.2.1 Starting the intiator
cd /etc/init.d./open-iscsi start
H.2.2 Discovering Available Targets
cd /usr/src/ open-iscsi-2.0-869.2/usr./iscsiadm –m discovery –t sendtargets –p 192.168.2.106:3260
Execution of the above command will result in the output of the available targets. On the Fedora
8 test bed this was as follows:
192.168.2.106:3260, 1 iqn.2008-04.com.example:storagedisk2.sys1.xyz
H.2.3 Login to Target
cd /usr/src/ open-iscsi-2.0-869.2/usr./iscsiadm –m node –T iqn.2008-04.com.example:storagedisk2.sys1.xyz –p 192.168.2.106:3260 –login
H.2.4 To logout of the Target
cd /usr/src/ open-iscsi-2.0-869.2/usr
175
./iscsiadm –m node –T iqn.2008-04.com.example:storagedisk2.sys1.xyz –p 192.168.2.106:3260 –logout
H.2.5 Stopping the Initiator
cd /etc/init.d./open-iscsi stop
I. iscsitarget-0.4.16
176
I.1 Installing the target software
tar xvf iscsitarget-0.4.16.tar.gzcd iscsitarget-0.4.16make KSRC=/usr/src/linuxmake KSRC=/usr/src/linux install
The iscsitarget-0.4.16 configuration file ietd.conf used during the testing on the Fedora 8 test bed
is included on the DVD accompanying this report. The following entries were changed in the
file:
# Example iscsi target configuration
# Targets definitions start with "Target" and the target name.# The target name must be a globally unique name, the iSCSI standard defines the "iSCSI# Qualified Name" as follows:## iqn.yyyy-mm.<reversed domain name>[:identifier]## "yyyy-mm" is the date at which the domain is valid and the identifier# is freely selectable. For further details please check the iSCSI spec.
Target iqn.2008-04.com.example:storage.disk2.sys1.xyz
# Logical Unit definition# You must define one logical unit at least.# Block devices, regular files, LVM, and RAID can be offered to the initiators as a block device.Lun 0 Path=/dev/vg/iscsidisk,Type=fileio
# various iSCSI parameters (not all are used right now, see also iSCSI spec for details)#InitialR2T YesInitialR2T Yes#ImmediateData NoImmediateData No#MaxRecvDataSegmentLength 8192MaxRecvDataSegmentLength 1024#DefaultTime2Wait 2DefaultTime2Wait 180#DefaultTime2Retain 20DefaultTime2Retain 180#DataPDUInOrder YesDataPDUInOrder Yes
177
#DataSequenceInOrder YesDataSequenceInOrder Yes
I.2 Running the target
In order to run the target software, the following commands should be used. It should be noted
that the target software should be started before the initiator software.
I.2.1 Starting the target
cd /etc/init.d./iscsi-target start
I.2.2 Stopping the Target
cd /etc/init.d./iscsi-target stop