Computers & Security, 16 (1997) 595-602

Penetration testing and system audit - Experience gained during the investigation of systems within the UK Andy Jones

Defence Evaluation and Research Agency, CIS3 Department, St. Andrews Road, Maloern, Worcestershire, wR14 3PS. UK.

Background In the main, Information Technology (IT) security within the UK has been achieved as a result of the production of a System Security Policy (SSP) or some similar document which defines the security require- ments for the system, based on the threat against it. The threat to any system is dependent on a number of factors, including the value of the data or processes, the attractiveness of the system as a target to potential attackers and the inherent vulnerability of the system. All of these factors have to be taken into account when deciding the strategy that will be adopted to secure the system. From the requirements laid down in the SSP, it has been the norm to procure systems and components that have, where it is necessary, under- gone evaluation and received certification by the national authorities to the appropriate levelThe per- son with overall responsibility for security within the relevant organization has then, after taking into account the security measures that have been imple- mented, accredited the system for use in the manner defined in the SW The person who accredits the system is, in so doing, identifying that the residual risk

0 British Crown Copyright 1997 /DEFU. Reproduction with the per- mission of the controller of Her Britannic Majesty’s Stationary Office.

to that system is acceptable and signifying will take responsibility for this residual risk.

The disadvantages of this approach are:

that they

It is largely a paper based, one off, exercise. It is unresponsive to changes in the threat, vulnera- bilities and known attack techniques. It is unresponsive to changes in the system itself. Any practical testing of the delivered system is most often designed to test that the countermeasures are in place, and that they operate correctly, not whether there are any exploitable vulnerabilities in the deliv- ered system.

As a result, it was felt that an alternative approach was needed; one which took into consideration what hap- pens in real life.The object of any approach must be to determine whether there are any exploitable vul- nerabilities in the delivered system, and then to rec- ommend action to redress the problem.This must take into account changes both in the system and to the environment throughout the life of the system.

It is notoriously difficult to convince the managers and financiers of organizations that there is a need to spend the required level of precious resources on the

0167-4048/97$17.00 0 Elsevier Science Ltd 595

Penetration testing and system audit/Andy Jones

security of their information systems, and that there is a real and a demonstrable risk. One way of demon- strating the threat to a system is to emulate the hack- er, one of the groups likely to mount an attack on the system. This type of testing has been given the name of penetration testing. Penetration tests on systems:

l Are based on known vulnerabilities and attack techniques.

l Demonstrate clearly and reproducibly the suscepti- bility of a system to an attack.

l Demonstrate the effectiveness of countermeasures. l Can be updated in line with the current threat, and

repeated.

There is a growing concern that a large number of system owners perceive that the process of generating a system security policy is an end in itself.That is to say that the ‘accreditation’ of the system is the end of the process rather than the first step in a process that encompasses the complete lifecycle of the system.

Introduction Since 1995, the Malvern Site of the Defence Evaluation and Research Agency (DERA) of the British Ministry of Defence (MOD), has carried out a series of penetration tests on IT systems belonging to a number of different organizations, none of which were owned by the MoD.These tests were carried out in isolation from research carried out in other countries or by other organizations within the UK, but in the knowledge that a body of research had been carried out and that a number of problems had been identified.

The tests confirmed a methodology for penetration testing and system audit that could, in due course, be applied to defence systems. The methodology was pragmatic, and would, therefore, produce proof of shortcomings in the existing security, rather than be theoretic and document based. The benefit being sought was the identification of changes that could be made that would provide immediate and demonstra- ble improvements to the security of the systems. The penetration testing and system audit was not intended to replace the UK ITSEC scheme, but was intended to perform a complementary function.

596

The purpose of the individual penetration tests and system audits was to determine whether the security measures being applied to the system were adequate to protect it from known vulnerabilities. The tests were carried out with the knowledge that the US Defense Information Systems Agency (DISA) Vulnerability Analysis and Assessment Programme (VAAP) had car- ried out a series of 38 000 attacks on US Department of Defence (DOD) systems between 1992 and 1995, of which 24 700 successfully penetrated the systems. Of the 38 000 VAAP tests carried out. only 267 were detected and reported.

The systems tested by DERA ranged from small local area networks contained within one building to large wide area networks which were geographically dis- tributed through a number of countries. The operat- ing systems and applications in use on the tested sys- tems covered a wide spectrum of those in normal commercial use.

Methodology At the start of each penetration test and system audit, the briefing given to the team that was to conduct the testing deliberately avoiding the introduction of pre- conceptions with regard to the system.The team were not briefed on concerns that the security staffs or accreditation authorities may have had with regard to the individual system. Additionally, the team was not briefed on the history of the system and viewed what existed at the time of the test, rather than considering the evolution of the system.

The audit of the system included:

A review of the System Security Policy. Determining the relevance of the SSP to the current configuration of the system and the threat. Reviewing the Security Operating Procedures (SyOPs) and checking that they were being implemented. The configuration of the system. System maintenance records. The physical security of system components.

The penetration testing of the systems tested them against the types of attack tools and methods that were

Computers & Security Vol. 16, No. 7

available in the public domain at the time of each of the tests, and applied the skills of the testing team to make the most effective use of them.The scenarios for the individual tests varied, depending on the type of network and the requirements of the system owners. It is important to note that all tests were carried out with the full knowledge and agreement of the system owner. During the tests, the vulnerabilities that were being tested for were those that were known to be in the public domain, and therefore were, or could have been, known to hackers. The penetration testing team placed itself in the role of an attacker and used the techniques available to hackers.

The team, depending on the requirements of the sys- tem owner, normally conducted the tests from a priv- ileged position, in that they were given access to the system to be tested. From there, they determined whether the networks and components were secure against unauthorized internal or external activity. The decision to operate from a privileged position, wher- ever possible, was taken for a number of reasons. First, it reduced the length of time that it took to conduct each test. Second, it removed the need to war-dial, strobe, etc. Third, it increased the confidence of the organization being tested that the tests would not leave them exposed to future ‘hacking’ attacks as a result of changes made to the system configuration by the penetration testing team. However, on a number of occasions, the system owner decided that there were more benefit to be gained from an external test, and the full range of techniques were applied to the system, including war-dialing and strobing.

This last point was to become significant. Penetration testing of systems had not been widely used on sys- tems within the UK and there was a degree of reser- vation by the system owners to allowing ‘outsiders’ to have access to their systems. Organizations were, in the main, keen to have their systems tested and to gain the benefits of the results, but were extremely cautious as to potential damage that might be caused to their sys- tems. A number of the individual system managers were also less than enthusiastic, as they felt that they would be embarrassed and shown to be inefficient if the tests identified flaws in the security of their systems.

As a result of the above concerns and sensitivities, all tests were conducted so that the system staffs could monitor the actions being taken. This has the benefit of allowing them to gain knowledge of the procedures being used and allowing them to ‘remain in control’ of the situation. Additionally, the tests were non-destruc- tive and did not alter the configuration, or affect the performance, of the systems. Finally, all magnetic media used in the testing could be erased or destroyed to meet the requirements of the system owners.

The equipment used by the test team consisted of two Sun Spare workstations and a 486 laptop computer.All of the software was either Commercial OffThe Shelf (COTS) operating systems and applications or toolk- its that contained the functionality of individual tools and scripts from information available to hackers (pri- marily derived from the Internet and from Bulletin Board Systems).

It is estimated that the total number of man days taken to conduct a full range of tests, make recommenda- tions for remedial actions, and re-test to verify the impact of those actions, on a typical system of 120 servers and 1200 workstations, was approximately 35 man days.This is broken down as:

l Reconnaissance - 3 man days. l Testing and system audit - 9 man days. l Report production including recommendations for

remedial actions, presentation to host organization and re-testing - 23 man days.

The initial steps taken in all of the penetration tests were:

l To identify those hosts and networks which could be seen.

l To identify those hosts that could be accessed. l To find the ways in which those systems could be

reached by: - Identifying which services the systems offer and, -Which version of the service they are using. l To isolate the known methods which could be used

to exploit identified vulnerabilities. l To identify what assistance was required and where

it could be obtained from.

597

Penetration testing and system audiVAndy Jones

Following this, the team then, having identified the points from which the system could be attacked and the sources of any additional information that they required, tested the system and identified weaknesses that remained in the systems, using the tools that were available at the time of the test.

It is important for the credibility of any testing that is undertaken that it is carried out with up to date tools. The use of six month old tools would only indicate the relative state of the security of a system six months prior to the test.

Findings

The results obtained from the penetration testing and system audits were not unexpected and compare with the findings of other research which has been carried out in both the UK and the USA. The team had a great deal of success and were able for the most part, to effect or simulate an intrusion into the systems under test. After analysis of the results of all of the pen- etration tests and audits that were carried out, it was apparent-that the main causes of the security weak- nesses within the systems fell into two areas. These areas have been identified as ‘System Management Issues’ and ‘Technical Issues’.To describe these more Illy:

System Management Issues -This relates to those shortcomings in the security of the IT systems that it was concluded could have been reduced or removed by improvements in the management of the organiza- tion or the system. Advice and good practice for the management of systems is widely available. However, the implementation of this information is not widespread or consistent.

Technical Issues -These were those security weak- nesses that were identified as being specific to a par- ticular operating system or set of applications. The technical shortcomings of the multitude of operating systems and applications is widely documented by a number of official and unofficial sources such as the various CERT (Computer Emergency Response Team) organizations, national agencies and even hack- er organisations such as 8LGM (the eight legged groove machine) and from bulletin boards.

System Management Issues Analysis of the results of the penetration tests and sys- tem audits revealed that an improvement in the man- agement of systems would be by far the most cost effective way of improving the security of those sys- tems.This is not a unique or unexpected finding, and is reflected in studies from the US. In the US General Accounting Office (GAO) report B-266140 of 1996, Chapter 4, which contains recommendations to the Secretary of Defense for ensuring that sufficient pri- ority, resources, and top-management attention are committed to establishing a more effective informa- tion systems security programme.

A number of problems, inherent in the way in which large organizations are managed, were identified in the systems reviewed and are detailed below, together with the remedial action that would need to be taken to respond to the situation.These are identified as:

Inadequate or non-existent security direction for users

Security policies had not been produced for a number of the systems that were tested. For others, while secu- rity policies had been developed, they were either inadequate or out of date. In addition, the intent of the policies had not been adequately interpreted or dis- seminated to the system staffs or users.

This is supported by previous research by the National Computer Centre (NCC) which identified, as a result of a poll of 9500 companies, that 54 percent of systems did not have a formal security policy document. It also matches the first two recommendations of chapter 4 of the US GAO report for an improvement in security policies and procedures and for a need to increase user awareness and accountability, mirroring their finding that “This - demonstrates the low priority top Defense management officials often give security,” and that “Currently, many of Defense’s policies relating to computer attacks are outdated and inconsistent. They do not set standards or mandate specific actions for important security activities such as vulnerability assessments, internal reporting of attacks, correction of vulnerabilities, and damage assessments.” There is a requirement for up-to-date, achievable and

598

Computers & Securit- Vol. 16, No. 7

coherent security policies for all systems. Additionally there should be an interpretation of the policy in operating procedures that define the responsibili- ties and duties of personnel and define the actions that they must take in order to implement the policy.

Security awareness

There was a poor level of awareness at all levels within the organizations reviewed, from high level organization management, through system management to users, of security issues relating to IT systems.

This is reflected in the US GAO document finding that “defence personnel lack sufficient awareness and technical training.” According to the Software Engineers Institute, many computer users do not understand the technology they are using, the vulner- abilities in the network environment they are working in, and the responsibilities they have for protecting critical information. Defence officials generally agreed that user awareness training was needed, but stated that installation commanders do not always understand computer security risk and, thus, do not always devote sufficient resources to the problem. The officials told us they are trying to overcome the lack of resources by low cost alternatives such as banners that warn indi- viduals of their security responsibilities when they turn on their computers. In addition, network and system administrators often do not know what their responsibilities are for protecting their systems, and for detecting and reacting to intrusions. Critical computer security responsibilities are often assigned to personnel as additional or ancillary duties.”

There is a requirement for computer security aware- ness training for personnel at all levels throughout an organization. This should extend down from the senior management of the organization, without whose support the resources will not be made avail- able to the system management. Additionally, if the organization management is not aware of the security issues involved, any policy which is produced is unlikely to be relevant or effective.

Training

Personnel from all levels, from the user through to the system administration and security staffs, were not adequately trained in security or system management to carry out their duties.There is a requirement for the level of security training for system staffs and users to be improved.

The findings of the US Joint Security Commission

USC), in its February 1994 report, ‘Redefining Security’, showed similar concerns, including that “the Department (of Defense) lacks comprehensive, consis- tent training for information systems security officers, and that Defense’s current information systems securi- ty training efforts produce inconsistent training quali- ty and, in some cases, a duplication of effort.” The report concluded that, “despite the importance of security awareness, training, and education pro- grammes, these programmes tend to be frequent and ready targets for budget cuts.”

The GAO report concluded that “The majority of system and network administrators are not adequately trained in security and do not have sufficient time to perform their duties.”

The level of security training for system staffs must be improved to ensure that the system staff are able to manage the system effectively. Training must be targeted to meet the specific needs of the different groups and be provided at an appropriate time. Training for the users should be improved, primarily to make them more aware of the security requirement and to prevent them from making unnecessary errors and to make them aware of their responsibilities for security.

System management

The general level of system security management was poor as system managers did not have the experience, training, level of skill or resources to adequately carry out the task.

The JSC report stated that, “According to Defense officials, installation commanders may not understand the risks associated with networked computers, and thus may not have devoted sufficient priority or

599

Penetration testing and system audit/Andy Jones

resources to address these problems.These officials also cite the lack of a professional job series for informa- tion security officials as a contributing factor to poor security practices at Defense installations. Until SYS-

terns security is supported by the personnel system, it will not be a full-time duty. As a result, security will continue to be the purview of part-time, inadequate- ly trained personnel.”

System management must be improved to meet the current threat. After an analysis of respective costs and benefits, it was concluded that an improvement in sys- tem management would produce the most cost effec- tive improvement in the security of the IT systems reviewed. If systems were well managed and regularly monitored, a large number of the problems with regard to the systems would be reduced or removed.

Resources

In the majority of cases there were inadequate resources available in terms of manpower to carry out the functions required in order to maintain the security of the system, and in terms of the funds to purchase tools to adequately manage the security of the systems.

This is in common with the US JSC report, which stated that: “Because of a lack of qualified personnel and a failure to provide adequate resources, many information systems security tasks are not being per- formed adequately. Too often critical security respon- sibilities are assigned as additional or ancillary duties.”

The resources available for the management of the systems must be improved both in terms of qualified manpower and funds to purchase tools and training, as without the manpower and tools required, the security management of the systems cannot improve.

Poor configuration of the systems

Systems either had not been correctly configured ini- tially or the configuration had subsequently been changed, or both. In addition, most of the systems reviewed had poor or non-existent password genera- tion or management policies. In a number of cases, even the default level of security offered by the provider had not been adopted.

600

The correct configuration of a system is dependent on the requirement of the security policy, the knowledge of the staff and the availability of resources. If the rec- ommendations raised in the previous sections were to be put into practice, it would be possible to configure the systems more effectively and to ensure that they remained configured in the optimal manner.

Systems contained known bugs

All of the systems reviewed contained known bugs for which fixes were available, but which had not been applied. This left the system vulnerable to known attack methods. Further investigation of this problem identified the main reasons as:

System administrators/managers did not know where to obtain patches for the system. It was perceived that the application of patches to software would invalidate the Accreditation/ Certification of the system. System administrators/managers did not have the experience to install the patches to the software. System administrators/managers were unwilling to install patches as they felt that this would be likely to introduce fresh vulnerabilities in their system and were unsure of the pedigree of the patches obtained.

There is a requirement to maintain the system in a secure manner and in order to achieve this, known bugs must be patched, with patches obtained from reliable sources, by staff who understand the impact of the actions that they are taking. In order to achieve this, they must be given access and guidance from a central focus, that is known and trusted. Additionally, system staffs must be trained and gain experience in the systems on which they are employed, in order to pro- vide them with the level of competence that they require to carry out the patching process with confidence.

Redundant software

Software which had been delivered with the system as a part of the package, but which was not required by the organization or for the operation of the system, had not been removed from the systems.The reasons for this failure to reduce the number of vulnerabilities

Computers & Security, Vol. 16, No. 7

(e.g. “if you don’t need sendmail, why leave yourself exposed to all of the sendmail bugs”) were:

l A lack of policy for the removal of such software. l A lack of competence and experience and thus con-

fidence by the system managers/administrators.

Any software which is not security relevant and the functionality of which is not required on the system, as it is configured, should be removed.

The failure to utilize facilities and tools supplied with the software

There was a widespread failure to utilize those facilities and tools that are supplied with the software or hardware when it was delivered. A typical example of this was the failure to unwrap, read or use the security manuals that were delivered as part of the system documentation.

The security and management tools provided with the system should be used unless more suitable tools are identified and utilized. Additionally, the advice provid- ed by the manufacturer on the configuration of the system should be implemented as a baseline and then modified to improve the configuration from that level, rather than starting from the default configuration.

Failure to monitor or audit the activity on the systems

In the majority of cases, there was no monitoring of system activity and audit trails were. either not activat- ed, or if they were activated, they were not checked. This was, in the main, due to lack of resources or ade- quate tools.The lack of adequate tools was partly as a result of the cost of such tools and partially as a lack of availability of suitable tools.

The US GAO report found that, “While Defense is attempting to react to attacks as it becomes aware of them it will not be in a strong position to deter them until it develops and implements more aggressive, proactive detection and reaction programmes,” and recommended the setting of minimum standards for ensuring that system and network security personnel

have sufficient time and training to properly do their jobs. In a separate paper by Dan Farmer, the author of the SATAN software tool, on the security of Internet connected systems, he concluded that “most organizations don’t have the resources or expertise to monitor their own critical systems effectively.” He then proposed that the most efficient solution would be for organizations to employ a third party to maintain the security of these systems.

The level of monitoring and audit of the systems which was being conducted at the time of the tests was not adequate to ensure that they remained con- figured in the optimally secure configuration or that there was no unauthorized activity on the system. There is a requirement to continuously monitor the systems to ensure that they remain configured in the optimally secure configuration.There is also a require- ment to constantly review the system activity to ensure that there is no unauthorized activity on the system. Periodic external audit is required in order to verify the security rules and procedures against external criteria and to utilize tools and techniques that are not available from within the organization.

Poor physical security

In a number of cases, components of the systems, i.e. routers (and in one case a server), were located in unlocked cupboards in corridors.As a result they were prone to theft and to the unauthorized connection of additional items of equipment.

All system components should be secured in such a manner that they cannot be physically accessed by personnel who do not have legitimate access.

Monitoring of the changing threat to systems

In the majority of cases the definition of the threat to a system was not subsequently reviewed in light of the changing threat environment. Therefore, in a number of cases, the security measures being applied to a sys- tem were in response to a threat that existed five or more years ago.

601

Penetration testing and system audit,lAndy Jones

There is a requirement to review the threat to a sys- tem at regular intervals or when the threat is known to have changed. A similar requirement is found in a military system when the state of readiness changes as a result of increased tension.

Technical Issues

As the result of the review of a number of systems with the same basic operating system (although in a number of versions or releases) a number of common factors were identified. These varied with the individ- ual operating system type, but were significant and showed where improvements could be made. In a number of cases, the flaws that existed were at least, in part, a reflection of the system management problems, but are reviewed again in the following sections against the operating system to which they applied.

The use of publicly available tools

Publicly available tools such as SATAN, CRACK and COPS had not been used by the system managers to test the systems for those flaws that are commonly known and understood.The major reasons were: An unwillingness by the management of the organi- zations and the systems to allow ‘hacker’ tools to be used on corporate systems. A lack of experience in the use of such tools and concern with regard to the effects that they might have on the system. A lack of confidence in where an unadulterated copy of such tools could be obtained.

Inefficient segmentation of the network The networks had not been segmented effectively and as a result were all running in an open mode with all traffic being broadcast to the whole community. This caused three primary problems. Firstly, it was possible to intercept data that should have been contained within a local segment of the network from anywhere on the wide area network. Secondly, a terminal from anywhere on the wide area network could emulate a terminal on what should have been the local segment of the net.Thirdly, there was a reduction in the service

availability for the network, as the whole of the wide area network was being flooded by al large quantity of unnecessary traffic.

Intruder detection

The intruder detection software which allows for a configurable number of incorrect access attempts to be made before the user/terminal is disabled was not activated or was poorly configured.

Insecure remote access

Despite it being a requirement on a number of the systems reviewed to access the systems remotely, none of the management groups had considered methods by which remote access could be achieved securely. Rconsole and Secure Console features provided by Novell were not implemented and as a result there was the possibility that privileged access could be obtained without authorization.

Conclusions Penetration testing and system audit, when carried out jointly, can provide an extremely cost effective method of identifying and remedying weaknesses in the orga- nization and the implementation of security of an Information Systems within an department. The pen- etration testing and system audit can be carried out at any time during the life of a system, with minimal dis- ruption to the staff and the operation of the system. It will provide an up to date assessment of the effective- ness of security measures in use to protect the system which cannot be achieved in any other way. If used to support the administration and security staffs and to help them to identify actions that need to be taken in order to improve the security of a system, the poten- tial benefit of this activity is high. The approach used must be threat-based and provide the system owners with a cost effective method of assessing that it is secure against the threat current at the time of the test.

Andy Jones IS an experienced military analyst and IT security spe- cialist with the Defence Evaluation and Research Agency. This paper was first presented at Compsec International ‘97 in London this

November.

602 0167-4048/97$17.00 0 1997 Elsevier Science Ltd