systems development and project management audit/assurance ... · web viewisaca® with more than...

Systems Development and Project Management Audit/Assurance Program

ISACA®

With more than 86,000 constituents in more than 160 countries, ISACA (www.isaca.org) is a recognized worldwide leader in IT governance, control, security and assurance. Founded in 1969, ISACA sponsors international conferences, publishes the ISACA Journal®, and develops international information systems auditing and control standards. It also administers the globally respected Certified Information Systems Auditor™ (CISA®) designation, earned by more than 60,000 professionals since 1978; the Certified Information Security Manager® (CISM®) designation, earned by more than 10,000 professionals since 2002; and the new Certified in the Governance of Enterprise IT™ (CGEIT™) designation.

DisclaimerISACA has designed and created Systems Development and Project Management Audit/Assurance Program (the “Work”), primarily as an informational resource for audit and assurance professionals. ISACA makes no claim that use of any of the Work will assure a successful outcome. The Work should not be considered inclusive of all proper information, procedures and tests or exclusive of other information, procedures and tests that are reasonably directed to obtaining the same results. In determining the propriety of any specific information, procedure or test, audit/assurance professionals should apply their own professional judgment to the specific circumstances presented by the particular systems or IT environment.

Reservation of Rights© 2009 ISACA. All rights reserved. No part of this publication may be used, copied, reproduced, modified, distributed, displayed, stored in a retrieval system or transmitted in any form by any means (electronic, mechanical, photocopying, recording or otherwise) without the prior written authorization of ISACA. Reproduction and use of all or portions of this publication are permitted solely for academic, internal and noncommercial use, and consulting/advisory engagements, and must include full attribution of the material’s source. No other right or permission is granted with respect to this work.

ISACA3701 Algonquin Road, Suite 1010Rolling Meadows, IL 60008 USAPhone: +1.847.253.1545Fax: +1.847.253.1443E-mail: [email protected] site: www.isaca.org

© 2009 ISACA. All rights reserved.

http://www.isaca.org/


ISBN 978-1-60420-082-9Systems Development and Project Management Audit/Assurance ProgramPrinted in the United States of AmericaISACA wishes to recognize: AuthorNorm Kelson, CISA, CGEIT, CPA, The Kelson Group, USA

Expert Reviewers Rafael Eduardo Fabius, CISA, Republica AFAP SA, UruguayJosé Manuel Ballester Fernández, Ph.D., CISA, CISM, CGEIT, IEEE, IT Deusto, SpainAnuj Goel, Ph.D., CISA, CISSP, Citigroup Inc., USALarry Marks, CISA, CGEIT, CFE, CISSP, CSTE, PMP, Resources Global Professionals, USABharath Nallapu, CISA, PMP, Smith, Nallapu & Associates LLP, USAKathy A. Rogers, CISA, USA

ISACA Board of DirectorsLynn Lawton, CISA, FBCS, FCA, FIIA, KPMG LLP, UK, International PresidentGeorge Ataya, CISA, CISM, CGEIT, CISSP, ICT Control SA, Belgium, Vice PresidentHoward Nicholson, CISA, CGEIT, City of Salisbury, Australia, Vice PresidentJose Angel Pena Ibarra, CGEIT, Consultoria en Comunicaciones e Info. SA & CV, Mexico, Vice PresidentRobert E. Stroud, CA Inc., USA, Vice PresidentKenneth L. Vander Wal, CISA, CPA, Ernst & Young LLP (retired), USA, Vice PresidentFrank Yam, CISA, CIA, CCP, CFE, CFSA, FFA, FHKCS, FHKIoD, Focus Strategic Group Inc.,

Hong Kong, Vice PresidentMarios Damianides, CISA, CISM, CA, CPA, Ernst & Young, USA, Past International PresidentEverett C. Johnson Jr., CPA, Deloitte & Touche LLP (retired), USA, Past International President Gregory T. Grocholski, CISA, The Dow Chemical Company, USA, DirectorTony Hayes, Queensland Government, Australia, DirectorJo Stewart-Rattray, CISA, CISM, CSEPS, RSM Bird Cameron, Australia, DirectorAssurance CommitteeGregory T. Grocholski, CISA, The Dow Chemical Company, USA, ChairPippa G. Andrews, CISA, ACA, CIA, Amcor, AustraliaRichard Brisebois, CISA, CGA, Office of the Auditor General of Canada, CanadaSergio Fleginsky, CISA, ICI, UruguayRobert Johnson, CISA, CISM, CISSP, Executive Consultant, USAAnthony P. Noble, CISA, CCP, Viacom Inc., USARobert G. Parker, CISA, CA, CMC, FCA, Deloittte & Touche LLP (retired), CanadaErik Pols, CISA, CISM, Shell International - ITCI, NetherlandsVatsaraman Venkatakrishnan, CISA, CISM, CGEIT, ACA, Emirates Airlines, UAE



Table of Contents

I. Introduction..........................................................................................................................................4II. Using This Document..........................................................................................................................5III. Controls Maturity Analysis..................................................................................................................8IV. Assurance and Control Framework.....................................................................................................9V. Executive Summary of Audit/Assurance Focus................................................................................11VI. Audit/Assurance Program..................................................................................................................14

1. Planning and Scoping the Audit....................................................................................................142. Understanding Supporting Infrastructure......................................................................................173. Initiation Phase...............................................................................................................................174. Planning Phase...............................................................................................................................245. Execution Phase.............................................................................................................................446. Closure Phase.................................................................................................................................677. Postimplementation Phase.............................................................................................................71

VII. Maturity Assessment..........................................................................................................................77VIII. Assessment Maturity vs. Target Maturity..........................................................................................97

I. Introduction

OverviewISACA has developed the IT Assurance FrameworkTM (ITAFTM) as a comprehensive and good-practice-setting model. ITAF provides standards that are designed to be mandatory and are the guiding principles under which the IT audit and assurance profession operates. The guidelines provide information and direction for the practice of IT audit and assurance. The tools and techniques provide methodologies, tools and templates to provide direction in the application of IT audit and assurance processes.

PurposeThe audit/assurance program is a tool and template to be used as a road map for the completion of a specific assurance process. The ISACA Assurance Committee has commissioned audit/assurance programs to be developed for use by IT audit and assurance practitioners. This audit/assurance program is intended to be utilized by IT audit and assurance professionals with the requisite knowledge of the subject matter under review, as described in ITAF, section 2200—General Standards. The audit/assurance programs are part of ITAF, section 4000—IT Assurance Tools and Techniques.

Control FrameworkThe audit/assurance programs have been developed in alignment with the IT Governance Institute® (ITGITM) Control Objectives for Information and related Technology (COBIT®)—specifically COBIT 4.1—using generally applicable and accepted good practices, and reflect ITAF, sections 3400—IT Management Processes, 3600—IT Audit and Assurance Processes, and 3800—IT Audit and Assurance Management.

Many organizations have embraced several frameworks at an enterprise level, including the Committee of Sponsoring Organizations of the Treadway Commission (COSO) Internal Control Framework. The importance of the control framework has been enhanced due to regulatory requirements by the US Securities and Exchange Commission (SEC) as directed by the US Sarbanes-Oxley Act of 2002 and similar legislation in other countries. They seek to integrate control framework elements used by the general audit/assurance team into the IT audit and assurance framework. Since COSO is widely used, it has been selected for inclusion in this audit/assurance program. The reviewer may delete or rename these



columns to align with the enterprise’s control framework.IT Governance Risk and ControlIT governance, risk and control are critical in the performance of any assurance management process. Governance of the process under review will be evaluated as part of the policies and management oversight controls. Risk plays an important role in evaluating what to audit and how management approaches and manages risk. Both issues will be evaluated as steps in the audit/assurance program. Controls are the primary evaluation point in the process. The audit/assurance program will identify the control objectives and the steps to determine control design and effectiveness.

Responsibilities of IT Audit and Assurance ProfessionalsIT audit and assurance professionals are expected to customize this document to the environment in which they are performing an assurance process. This document is to be used as a review tool and starting point. It may be modified by the IT audit and assurance professional; it is not intended to be a checklist or questionnaire. It is assumed that the IT audit and assurance professional holds the Certified Information Systems Auditor (CISA) designation, or has the necessary subject matter expertise required to conduct the work and is supervised by a professional with the CISA designation and necessary subject matter expertise to adequately review the work performed.

II. Using This Document

This audit/assurance program was developed to assist the audit and assurance professional in designing and executing a review over the various phases of the project. The audit/assurance program is segmented according to phase. The audit and assurance professional should customize and select those phases within the scope of the specific audit/assurance review. In addition, the evaluation of specific application-based internal controls is often within the scope, and is addressed in the Internal Controls section of this program. The audit and assurance professional is encouraged to utilize sections of other audit/assurance programs addressing specific applications, IT operations, and other COBIT-related subjects that are impacted by the project. Details regarding the format and use of the document follow.

Work Program StepsThe first column of the program describes the steps to be performed. The numbering scheme used provides built-in work paper numbering for ease of cross-reference to the specific work paper for that section. The physical document was designed in Microsoft® Word. The IT audit and assurance professional is encouraged to make modifications to this document to reflect the specific environment under review.

Steps 1 and 2 are part of the fact gathering and pre-fieldwork preparation. Because the pre-fieldwork is essential to a successful and professional review, the steps have been itemized in this plan. The first-level steps, e.g., 1.1, are in bold type and provide the reviewer with a scope or high-level explanation of the purpose for the substeps.

Beginning in step 3, the steps associated with the work program are itemized. To simplify the use of the program, the audit/assurance program describes the audit/assurance objective—the reason for performing the steps in the topic area. The specific controls follow and are shown in blue type. Each review step is listed below the control. These steps may include assessing the control design by walking through a process, interviewing, observing, or otherwise verifying the process and the controls that address that process. In many cases, once the control design has been verified, specific tests need to be performed to provide assurance that the process associated with the control is being followed. Test objectives are shown in green type. The specific test process follows the test objective.



The Systems Development and Project Management Audit/Assurance Program is intended to be utilized during the various phases of the project. In all phases, project management is addressed. However, in specific systems, development processes and controls will vary based on that phase. Those control areas not applicable, are grayed out. This gives the professional the opportunity to to add them back into the plan if appropriate.

The maturity assessment, which is described in more detail later in this document, makes up the last section of the program.

The audit/assurance plan wrap-up—those processes associated with the completion and review of work papers, preparation of issues and recommendations, report writing and report clearing—has been excluded from this document, since it is standard for the audit/assurance function and should be identified elsewhere in the enterprise’s standards.

COBIT Cross-referenceThe COBIT cross-reference provides the audit and assurance professional with the ability to refer to the specific COBIT control objective that supports the audit/assurance step. The COBIT control objective should be identified for each audit/assurance step in the section. Multiple cross-references are not uncommon. Processes at lower levels in the work program are too granular to be cross-referenced to COBIT. The audit/assurance program is organized in a manner to facilitate an evaluation through a structure parallel to the development process. COBIT provides in-depth control objectives and suggested control practices at each level. As the professional reviews each control, he/she should refer to COBIT 4.1 or the IT Assurance Guide: Using COBIT for good-practice control guidance.

COSO ComponentsAs noted in the introduction, COSO and similar frameworks have become increasingly popular among audit and assurance professionals. This ties the assurance work to the enterprise’s control framework. While the IT audit/assurance function has COBIT as a framework operational audit and assurance professionals use the framework established by the enterprise. Since COSO is the most prevalent internal control framework, it has been included in this document and is a bridge to align IT audit/assurance with the rest of the audit/assurance function. Many audit/assurance enterprises include the COSO control components within their report, and summarize assurance activities to the audit committee of the board of directors.

For each control, the audit and assurance professional should indicate the COSO component(s) addressed. It is possible, but not generally necessary, to extend this analysis to the specific audit step level.

The original COSO internal control framework contained five components. In 2004, COSO was revised as the Enterprise Risk Management (ERM) Integrated Framework and extended to eight components. The primary difference between the two frameworks is the additional focus on ERM and integration into the business decision model. ERM is in the process of being adopted by large enterprises. The two frameworks are compared in figure 1.

The original COSO internal control framework addresses the needs of the IT audit and assurance professional: control environment, risk assessment, control activities, information and communication, and monitoring. As such, ISACA has elected to utilize the five component model for these audit/assurance programs. As more enterprises implement the ERM model, the additional three columns can be added, if relevant. When completing the COSO component columns, consider the definitions of the components as described in figure 1.



Figure 1—Comparison of COSO Internal Control and ERM Integrated FrameworksInternal Control Framework ERM Integrated Framework

Control Environment: The control environment sets the tone of an organization, influencing the control consciousness of its people. It is the foundation for all other components of internal control, providing discipline and structure. Control environment factors include the integrity, ethical values, management’s operating style, delegation of authority systems, as well as the processes for managing and developing people in the organization.

Internal Environment: The internal environment encompasses the tone of an organization, and sets the basis for how risk is viewed and addressed by an entity’s people, including risk management philosophy and risk appetite, integrity and ethical values, and the environment in which they operate.

Objective Setting: Objectives must exist before management can identify potential events affecting their achievement. Enterprise risk management ensures that management has in place a process to set objectives and that the chosen objectives support and align with the entity’s mission and are consistent with its risk appetite.Event Identification: Internal and external events affecting achievement of an entity’s objectives must be identified, distinguishing between risks and opportunities. Opportunities are channeled back to management’s strategy or objective-setting processes.

Risk Assessment: Every entity faces a variety of risks from external and internal sources that must be assessed. A precondition to risk assessment is establishment of objectives, and thus risk assessment is the identification and analysis of relevant risks to achievement of assigned objectives. Risk assessment is a prerequisite for determining how the risks should be managed.

Risk Assessment: Risks are analyzed, considering the likelihood and impact, as a basis for determining how they could be managed. Risk areas are assessed on an inherent and residual basis.

Risk Response: Management selects risk responses–avoiding, accepting, reducing or sharing risk–developing a set of actions to align risks with the entity’s risk tolerances and risk appetite.

Control Activities: Control activities are the policies and procedures that help ensure management directives are carried out. They help ensure that necessary actions are taken to address risks to achievement of the entity's objectives. Control activities occur throughout the organization, at all levels and in all functions. They include a range of activities as diverse as approvals, authorizations, verifications, reconciliations, reviews of operating performance, security of assets and segregation of duties.

Control Activities: Policies and procedures are established and implemented to help ensure the risk responses are effectively carried out.

Information and Communication: Information systems play a key role in internal control systems as they produce reports, including operational, financial and compliance-related information that make it possible to run and control the business. In a broader sense, effective communication must ensure information flows down, across and up the organization. Effective communication should also be ensured with external parties, such as customers, suppliers, regulators and shareholders.

Information and Communication: Relevant information is identified, captured, and communicated in a form and timeframe that enable people to carry out their responsibilities. Effective communication also occurs in a broader sense, flowing down, across, and up the entity.

Monitoring: Internal control systems need to be monitored—a process that assesses the quality of the system’s performance over time. This is accomplished through ongoing monitoring activities or separate evaluations. Internal control deficiencies detected through these monitoring activities should be reported upstream and corrective actions should be taken to ensure continuous improvement of the system.

Monitoring: The entirety of enterprise risk management is monitored and modifications made as necessary. Monitoring is accomplished through ongoing management activities, separate evaluations or both.

Information for figure 1 was obtained from the COSO web site www.coso.org/aboutus.htm.

Reference/HyperlinkGood practices require the audit and assurance professional to create a work paper for each line item, which describes the work performed, issues identified and conclusions. The reference/hyperlink is to be used to cross-reference the audit/assurance step to the work paper that supports it. The numbering system of this document provides a ready numbering scheme for the work papers. If desired, a link to the work paper can be pasted into this column.

Issue Cross-referenceThis column can be used to flag a finding/issue that the IT audit and assurance professional wants to


http://www.coso.org/aboutus.htm


further investigate or establish as a potential finding. The potential findings should be documented in a work paper that indicates the disposition of the findings (formally reported, reported as a memo or verbal finding, or waived).

CommentsThe comments column can be used to indicate the waiving of a step, or other notations. It is not to be used in place of a work paper describing the work performed.

III. Controls Maturity Analysis

One of the consistent requests of stakeholders who have undergone IT audit/assurance reviews is a desire to understand how their performance compares to good practices. Audit and assurance professionals must provide an objective basis for the review conclusions. Maturity modeling for management and control over IT processes is based on a method of evaluating the organization, so it can be rated from a maturity level of nonexistent (0) to optimized (5). This approach is derived from the maturity model that the Software Engineering Institute (SEI) of Carnegie Mellon University defined for the maturity of software development.

The IT Assurance Guide Using COBIT, Appendix VII—Maturity Model for Internal Control, in figure 2, provides a generic maturity model showing the status of the internal control environment and the establishment of internal controls in an enterprise. It shows how the management of internal control, and an awareness of the need to establish better internal controls, typically develops from an ad hoc to an optimized level. The model provides a high-level guide to help COBIT users appreciate what is required for effective internal controls in IT and to help position their enterprise on the maturity scale.

Figure 2—Maturity Model for Internal ControlMaturity Level Status of the Internal Control Environment Establishment of Internal Controls0 Non-existent There is no recognition of the need for internal control.

Control is not part of the organization’s culture or mission. There is a high risk of control deficiencies and incidents.

There is no intent to assess the need for internal control. Incidents are dealt with as they arise.

1 Initial/ad hoc There is some recognition of the need for internal control. The approach to risk and control requirements is ad hoc and disorganized, without communication or monitoring. Deficiencies are not identified. Employees are not aware of their responsibilities.

There is no awareness of the need for assessment of what is needed in terms of IT controls. When performed, it is only on an ad hoc basis, at a high level and in reaction to significant incidents. Assessment addresses only the actual incident.

2 Repeatable but Intuitive

Controls are in place but are not documented. Their operation is dependent on the knowledge and motivation of individuals. Effectiveness is not adequately evaluated. Many control weaknesses exist and are not adequately addressed; the impact can be severe. Management actions to resolve control issues are not prioritized or consistent. Employees may not be aware of their responsibilities.

Assessment of control needs occurs only when needed for selected IT processes to determine the current level of control maturity, the target level that should be reached and the gaps that exist. An informal workshop approach, involving IT managers and the team involved in the process, is used to define an adequate approach to controls for the process and to motivate an agreed-upon action plan.

3 Defined Controls are in place and adequately documented. Operating effectiveness is evaluated on a periodic basis and there is an average number of issues. However, the evaluation process is not documented. While management is able to deal predictably with most control issues, some control weaknesses persist and impacts could still be severe. Employees are aware of their responsibilities for control.

Critical IT processes are identified based on value and risk drivers. A detailed analysis is performed to identify control requirements and the root cause of gaps and to develop improvement opportunities. In addition to facilitated workshops, tools are used and interviews are performed to support the analysis and ensure that an IT process owner owns and drives the assessment and improvement process.

4 Managed and Measurable

There is an effective internal control and risk management environment. A formal, documented evaluation of controls occurs frequently. Many controls are automated and regularly reviewed. Management is likely to detect most control issues, but not all issues are routinely identified. There is consistent follow-up to address identified control weaknesses. A limited, tactical use of technology is applied to automate controls.

IT process criticality is regularly defined with full support and agreement from the relevant business process owners. Assessment of control requirements is based on policy and the actual maturity of these processes, following a thorough and measured analysis involving key stakeholders. Accountability for these assessments is clear and enforced. Improvement strategies are supported by business cases. Performance in achieving the desired outcomes is consistently monitored. External control reviews are organized occasionally.



Figure 2—Maturity Model for Internal ControlMaturity Level Status of the Internal Control Environment Establishment of Internal Controls

5 Optimized An enterprisewide risk and control program provides continuous and effective control and risk issues resolution. Internal control and risk management are integrated with enterprise practices, supported with automated real-time monitoring with full accountability for control monitoring, risk management and compliance enforcement. Control evaluation is continuous, based on self-assessments and gap and root cause analyses. Employees are proactively involved in control improvements.

Business changes consider the criticality of IT processes and cover any need to reassess process control capability. IT process owners regularly perform self-assessments to confirm that controls are at the right level of maturity to meet business needs and they consider maturity attributes to find ways to make controls more efficient and effective. The organization benchmarks to external best practices and seeks external advice on internal control effectiveness. For critical processes, independent reviews take place to provide assurance that the controls are at the desired level of maturity and working as planned.

The maturity model evaluation is one of the final steps in the evaluation process. The IT audit/assurance professional can address the key controls within the scope of the work program, and formulate an objective assessment of the maturity level of the control practices. The maturity assessment can be a part of the audit/assurance report and can be used as a metric from year to year to document progression in the enhancement of controls. However, it must be noted that the perception as to the maturity level may vary between the process/IT asset owner and the auditor. Therefore, an auditor should obtain the concerned stake holder’s concurrence before submitting the final report to the management.

At the conclusion of the review, once all findings and recommendations are completed, the professional assesses the current state of the COBIT control framework, and assigns it a maturity level using the six-level scale. Some practitioners utilize decimals (x.25, x.5, x.75) to indicate gradations in the maturity model. As a further reference, COBIT provides a definition of the maturity designations by control objective. While this approach is not mandatory, the process is provided as a separate section at the end of the audit/assurance program for those enterprises that wish to implement it. It is suggested that a maturity assessment be made at the COBIT control level. To provide further value to the client/customer, the professional can also obtain maturity targets from the client/customer. Using the assessed and target maturity levels, the professional can create an effective graphic presentation which describes the achievement or gaps between the actual and targeted maturity goals. A graphic is provided as the last page of the document (section VIII), based on sample assessments.

IV. Assurance and Control Framework

ISACA IT Assurance Framework and StandardsSystems development and project management are included in ITAF and the following are the relevant sections: 3420—IT Project Management 3450—IT Processes 3490—IT Support of Regulatory Compliance 3630.8—Systems Development Life Cycle 3650—Auditing Application Controls 3657—Auditing Alternative Software Development Strategies

ISACA has long recognized the specialized nature of IT assurance and strives to advance globally applicable standards. Guidelines and procedures provide detailed guidance on how to follow those standards. IS Auditing Guidelines G23 System Development Life Cycle (SDLC) Review, G26 Business Process Reengineering (BPR) Project Reviews and G29 Post-implementation Review, and IS Auditing Procedure P10 Business Application Change Control are relevant to this audit/assurance program.



ISACA Controls FrameworkCOBIT is an IT governance framework and supporting tool set that allows managers to bridge the gap among control requirements, technical issues and business risks. COBIT enables clear policy development and good practice for IT control throughout enterprises.

Utilizing COBIT as the control framework on which IT audit/assurance activities are based aligns IT audit/assurance with good practices as developed by the enterprise.

The COBIT Acquire and Implement (AI) domain addresses good practices for systems development, the Plan and Organize (PO) domain, IT process PO10 addresses managing projects. The COBIT areas for this evaluation include: PO10 Manage projects—A program and project management framework for the management of all

IT projects is established. The framework ensures the correct prioritization and coordination of all projects. The framework includes a master plan, assignment of resources, definition of deliverables, approval by users, a phased approach to delivery, QA, a formal test plan, and testing and post-implementation review after installation to ensure project risk management and value delivery to the business. This approach reduces the risk of unexpected costs and project cancellations, improves communications to and involvement of business and end users, ensures the value and quality of project deliverables, and maximizes their contribution to IT-enabled investment programs.

AI1 Identify automated solutions—The need for a new application or function requires analysis before acquisition or creation to ensure that business requirements are satisfied in an effective and efficient approach. This process covers the definition of the needs, consideration of alternative sources, review of technological and economic feasibility, execution of a risk analysis and cost-benefit analysis, and conclusion of a final decision to ‘make’ or ‘buy.’ All these steps enable organizations to minimize the cost to acquire and implement solutions while ensuring that they enable the business to achieve its objectives.

AI2 Acquire and maintain application software—Applications are made available in line with business requirements. This process covers the design of the applications, the proper inclusion of application controls and security requirements, and the development and configuration in line with standards. This allows organizations to properly support business operations with the correct automated applications.

AI3 Acquire and maintain technology infrastructure—Organizations have processes for the acquisition, implementation and upgrade of the technology infrastructure. This requires a planned approach to acquisition, maintenance and protection of infrastructure in line with agreed-upon technology strategies and the provision of development and test environments. This ensures that there is ongoing technological support for business applications.

AI4 Enable operation and use—Knowledge about new systems is made available. This process requires the production of documentation and manuals for users and IT, and provides training to ensure the proper use and operation of applications and infrastructure.

AI5 Procure IT resources—IT resources, including people, hardware, software and services, need to be procured. This requires the definition and enforcement of procurement procedures, the selection of vendors, the setup of contractual arrangements, and the acquisition itself. Doing so ensures that the organization has all required IT resources in a timely and cost-effective manner.

AI7 Install and accredit solutions and changes—New systems need to be made operational once development is complete. This requires proper testing in a dedicated environment with relevant test data, definition of rollout and migration instructions, release planning and actual promotion to production, and a post-implementation review. This assures that operational systems are in line with the agreed-upon expectations and outcomes.

Refer to the IT Governance Institute’s COBIT Control Practices: Guidance to Achieve Control Objectives for Successful IT Governance, 2nd Edition, published in 2007, for the related control practice value and



risk drivers.V. Executive Summary of Audit/Assurance Focus

Systems Development and Project ManagementThe systems development methodology (sometimes referred to as the systems development life cycle) is composed of the phases deployed in the development or acquisition of a software system. Experience has shown that development of a new system cannot be executed in a vacuum. The business must be involved as the driver, owner and overall manager of the project. The IT development team is a member of this overall project team that is responsible for executing technical development of the business plan. Therefore, the key to success of any IT initiative is to consider the project as part of a larger scope, which is the implementation of a business solution. To keep the project on track, it is necessary to provide project management, a structured set of activities concerned with delivering to the enterprise a defined capability (that is necessary but not sufficient to achieve a required business outcome) based on an agreed-upon schedule and budget. Many entities use the expression “program” to describe a business initiative. The definition of program, a structured grouping of interdependent projects that includes the full scope of business, process, people, technology and organizational activities that are required (both necessary and sufficient) to achieve a clearly specified business outcome, is a superset of a project.

Systems development has traditionally used the waterfall approach, a procedure-focused development cycle with formal sign-off at the completion of each level. The processes include: Feasibility study Requirements study Requirements definition Detailed design Programming Testing Installation/accrediation Postimplementation review

With the advent of prototyping, fourth-generation programming language (4GL) application development tools and other approaches, more dynamic frameworks were required. These include: PMBOK,1 PRINCE2,2 Agile,3 RUP4 and Spiral.5 While each framework utilizes different naming conventions, the key processes are common to all: Initiation—Preparation of the initial concept, business case, scope, creation of project team, and

preparation and approval of budget and capital expenditure requests Planning—Preparation of the detailed plan, requirements definition, acquisition cycle and acquisition

of external consulting assistance Execution—Execution of project plan once planning phase is completed to the go-live phase.

Subphases of the phase include:– Creating business processes (automated and manual), instructions and training– Establishing controls

1 Project Management Body of Knowledge is a project management standard developed by the Project Management Institute (PMI).

2 Projects in a Controlled Environment was developed by The Open Geospatial Consortium Inc., an international industry consortium of companies, government agencies and universities participating in a consensus process to develop publicly available interface specifications.

3 RAD, developed by James Martin, was built on prototyping.4 Rational Unified Process is an iterative software development process framework created by the Rational Software

Corporation, a division of IBM.5 This software development process combines elements of both design and prototyping-in-stages, in an effort to combine

advantages of top-down and bottom-up concepts.


http://en.wikipedia.org/wiki/Top-down_and_bottom-up_design

http://en.wikipedia.org/wiki/Prototyping

http://en.wikipedia.org/wiki/Design

http://en.wikipedia.org/wiki/Software_development_process

http://en.wikipedia.org/wiki/Rational_Software

http://en.wikipedia.org/wiki/Software_development_process

http://en.wikipedia.org/wiki/Iterative


– Establishing conversion and transition processes, balancing routines, and verification of process accuracy and completeness

– Testing:- Business processes- Conversion- Stress testing the processes- Fallbacks

Implementation Reconciliation of conversion Go live—Activities associated with the commencement of the new process Closure—Closing project, accounting and costs finalized Postimplementation:

– Review of the project success– Financial review of the business case vs. results

The decision to perform reviews at each phase is dependent upon the risks identified and the reliance being placed on the application. The most important phases to audit/assurance professionals include planning, execution and postimplementation. The initiation phase may require review if the business case is questioned. It is recommended that the go-live phase be reviewed after the fact as a part of a postimplementation review to ensure that the audit/assurance process does not interfere with the go-live activities.

The systems development process occurs over a period of time, which can range from a few short weeks for a small project to several years for large-scale projects and/or programs. For larger projects with extended durations, it may be necessary to schedule multiple reviews of the same project. The timing of the review needs to be in alignment with the development schedule and key milestone dates.

Business Impact and RiskApplications developed or acquired are the backbone of the business’s operations. These systems (both automated and manual) provide input into the financial reporting systems, are the source for analysis related to business decisions, and either perform or control processes critical to the business’s livelihood. Failure to implement good-practice systems development and project management may result in: Inappropriate supplier selection Solution failing to meet business and/or user requirements, not performing as expected, or unable to

integrate with the strategic IT plan, information architecture and technology direction Misunderstanding of project objectives and requirements Insufficient stakeholder participation in defining requirements and reviewing deliverables Incorrect solution selected or significant missing requirements discovered later in the project,

causing costly reworking and implementation delays Alternate solutions not identified High costs of fragmented solutions Contractual discrepancies and gaps between business expectations and supplier capabilities Management unaware of risks in the acquisition of software Gaps between controls and actual threats or risks System security and confidentiality compromised Invalid transactions or transactions processed incorrectly Costly compensating controls Reduced system availability and questionable integrity of information Poor software quality, inadequate testing and a high number of failures Disorganized and ineffective approach to project management, inappropriate priorities, delayed



critical functions, inappropriate priorities Inadequate budgets and resources Failure to respond to project issues with optimal and approved decisions Unclear responsibilities and accountabilities for ensuring cost control and project success Increased reliance on key staff, problems in daily operations, help desk overload Inability to implement new system or ability to back-out new system and restore old system

Objectives and ScopeObjectives—The objectives of the systems development and project management audit/assurance review are to: Provide management with an independent assessment of the progress, quality and attainment of

project/program objectives at defined milestones within the project/program Provide management with an evaluation of the internal controls of proposed business processes at a

point in the development cycle where enhancements can be easily implemented and processes adapted

Satisfy process audit/assurance objectives in reviewing the process before it goes live, place future reliance on the process based upon the assurance work performed while the application is under development, and implement integrated computer-assisted audit techniques (CAATs) as part of the design of the application

Scope—The review will focus upon the (initiation/planning/execution/closure/postimplementation) phase of the systems development process for the {insert application name}. It will rely upon the systems development methodology to provide a design, development, and testing methodology and the project management methodology to provide accurate and efficient planning, budgeting and cost control. Within each phase, the review will address: Governance Project management Budget Internal controls Business process Third-party providers and other external influences

The scoping of the systems development and project management program must be further modified as appropriate for the application and process under development. It may be necessary to take steps from the Generic Applications audit program to supplement this program as necessary.

Minimum Audit SkillsThe IT audit and assurance professional must have an understanding of the good-practice systems development and project management processes. Those professionals having achieved CISA certification should have these skills. Technical skills necessary to perform some audit steps may require specific understanding of the operating system environments in use, library management systems and computer-assisted audit techniques (CAATs).



VI. Audit/Assurance Program

Audit/Assurance Program StepCOBIT Cross-

reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

1 . PLANNING AND SCOPING THE AUDIT

1.1 Define audit/assurance objectives.The audit/assurance objectives are high level and describe the overall audit goals.

1.1.1 Review the audit/assurance objectives in the introduction to this audit/assurance program.

1.1.2 Modify the audit/assurance objectives to align with the audit/assurance universe, annual plan and charter.

1.2 Define boundaries of review.The review must have a defined scope. The reviewer must understand the operating environment and prepare a proposed scope, subject to a later risk assessment.

1.2.1 Perform a high-level walkthrough of the project to be reviewed.

1.2.1.1 Determine what phase the project is in.

1.2.1.2 Obtain a list of implemented changes to production for the test period.

1.2.1.3 Determine the applications and/or operating environments serviced by the project.

1.2.1.4 Obtain a list of the Sarbanes-Oxley critical applications.

1.2.1.5 Obtain a list of implemented new systems or major enhancements to existing systems for the period under test.

1.2.2 Establish initial boundaries of the audit/assurance review.

1.2.2.1 Identify limitations and/or constraints affecting audit of specific systems.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

1.2.3 Review all phases and steps within this audit/assurance program.

1.2.3.1 Select those steps within the phase that are applicable to the scope and boundaries established.

1.2.3.2 Consider additional steps as required based on the scope established.

1.2.3.3 Consider using additional audit/assurance programs for application specific and IT operational issues.

1.3 Define assurance.The review requires two sources of standards. The corporate standards defined in policy and procedure documentation establish the corporate expectations. At minimum, corporate standards should be implemented. The second source, a good-practice reference, establishes industry standards. Enhancements should be proposed to address gaps between the two.

1.3.1 Obtain company systems development standards, systems development methodology manual, project management standards and project methodology manual.

1.3.2 Determine if COBIT and the appropriate systems development framework will be used as a good-practice reference.

1.4 Identify and document risks.The risk assessment is necessary to evaluate where audit resources should be focused. In most entities, audit resources are not available for all processes. The risk-based approach ensures utilization of audit resources in the most effective manner.

1.4.1 For the project identified as potentially being inscope:

1.4.1.1 Identify the business risk associated with the application being developed.

1.4.1.2 Identify the technology risks associated with the application being developed.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

1.4.1.3 Evaluate business and technology risks.

1.4.2 Based on the risk assessment, identify changes to the scope.

1.4.3 Discuss the risks with IT, business and operational audit management, and adjust the risk assessment.

1.4.4 Based on the risk assessment, revise scope.

1.5 Define the change process.The initial audit approach is based upon the reviewer’s understanding of the operating environment and associated risks. As further research and analysis is performed, changes to the scope and approach will result.

1.5.1 Identify the senior IT assurance resource responsible for the review.

1.5.2 Establish the process for suggesting and implementing changes to the audit/assurance program and required authorizations.

1.6 Define assignment success.The success factors need to be identified. Communication among the IT audit/assurance team, other assurance teams and the enterprise is essential.

1.6.1 Identify the drivers for a successful review (This should exist in the assurance function’s standards and procedures).

1.6.2 Communicate success attributes to the process owner or stakeholder and obtain agreement.

1.7 Define audit/assurance resources required.The resources required are defined in the introduction to this audit/assurance program.

1.7.1 Determine audit/assurance skills necessary for review.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

1.7.2 Determine estimated total resources (hours) and time frame (start and end dates) required for review.

1.8 Define deliverables.The deliverable is not limited to the final report. Communications between the audit/assurance teams and the process owner are essential to assignment success.

1.8.1 Determine the interim deliverables, including initial findings, status reports, draft reports, due dates for responses and the final report.

1.9 CommunicationsThe audit/assurance process is clearly communicated to the customer/client.

1.9.1 Conduct an opening conference to discuss the review objectives with the executive responsible for operating systems and infrastructure.

2 . UNDERSTANDING SUPPORTING INFRASTRUCTURE

2.1 The systems development and project management process are supported by entity standards, processes, and procedures. To properly evaluate the process, the supporting infrastructure needs to be reviewed and evaluated.

2.1.1 Review project management documentation and establish an understanding of the process and its controls.

2.1.2 Review systems development documentation and establish an understanding of the process and its controls.

3 . INITIATION PHASEThe initiation phase addresses preparation of the initial concept, business case, scope, creation of project team, and preparation and approval of budget and capital expenditure requests. If a third-party is required during the initiation phase, procurement of that service is a part of this phase.

3.1 GovernanceAudit/assurance objective: Management should provide adequate governance over the project to ensure that the project is adequately defined and approved by senior management and the




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

business, and technical resources are assigned. Procedures should be defined to keep management informed of the progress. Communications and escalation procedures should be in place to allow management to respond to issues as they arise.

3.1.1 Business caseControl: A business case has been prepared and reviewed by management. The business case is the rationale for initiating the project, expected benefits, estimated costs, and key attributes to evaluate the success of the project.

PO10.1PO10.2ME4.2ME4.3

X

3.1.1.1 Obtain the business case document.

3.1.1.2 Review the business case to determine if it describes the reason for the request, expected benefits, estimated cost, cost savings or revenue generation, estimated return on investment (ROI) and key performance indicators (KPIs) for the project.

3.1.1.3 Interview the stakeholders to determine their involvement in the process.3.1.1.4 Review the basis for the business case to determine if the assumptions used to justify the

project were supported.

3.1.2 Scope managementControl: The initial scope of the project has been established through a feasibility study, alignment with the IT architecture and the development of an initial high-level project plan.

AI1.1AI1.2AI1.3AI1.4

X X

3.1.2.1 Obtain the initial feasibility study.3.1.2.2 Based upon the feasibility document and interviews with management, determine if the

scope of the project was appropriately defined.3.1.2.3 If a feasibility study had not been performed, determine how the scoping decisions are

achieved and whether they support the objectives of the enterprise.3.1.2.4 Determine if appropriate levels of management have reviewed the initial scope.

3.1.3 Roles and responsibilitiesControl: The responsibility for the project is assigned to senior stakeholders from the affected business units and IT.

ME1.5AI1.4 X X X X

3.1.3.1 Determine if a steering committee has been established to oversee this project.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

3.1.3.2 Determine the executive sponsor and chairperson of the steering committee.3.1.3.3 Determine if the chairperson has adequate authority to lead this project.3.1.3.4 Determine if the business units affected by the project are represented in the steering

committee at the appropriate executive level.3.1.3.5 Determine the role of the steering committee; review the charter or similar document.3.1.3.6 Determine if the senior project team has been assigned and report to the steering committee.

Identify the project leader. If the leader is not from the business (e.g., IT is leading the process), determine the effect of this leadership on the project.

3.1.4 ApprovalsControl: The project is approved by senior management, depending on the investment and criticality of the project, and the approvals are documented.

PO5.1PO5.4PO10.3PO10.4AI1.4

X X X

3.1.4.1 Determine who the senior executive responsible for the project is, obtain evidence of his/her approval of the project, and determine if he/she is at the appropriate authority level to approve the project.

3.1.4.2 Determine if a project funding request has been prepared and approved.3.1.4.3 Determine if the approvals are in compliance with organizational policies.3.1.4.4 Determine if the budget has the appropriate capital expenditure and/or expense categories

based on accounting practices.

3.1.5 Return on investment and key performance indicatorsControl: Metrics to objectively evaluate the success of a project are established.

PO5.1PO5.5PO10.3ME4.3

X X

3.1.5.1 Determine if the expected return on investment has been defined in the business case. Evaluate the effectiveness of this metric and the objectivity of its calculation.

3.1.5.2 Determine if key performance indicators have been established to measure the performance of the project team and the project.

3.1.6 Escalation management X X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

Control: Escalation of serious project issues should be directed to the steering committee and senior management on a timely basis; the escalation should be documented and resolution monitored.

PO10.13

3.1.6.1 Obtain the escalation procedure and evaluate the control design to ensure timely response.

3.1.7 Buy or build decisionsControl: Buy or build decisions are based on fact. Supporting documentation supports the buy or build decision.

AI1.3AI2.5ME4.2

X X

3.1.7.1 Determine if the project includes a buy or build decision.

3.1.7.2 If a buy/build decision is to be or was made, determine the objectivity of the criteria used in making the decision.

3.1.7.3 Determine that the decision to buy or build was not influenced by compensation or other rewards.

3.2 Project managementAudit/assurance objective: The project management approach should be commensurate with the size, complexity and regulatory requirements of the project. The project management controls should ensure adequate oversight of the project (financial, meeting deadlines, etc.), appropriate involvement by the stakeholders, iterative evaluation of risks, monitoring of issues, and escalation of issues where required.

3.2.1 Integration of business/information managementControl: The business and information management teams are integrated, information requirements are clearly documented, project objectives are aligned with the business and information strategies; and all affected business units are involved in the project. The steering committee reviews the effectiveness of the integration.

PO8.1PO10.3PO10.6ME4.2

X X X X

3.2.1.1 Interview team leaders representing the business units and information technology. Determine if the project teams are in alignment with their separate organization’s strategies.

3.2.1.2 Obtain the project organization chart and determine leadership positions.3.2.1.3 If IT is managing the project, determine the process to ensure full participation and

agreement with the business units.

3.2.2 Composition of project team PO10.3 X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

Control: The project team consists of a project team leader with appropriate project management experience and the team consists of the skill sets and authority levels from their respective business units.

PO10.8

3.2.2.1 Evaluate the experience of the project team leader. Consider his/her professional experience, knowledge of the business processes being developed, leadership skills and authority granted by the steering committee.

3.2.2.2 Determine if there are any restrictions or encumbrances that would preclude the project team leader from effectively leading the project.

3.2.2.3 Review skills of the project team. Determine if the appropriate skills are represented. Consider business knowledge, representation of affected business units, IT experience, etc.

3.2.3 Risk and issue managementControl: Risk analysis has been applied to the project during the initial phase; risks have been identified. Where risks can be mitigated, appropriate processes have been implemented; where risks are inherent to the process, appropriate monitoring processes are in place.

PO9.1PO10.2PO10.9PO10.13

AI1.2

X X X X

3.2.3.1 Determine if the project team has prepared an initial risk assessment.3.2.3.2 Review the risk assessment to determine if the assessment is comprehensive and risks are

clearly identified. Consider reputational, business, operational, financial, regulatory and organizational risks.

3.2.3.3 Review the monitoring process to ensure that the steering committee and senior project sponsors have reviewed the risk assessment.

3.2.4 Escalation proceduresControl: Escalation procedures are established to include monitoring by the steering committee.

PO10.3 X X X

3.2.4.1 Review escalation procedures to verify the existence of an escalation plan and the inclusion of a documented review process by project management and the steering committee.

3.2.5 Quality managementControl: Project sponsor has defined specific quality expectations and criteria.

PO8.6PO10.10

AI2.8X X X

3.2.5.1 Review quality management requirements; identify the quality assurance process. Consider user acceptance criteria, review process for financial models; review significant decisions,




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

programming, design oversight, etc.3.2.5.2 Review management approval of quality plan and the communications with project team

and stakeholders.

3.2.6 Use of development methodologyControl: The project utilizes the organization’s development methodology and the methodology is appropriate for the size, scope and architecture of the project.

PO10.2PO10.3 X X

3.2.6.1 Determine the systems development methodology used for the project.3.2.6.2 Based on the project’s size, scope and architecture, verify that the systems development

methodology is comprehensive to achieve the goals of the project and is not too complex for the project.

3.2.7 Change management

3.2.8 Planning and control

3.2.9 Progress control

3.2.10 Expense and time management

3.2.11 CommunicationsControl: A communications plan is established to provide stakeholders and project leadership with appropriate information to ensure that the project meets functionality, budgetary and timeline goals.

PO10.1PO10.2 X X

3.2.11.1 Obtain the communications plan.

3.2.11.1.1 Verify that the appropriate stakeholders, project participants and sponsors are included in the distribution lists.

3.2.11.1.2 Verify that the communications plan includes the method of communications (e-mail, presentation, formal report, status report, etc.) and content to be issued, including their frequency.

3.2.11.1.3 Determine if the plan includes exception reports.

3.3 BudgetAudit/assurance objective: The budget and accounting processes should be accurate, complete




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

and provide the information necessary to manage the project.

3.3.1 Budget status

3.3.2 AccountingControl: The recognition of expenses vs. capital expenditure is in compliance with tax and accounting principles.

ME1.2 X

3.3.2.1 Determine if capital expenditure requests for the project have been authorized by the appropriate approver.

3.3.2.2 Determine if the treatment of the capital expenditure is in compliance with accounting and tax requirements.

3.3.2.3 Determine if initial project work has been expensed with the intent to later transfer the expenses to the capital expenditure account.

3.3.3 Time accounting

3.4 Internal controls

3.4.1 Internal control requirements

3.4.2 Internal control risk assessment

3.4.3 CAATs development

3.5 Adequacy of testing

3.6 Implementation

3.7 Third-party ProvidersAudit/Assurance Objective: The use and procurement of third-party assistance should be defined, and the criteria for obtaining and evaluating third-party services objectively established and documented.

3.7.1 Service ProvidersControl: Service provider requirements are clearly stated; the criteria for soliciting and accepting services are documented and follow a prescribed procedure.

AI5.1AI5.2AI5.3AI5.4

X

3.7.1.1 Determine if a business case has been documented for acquiring the services of a third-party




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

provider.3.7.1.2 Determine if internal alternatives have been considered.3.7.1.3 Review the documented criteria for soliciting and accepting third-party services.3.7.1.4 Determine if a third-party is necessary for the initial and planning phases.3.7.1.5 If a third-party is necessary for the planning phase, review the procurement process and

determine if procedures have been followed.

4 . PLANNING PHASEThe planning phase addresses the preparation of the project’s detail plan. During this phase, the requirements definition, describing the specific data attributes and business processes, is prepared. If the solution involves the purchase of software, an acquisition cycle is initiated. This involves soliciting requests for proposals (RFPs), evaluating vendor responses, selecting a provider and negotiating a contract. If the solution is to be designed in-house, the planning phase involves a detailed design of the solution for use by the developers in the execution phase.

4.1 GovernanceAudit/assurance objective: Management should provide adequate governance over the project to ensure that the project is adequately planned and the business and technical resources are assigned. Procedures should be defined to keep management informed of the progress. Communications and escalation procedures should be in place to allow management to respond to issues as they arise.

4.1.1 Business CaseControl: On a regular basis, the project team leadership monitors and provides reports to executive sponsors on the continued alignment of the project plan with the business case.

PO10.1AI1.1ME1.5ME4.2ME4.3

X X X

4.1.1.1 Verify that the stakeholders have received a formally documented statement defining the objective, scope and business value of the project before work has begun in the planning phase.

4.1.1.2 Verify that the project has been agreed upon and that there is documented acceptance by key stakeholders, executive sponsors and the steering

X X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

committee.

4.1.1.3 Review project team procedures to verify that the project team routinely analyzes the alignment of the project plan and the business case.

4.1.1.4 Review project team communications with the executive sponsors to determine that the project plan and business case are in alignment, or that the communications describe changes to either the business case or the plan to maintain alignment.

4.1.2 Scope ManagementControl objectives: The scope of the project is clearly defined and a project plan has been developed that clearly identifies the phases, processes and subprocesses. Responsibility for managing scope changes is defined and procedures are in place to obtain approval of scope changes from the project steering committee or executive sponsors.

PO10.5 X X X

4.1.2.1 Obtain the project plan.

4.1.2.1.1 Review the project plan and evaluate the detail of the plan. At minimum, a detailed plan of phases, processes, subprocesses, milestones and resource assignments should be documented. This plan may change over time, and should be considered a living document.

4.1.2.1.2 Based on the reviewer’s knowledge of the project, assess the completeness of the project plan.

4.1.2.2 Obtain the scope change procedure and determine if the procedure requires steering committee involvement when the scope is changed. For the sample of selected changes, obtain the project notebook for all sampled changes to verify that each has: Been entered into the project management tool A cost-benefit analysis performed if within the threshold that requires

such analysis An analysis of effect on the overall project, performed and approved by




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

the project manager and executive sponsor (if above established threshold)

An analysis of resources performed and approved4.1.2.2.1 Test objective: To verify that scope changes are implemented with the knowledge and approval of the steering committee or executive sponsors

4.1.2.2.1.1 Select a representative sample of scope changes, focusing on those with the greatest impact to the project.

4.1.2.2.1.2 Verify the following for the scope change: Documented approval of management Timeliness of approval

4.1.2.2.2 Determine if the process has been delayed or circumvented.

4.1.2.3 Determine that “out of scope” areas are clearly identified.

4.1.3 Roles and responsibilitiesControl: Roles and responsibilities of the project team are clearly identified; appropriate subject matter experts and stakeholders are included on the project team; and the division of responsibilities is appropriate for the project and entity level organizational structure (including separation of duties).

PO7.3PO10.3 X X

4.1.3.1 Determine if the following roles are identified: Owner, executive sponsor, project leader, steering committee representative (from the project team).

4.1.3.2 Determine if the project team includes members from all affected business units, systems development, IT operations, information security, internal audit, legal, compliance, suppliers, and other appropriate areas.

4.1.3.3 Determine the appropriateness of the division of responsibilities among the organization’s executive group (steering committee), the executive sponsor, the project leader and third-party suppliers (if applicable).

4.1.3.4 Determine who is responsible for the overall project, including delivery of the approved project scope, budget, and timing. Evaluate if the




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

responsibility has been assigned at the appropriate executive level.

4.1.3.5 Determine if the project leader has appropriate assigned responsibilities, including quality management, budgetary authority (resources and expenses), deliverables and go/no-go decisions.

4.1.4 ROI and KPIsControl: The calculations for determining project ROI and KPIs are approved by the steering committee and executive sponsor, are objective, and provide meaningful status of the project and a measure of its success.

PO10.3PO13.13ME4.3

X X X

4.1.4.1 Determine the attributes for calculating the project’s ROI. Consider if the attributes are objective, repeatable and include relevant information that can be compared from period to period.

4.1.4.2 Determine if KPIs objectively represent progress of the project. Consider if the KPIs fairly compensate the project team (if applicable) and the compensation will not affect the team’s objectivity in reporting (avoid conflicts of interest).

4.1.5 Escalation managementControl: Steering committee and executive sponsors receive and act upon issues escalated by the project team.

PO10.13 X X X X

4.1.5.1 Determine escalation issues identified in escalation procedures as seen in step 4.2.4 that are presented to the steering committee, with appropriate disposition.

4.1.6 Functional analysis supports buy or build decisionsControl: The buy or build decision is based upon business and functional requirements, with appropriate procurement procedures and steering committee authorization.

AI1.1AI1.2AI2.3AI2.5AI3.1AI5.1AI5.2ME4.2

X X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.1.6.1 Determine the process for deciding whether to buy or build.

4.1.6.2 Determine if the build alternative was planned out and if costs associated with this alternative were considered.

4.1.6.3 Determine if the procurement process included a request for proposal, a methodical selection of vendor candidates and established criteria for evaluating the responses.

4.1.6.4 Determine if the project team—including business unit team members knowledgeable of the business process being replaced, and IT team—members knowledgeable of the IT strategic direction and architecture are actively involved in the vendor evaluation process.

4.1.6.5 Determine if legal, compliance and internal audit staff have reviewed the procurement agreements.

4.1.6.6 Determine if the final decision to buy or build provides for active steering committee participation and approval.

4.1.6.7 Confirm through interviews with key staff members that all requirements and acceptance criteria have been considered, evaluated, prioritized and documented in a way that is understandable to stakeholders and sponsors.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.1.6.8 Confirm through interviews with key staff members that application and infrastructure technical requirements meet the needs of the organization’s information architecture standards and strategic direction.

4.1.6.9 Verify through interviews with key IT staff members that exceptions and/or deviations from the information architecture standards and strategic direction have documented approval from senior IT management and strategic committees.

4.1.6.10 Verify whether a feasibility study exists that sets out an alternative course of action that will satisfy business technical and functional requirements.

4.1.6.11 Verify that the feasibility study considers the potential cost-benefit analysis of each of the identified alternatives and system functionality.

4.2 Project managementAudit/assurance objective: The project management activity should provide appropriate oversight and process to ensure the timely execution of the plan, mitigation of risks as they are identified, issues are resolved or escalated to the appropriate management level, quality of process is maintained, costs are monitored and minimized, and a go/no-go decision is made at each critical milestone.

4.2.1 Integration of business/information managementControl: The business and information management teams remain integrated as the project team creates the project plan; all affected business units are involved in the

PO10.3PO10.6ME4.2

X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

project. The steering committee monitors the effectiveness of the integration.

4.2.1.1 Interview team members, review team meeting minutes, and observe team meetings to determine full participation of team members.

4.2.1.2 Determine if team members appear to be intimidating or ignoring other team members.

4.2.1.3 Determine if the project manager has reported any conflicts in the integration process.

4.2.2 Composition of project teamControl: The project team consists of the appropriate resources, with the knowledge of the business process and automated solution, to effectively plan the project.

PO7.3PO7.4PO10.3PO10.8

4.2.2.1 Determine if the planning phase project team is appropriate in terms of skill sets, knowledge of the process and leadership for the size and scope of the project.

4.2.2.1.1 Obtain and evaluate the resumes/professional experience of the team members.

4.2.2.1.2 Obtain an organization chart and evaluate effectiveness of the project’s organization.

4.2.2.2 Determine if the team members are available to participate in project activities and that other duties are not impeding project progress.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.2.2.1 Interview project manager and staff to determine availability.

4.2.2.2.2 Review meeting minutes for attendance issues.

4.2.2.3 Determine if contingency plans are in place to replace team members, either permanently or on an interim basis.

4.2.3 Risk and issue managementControl: Risk analysis has been applied to the project during the planning phase; risks have been identified. Where risks can be mitigated, appropriate processes have been implemented; where the risks are inherent to the process, appropriate monitoring processes are in place. Issues identified during the planning are reported, and issues are monitored and closed.

PO10.2PO10.9PO10.13

AI1.2

X X X X

4.2.3.1 Determine if risks are classified in terms of impact (reputation, operations, finance/budgetary, completion date, regulations, etc.) and likelihood of occurrence (probability).

4.2.3.2 Determine if a holistic approach to the risk analysis is used.

4.2.3.3 Determine if the stakeholders establish the risk tolerance levels and have been involved in the risk analysis.

4.2.3.4 Determine if stakeholders are notified when their tolerance levels have been exceeded.

4.2.3.5 Test objective: To verify that risks have been evaluated and reported to stakeholders.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.3.5.1 Obtain the initial risk assessment to determine the quality and scope of the risk assessment.

4.2.3.5.2 Determine if new risks have been identified during the planning process. If so, determine how the risk impact and likelihood were assessed, how the stakeholders were notified, and the disposition of the risk (acceptance of risk or mitigation of process that triggered the risk).

4.2.3.6 Determine if an issue monitoring system is in use.

4.2.3.7 Determine if issues are documented, reviewed and monitored.

4.2.3.8 Test objective: To verify that issues are identified, documented and monitored.

4.2.3.8.1 Interview project management and several team members to document known issues identified during the planning phase.

4.2.3.8.2 Obtain the issues log and compare the known issues to those reported.

4.2.4 Escalation proceduresControl: Escalation procedures are utilized to inform the project team and the steering committee, where appropriate.

PO10.3 X X X

4.2.4.1 Verify that an escalation procedure is in use.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.4.2 Determine if any issues have required escalation; if so, trace the process of the escalation.

4.2.4.3 Determine if any escalated issues remain open; and if so, determine the effect on the project.

4.2.5 Quality managementControl: The project process has defined quality assurance (QA) procedures.

PO8.1PO10.10

AI2.8X X

4.2.5.1 Determine, if for significant decisions, there is a QA plan addressing review of presentations, criteria and assumptions.

4.2.5.2 Evaluate the robustness of the QA plan; consider approvals of deliverables, documentation review, peer reviews, assumption reasonableness and project management process.

4.2.5.3 Determine if the QA roles have been established.

4.2.5.4 Determine if a QA review has been performed for the current phase. If so, determine the level of documentation and management review.

4.2.5.5 Verify with business sponsor that quality reviews are being performed for business functional and technical requirements and feasibility study reports, and that the original acceptance criteria are considered in the quality assessment.

4.2.5.6 Verify that the quality plan clearly identifies ownership, processes and




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

metrics to provide QA of the project deliverables that constitute the project quality system.

4.2.5.7 Verify that the quality plan outlines the requirements for independent validation and verification of the business and technical solution.

4.2.6 Use of development methodologyControl: The project utilizes the organization’s development methodology and the methodology is appropriate for the size, scope and architecture of the project.

AI2.1AI2.2 X

4.2.6.1 Determine the systems development methodology being utilized for this project.

4.2.6.2 Confirm with key IT staff members that a high-level design specification is defined that translates the business requirements for software development.

4.2.6.3 Test objective: To verify that high-level specifications translate to the business requirements.

4.2.6.3.1 Obtain the project design specifications and determine that they address business requirements.

4.2.6.4 Evaluate the use of the systems development methodology against the size, scope and architecture.6

4.2.6.5 Determine if the systems development methodology is being used as

6 Note: It is not the reviewer’s position to evaluate which system development methodology to use, but rather to determine if the process fits the project.© 2009 ISACA. All rights reserved.



reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

intended.

4.2.6.5.1 Select several subphases of the planning phase; determine if the process is being followed.

4.2.7 Change managementControl: A change management procedure has been implemented that documents and obtains approval for changes in the scope, business case or key attributes of the project.

PO10.11AI6.1AI6.3AI6.4

X

4.2.7.1 Obtain the change management procedure.

4.2.7.1.1 Verify that the change procedure requires:

4.2.7.1.1.1 A documented request for a change

4.2.7.1.1.2 Thresholds where the project manager can approve the change without executive sponsor or steering committee authorization

4.2.7.1.1.3 Thresholds where the change must be submitted for approval to the executive sponsor and/or the steering committee

4.2.7.2 Determine the process for changes affecting the business case.

4.2.7.2.1 Verify that the benefits for the change have been analyzed, including their effect on key risks, costs and delivery dates.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.7.2.2 Verify that business case changes require executive sponsor and/or executive committee approval.

4.2.7.3 Test objective: To verify that change management procedures have been followed.

4.2.7.3.1 Select changes from the change management log.

4.2.7.3.1.1 Verify that the appropriate description of the change has been documented.

4.2.7.3.1.2 Verify that the effect on the business case and project has been documented.

4.2.7.3.1.3 Determine if a new risk assessment was required and whether it has been performed.

4.2.7.3.1.4 Verify that the appropriate approvals were obtained, based upon thresholds.

4.2.7.3.1.5 For any projects not successfully implemented in production, verify that a backout was properly documented and successfully executed by IT on a timely basis.

4.2.8 Planning and controlControl: The planning and control of the project includes effective time control, a project plan with milestones, deliverables, a sequence of process, resource projections and activity dependency.

PO10.6 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.8.1 Determine the comprehensiveness of the project plan. Consider the level of detail for each phase, subphase and task, resources required, deliverables, milestones and dependencies.

4.2.8.1.1 Verify that the milestones and tasks can be achieved with the resources available.

4.2.8.1.2 Verify that resource utilization can be realized (resources working double shifts or maintaining regular responsibilities while being assigned to the project team).

4.2.8.1.3 Verify that deliverables have been translated into activities.

4.2.8.1.4 Determine that project assumptions and constraints are documented and included in the plan.

4.2.8.1.5 Determine that each task has a clearly stated objective and goal.

4.2.8.2 Determine if changes to the project planning documents require QA and management review.

4.2.8.3 Determine if resource time is clearly established. Verify that no resources are unassigned.

4.2.9 Milestone go/no-go decisionsControl: At major milestones, management exercises and documents go/no-go decisions.

PO10.6 X X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.9.1 Determine if management reviews significant milestones, and makes a go/no-go decision to go to the next subphase or task based on the successful completion of the previous task or subphase, which is appropriately authorized.

4.2.9.2 Test objective: To verify that go/no-go decisions are being made at appropriate milestones, approved by the authorized management and properly documented.

4.2.9.2.1 Select several milestones.

4.2.9.2.2 Determine if a go/no-go decision process was made at that milestone.

4.2.9.2.3 Evaluate if the milestone required a go/no-go decision.

4.2.9.2.4 Review the documentation for the decision-making process to ensure appropriate approval and description.

4.2.10 Progress controlControl: Progress defined as meeting milestones and budgets are maintained and reported.

PO10.2PO10.3PO10.7

X X X

4.2.10.1 Determine if resource time reports, task completion percentages and deliverables are recorded in the project management document.

4.2.10.2 Determine the reporting cycle for progress control, adequacy of the distribution list and the review process for the progress reports.

4.2.11 Expense and time managementControl: Expense and time management are accurately recorded and approved.

PO10.2PO10.3PO10.7

X X

4.2.11.1 Determine how resources record and expense time to the project.

4.2.11.1.1 Verify that all team members record their time.7

7 Some entities record systems development time, but not business unit time, especially when those costs are not capitalized. Failure to record all time will affect the return on investment calculation and a benchmark for future business unit involvement.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.2.11.1.2 Verify that all costs are recorded.

4.2.11.1.3 Verify that external consultants document their time.

4.2.11.1.4 Determine how and expenses are approved.


PO10.2PO10.3PO10.7

X X X

4.2.12.1 Determine that the communications plan provides for status reports and exception reports that move up the project management hierarchy.

4.2.12.2 Determine that the communications plan provides for cross-team reporting.

4.2.12.3 Determine that the communications frequency and content are in alignment with the documented communications plan.

4.3 BudgetAudit/assurance objective: The budget and accounting processes should be accurate, complete and provide the information necessary to manage the project.

4.3.1 Budget statusControl: The project budget is defined, segregated from other projects and is in alignment with the business case.

PO5.2PO10.2PO10.3PO10.7PO10.13

X

4.3.1.1 Determine that project costs have been clearly identified, documented and approved by the executive sponsor and/or the steering committee.

4.3.1.2 Verify that the budget was established based upon a cost estimation process.

4.3.1.3 Determine if the budget is equal to the business case estimate. If not, determine if the variance has been approved by the executive sponsor and/or steering committee.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.3.1.4 Discuss the budget and deliverables with key members of the project team and executive sponsor. Determine if there are known gaps that would impact the budget.

4.3.1.5 Determine that the project has its own cost center that is not shared with other projects.

4.3.1.6 Determine if the following phases have separate components in the budget: analysis, design and selection; testing (including user acceptance testing [UAT]); implementation and rollout; training; conversion; and infrastructure modifications.

4.3.1.7 Determine if there is a contingency built into the budget and if it is within accepted limits.

4.3.2 AccountingControl: The accounting of the project is in compliance with expense and capitalization requirements.

PO10.2PO10.3PO10.7PO10.13

X

4.3.2.1 Determine if the appropriate costs are being capitalized or expensed based upon standard accounting principles.

4.3.2.2 Test objective: To verify that the costs are properly allocated.4.3.2.2.1 For a selected period, trace the expense and resource charges

through the accounting system. Determine if the allocations are within accounting guidelines.

4.4 Internal controlsAudit/assurance objectives: Internal controls, which ensure the accurate and complete processing of data, should be designed in the planning phase, when it is most efficient to modify, enhance or aggregate controls to provide a streamlined, but thorough, approach to internal control of application data and processes.

4.4.1 Internal control requirementsControl: Internal control requirements are established during the planning phase as part of the high-level system design or requirements definition.

AI2.3AI2.4 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.4.1.1 Obtain an understanding of the intended functionality of the new application.

4.4.1.2 Based on the application’s function, determine if the project team has identified the key controls required for oversight. Consider financial reporting, operations, security and regulatory requirements.

4.4.1.3 Review the requirements document for design controls, and identify instances where authorization, input, processing output and boundary controls, security, data integrity, audit trails, access control and database integrity controls are inadequate.

4.4.1.4 Review plans for implementing automated controls in packaged application software and determine that business process control requirements are adequately addressed.

4.4.1.5 Determine that control requirements are approved.4.4.2 Internal control risk assessment

Control: A control risk assessment was prepared by the project team to identify internal control requirements based upon processing, operational and financial risks.

PO9.1PO10.9ME2.1

X X

4.4.2.1 Review the project team’s control risk assessment to determine if the controls identified are placed properly and the assessment is complete.

4.4.2.2 Supplement the control risk assessment with IT audit/assurance-identified risks.

4.4.3 CAATs developmentControl: IT audit/assurance will utilize the reviews of the systems development phases to understand and test the application, and to allow reliance on the internal controls of the completed and implemented application.

ME2.4ME2.5 X X X

4.4.3.1 Determine the level of reliance to be placed on the IT audit/assurance review of the development process. If sufficient testing will be performed and change management is at a good-practice level, the delivered system can be relied upon for audit/assurance purposes.

4.4.3.2 Determine if integrated computer-assisted audit techniques (CAATs) will be built into the system to facilitate later audit/assurance activities.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.5 Adequacy of testingAudit/assurance objective: The project plan should provide for adequate testing at the various stages of development, including definition of the types of tests to be performed, the timeframe for testing and documentation requirements. At minimum, testing should include unit testing, integration testing, UAT, integration of manual and automated processes, conversion testing and stress testing. Consider parallel testing or separate operating platform testing prior to implementation.

4.5.1 Testing requirementsControl: Testing requirements are established and include documentation and review standards.

AI1.2AI2.3 X

4.5.1.1 Determine if testing requirements have been established for each type of testing, including test objective, scope, size, timing, scripting (where appropriate), documentation, review and approval.

4.5.1.2 Determine if the planning process includes unit, integration, user acceptance, stress and conversion testing.

4.5.1.3 Determine if the planning process has defined how UAT will be performed: on a separate platform or on the production system.

4.5.1.4 Determine if a test suite of transactions will be developed and be available later for regression testing after major system updates.

4.5.2 Project planControl: The project plan provides adequate time for testing and remediation based upon test results.

PO8.3PO10.2

4.5.2.1 Review the project plan and determine if adequate time has been provided for testing.

4.5.2.2 Determine what provisions have been made if the delivery of the UAT system is late—whether delays in development shorten or otherwise limit the testing time available prior to implementation.

X

4.5.3 Testing contentControl: Test scripts and volumes are adequate to ensure accurate, effective and complete results.

PO10.2 X

4.5.3.1 Review test plans to ensure full testing of the system.4.5.3.2 Review plans to reconcile test results to known results (either based on an




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

independent process or manual calculation).4.5.3.3 Review stress testing plans to ensurethat a realistic volume is used to

stress-test the system for current and future volume.4.6 ImplementationAudit/assurance objective: The implementation plan should be developed to ensure minimum disruption during the initial implementation of the new system and thorough vetting of the processes prior to final “throwing the switch” to the new system.

4.6.1 Pilot test planControl: Pilot implementations of the new processes are utilized to minimize the risks of a full roll-out of the application.

PO10.7AI7.3AI7.4

X

4.6.1.1 Review the plans for pilot tests. Determine the scope and evaluate the effectiveness of the pilot test.

4.6.1.2 Determine that a review process has been established to ensure that pilot test results adequately address issues identified during the test process and that these issues are included in the issue monitoring system.

4.6.1.3 Determine that pilot objectives are clearly stated to permit objective assessment of successes after the completion of the pilots.

4.6.1.4 Determine if go/no-go evaluation is included in the pilot implementation plan.

4.6.2 Readiness assessmentControl: A readiness assessment is part of the implementation plan to ensure that the system is ready for the implementation phase.

PO10.6PO10.7 X X X X

4.6.2.1 Determine if a readiness assessment is required prior to moving the project to the next phase.

4.6.2.2 Determine if the readiness assessment includes an inventory of open issues, a risk analysis of the impact to the business of moving to implementation, approval from key stakeholders; and report to executive sponsor and/or steering committee.

4.6.3 Resource planningControl: Resource planning is included in the planning process to ensure adequate coverage of essential processes during implementation.

PO10.6PO10.8 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.6.3.1 Determine that a resource plan has been completed to address the resources required to convert and implement the new system.

4.6.3.2 Determine if the plan addresses coverage of existing workload as well as the new processes and interim conversion processes.

4.6.3.3 Determine if the plan allows for contingencies, vacations, illnesses, etc.4.6.4 Conversion plan

Control: A conversion plan is part of the overall planning activity and includes documented conversion specifications, a dress rehearsal of the conversion, a backout plan in the event the conversion is not successful, and a reconciliation of data between the new and old systems.

PO10.6AI7.5 X

4.6.4.1 Review the conversion plan for completeness. Determine if the following tasks are included in the plan:

4.6.4.1.1 Conversion requirements and specifications

4.6.4.1.2 A dress rehearsal plan for testing the conversion process using a copy of the full data volume to be converted

4.6.4.1.3 A backout plan in the event the conversion is not successful

4.6.4.1.4 An effective reconciliation between the new and old systems4.6.4.2 The steering committee and/or executive sponsor has reviewed and

approved the conversion plan.4.6.5 Communication plan

Control: A communication plan informs stakeholders and management of the progress of the roll-out.

AI7.5 X X

4.6.5.1 Review the communication plan. Determine the frequency, distribution list and content of the communication plan to ensure that it informs the appropriate interested parties of the roll-out on a timely basis.

4.6.6 Training planningControl: An appropriate training program has trained affected functions prior to implementation.

AI7.1 X

4.6.6.1 Review the training programs for the various affected parties.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.6.6.1.1 Determine that transaction processors have been properly trained in the use of the new system.

4.6.6.1.2 Determine that IT operations have been trained in the scheduling and processing of the new system.

4.6.6.1.3 Determine that the help desk is prepared to assist users in the use of the new system.

4.6.7 Transition planControl: A transition plan is created to address interim processes that are required until the new system is fully operational and integrated with other systems.

AI7.3 X

4.6.7.1 Determine if, as part of the planning process, the project team has identified interim processes that will be required due to temporary interfaces or processes until full integration of processes is achieved.

4.6.7.2 Determine if additional resources have been included in the plan to augment internal resources during the interim process period.

4.6.7.3 Determine if the costs of additional resources have been included in the budget.

4.6.8 Backout planControl: The backout plan is prepared with appropriate review, approval and decision points to initiate the plan.

AI7.3 X

4.6.8.1 Review the backout plan.4.6.8.2 Evaluate the backout plan for objective decision points to make a decision

to back out of the conversion.4.6.8.3 Determine if the backout plan includes active participation by project

management, major stakeholders and executive sponsor. 4.7 Third-party providersAudit/assurance objectives: Third-party providers should be selected and managed effectively to provide maximum ROI, should be adequately vetted, and contracts should provide for measurable deliverables and safeguarding of entity intellectual property.8

8 It is recommended that a separate outsourcing assurance/audit evaluation be performed for the acquisition of large-scale third-party services. The ISACA/ITGI Outsourced IT Environments Audit/Assurance Program can be used for that purpose.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

4.7.1 Vendor selectionControl: Criteria for vendor selection are predefined prior to selection, the selection and contract negotiation are performed according to policy, and the criteria and selection process are objective.

AI5.3 X

4.7.1.1 Determine the selection criteria used for the selection of a third-party vendor.4.7.1.2 Determine that the selection process included the evaluation of these

criteria against the vendors’ responses to the RFP or other selection processes.

4.7.1.3 Review the documented decision process and justification for the selection of the final vendor.

4.7.1.4 Determine if standard and required entity contract provisions were included in the contract.

4.7.1.5 Determine if the contract authorization was in accordance with enterprise’s bill of authority for contract amount and scope.

4.7.1.6 Determine whether software acquisition contracts include and enforce the rights and obligations of all parties addressing ownership and licensing of intellectual property, maintenance, warranties, arbitration procedures, upgrade terms, fitness for purpose, security, escrow and access rights.

4.7.2 SLAs and contract fulfillmentControl: SLAs are defined and objective to permit monitoring of vendor activities, compliance with contract and assignment of penalties for failure to comply with the contract.

AI5.3DS1.1DS1.2DS1.3DS1.4DS1.5DS1.6

X

4.7.2.1 Determine if SLAs are documented. If documented, determine if SLAs are as described in the contract.

4.7.2.2 Review SLAs to determine if they utilize metrics that are monitored and measured.

4.7.2.3 Determine if SLAs are appropriate to measure the activity performed by vendor.

4.7.2.4 Determine if SLAs permit assessment of penalty for nonperformance of




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

contracted worked.4.7.3 Intellectual property

Control: Vendors sign confidentiality agreements restricting their access to and ability to disseminate the enterprise’s intellectual property. The vendor’s access is limited to information necessary to perform its contracted duties.

AI5.3

4.7.3.1 Determine if vendors sign confidentiality agreements relating to intellectual property.

4.7.3.2 Determine if management reviews vendor access to intellectual property.

5 . EXECUTION PHASEThe execution phase of the project plan usually is of the longest duration and includes all processes after the completion of the planning phase until the go-live phase. Execution subphases include: If the project under development is purchased, a gap analysis to determine modifications Design of the new system including:

– New business process (manual and automated)– New controls– Development of interfaces to bridge the new system with existing retained applications– Development of interim processes if the new system must operate with the older system

until all processes are implemented– Conversion utilities to convert the new system

Development of the applications (programming) Establishment of conversion and transition processes, balancing routines, and verification of

process accuracy and completeness Testing:

– Business processes (unit testing and integration testing)– UAT– Conversion testing– Stress testing the processes and equipment– Backout if the conversion must be postponed

Implementation Reconciliation of conversion




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.1 GovernanceAudit/assurance objective: Management should provide governance over the project to ensure that the project is adequately monitored. The business and technical resources should be assigned to ensure planned progress of the project. Procedures should be defined to keep management informed of the progress. Communications and escalation procedures should be in place to allow management to respond to issues as they arise.

5.1.1 Business caseControl: The project team leadership, on a regular basis, monitors and provides reports to executive sponsors on the continued alignment of the project plan with the business case and any changes in satisfying the business case.

PO10.1AI1.1ME1.5ME4.2ME4.3

X X X

5.1.1.1 Review project change logs to determine if significant changes have been made to the project. Interview project manager to determine impact of changes. Determine if changes have affected the business case. Interview the project team manager to determine if any changes to the project plan have impacted the business case.

5.1.1.2 If significant changes have been implemented, review steering committee meeting minutes and other documentation to determine evidence of management’s governance oversight.

5.1.2 Scope managementControl objectives: The scope of the project is clearly defined—a project plan is maintained and updated that clearly identifies the phases, processes and subprocesses. Responsibility for managing scope changes is defined and procedures are in place to obtain approval of scope changes from the project steering committee or executive sponsors.

PO10.5 X X X

5.1.2.1 Review the project plan and identify changes to the plan. Determine if changes from this project or other interfacing projects affect the scope of this project.

5.1.2.1.1 If scope has changed, verify that the project plan has been updated.

5.1.2.1.2 If the scope has changed, determine that appropriate steering




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

committee and executive sponsor authorization was obtained. 5.1.2.1.3 Test objective: To verify that scope changes are implemented with the knowledge and approval of the steering committee or executive sponsors.

5.1.2.1.3.1 Select a representative sample of scope changes, focusing on those with the greatest impact to the project.

5.1.2.1.3.2 Verify the following for the scope change: Documented approval of management Timeliness of approval Determine if the process had been delayed or

circumvented5.1.3 Roles and responsibilities

Control: Roles and responsibilities of the project team continue to be clearly identified, appropriate subject matter experts and stakeholders are actively participating on the project team, and the division of responsibilities is appropriate for the project and entity-level organizational structure (including separation of duties).

PO7.1PO7.3PO10.3

X X

5.1.3.1 Determine if roles and responsibilities issues have arisen during the project requiring senior management intervention to ensure timely and effective involvement by all affected business units, and to ensure systems development, IT operations, information security, internal audit, legal, compliance, suppliers and other appropriate areas are actively involved in the project.

5.1.4 ROI and KPIsControl: The calculations for determining project ROI and KPIs are updated and reported to the steering committee and executive sponsor as scope or other components that affect performance or ROI changes.

PO10.3PO13.13ME4.3

X X X

5.1.4.1 Determine if the attributes for calculating the project’s ROI and KPIs have changed due to project modifications.

5.1.5 Escalation managementControl: Steering committee and executive sponsors are receiving and acting upon

PO10.13 X X X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

issues escalated by the project team.5.1.5.1 Determine if there are any open escalation issues identified in escalation procedures without

appropriate disposition.5.1.5.2 Functional analysis supports buy or build decisions

5.2 Project managementAudit/assurance objective: The project management activity should provide appropriate oversight and process to ensure the timely execution of the plan, mitigation of risks as they are identified, issues are resolved or escalated to the appropriate management level, quality of process is maintained, costs are monitored and minimized, and a go/no-go decision is made at each critical milestone.

5.2.1 Integration of business/information managementControl: The business and information management teams remain integrated as the project team executes the project plan; all affected business units are involved in the project. The steering committee monitors the effectiveness of the integration.

PO10.3PO10.6ME4.2

X X

5.2.1.1 Interview team members, review team meeting minutes, and observe team meetings to determine full participation of team members.

5.2.1.2 Determine if team members appear to be intimidating or ignoring team members.5.2.1.3 Determine if the project manager has reported any conflicts in the integration process.

5.2.2 Composition of project teamControl: The project team consists of the appropriate resources, with the knowledge of the business process and automated solution, to effectively plan the project.

PO7.1PO7.2PO7.3PO10.3PO10.8

X

5.2.2.1 Determine if the execution phase project team is appropriate in terms of skill sets, knowledge of the process and leadership for the size and scope of the project.

5.2.2.2 Determine if the team members are available to participate in project activities and that other duties are not impeding project progress.

5.2.2.2.1 Interview project manager and staff to determine availability.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.2.2.2 Review meeting minutes for attendance issues.

5.2.2.3 Determine if contingency plans are in place and have been implemented to replace team members, either permanently or on an interim basis.

5.2.3 Risk and issue managementControl: Risk analysis continues to be applied to the project during the execution phase as risks are identified. Where risks can be mitigated, appropriate processes have been implemented; where the risks are inherent to the process, appropriate monitoring processes are in place. Issues identified during the planning are reported, and issues are monitored and closed.

PO10.2PO10.9PO10.13

AI1.2

X X X X

5.2.3.1 Determine if risks have changed during the execution process. If changes are noted, determine if appropriate impact (reputation, operations, finance/budgetary, completion date, regulations, etc.) analysis and likelihood of occurrence (probability) have been applied.

5.2.3.2 If risks have changed, determine if the stakeholders have been involved in the process and have approved the changes in risks.

5.2.3.3 Determine if stakeholders have been notified when their tolerance levels have been exceeded.

5.2.3.4 Test objective: To verify that risks have been evaluated and reported to stakeholders.5.2.3.4.1 Obtain the initial risk assessment to determine the quality and

scope of the risk assessment.5.2.3.4.2 Determine if new risks have been identified during the

execution process. If so, determine how the risk impact and likelihood were assessed, how the stakeholders were notified, and the disposition of the risk (acceptance of risk or mitigation of process that triggered the risk).

5.2.3.5 Determine if issues are documented, reviewed and monitored using the issue monitoring system.

5.2.3.6 Test objective: To verify that issues are identified, documented and monitored.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.3.6.1 Interview the project management and several team members to document known issues identified during the planning phase.

5.2.3.6.2 Obtain the issues log and compare the known issues to those reported.

5.2.4 Escalation proceduresControl: Escalation procedures are followed to inform the project team and the steering committee, where appropriate.

PO10.3 X X X

5.2.4.1 Determine if any issues have required escalation; if so, trace the process of the escalation.

5.2.4.2 Determine if any escalated issues remain open; and if so, determine the effect on the project.

5.2.5 Quality managementControl: The project process has defined QA procedures.

PO8.6PO10.10

AI2.8X X

5.2.5.1 Determine if the QA plan addressing review of presentations, criteria and assumptions has been followed.

5.2.5.2 Determine if a QA review has been performed for the current phase. If so, determine the level of documentation and management review.

5.2.5.3 Determine if QA reviews are performed independent of the development team.

5.2.5.4 Test objective: To verify that QA reviews are performed by independent reviewers and their results are analyzed, approved and corrective actions taken were appropriate.

5.2.5.4.1 Review relevant documentation to verify the existence of routine QA reviews.

5.2.5.5 Determine if the process of monitoring software quality has been defined and is performed on a regular basis.

5.2.5.5.1 Test objective: To verify that software development is reviewed on a routine basis for quality.

5.2.5.5.1.1 Select several programs within the project




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

development process. Verify that a QA review was performed on the programs and results reported.

5.2.5.5.1.2 Identify instances where QA was not performed or where no action was taken based on negative review.

5.2.6 Use of development methodologyControl: The project utilizes the enterprise’s development methodology.

AI2.2AI2.7AI7.2AI7.3AI7.5AI7.7AI7.8AI7.9

X

5.2.6.1 Determine if the systems development methodology is being utilized for this project as intended.

5.2.6.2 Perform code walk-through and examine documentation associated with data inputs and outputs to determine whether proper storage, location and retrieval methods are implemented according to data dictionary standards.

5.2.6.3 Examine information architecture and data dictionary documentation to identify deviations from the data dictionary standards in the program design.

5.2.6.4 Inquire of key staff members whether data dictionary standards are being used and compare actual performance of data inputs/outputs with responses from key staff members.

5.2.6.5 Confirm with key staff members that source data collection design is specified that incorporates controls over computed and stored data.

5.2.6.6 Perform code walk-through and inspect plans to confirm that data are collected and validated for processing transactions.

5.2.6.7 Confirm with key IT staff members that adequate redundancy, failure recovery and backup arrangements are defined and included in the detailed design specification.

5.2.6.8 Review the backup plan and procedures to determine that they adequately address the




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

availability requirements of the new system and are cost-effective.5.2.6.9 Review project documentation to determine if good practices, such as availability, control

and auditability, security, and network requirements, are considered.5.2.6.10 Inquire of key staff members and inspect relevant project documentation to determine

whether processing steps, including transaction types, and processing rules including logic transformations or specific calculations are defined and included in the detailed design specifications.

5.2.6.11 Confirm with key IT staff members that all identified output data requirements are properly defined.

5.2.6.12 Review detailed design documentation to determine that pertinent design details, such as different types of recipients, usage, details required, frequency and method of generation, are considered.

5.2.6.13 Review detail design requirement documentation to determine if the availability, completeness, integrity and confidentiality of output data as well as the impact of data outputs to other programs are appropriately addressed.

5.2.6.14 Confirm with key staff members that the interface between the user and the system application is defined and included in the detailed design specification.

5.2.6.15 Inspect the detailed design specifications to confirm that they adequately address user interface requirements.

5.2.6.16 Review documents such as system design analysis reports or system design change requests to confirm that the system design reassessment procedures are followed (e.g., change in system design needs to be approved by business and IT sponsors).

5.2.6.17 Review detailed design specification documentation to determine if it was prepared in conformance with organization and industry standards and the information architecture.

5.2.6.18 Confirm with IT and business stakeholders that a design walk-through has been conducted before development commenced.

5.2.6.19 Review the detailed design specifications to confirm that a design walk-through is conducted for all stakeholders and that stakeholder sign-off has been initiated before development (e.g., signature and date or e-mail confirmation).

5.2.6.20 Confirm with key staff members that all development activity has been established to ensure adherence to development standards and that developed software is based on agreed-upon specifications to meet business, functional and technical requirements.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.6.21 Inspect relevant documentation (such as design, code review and walk-throughs) to identify exceptions to specifications and standards.

5.2.6.22 Obtain and review assessment documentation of the developed software to confirm adequacy.

5.2.6.23 Confirm with key staff members that technical authorities and operations management applications are ready and suitable for migration to the production environment.

5.2.6.24 Perform a walk-through of code and identify problems/exceptions.5.2.6.25 Inspect relevant documentation (such as design, code review and walk-throughs) to

identify exceptions to specifications and standards.5.2.6.26 Confirm with key staff members that technical authorities and operations management

applications are ready and suitable for migration to the production environment.5.2.6.27 Perform a walk-through of code and identify problems/exceptions.5.2.6.28 Inquire of key staff members to determine compliance with all obligations and

requirements.5.2.6.29 Review contractual obligations and licensing requirements relating to third-party

developers.

5.2.7 Change managementControl: A change management procedure is being utilized to document changes and approval in the scope, business case or key attributes of the project.

PO10.11AI6.1AI6.2AI6.3AI6.4

X X

5.2.7.1 Review procedures for program transfer to verify that a formal process exists that requires documented approval from user management and system development.

5.2.7.2 Confirm that the approval process identifies effective dates for promotion of new systems, applications or infrastructure to production.

5.2.7.3 Inquire whether and confirm that the approval process includes a formal documented sign-off from business process owners, third parties and IT stakeholders as appropriate (e.g., development group, security group, database management, user support and operations group).




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.7.4 Confirm procedures for updating copies of system documentation and relevant contingency plan.

5.2.7.5 Inquire of key staff members concerning procedures for updating all source program libraries and procedures for labeling and retaining prior versions.

5.2.7.6 Test objective: To verify that change management procedures have been followed.

5.2.7.6.1 Select changes from the change management log.

5.2.7.6.1.1 Verify that the appropriate description of the change had been documented.

5.2.7.6.1.2 Verify that the effect on the business case and project had been documented.

5.2.7.6.1.3 Determine if a new risk assessment was required and has been performed.

5.2.7.6.1.4 Verify that the appropriate approvals were obtained based upon thresholds.

5.2.8 Planning and controlControl: The planning and control of the project includes effective time control, a project plan with milestones, deliverables, a sequence of process, resource projections and activity dependency.

PO10.6PO10.7PO10.11PO10.13

X

5.2.8.1 Determine that the project plan is updated with scope, resource and milestone changes.5.2.8.1.1 Verify that the milestones and tasks can be achieved with the

resources available.5.2.8.1.2 Verify that resource utilization can be realized (resources

working double shifts or maintaining regular responsibilities while being assigned to the project team).

5.2.8.1.3 Verify that deliverables have been translated into activities.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.8.1.4 Determine that project assumptions and constraints are documented and included in the plan.

5.2.8.1.5 Determine that each task has a clearly stated objective and goal.5.2.8.2 Verify that the status of each subphase is updated on a timely basis to show

accurate percentage completion and any delays. 5.2.9 Milestone go/no-go decisions

Control: At major milestones, management exercises and documents go/no-go decisions.

PO10.6 X X X

5.2.9.1 Determine if management reviews significant milestones, and makes a go/no-go decision to go onto the next subphase or task based upon the successful completion of the previous task or subphase and that decisions are appropriately authorized.

5.2.9.2 Test objective: To verify that go/no-go decisions are being made at appropriate milestones, approved by the authorized management and properly documented.

5.2.9.2.1 Select several milestones.5.2.9.2.2 Determine if a go/no-go decision process was made at that

milestone.5.2.9.2.3 Evaluate if the milestone required a go/no-go decision.5.2.9.2.4 Review the documentation for the decision-making process to

ensure appropriate approval and description.5.2.10 Progress control

Control: Progress, defined as meeting milestones and budgets, is maintained and reported.

PO10.2PO10.3PO10.7

X X X

5.2.10.1 Determine if resource time reports, task completion percentages, and deliverables are recorded in the project management document.

5.2.10.2 Determine the reporting cycle for progress control, adequacy of the distribution list and the review process for the progress reports.

5.2.11 Expense and time managementControl: Expenses and time management are accurately recorded and approved.

PO10.2PO10.3PO10.7

X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.2.11.1 Determine how resources record and expense time to the project.5.2.11.1.1 Verify that all team members record their time.9

5.2.11.1.2 Verify that all costs are recorded.5.2.11.1.3 Verify that external consultants document their time.5.2.11.1.4 Determine how and expenses are approved.


PO10.2PO10.3PO10.7

X X X

5.2.12.1 Determine that the communications plan providing for status reports and exception reports up the project management hierarchy is executed on a timely basis and reports are distributed as planned.

5.2.12.2 Determine that the communications plan for cross-team reporting is being followed.5.2.12.3 Determine that the communications frequency and content are in alignment with the

documented communications plan.5.3 BudgetObjective: The budget and accounting processes should be accurate, complete and provide the information necessary to manage the project.

5.3.1 Budget designControl: The project budget is defined, segregated from other projects and is in alignment with the business case.

PO10.2PO10.3PO10.7PO10.13

X

5.3.1.1 Verify that the budget reflects any changes in scope or identified issues.5.3.1.2 Determine if the budget is equal to the business case estimate. If not, determine if the

variance has been approved by the executive sponsor and/or steering committee.5.3.1.3 Discuss the budget and deliverables with key members of the project team and executive

sponsor. Determine if there are known gaps that would impact the budget.5.3.2 Budget status PO10.2 X

9 Some entities record systems development time, but not business unit time, especially when those costs are not capitalized. Failure to record all time will affect the return on investment calculation and a benchmark for future business unit involvement.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

Determine if the budget and actual costs (including resources and expenses) are in alignment with the percentage completion.

PO10.3PO10.7PO10.13

5.3.2.1 Obtain the budget and actual costs.5.3.2.2 Verify that the actual costs at the current stage of the project (based upon percentage

completion) are in line with the budget for the same stage of the project.5.3.2.3 Based upon the results of the actual versus budget, determine if management exception

reporting has been initiated if the project is behind.


X

5.3.3.1 Determine if the appropriate costs are being capitalized or expenses based upon standard accounting principles.

PO10.2PO10.3PO10.7PO10.13

5.3.3.2 Test objective: To verify that the costs are properly allocated.5.3.3.2.1 For a selected period, trace the expenses and resources charges

through the accounting system. Determine if the allocations are within accounting guidelines.

5.4 Internal controlsAudit/assurance objectives: Internal controls, which ensure the accurate and complete processing of data, should be developed to provide an efficient, streamlined and thorough approach to internal control of application data and processes.

5.4.1 Design Subphase

5.4.1.1 Internal control requirementsControl: Detailed internal control requirements are established during the design subphase as part of the detailed design.

AI2.3 X

5.4.1.1.1 Obtain an understanding of the intended functionality of the new application.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.4.1.1.1.1 If it is an acquired system, review gap analysis of system’s functionality to system’s requirements. Determine if modification list or parallel system provides required functionality.

5.4.1.1.2 Based on the application’s function, determine if the project team has identified the key controls required for oversight. Consider financial reporting, operations, security and regulatory requirements.

5.4.1.1.2.1 Review the identified key controls and recommend additional controls as necessary to provide good practices.

5.4.1.1.2.2 Interview stakeholders to identify their view of internal control requirements and ensure that their needs are included in the design and the audit/assurance process.

5.4.1.1.3 Prepare work papers for future audit/assurance reviews.

5.4.1.1.3.1 Expand this audit/assurance program to address the application’s processes.

5.4.1.1.3.2 Prepare control objectives for each control area.

5.4.1.1.3.3 For each control objective, document how each control will be executed in the new system. Consider the following areas: interfaces with other applications, critical processes or calculations, and manual interfaces.

5.4.1.2 Internal control risk assessmentControl: A controls risk assessment prepared by the project team during the planning phase is updated to identify detailed internal control requirements based upon processing, operational and financial risks.

PO10.9 X X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.4.1.2.1 Review the project team’s updated control risk assessment to determine if the controls identified are placed properly and the assessment is complete.

5.4.1.2.2 Supplement the control risk assessment with IT audit/assurance identified risks.

5.4.1.3 CAATs developmentControl: IT audit/assurance will utilize the reviews of the systems development phases to understand and test the application, to allow reliance on the internal controls of the completed and implemented application.

X X X

5.4.1.3.1 Identify the design of the CAATs. IT audit/assurance is a stakeholder in the design process and is part of the project team for this process.

5.4.1.3.2 Determine that the CAATs design will satisfy future IT audit/assurance needs.

5.4.2 Development/Prototyping Subphase5.4.2.1 Internal control requirements

Control: Internal control requirements established during the project process have been mapped to and included in specific processing solutions.

AI2.3 X

5.4.2.1.1 Identify controls that IT audit/assurance considers of high risk, requiring monitoring during the development phase.

5.4.2.1.1.1 Expand this audit/assurance program to reflect the monitoring and review process.

5.4.2.1.2 Review issue monitoring reports to determine if control issues have been identified.

5.4.2.1.2.1 If control issues have been identified, evaluate whether the scope of the audit/assurance project should be expanded to include these areas.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.4.2.1.2.2 If expanded scope is required and approved, prepare additional audit/assurance steps within this program.

5.4.2.1.3 If prototyping is being used as a development methodology, verify that the prototyping activities are focused on the development subphase. Design should not be modified without appropriate reviews and approvals.

5.4.2.2 Internal control risk assessment

5.4.2.3 CAATs developmentControl: IT audit/assurance participates in the development of CAATs processes.

X X X

5.4.2.3.1 Establish milestones in the CAATs development requiring IT audit/assurance participation.

5.4.2.3.2 Participate in the development as required by the project team. 5.4.2.3.3 Perform review of CAATs functionality at milestones and

ensure audit/assurance objectives will be met by the CAATs.

5.4.3 Testing

5.4.3.1 Internal control requirementsControl: Internal control requirements established during the project have been mapped to their implementation and thoroughly tested.

AI2.3AI7.2AI7.4AI7.7AI7.8

X

5.4.3.1.1 Review user acceptance testing plans to ensure thoroughness of testing.

5.4.3.1.1.1 Review user test scripts to ensure that both automated and manual processes that constitute the system of internal controls are tested.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.4.3.1.1.2 Review the quality of the test scripts to ensure that they can be executed in the future as the processes change and that the level of detail is adequate to allow the scripts to be repeatable in the future.

5.4.3.1.2 Review the results of the tests to ensure identified testing issues are registered in the issue management system, the issues are risk-assessed, and remediation is appropriately scheduled.

5.4.3.1.3 Review the issue monitoring reports to ensure that reported issues have been remediated on schedule with the solutions defined in the remediation documentation.

5.4.3.1.4 Determine if independent testing is required by IT audit/assurance to satisfy audit/assurance objectives.

5.4.3.1.4.1 If independent testing is required, determine if testing can be re-performance of the UAT or if new scripts need to be created and executed.

5.4.3.1.4.2 Prepare test scripts, if necessary.

5.4.3.1.4.3 Perform tests and document results.

5.4.3.2 Internal control risk assessment

5.4.3.3 CAATs developmentControl: IT audit/assurance thoroughly tests CAATs processes as a UAT.

X X X

5.4.3.3.1 Determine the level of reliance to be placed on the IT audit/assurance review of the development process. If sufficient testing will be performed and change management is at a good-practice level, the delivered system can be relied upon for




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

audit/assurance purposes.

5.4.3.3.2 Determine if integrated CAATs will be built into the system to facilitate later audit/assurance activities.

5.4.4 Conversion

5.4.4.1 Internal control requirementsControl: Conversion internal control requirements established during the project have been designed, implemented and tested.

AI2.3 X

5.4.4.1.1 Review conversion controls to ensure that data will be converted accurately and completely.

5.4.4.1.2 Review conversion test run to ensure that the process is ready for the final conversion prior to the system going live.

5.4.4.1.3 Review the final conversion control/reconciliation reports and conversion summary prepared for management to identify any control issues identified during the conversion process.

5.4.4.2 Assess internal control risk

5.4.4.3 Develop CAATs

5.5 Adequacy of testingAudit/assurance objective: The execution phase should exhibit adequate testing at the various stages of development, including definition of the types of tests to be performed, the timeframe for testing and documentation requirements. At minimum, testing should include unit testing, integration testing, UAT, integration of manual and automated processes, conversion testing and stress testing. Parallel testing or separate operating platform testing prior to implementation should be considered.

5.5.1 Testing requirementsControl: Testing is performed according to project and enterprise standards and requirements and the testing is documented and reviewed.

AI7.2AI7.7 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.5.1.1 Determine if testing requirements have been performed for each type of testing according to the subphase (unit, integration, user acceptance, stress and conversion testing). Tests should include test objective, scope, size, timing, scripting (where appropriate), documentation, review and approval.

5.5.1.2 Determine if the test suite of transactions will be available later for regression testing after major system updates.

5.5.1.3 Verify approval of the test plan by the business owner.

5.5.1.4 Confirm that the project lead has signed-off on test results.

5.5.1.5 Verify that the test scripts and test results have been properly reviewed and approved by management and the business owner.

5.5.2 Project PlanControl: The project plan provides adequate time for testing and remediation based upon test results.

AI7.2AI7.7 X

5.5.2.1 Review the project plan and determine if adequate time has been provided for testing.

5.5.2.2 Determine if the delivery of the system UAT is late, resulting in limited testing or remediation identified by testing not included in the production release.

5.5.3 Testing contentControl: Test scripts and volumes are adequate to ensure accurate, effective and complete results.

AI7.2AI7.7 X

5.5.3.1 Review test results to ensure full testing of system.

5.5.3.2 Review test results to known results (either based on an independent process or manual calculation).

5.5.3.3 Review stress testing results to ensure that realistic volume is used to stress




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

test the system for current and future volume.

5.6 ImplementationAudit/assurance objective: The implementation plan should be developed to ensure minimum disruption during the initial implementation of the new system and thorough vetting of the processes prior to final “throwing the switch” to the new system.

5.6.1 Pilot test planControl: Pilot implementations of the new processes are utilized to minimize the risks of a full roll-out of the application.

PO10.7AI7.3AI7.4

X

5.6.1.1 Review the results from the pilot tests. Determine if the scope of the pilot is complete and evaluate the effectiveness of the pilot test.

5.6.1.2 Determine that a review process was effective in ensuring that pilot test results adequately address issues identified during the test process and that these issues are included in the issue monitoring system.

5.6.1.3 Determine that pilot objectives were achieved after the completion of the pilots.

5.6.1.4 Determine if go/no-go evaluation was considered at the conclusion of the pilot.

5.6.2 Readiness assessmentControl: A readiness assessment is part of the implementation plan to ensure that the system is ready for the implementation phase.

PO10.6PO10.7AI7.3

X X X X

5.6.2.1 Determine if a readiness assessment was performed prior to moving the project to the implementation phase.

5.6.2.2 Determine if the readiness assessment included an inventory of open issues, a risk analysis of the impact to the business of moving to implementation, approval from key stakeholders and a report to executive sponsor and/or steering committee.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.6.3 Resource planningControl: Resource planning is monitored during the execution process to ensure adequate coverage of essential processes during implementation.

PO10.6PO10.7AI7.3

X

5.6.3.1 Determine that the resource plan is maintained and monitored to address the resources required for the current subphase.

5.6.3.2 Determine if any resource planning issues were identified; evaluate the effectiveness of the disposition and any risks to the project.

5.6.4 Conversion planControl: A conversion plan has been thoroughly tested and data reconciled prior to implementation.

PO10.6AI7.5 X

5.6.4.1 Review the results of the conversion plan, tests and data reconciliation for completeness and readiness.

5.6.4.2 Consider further independent tests of the conversion reconciliation if it is considered high risk or the conversion tests don’t adequately document the assurance processes.

5.6.4.3 Determine if the steering committee and/or executive sponsor have reviewed and approved the conversion prior to going live.

5.6.5 Communications planControl: A communications plan informs stakeholders and management of the progress of the roll-out.

AI7.5 X X

5.6.5.1 Review the communications plan. Determine that communications are in compliance with the plan including frequency, distribution list and content of the communications plan.

5.6.6 Training planningControl: The training program has trained affected functions prior to implementation.

AI4.2AI4.4AI7.1

X

5.6.6.1 Review the results of the training programs for the various affected parties.© 2009 ISACA. All rights reserved.



reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

5.6.6.1.1 Interview the transaction processors to verify they have been properly trained in the use of the new system.

5.6.6.1.2 Interview the IT operations staff to verify they have been trained in the scheduling and processing of the new system.

5.6.6.1.3 Interview the help desk staff to verify that they are prepared to assist users in the use of the new system.

5.6.6.2 Review training and implementation materials for required content.

5.6.6.3 Review support materials (e.g., training materials, user manuals, procedure manuals, online help, help desk written procedures, etc.) for adequacy.

5.6.7 Transition planControl: A transition plan has been implemented to address interim processes that are required until the new system is fully operational and integrated with other systems.

AI4.1AI7.3 X

5.6.7.1 Determine if interim processes that are required due to temporary interfaces or processes until all full integration of processes are functioning.

5.6.8 Backout planControl: The backout plan has been prepared with appropriate review, approval and decision points to initiate the plan.

AI4.1AI7.3 X

5.6.8.1 Determine if the backout plan was initiated.

5.6.8.2 If the backout plan has been initiated, determine that processing has returned to a steady state using the old systems.

5.7 Third-party providersAudit/assurance objectives: Third-party providers should be managed effectively to provide maximum ROI, and ensure deliverables are achieved and payments to vendors are paid upon achievement of contract provisions.

5.7.1 Vendor selection

5.7.2 SLAs and contract fulfillment AI5.2 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

Control: SLAs are fulfilled and the vendor is in compliance with contract, and assignment of penalties for failure to comply with the contract have been levied and collected.

5.7.2.1 Determine that SLAs have been achieved.

5.7.2.1.1 Obtain SLA agreement and SLA monitoring documents.

5.7.2.1.2 Verify that SLAs have been achieved by comparing the agreements to the documents.

5.7.2.2 Determine that remediation procedures have been implemented where SLAs have not been realized.

5.7.2.2.1 Verify that SLAs that have not been achieved have been included in the vendor remediation plan.

5.7.2.3 Determine if penalties have been assessed and collected in compliance with the contract.

5.7.2.3.1 Review the penalties assessed. Determine if they have been paid. If not, determine current status.

5.7.2.4 Determine if further escalation has been initiated, needs to be initiated or legal issues need to be escalated.

5.7.3 Vendor invoicingControl: Vendor invoicing is reviewed regularly and disputed charges are returned to the vendor.

AI5.4 X

5.7.3.1 Determine if vendors provide timesheets to reflect work performed. Determine that the timesheets are approved by project team management.

5.7.3.2 Determine if vendor invoicing is accurate according to the contract and supported by timesheets or other deliverables.

5.7.4 Intellectual property

5.7.5 Integration of vendor-provided application software into the organization AI2.5




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

Control: Vendor provided application software is subject to QA testing and further analysis to ensure that it will integrate into the organization’s architecture.

5.7.5.1 Confirm with key staff members whether the application software is customized and configured utilizing good practice as advised by vendors and conforming with internal architecture standards.

5.7.5.2 Inspect good practices supplied by vendors, compare with the implementation strategy, and identify inappropriate configuration and customization.

6 . CLOSURE PHASEClosure is the process of closing the project, finalizing the costs and accounting treatment, and transferring the remaining processes to operations functions.

6.1 GovernanceAudit/assurance objective: Governance over the project should be achieved through management’s oversight.

6.1.1 Business caseControl: All business case modifications have been approved by the executive sponsor and/or steering committee.

PO10.1PO10.14

AI1.1ME1.5ME4.2ME4.3

X X X

6.1.1.1 Interview the executive sponsor to ensure that the business case and the expected business processes/features were delivered with the application.

6.1.1.2 Review the scope change log and verify that all major changes affecting the business case have been included.

6.1.2 Scope management

6.1.3 Roles and responsibilitiesControl: The executive sponsor has approved and formally documented the closure of the project.

PO10.3 X




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

6.1.3.1 Verify the formal closure of the project by the executive sponsor.

6.1.4 Return on investment and KPIs

6.1.5 Escalation management

6.2 Project managementAudit/assurance objective: The project management activity should be ended, with all active project follow-up transferred to operations or business units.

6.2.1 Integration of business/information management

6.2.2 Composition of project team

6.2.3 Risk and issue managementControl: Open issues have been transferred to line operations.

PO10.2PO10.9PO10.13PO10.14

AI1.2

X

6.2.3.1 Verify that open issues, including needed revisions, and open fixes (sometimes referred to as a “punch list”), are transferred to the appropriate operating group.

6.2.4 Escalation procedures

6.2.5 Quality management

6.2.6 Use of development methodology


6.2.8 Planning and controlControl: Project management verifies that all deliverables have been completed.

PO10.6PO10.14 X

6.2.8.1 Verify that the project manager has formally documented receipt of all expected deliverables.

6.2.9 Milestone go/no go decisions




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

6.2.10 Progress Control

6.2.11 Expense and time managementControl: Expense and time management processes are closed, so no additional resource or expenses charges can be allocated to the project.

PO10.2PO10.3PO10.7

X

6.2.11.1 Verify that the time and expense has closed the project.

6.2.11.2 Verify that no charges have been applied to the project since the closure date.

6.2.12 CommunicationsControl: The stakeholders have been notified of the closure of the project.

PO10.2PO10.3PO10.7

X X X

6.2.12.1 Verify that stakeholders are aware that the project has been closed.

6.3 BudgetObjective: The budget and accounting processes should be accurate, complete and provide the information necessary to allocate final costs to the project.

6.3.1 Budget statusControl: The project budget is finalized with all costs. The budget to actual is prepared, with variance explanations.

PO10.2PO10.3PO10.7PO10.13PO10.14

X

6.3.1.1 Review the final budget figures. Determine that all costs have been applied.

6.3.1.2 Determine if all scope changes have been included in the final financial report.

6.3.1.3 Perform a cut-off audit to ensure that all costs are included.





reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

6.3.1.5 Discuss the budget and deliverables with key members of the project team and executive sponsor. Determine if the final budget figures were in line with the expectations.


PO10.2PO10.3PO10.7PO10.13PO10.14

X

6.3.2.1 Determine if the appropriate costs have been capitalized or expensed based upon standard accounting principles.

6.3.2.2 Determine if additional funding requests had been approved and reflected in the accounting for the project.

6.4 Internal controls

6.4.1 Internal control requirements




6.5.1 Testing requirements

6.5.2 Project plan

6.5.3 Testing content

6.6 Implementation

6.6.1 Pilot test plan

6.6.2 Readiness assessment

6.6.3 Resource planning




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

6.6.4 Conversion plan

6.6.5 Communication plan

6.6.6 Training planning

6.6.7 Transition plan

6.6.8 Backout plan

6.7 Third-party providersAudit/assurance objectives: Third-party providers should be paid according their contracts, remediation processes concluded, penalties collected and all deliverables due from the vendors received.


6.7.2 SLAs and contract fulfillmentControl: Contract provisions have been reviewed, all deliverables have been reviewed and accepted, open contract issues have been reviewed by project management and the executive sponsor, if necessary.

AI5.2 X

6.7.2.1 Determine if the project manager has reviewed all deliverables to determine vendor satisfaction of contract.

6.7.2.2 Determine if legal has reviewed the delivery of contract deliverables and agreed to successful completion of the contract.

6.7.3 Vendor invoicing

6.7.4 Intellectual propertyControl: Vendor’s access to enterprise information is deactivated and all enterprise property returned.

IA2.5 X

6.7.4.1 Determine if vendors were required to return enterprise property (badges, documents, etc.)

6.7.4.2 Determine if vendor access to enterprise information systems has been




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

suspended or deleted.

7 . POSTIMPLEMENTATION PHASEThe postimplementation review addresses the: Review of the project success Financial review of the business case vs. results Lessons learned and improvements for the future

7.1 GovernanceAudit/assurance objective: The business case should be achieved, (i.e. project costs are within budget and management has provided governance over the project).

7.1.1 Business caseControl: The project team leadership, on a regular basis, monitors and provides reports to the executive sponsor on the continued alignment of the project plan with the business case.

AI7.9 X X X

7.1.1.1 Interview the executive sponsor to ensure that the business case and the expected business processes/features were delivered with the application.

7.1.2 Scope management

7.1.3 ROI and KPIsControl: The project’s ROI and KPIs have been reviewed by the steering committee and executive sponsor.

AI7.9 X X X

7.1.3.1 Review the calculation of the project’s ROI.

7.1.3.2 Review the final KPIs for accuracy.

7.1.3.3 Determine if the key metrics have been reviewed and approved by the steering committee and the executive sponsor.

7.1.4 Escalation management

7.2 Project management

7.2.1 Integration of business/information management © 2009 ISACA. All rights reserved.



reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

7.2.2 Risk and issue management

7.2.3 Escalation procedures

7.2.4 Quality management

7.2.5 Use of development methodology


7.2.7 Planning and control

7.2.8 Milestone go/no go decisions

7.2.9 Progress control

7.2.10 Expense and time management

7.2.11 CommunicationsControl: The stakeholders have been received and reviewed ROI and key performance metrics.

AI7.9 X X X

7.2.11.1 Verify that the ROI and key performance metrics have been provided to the key stakeholders, including the executive sponsor and steering committee.

7.2.11.2 Determine if there is documented evidence of executive review of the metrics.

7.3 BudgetObjective: The budget and accounting processes should be accurate, complete and provide the information necessary to allocate final costs to the project.

7.3.1 Budget statusControl: The project budget is finalized with all costs. The budget to actual is prepared, with variance explanations. Management analyzes variances and evaluates how negative variances can be minimized in the future.

AI7.9 X

7.3.1.1 Determine the project summary report received by management.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

7.3.1.2 Determine if the level of detail provides management with the explanations necessary to evaluate variances.

7.3.1.3 Perform a cut-off audit to ensure that all costs are included.


7.3.1.5 Discuss the budget and deliverables with key members of the project team and executive sponsor. Determine if the final budget figures were in line with the expectations.


AI7.9 X

7.3.2.1 Determine if the appropriate costs have been capitalized or expensed based upon standard accounting principles.

7.3.2.2 Determine if additional funding requests have been approved and reflected in the accounting for the project.

7.4 Internal controlsAudit/assurance objectives: Internal controls, which ensure the accurate and complete processing of data, should be designed in the planning phase, when it is most efficient to modify, enhance or aggregate controls to provide a streamlined but thorough approach to internal control of application data and processes.

7.4.1 Internal control requirementsControl: Internal control requirements were reviewed and implemented during the development process and internal control deficiencies have not been identified after implementation.

X

7.4.1.1 Verify that internal control requirements identified in the planning and design phases have been implemented.




reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

7.4.1.2 Determine if control deficiencies identified after development, during testing, implementation or after implementation remain. Determine if a remediation plan has been developed and implemented.

7.4.1.3 Review the identified key controls and recommend additional controls as necessary to provide good practices.



7.4.3.1 Determine the level of reliance to be placed on the IT audit/assurance review of the development process. If sufficient testing will be performed and change management is at a good-practice level, the delivered system can be relied upon for audit/assurance purposes.

7.4.3.2 Determine if integrated CAATs will be built into the system to facilitate later audit/assurance activities.


7.5.1 Testing requirements

7.5.2 Project plan

7.5.3 Testing content

7.6 Implementation

7.6.1 Pilot test plan

7.6.2 Readiness assessment

7.6.3 Resource planning

7.6.4 Conversion plan

7.6.5 Communication plan

7.6.6 Training planning© 2009 ISACA. All rights reserved.



reference

COSO

Reference

Hyper-link

IssueCross-

referenceComments

Con

trol

Env

ironm

ent

Ris

k A

sses

smen

t

Con

trol A

ctiv

ities

Info

rmat

ion

and

Com

mun

icat

ion

Mon

itorin

g

7.6.7 Transition plan

7.6.8 Backout plan

7.7 Third-party providersAudit/assurance objectives: Third-party providers should be paid according their contracts, remediation processes concluded, penalties collected and all deliverables due from the vendors received.


7.7.2 SLAs and contracts fulfillmentControl: Contract provisions have been achieved, all deliverables have been reviewed and accepted, open contract issues have been reviewed by project management and executive sponsor, if necessary.

AI5.3 X

7.7.2.1 Determine if open issues remain with the vendor.

7.7.2.2 Determine if remediation plans are in effect to address open issues.

7.7.3 Intellectual property



VII. Maturity Assessment

The maturity assessment is an opportunity for the reviewer to assess the maturity of the processes reviewed. Based on the results of audit/assurance review, and the reviewer’s observations, assign a maturity level to each of the following COBIT control practices.

COBIT Control PracticeAssessed Maturity

Target Maturity

ReferenceHyper-link

Comments

PO10.1 Program Management Framework1. Define and document the programme, including all the projects required to achieve the

programme’s expected business outcomes. Specify required resources, including funding, project managers, project teams, IT resources and business resources where applicable. Gain formal approval of the document from key business and IT stakeholders.

2. Assign accountability clearly and unambiguously for each project, including achieving the benefits, controlling the costs, managing the risks and co-ordinating the project activities.

3. Determine the interdependencies of multiple projects in the programme, and develop a schedule for their completion that will enable the overall programme schedule to be met.

4. Determine programme stakeholders inside and outside the enterprise, and establish and maintain appropriate levels of co-ordination, communication and liaison with these parties.

5. Verify periodically with the business that the current programme as designed will meet business requirements and make adjustments as necessary. Review progress of individual projects and adjust the availability of resources as necessary to meet scheduled milestones.

PO10.2 Project Management Framework1. Ensure that the project management framework is consistent with, and is an integral

component of, the organisation’s programme management framework.2. Ensure that the project management framework includes:

• Guidance on the role and use of the programme or project office• A change control process for recording, evaluating, communicating and authorising

changes to the project scope, project requirements or system design• Requirements for integrating the project within the overall programme

3. Ensure that the project management method covers, at minimum, the initiating, planning, executing, controlling and closing project stages, as well as checkpoints and approvals.

PO10.3 Project Management Approach1. Prior to each project’s initiation, establish a project management governance structure

appropriate to the project’s size, complexity and risks, including legal, regulatory and reputational risks.

2. Assign each IT project one or more sponsors with sufficient authority to manage execution of the project within the overall programme.

3. Define the responsibility and accountability of the programme sponsor, the project manager,




Target Maturity

ReferenceHyper-link

Comments

and, as necessary, the steering committee and project management office.4. To track the execution of a project, put in place mechanisms such as regular reporting and

stage reviews that are the responsibility of the project manager to complete in a timely manner.

PO10.4 Stakeholder Commitment1. Obtain the commitment and participation of key stakeholders, including management of the

affected user department and key end users in the initiation, definition and authorisation of a project.

2. Outline during project initiation ongoing key stakeholder commitment and roles and responsibilities for the duration of the project life cycle. Ongoing involvement includes, but is not limited to, project approval, project phase approval, project checkpoint reporting, project board representation, project planning, product testing, user training, user procedures documentation and project communication material development.

PO10.5 Project Scope Statement1. Provide to the stakeholders a clear, written statement defining the nature, scope and business

benefit of every project to create a common understanding of project scope amongst stakeholders.

2. Ensure that key stakeholders and programme and project sponsors within the organisation and IT agree upon and accept the requirements for the project, including definition of project success (acceptance) criteria and key performance indicators.

3. Ensure that the project definition describes the requirements for a project communication plan that identifies internal and external project communications.

4. With the approval of stakeholders, maintain the project definition throughout the project, reflecting changing requirements.

PO10.6 Project Phase Initiation1. Gain approval and sign-off on the deliverables produced in each project phase from

designated managers and customers of the affected business and IT functions.2. Base the approval process on clearly defined acceptance criteria agreed to by key

stakeholders prior to work commencing on the project phase deliverable.3. Assess whether the project is on schedule, within budget and aligned with the agreed-upon

scope. Assess identified variances and identify the impact on the project plan and realisation of expected benefits.

4. Assess the project at agreed-upon major review points, and make formal ‘stop/go’ decisions based on predetermined critical success criteria.

PO10.7 Integrated Project Plan1. Develop a project plan that provides information to enable management to control project

progress. The plan should include details of project deliverables, required resources and




Target Maturity

ReferenceHyper-link

Comments

responsibilities, clear work breakdown structures and work packages, estimates of resources required, milestones, key dependencies, and identification of a critical path. Identify interdependencies of resources (e.g., key personnel) and deliverables with other projects.

2. Maintain the project plan and any dependent plans to ensure that they are up to date and reflect actual progress and material changes.

3. Ensure that there is effective communication of project plans and progress reports amongst all projects and with the overall programme. Ensure that any changes made to individual plans are reflected in the other plans.

PO10.8 Project Resources1. Identify resource needs for the project and clearly map out appropriate roles and

responsibilities, with escalation and decision-making authorities agreed upon and understood.

2. Identify required skills and time requirements for all individuals involved in the project phases in relation to defined roles. Staff the roles based on available skills information (e.g., IT skills matrix).

3. Utilise experienced project management and team leader resources with skills appropriate to the size, complexity and risk of the project.

4. Consider and clearly define the roles and the responsibilities of other involved parties, including finance, legal, procurement, human resources, internal audit and compliance.

5. Clearly define and agree upon the responsibility for procurement and management of third-party products and services, and manage the relationships.

PO10.9 Project Risk Management1. Establish a formal project risk management framework that includes identifying, analyzing,

responding to, mitigating, monitoring and controlling risks.2. Assign to appropriately skilled personnel the responsibility for executing the organisation’s

project risk management framework within a project. Consider allocating this role to an independent team, especially if an objective viewpoint is required or a project is considered critical.

3. Perform the project risk assessment of identifying and quantifying risks continuously throughout the project. Manage and communicate risks appropriately within the project governance structure.

4. Reassess project risks periodically, including at entry into each major project phase and as part of major change request assessments.

5. Identify risk and issue owners for responses to avoid, accept or mitigate risks.6. Maintain and review a project risk register of all potential project risks, and maintain a log of

all project issues and their resolution. Analyse the log periodically for trends and recurring problems, to ensure that root causes are corrected.




Target Maturity

ReferenceHyper-link

Comments

PO10.10 Project Quality Plan1. Identify ownership and responsibilities, quality review processes, success criteria and

performance metrics, to provide quality assurance for the project deliverables.2. Define any requirements for independent validation and verification of the quality of

deliverables in the plan.PO10.11 Project Change Control1. Establish a standard change request form and request process requiring documentation of the

requested change and the expected benefits of the change. The program management team should designate the individuals (business stakeholders, IT personnel) authorised to make project change requests.

2. Review change requests and estimate the potential effects on the project, including resource requirements and impact on schedule. Document the estimated project impact in the change request.

3. Review the completed change request and document the approval or denial of the request by key stakeholders, including business project sponsor and IT project manager.

4. Consider and approve at the programme level all approved project change requests based on an assessment of the effect the change will have on the other projects. If the requested change should not be implemented, share the reasons with the requesting project management team so they can evaluate alternative approaches.

5. Update the project and programme plans for all approved changes, and communicate approved changes to all business and IT stakeholders in a timely manner.

PO10.12 Project Planning of Assurance Methods1. Define the assurance tasks required to ensure compliance with internal controls and security

requirements that impact the systems or processes in the scope of the project. Include key compliance stakeholders in the definition and approval of assurance tasks.

2. Determine and document how the assurance tasks will be performed. Include appropriate subject matter specialists (e.g., audit, security or compliance) in the process.

PO10.13 Project Performance Measurement, Reporting and Monitoring1. Establish and use a set of project criteria as part of the programme management framework,

including, but not limited to, scope, schedule, quality, cost and level of risk.2. Measure project performance against key project performance criteria. Analyse deviations

from established key project performance criteria for cause, and assess positive and negative effects on the programme and its component projects. Report to identified key stakeholders progress for the programme and component projects, deviations from established key project performance criteria, and positive and negative effects on the programme and its component projects.

3. Monitor changes to the program and review existing key project performance criteria to




Target Maturity

ReferenceHyper-link

Comments

determine if they still represent valid measures of progress. Document and submit any necessary changes to the programme’s key stakeholders for their approval before adoption. Communicate revised criteria to project managers for use in future performance reports.

4. Recommend, implement and monitor remedial action, when required, in line with the program and project governance framework.

PO10.14 Project Closure1. Define and apply key steps for project closure, including post-implementation reviews that

assess whether a project attained desired results and benefits.2. Plan and execute post-implementation reviews to determine if projects delivered expected

benefits and to improve the project management and system development process methodology.

3. Identify, assign, communicate and track any uncompleted activities required to achieve planned programme project results and benefits.

4. Collect from the project participants and reviewers the lessons learned and key activities that led to delivered benefits. Analyse the data and make recommendations for improving the project management method for future projects.

AI1.1 Definition and Maintenance of Business Functional and Technical Requirements1. Define and implement a requirements definition and maintenance procedure and a

requirements repository that are appropriate for the size, complexity, objectives and risks of the business initiative that the organisation is considering undertaking. This procedure should take into account the nature of the enterprise’s business, strategic direction, strategic and tactical IT plans, in-house and outsourced business and IT processes, emerging regulatory requirements, people skills and competencies, structure, business case, and enabling technology.

2. Confirm that all stakeholder requirements, including relevant acceptance criteria, are considered, captured, prioritised and recorded in a way that is understandable to the stakeholders, business sponsors and technical implementation personnel.

3. Confirm that the functional and technical requirements are considered, captured and prioritised.

4. Confirm that the requirements include aspects regarding:• Continuity• Legal and regulatory compliance• Performance• Reliability• Compatibility• Auditability• Security and risk management




Target Maturity

ReferenceHyper-link

Comments

• Availability• Ergonomics• Operability and usability• Safety• Documentation (end user, operations, deployment, configuration)

AI1.2 Risk Analysis Report1. Use a holistic approach to risk analysis, ensuring that business, technology and project risks

are properly identified, examined, assessed and understood by all stakeholders.2. Consider as part of the risk analysis the impact of the project (development or acquisition,

implementation, operation, retirement, and disposal) on the organisation’s risk profile and threats to data integrity, security, availability, privacy, and compliance with laws and regulations.

3. Consider appropriate business, functional, technical and project risk mitigation activities as part of the definition of the possible solutions.

AI1.3 Feasibility Study and Formulation of Alternative Courses of Action1. Define and execute a feasibility study that clearly and concisely describes the key alternative

courses of action that will satisfy the business and functional requirements with an evaluation of their technological and economic feasibility. Identify required actions for the acquisition or development, and take into account scope and/or time and/or budget limitations.

2. Review the alternative courses of action with all stakeholders, and select the most appropriate one based on feasibility criteria, including risks and cost.

3. Translate the preferred course of action into a high-level acquisition/development plan identifying resources to be used and stages requiring a go or no-go decision.

AI1.4 Requirements and Feasibility Decision and Approval1. Obtain sign-off from the business sponsor and technical authority for the proposed approach,

and gather feedback requiring further feasibility analysis.2. Perform quality reviews at the end of each key project stage to assess the results against the

original acceptance criteria. Business sponsors and other stakeholders should sign off on each successful quality review.

AI2.1 High-level Design1. Establish a high-level design specification that translates the business requirements for the

software development based on the organisation’s technological direction and information architecture model.

2. Confirm that the design approach and documentation are compliant with the organisation’s design standards.

3. Involve appropriately qualified and experienced users in the design process to draw on their




Target Maturity

ReferenceHyper-link

Comments

expertise and knowledge of existing systems or processes.4. Confirm that the design is consistent with the business plans, strategies, applicable

regulations and IT plans.5. Ensure that the high-level design is approved and signed off on by IT stakeholders (e.g.,

human/computer interaction, security and other experts) to ensure that their inputs have been recognised and the design, as a whole, constitutes a solution that the organisation can deliver, operate and maintain. Establish that no project proceeds to the business approval process without appropriate review and sign-off by IT stakeholders.

6. Submit the final high-level design after QA sign-off to the project sponsor/business process owner, and obtain approval and sign-off. Establish that no project proceeds to development without appropriate sign-off by the business.

AI2.2 Detailed Design1. Classify data inputs and outputs according to information architecture and data dictionary

standards.2. Assess the impact on existing applications and infrastructure during the process of gathering

requirements and designing the solution, and design appropriate integration approaches. Address integration of the planned application system with existing or planned co-operating applications and infrastructure, including packaged software acquired from third parties. Consider the impact of differing update cycles.

3. Specify the source data collection design, documenting the data that must be collected and validated for processing transactions as well as the methods for validation. Consider data inputs from existing programs, packaged software, external parties, web forms, etc.

4. Define system availability requirements, and design appropriate redundancy, failure recovery and backup processing arrangements.

5. Define file and database requirements for storage, location and retrieval of data. Consider availability, control and auditability, security, and network requirements.

6. Define the processing steps, including specification of transaction types and processing rules incorporating logic transformations or specific calculations. Consider availability, control and auditability, logging, and audit trails.

7. Based on the user requirements and taking into account the different types of recipients, usage, details required, frequency, method of generation and other design details, define the data requirements for all identified outputs. Appropriate design requirements should guarantee the availability, completeness, integrity and confidentiality of output data. Consider the impact of data outputs to other programs, external parties, etc.

8. Design the interface between the user and the system application so that it is easy to use and self-documenting. Consider the impact of system-to-system interface design on infrastructure performance, including the capacity of personal computing devices and network bandwidth




Target Maturity

ReferenceHyper-link

Comments

and availability.9. Reassess system design whenever significant technological and/or logical discrepancies

occur during design, development and maintenance. Results of the reassessment should be subject to the normal approval cycle.

10. Prepare and document detailed design specifications in accordance with organisational and industry-accepted specification standards and the information architecture.

11. Conduct a design walk-through with IT and business stakeholders before development is initiated, as a part of the sign-off process for the design specifications. Various aids can be used to assist with the sign-off, including prototypes, to aid stakeholder understanding of the final design.

AI2.3 Application Control and Auditability1. Define all automated application controls (authorisation, input, processing and output) based

on business process control requirements provided in the requirements documentation.2. Define how business processes will need to be adjusted to use the automated control

functions provided in purchased/packaged application software.3. Confirm the design specifications for all automated application controls with IT technical

authorities and business process owners, and obtain their approval and sign-off.4. Confirm that the design includes automated controls within the application that support

general control objectives (such as security, data integrity and audit trails), including access control mechanisms and database integrity controls. Confirm that the design has received sign-off from relevant technical design authorities and approval of the business process owner.

5. Assess design specifications of automated application and general controls against internal audit, control, and risk management standards and objectives. Consider the effect of compensating controls outside the application software realm.

AI2.4 Application Security and Availability1. Design approaches and solutions to security and availability that adequately meet the defined

requirements. These approaches should take into account the organizational security architecture and policies, industry security and privacy best practices, and regulatory and compliance requirements for security and privacy.

2. Consider the security and availability infrastructure already in place. Where possible, build on and extend these capabilities.

3. Consider access rights and privilege management, protection of sensitive information at all stages, authentication and transaction integrity, and automatic recovery.

4. Define how the solutions for security and availability in the infrastructure will be integrated with the application, paying particular attention to transactions, local and wide area networks (e.g., Internet), shared and federated databases, access control mechanisms, load detection,




Target Maturity

ReferenceHyper-link

Comments

and recovery mechanisms.5. Confirm the design of security, availability, access management, authentication and

protection of transaction integrity with IT technical authorities and, as appropriate, subject matter experts. Obtain their sign-off on and approval of the design. Also confirm with business process owners that the design meets their security and availability requirements using non-technical walk-throughs, where necessary, to confirm understanding.

AI2.5 Configuration and Implementation of Acquired Application Software1. Assess the impact of any major upgrade and classify it according to agreed-upon objective

criteria (such as business requirements), based on the outcome of analysis of the risk involved (such as impact on existing systems and processes or security), cost-benefit justification and other requirements. Follow normal system development and implementation processes as appropriate for the nature of the change.

2. Consider interoperability with existing applications and databases, appropriate user interfaces, and efficient utilisation of technology resources (e.g., security framework and standards, availability, access management, auditability, networks and storage) in the design specification.

3. Consider the impact of customisation and configuration on the performance and efficiency of the acquired packaged application software and on existing applications, operating systems and other infrastructure.

4. Consider the effect of contractual terms with the vendor on the design of customisation and configuration.

5. Consider the availability of source code for purchased/packaged applications in the customisation and configuration process. Review contractual arrangements with the vendor. Consider escrow arrangements where the source code is not available. Assess the risks in the event that the acquired application packaged software is no longer available at the expiry of a contract or for other reasons.

6. Ensure that testing procedures cover verification of acquired application control objectives (such as functionality, interoperability with existing applications and infrastructure, system performance efficiency, integrated capacity and load stress testing, and data integrity).

7. Conduct a design walk-through with IT and business stakeholders before customization is initiated, as a part of the sign-off process for the customisation and configuration of application software specifications.

8. Consider whether the implications of customisation and configuration require reassessment of strategies for acquired application packaged software.

9. Obtain approval of business process owners for detailed design specifications for customisation and configuration of application software.

10. Ensure that user and operational manuals (online help) are complete and updated where




Target Maturity

ReferenceHyper-link

Comments

necessary to account for any customisation or special conditions unique to the implementation.

11. Consider when the effect of cumulative customizations and configurations (including minor changes that were not subjected to formal design specifications) require a high-level reassessment of the acquired solution and associated functionality. Assess whether these changes trigger the development of a detailed design specification for customisation and configuration of the application software. Assess whether these changes restrict the ability of the organization to adopt vendor upgrades to purchased applications packaged software.

AI2.6 Major Upgrades to Existing Systems1. Assess the impact of any major upgrade and classify it according to specified objective

criteria (such as business requirements), based on the outcome of analysis of the risk involved (such as impact on existing systems and processes or security), cost-benefit justification and other requirements. Follow normal system development and implementation processes as appropriate for the nature of the change.

2. Obtain agreement on and approval of the implementation of the development and implementation process with the business process sponsor and other affected stakeholders. Ensure that the business process owners understand the effect of designating changes as maintenance or major upgrades.

AI2.7 Development of Application Software1. Establish development procedures to ensure that the development of application software

adheres to organisational development standards.2. Ensure that application software is developed based on agreed-upon specifications and

business, functional and technical requirements.3. Establish agreed-upon stages of the development process (development checkpoints). At the

end of each stage, facilitate formal discussions of approved criteria with the stakeholders. Obtain approval and sign-off from all stakeholders following successful completion of functionality, performance and quality reviews before finalizing stage activities. At the final stage, confirm with IT technical authorities and operations management that the applications are ready and suitable for migration to the production environment.

4. Assess the adequacy of software developed in terms of its compatibility and ease of integration with existing applications and infrastructure.

5. When third-party developers are involved with the applications development, establish that they adhere to contractual obligations and organisational development standards and that licensing requirements have been addressed.

6. Monitor all development activities and track change requests and design, performance and quality reviews, ensuring active participation of all impacted stakeholders, including




Target Maturity

ReferenceHyper-link

Comments

business process users and IT technology representatives.7. Ensure that requested changes arising within IT or from the business process owner are

tracked.8. Consider the effect of dynamic, non-sequential development techniques (e.g., rapid

application development, extreme programming) on the monitoring of the application development progress and approval of application software by stakeholders.

AI2.8 Software Quality Assurance1. Define a software QA plan. Ensure that the plan includes:

• Specification of quality criteria• Validation and verification processes• Definition of how quality will be reviewed• Necessary qualifications of quality reviewers• Roles and responsibilities for the achievement of qualityConsider:• The effect of embedding quality within the development process• The presence or absence of formal review by independent QA teams• Ensuring that reviewers are independent from the development team

2. Design a process that monitors the software quality based on:• Project requirements• Enterprise policies• Adherence to site development systems methodologies• Quality management procedures and acceptance criteria

3. Employ code inspection, programme walk-throughs and testing of applications. Report on outcomes of the monitoring process and testing to the application software development team and IT management.

4. Monitor all quality exceptions. Ensure that corrective actions are taken. Maintain a record of all reviews, results, exceptions and corrections. Repeat quality reviews, where appropriate, based on the amount of rework and corrective action.

AI2.9 Applications Requirements Management1. Design a process for standardising, tracking, recording and approving all

change requests during development of application systems.2. Assess the impact of all project change requests, and categorise and

prioritise them accordingly.3. Track changes to requirements for development projects, enabling all

stakeholders to monitor, review and approve the changes. Ensure that the outcomes of the change process are fully understood and agreed to




Target Maturity

ReferenceHyper-link

Comments

by the stakeholders.AI2.10 Application Software Maintenance1. Design an effective and efficient process for application software

maintenance activities. Prioritise maintenance activities, paying attention to business needs and resource requirements. Ensure that all changes in software comply with the formal change management process, including impact on other systems and infrastructure. Ensure that risk and security requirements and interdependencies are addressed.

2. Monitor all maintenance changes. If appropriate, aggregate maintenance tasks into a single ‘change’ to make management and control easier. Ensure that any major maintenance is categorised and managed as a formal redevelopment.

3. Establish the review and approval of all emergency or any other changes applied without adherence to the formal change process.

4. Ensure that the pattern and volume of maintenance activities are analysed periodically for abnormal trends indicating underlying quality or performance problems.

5. Establish processes to ensure that all maintenance activity is completed successfully and thoroughly. Track maintenance activities to ensure completion. Where necessary, update user systems and operational documentation.

AI3.1 Technological Infrastructure Acquisition Plan1. Create and maintain a plan for the acquisition, implementation and

upgrade of technology infrastructure that meets established business functional and technical requirements and is in accord with the organisation’s technology direction. The plan should also consider future flexibility for capacity additions, transition costs, technical risks and, for technology upgrade purposes, the lifetime of the investment. The plan should be integrated with the organisation’s strategic and operational planning processes.

2. Ensure that the plan includes a financial appraisal stating the ROI over the expected lifetime of the infrastructure.

3. Review all acquisition plans considering risks, costs, benefits and technical conformance with corporate technology standards. Any deviations should be authorised by the IT architecture board. Approve all reviewed and accepted plans with a formal sign-off.

4. Establish a feedback process to support continuous improvement and © 2009 ISACA. All rights reserved.



Target Maturity

ReferenceHyper-link

Comments

raise any suggested changes to the technology infrastructure plan, technology guidelines and standards.

AI3.2 Infrastructure Resource Protection and Availability1. Back up and secure all infrastructure data and software prior to

installation or maintenance tasks.2. Test whether the application software environment is separated from, but

sufficiently similar to, production to verify functionality and establish its security, availability or integrity conditions. This ensures that they operate appropriately and are in compliance with requirements established within the acquisition and maintenance framework for technology infrastructure. Analyse and follow vendor recommendations.

3. Assess all the security aspects associated with system software installation and maintenance processes, especially the modification of original passwords assigned by service providers and the setup of parameters that may affect security, such as vendor-established default parameter settings.

4. Monitor when temporary access is granted to allow installation, and ensure that passwords are changed as installation is completed.

5. Monitor that only appropriately licenced software is tested and installed. Review the process to ensure that system software installation is performed in accordance with vendor guidelines and any deviations are discussed with the vendor to assess potential impact.

6. Control movement of programs and data amongst libraries by ensuring that this is performed by an independent group (e.g., librarian).

7. Enforce acceptance procedures using objective acceptance criteria to ensure that product performance (including security and functionality) is consistent with the agreed-upon specifications and/or SLA requirements.

8. Provide appropriate training to personnel who use sensitive infrastructure components.

9. Monitor and log access and maintenance of sensitive infrastructure components, and ensure that these are regularly reviewed.

AI4.1 Planning for Operational Solutions1. Define and document the operational procedures in advance of

implementation, and establish them during acceptance tests to ensure that they are complete, accurate and usable.

2. Define and document the user procedures in advance of implementation, and establish them during acceptance tests to ensure that they are




Target Maturity

ReferenceHyper-link

Comments

complete, accurate and usable.AI4.2 Knowledge Transfer to Business Management1. Establish ownership of the system. Define management and

administrative functions, security and control procedures, and training requirements.

2. Create management documentation, including roles and responsibilities, segregation of duties, business continuity considerations, access and privilege controls, administration procedures, and internal control procedures.

3. Involve business management in the creation of management documentation, and integrate any procedures with existing management and control procedures.

4. Provide training to business management on how to manage the system effectively.

5. Collect regular feedback from business management on the adequacy of the supporting documentation, procedures and related training.

AI4.3 Knowledge Transfer to End Users1. Create all the required user instructions, documentation, procedures and

training materials on a timely basis to enable efficient and effective use of the new system.

2. Create informative and understandable end-user documentation and reference materials, designed for all levels of expertise, written in plain language and easily accessible (e.g., electronic documentation).

3. Involve end-user groups in the creation of end-user support documentation, and integrate any procedures with existing end-user procedures.

4. Provide training to end users on how to use the system effectively.5. Assess end-user documentation (such as procedure manuals, online help

and help desk support material) for content and quality as part of user acceptance testing of the system.

6. Collect regular feedback from end users on the adequacy of the end-user documentation, procedures and related training.

AI4.4 Knowledge Transfer to Operations and Support Staff1. Create informative and understandable system maintenance and support

documentation that is written in plain language and is easily accessible (e.g., service desk scenarios and electronic documentation).

2. Involve operations and support staff in the creation of maintenance and © 2009 ISACA. All rights reserved.



Target Maturity

ReferenceHyper-link

Comments

support documentation, and integrate any procedures with existing operational procedures.

3. Provide training to operations support staff on how to support the new system effectively. Include the business purpose of the system and service levels required.

4. Assess operations documentation (such as procedure manuals, online help, FAQs and help desk support material) for content and quality as part of user acceptance testing of the system.

5. Collect regular feedback from operations and support staff on the adequacy of the operations documentation, procedures and related training.

AI5.1 Procurement Controls1. Define IT procurement policies and procedures in alignment with the

organisation’s procurement policies and procedures. The IT procurement policies and procedures should address specific concerns such as:• Legislative requirements• Compliance with the organisation’s IT acquisition policy• Involvement of IT legal contract expertise• Licensing and leasing requirements• Technology upgrade clauses• Escrow arrangements• Vendor software support and security arrangements• Ensuring involvement of the business• Total cost of ownership• Acquisition plan for major acquisitions• Recording of assets

2. Define and implement standard procurement procedures that use selection approaches responsive to the risks associated with the procurement.

3. Define and implement required approvals at key decision points during the procurement processes. Obtain approval from senior management in advance, if the existing policy will not be followed.

4. Record receipt of all hardware and software acquisitions in an asset inventory, and assess the quality before making any payment.

AI5.2 Supplier Contract Management1. Establish supplier contract management policies and procedures in

accordance with legal terms and conditions. The policies and procedures © 2009 ISACA. All rights reserved.



Target Maturity

ReferenceHyper-link

Comments

should address specific concerns such as:• Supplier responsibilities• Client responsibilities• Supplier SLAs• Monitoring and reporting against SLAs• Transition arrangements• Notification and escalation procedures• Security standards, records management and control requirements• Required supplier QA practices• Right to audit• Penalties or incentives relating to agreed-upon service levels• Intellectual property rights• Provision for independent assurance• Technology upgrade clauses

All contracts and contract changes should be reviewed by legal advisors.2. Define and implement a policy and related procedures to establish,

change and terminate supplier contracts. Consult the appropriate stakeholders, including legal, purchasing, audit, business and IT representatives.

3. Perform review of supplier internal controls by management or independent third parties based on contracts with key service suppliers.

4. Obtain and review contracts for clauses relating to third-party reviews and obtain reports from such reviews.

5. When defining the contract remedies, consider software escrow agreements and alternative suppliers or standby agreements in the event of supplier failure.

6. Enquire whether remedies were considered when defining the contract.AI5.3 Supplier Selection1. Review all requests for information (RFIs) and requests for proposal

(RFPs) to ensure that they:• Clearly define requirements• Include a procedure to clarify requirements• Allow vendors sufficient time to prepare their proposals• Clearly define award criteria and the decision process

2. Evaluate RFIs and RFPs in accordance with the approved evaluation process/criteria, and maintain documentary evidence of the evaluations.




Target Maturity

ReferenceHyper-link

Comments

Verify the references of candidate vendors.3. Select the supplier that best fits the RFP, document and communicate the

decision, and sign the contract.4. In the specific case of software acquisition, include and enforce the rights

and obligations of all parties in the contractual terms. These rights and obligations may include ownership and licencing of intellectual property, maintenance, warranties, arbitration procedures, upgrade terms, and fit for purpose, including security, escrow and access rights.

5. In the specific case of acquisition of development resources, include and enforce the rights and obligations of all parties in the contractual terms. These rights and obligations may include ownership and licencing of intellectual property; fit for purpose, including development methodologies; languages; testing; quality management processes, including required performance criteria; performance reviews; basis for payment; warranties; arbitration procedures; human resource management; and compliance with the organisation’s policies. Obtain legal advice on resource development acquisition agreements regarding ownership and licencing of intellectual property.

6. In the specific case of acquisition of infrastructure, facilities and related services, include and enforce the rights and obligations of all parties in the contractual terms. These rights and obligations may include service levels, maintenance procedures, access controls, security, performance review, basis for payment and arbitration procedures.

AI5.4 IT Resources Acquisition1. Review the technology delivery life cycle and related deliverables and approve deliverables

at key points in the acquisition cycle. Ensure that technology updates are available within agreed-upon time frames.

2. Obtain broad licencing rights and ownership wherever possible and ensure that the supplier is in compliance with applicable regulations.

3. Confirm that the technology acquired delivers as required, proposed and contractually agreed upon, through oversight, inspection and testing. Oversee the delivery of technology components of identified automated solutions. Test acquired software/hardware resources with the standard test procedures, in appropriate environments and using representative data. Maintain test data confidentiality where applicable. Review documentation and knowledge transfer to enable efficient future maintenance.

AI7.1 Training1. For systems development, implementation or modification projects, a training plan is an

integral part of the overall project master plan. Ensure that the plan clearly identifies learning © 2009 ISACA. All rights reserved.



Target Maturity

ReferenceHyper-link

Comments

objectives, resources, key milestones, dependencies and critical path tasks impacting the delivery of the training plan. The plan should consider alternative training strategies depending on the business needs, risk level (e.g., for mission-critical systems, a formal system of user accreditation and reaccreditation may be appropriate), and regulatory and compliance requirements (e.g., impact of varying privacy laws may require adaptation of the training at a national level).

2. Ensure that the training plan identifies and addresses all impacted groups, including business end users, IT operations, support and IT application development training, and service providers. The training plan should incorporate the delivery of the training in a timely manner. It should also identify staff members who must be trained and those for whom training is desirable.

3. Consider alternative training strategies that satisfy the training requirements, and select the most cost-effective approach that aligns with the organisation’s training framework. Alternative strategies include train the trainer, end-user accreditation and intranet-based training.

4. Confirm that there is a process to ensure that the training plan is executed satisfactorily. Complete the documentation detailing compliance with the training plan. Examples of information include lists of staff members invited to attend the training, attendees, evaluations of achievement of learning objectives and other feedback.

5. Monitor training to obtain feedback that could lead to potential improvements in either the training or the system.

6. Monitor all planned changes to ensure that training requirements have been considered and suitable plans created. Consider postponing the change if training has not been performed and the lack of training would jeopardise the implementation of the change.

AI7.2 Test Plan1. Develop and document the test plan, which aligns to the project quality plan and relevant

organisational standards. Communicate and consult with appropriate business process owners and IT stakeholders.

2. Ensure that the test plan reflects an assessment of risks from the project and that all functional and technical requirements are tested. Based on assessment of the risk of system failure and faults on implementation, the plan should include requirements for performance, stress, usability, pilot and security testing.

3. Ensure that the test plan addresses the potential need for internal or external accreditation of outcomes of the test process (e.g., financial regulatory requirements).

4. Ensure that the test plan identifies necessary resources to execute testing and evaluate the results. Examples of resources include construction of test environments and staff for the test group, including potential temporary replacement of test staff in the production or




Target Maturity

ReferenceHyper-link

Comments

development environments. Ensure that stakeholders are consulted on the resource implications of the test plan.

5. Ensure that the test plan identifies testing phases appropriate to the operational requirements and environment. Examples of such testing phases include unit test, system test, integration test, user acceptance test, performance test, stress test, data conversion test, security test, operational readiness, and backup and recovery tests.

6. Confirm that the test plan considers test preparation (including site preparation), training requirements, installation or an update of a defined test environment, planning/performing/documenting/retaining test cases, error and problem handling, correction and escalation, and formal approval.

7. Ensure that the test plan establishes clear criteria for measuring the success of undertaking each testing phase. Consult the business process owners and IT stakeholders in defining the success criteria. Determine that the plan establishes remediation procedures when the success criteria are not met (e.g., in a case of significant failures in a testing phase, the plan provides guidance on whether to proceed to the next phase, stop testing or postpone implementation).

8. Confirm that all test plans are approved by stakeholders, including business process owners and IT, as appropriate. Examples of such stakeholders are application development managers, project managers and business process end users.

AI7.3 Implementation Plan1. Define a policy for numbering and frequency of releases.2. Confirm that all implementation plans are approved by stakeholders, including technical and

business.3. Create an implementation plan reflecting the outcomes of a formal review of technical and

business risks. Include with the implementation plan:• The broad implementation strategy• The sequence of implementation steps• Resource requirements• Interdependencies• Criteria for management agreement to the production implementation• Installation verification requirements• Transition strategy for production support

Align the implementation plan with the business change management plan.

4. Obtain commitment from third parties to their involvement in each step of the implementation.

5. Identify and document the fallback and recovery process.




Target Maturity

ReferenceHyper-link

Comments

AI7.4 Test Environment1. Ensure that the test environment is representative of the future operating landscape, including

likely workload stress, operating systems, necessary application software, database management systems, and network and computing infrastructure found in the production environment.

2. Ensure that the test environment is secure and incapable of interacting with production systems.

3. Create a database of test data that are representative of the production environment. Sanitise data used in the test environment from the production environment according to business needs and organisational standards (e.g., consider whether compliance or regulatory requirements oblige the use of sanitised data).

4. Protect sensitive test data and results against disclosure, including access, retention, storage and destruction. Consider the effect of interaction of organisational systems with those of third parties.

5. Put in place a process to enable proper retention or disposal of test results, media and other associated documentation to enable adequate review and subsequent analysis as required by the test plan. Consider the effect of regulatory or compliance requirements.

AI7.5 System and Data Conversion1. Define a data conversion and infrastructure migration plan. Consider, for example, hardware,

networks, operating systems, software, transaction data, master files, backups and archives, interfaces with other systems (both internal and external), procedures and system documentation, in the development of the plan.

2. Ensure that the data conversion plan incorporates methods for collecting, converting and verifying data to be converted, and identifying and resolving any errors found during conversion. This includes comparing the original and converted data for completeness and integrity.

3. Confirm that the data conversion plan does not require changes in data values unless absolutely necessary for business reasons. Document changes made to data values, and secure approval from the business process data owner.

4. Consider real-time disaster recovery, business continuity planning, and reversion in the data conversion and infrastructure migration plan where risk management, business needs, or regulatory/compliance requirements demand.

5. Co-ordinate and verify the timing and completeness of the conversion cutover so there is a smooth, continuous transition with no loss of transactions. Where necessary, in the absence of any other alternative, freeze live operations.

6. Ensure that there is a backup of all systems and data taken at the point prior to conversion, audit trails are maintained to enable the conversion to be retraced, and there is a fallback and




Target Maturity

ReferenceHyper-link

Comments

recovery plan in case the conversion fails. Ensure that retention of backup and archived data conforms to business needs and regulatory or compliance requirements.

AI7.6 Testing of Changes1. Ensure that testing of changes is undertaken in accordance with the testing plan. Ensure that

the testing is designed and conducted by a test group independent from the development team. Consider the extent to which business process owners and end users are involved in the test group. Ensure that testing is conducted only within the test environment.

2. Ensure that the tests and anticipated outcomes are in accordance with the defined success criteria set out in the testing plan.

3. Consider using clearly defined test instructions (scripts) to implement the tests. Ensure that the independent test group assesses and approves each test script to confirm that it adequately addresses test success criteria set out in the test plan. Consider using scripts to verify the extent to which the system meets security requirements. Consider the appropriate balance between automated scripted tests and interactive user testing.

4. Undertake tests of security in accordance with the test plan. Measure the extent of security weaknesses or loopholes. Consider the effect of security incidents since construction of the test plan. Consider the effect on access and boundary controls.

5. Undertake tests of system and application performance in accordance with the test plan. Consider a range of performance metrics (e.g., end-user response times and database management system update performance).

6. When undertaking testing, ensure that the fallback and rollback elements of the test plan have been addressed.

7. Identify, log and classify (e.g., minor, significant and mission-critical) errors during testing. Ensure that an audit trail of test results is available. Communicate results of testing to stakeholders in accordance with the test plan to facilitate bug fixing and further quality enhancement.

AI7.7 Final Acceptance Test1. Ensure that the scope of final acceptance evaluation activities covers all components of the

information system (e.g., application software, facilities, technology, user procedures, operations procedures, monitoring and support).

2. Ensure that the categorised log of errors found in the testing process has been addressed by the development team. Ensure that the cause of errors has been remediated (e.g., by appropriate changes to the application, configuration or workaround, and/or delayed correction where the error is minor).

3. Ensure that the final acceptance evaluation is measured against the success criteria set out in the testing plan. Ensure that the review and evaluation process is appropriately documented.

4. Document and interpret the final acceptance testing results, and present them in a form that is




Target Maturity

ReferenceHyper-link

Comments

understandable to business process owners and IT so an informed review and evaluation can take place.

5. Ensure that business process owners, third parties (as appropriate) and IT stakeholders formally sign off on the outcome of the testing process as set out in the testing plan. Such approval is mandatory prior to promotion to production.



VIII. Assessment Maturity vs. Target Maturity


systems development and project management audit/assurance ... · web viewisaca® with more than...

Documents