city auditor’s office 2017 annual report

ISC: UNRESTRICTED

AC2018-0019

Attachment

City Auditor’s Office 2017 Annual Report

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

It is my ongoing pleasure, since 2013, to serve City Council and Calgarians as your City Auditor. I am privileged to lead a City Auditor’s Office (CAO) of highly experienced and effective professionals, who year over year have provided added value to The City of Calgary (The City) through the consistent delivery of independent and objective assurance, advisory and investigative work.

The work we do is critical to Audit Committee, a Committee of Council, as it supports their increasingly important role of providing effective City governance through effective oversight and risk management. We also continue to provide a comprehensive Whistle-blower Program that operates with high integrity, and is available to both City employees and citizens.

Results of our work are brought in the form of recommendations and action plan commitments that support The City’s common purpose to make life better every day for the citizens of today and tomorrow. Our success is visible in supporting positive change, both in immediate improvements as well as year over year gains. It is these positive change success stories that I am pleased to share with you.

In 2017, we delivered 48 valued recommendations from our audits, and monitored the closure of 79 action plans. We completed 38 Whistle-blower investigations which in turn generated 48 corrective actions. In addition we provided ongoing advisory services focused on fostering best practices, innovation, and efficiency. We accomplished these positive changes with a staff of 15 and a budget spend of $2.7M.

We continue to hold ourselves accountable to delivering our work in accordance with our approved Audit Plan and our professional and internal performance standards. In April, 2017, our audit practices were formally confirmed with the external assessment report of ‘Generally Conforms’ to the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing by the Institute of Internal Auditors. We are pleased to share details of these positive results in this report.

As we move ahead into 2018, we will continue our valued work in accordance with our approved 2017/2018 Audit Plan. In sync with the next four year service based budgetary process, we will reassess and revise our audit universe and establish a 2019/2020 audit plan that reflects current risks associated with Council Priorities, service strategies, capital projects and other significant initiatives. We remain steadfast in our commitment to include in our audit plan as much as we can, as efficiently and effectively as we can, within the budget we are provided.

The CAO is your independent and objective body and is committed to delivering the highest standards and best practices of a high performing audit office in the public sector.

Katharine Palmer, CIA, CFE, MBA City Auditor

Message

from

The City

Auditor

Message

from

The City

Auditor

Message

from

The City

Auditor

Message

from

The City

Auditor

Message

from

The City

Auditor

Message

from

The City

Auditor

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Table of Contents

Message from The City Auditor ......................................................................... 2

1.0 Our Coverage .................................................................................................. 4

1.1 Annual Activities......................................................................................... 4

1.2 Audits Completed ...................................................................................... 5

1.3 Advisory Services ...................................................................................... 9

1.4 Investigative Services………………………………………………………...10

2.0 Our Impact .................................................................................................... 13

2.1 Audit Recommendations .......................................................................... 13

2.2 Investigation Recommendations.............................................................. 15

3.0 Our Commitment to Value Add ................................................................... 16

3.1 Key Performance Indicators .................................................................... 16

3.2 Professional Work Standards .................................................................. 17

3.3 Budget ..................................................................................................... 18

4.0 Appendices ................................................................................................... 19

Appendix A – Audit Activity Status as at December 31, 2017 ....................... 19

Appendix B – Audit Recommendation Follow-up .......................................... 21

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

1.0

Our Coverage

1.1 Annual Activities

During 2017, the City Auditor’s Office (CAO) audit, advisory and investigative services provided significant interaction with 80% of the Business Units within The City of Calgary (The City). Additionally, due to the nature of our audit work, there were many touch points with several of our City subsidiaries and partners. Our extensive coverage, as highlighted in red text, provided us with greater insight into challenges and opportunities faced by the corporation, and increased our agility to provide valued advice in response to key risks.

The CAO was able to achieve this coverage due to the effective collaborative relationship that exists with Administration and the office’s team of 15 professional staff. During Q1 2017, the CAO also reviewed and made changes to both the City Auditor’s and Whistle-blower’s internal and external websites.

*This is not intended to be an exhaustive list of all subsidiaries and partners.

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

1.2 Audits Completed

The CAO is focused on optimizing audit efficiency through increased utilization of tools, and review and revision of practices and processes. Data analytics is an integral part of the CAO. Analytics provides insight into process anomalies, trends and risk indicators through the extraction and analysis of transactional or unstructured data. The initial objective was to expand the use of existing audit data analytics to create a bank of 20-30 analysis reports that could be utilized to monitor compliance to policies (HR, Finance, Legal and others) and to assess risk indicators across the organization. Looking forward, we plan to embed more data analytics into our audit work and increase the use of data analytics and Computer-Aided Audit Tools (CAATs) to make our audits more efficient.

During 2017, the CAO finalized ten audits and initiated an additional eight audits. Full details of the status of all audits at year-end can be found in Appendix A. Summaries of finalized audits are set out below.

Deputy City Manager’s Office

Corporate Structures List The Corporate Structures List (CSL) initiative is constructing a comprehensive repository of structures by Business Unit, along with basic attributes for each structure. The objective of the audit was to assess the effectiveness of the CSL as a tool to manage the facility portfolio at a corporate level. This was achieved by assessing time frames and criteria for moving the initiative to a sustainment phase, and the design of controls that ensure information quality. As the list was under development, and subject to ongoing additions and alternations, we did not test its accuracy.

The CSL tool provides a foundation for managing The City’s facility portfolio at a corporate level. However, the initiative does not have formal objectives to provide clarity of purpose and balance the different stakeholder needs. The information quality controls in place provide sufficient checks at the CSL’s current development phase. As the initiative moves into sustainment, preventative controls are needed, and the detective information quality controls Facility Management are currently implementing will need to be formalized. We raised six recommendations to support CSL as the initiative moves into sustainment phase.

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Chief Financial Officer

Human Resources – Succession Planning Effective succession planning helps to build The City’s resiliency by addressing continuity risk for critical positions while also increasing leadership capacity, employee engagement, retention, and productivity. The objective of the audit was to assess the effectiveness of the succession planning process across the organization by assessing the established process for General Manager and Director positions and processes utilized in a sample of Business Units addressing Manager, Supervisor and technical/professional positions.

Overall, our audit testing determined that The City’s succession planning process is designed and operating effectively. We identified two areas where processes could be improved. Firstly, although Business Units are encouraged to customize succession planning to meet their needs, we noted inconsistent awareness and use of Human Resources (HR) succession management guidance and tools. Secondly, the process is manual and not integrated with other HR systems. In addition, we brought forward opportunities for improvement related to development opportunities, candidate readiness and inclusion, and cross-departmental sharing of best practices. Four recommendations were raised to improve efficiency of the succession planning process, strengthen the talent pool available to The City, and help identify and develop early and diverse talent.

Information Technology (IT) Follow-Up Audit The objective of this audit was to assess the effectiveness of management’s actions to mitigate business risks in response to CAO IT audit recommendations raised over the last five years. We evaluated the effectiveness of current status implementation of management actions through the assessment of risk mitigation approaches, residual risk exposure, and, where appropriate, raised opportunities to mitigate undesired risks. This audit conducted a follow-up of nine management actions that were deemed high risk due to the nature of changing or new technology, recently established IT investment governance model and IT security governance. We assessed five of the nine management actions as effectively implemented to mitigate the business risks. For the remaining four management actions, five recommendations were raised to support further timely risk mitigation.

Community Services

Calgary Neighbourhoods’ Support of Community Associations The City engages in partnerships with Community Associations (CAs) as a way to increase the quality of life for Calgarians and provide them with a means of formal representation and advocacy to The City. The City plays a role in contributing to their success, which is demonstrated through investment of land and resources. The Calgary Neighbourhoods (CN) Business Unit is responsible for providing a central line of support to CAs as well as performing critical risk assessment and risk mitigation work to protect The City’s interest. The audit objective was to assess the design of key controls in place to identify, assess, communicate and support timely mitigation of risks to CAs’ sustainability, including the reporting and escalation process.

We reviewed the design of controls based on the COSO Internal Control Framework related to CN’s Review Process to assess CA sustainability. We determined that the design of the process to identify CAs at risk and allocate resources is effective and includes the key components of an internal control system related to a CA’s control environment, control activities and risk assessments.

Annually, Community Services reports to Audit Committee and Council on the status of CAs operating on City owned land (Annual Status Report) and provides additional details and risk mitigation strategies for CAs that have a financial status of “Organization of Concern”.

The audit identified that the communication and monitoring components of the internal control system should be strengthened to support common understanding of CA challenges and better equip oversight bodies in creating

Adding Value: HR

Succession Planning

“The audit was helpful for us

in getting validation and

prioritization of this work, and

was helpful for us to move it

forward”. – Manager, Talent

Management, HR

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

policy and allocating resources. We raised two recommendations to improve the Annual Status Report which will direct attention to areas of high risk and provide oversight bodies with relevant information identified through the Review Process, including aging facilities, life-cycle costs and organizational health.

9-1-1 Call CentreCalgary 9-1-1 (C9-1-1) is the 9-1-1 call centre for The City, acting as the first point of contact for citizens in need of emergency assistance. C9-1-1 answers and evaluates 9-1-1 emergency and non-emergency calls, and dispatches the appropriate agencies to respond: Police, Fire or Emergency Medical Services (EMS). The objective of the audit was to evaluate the effectiveness of controls in place to support the achievement of call handling key performance indicators (KPIs). The audit assessed the design and operation of key controls to mitigate the risk of delays or problems in the call handling process for Police 9-1-1 calls up to the point the call was passed over for dispatch. We concluded that the design of C9-1-1’s key controls support the achievement of call handling KPIs and mitigate the risk of delays or problems in the call handling process. However, as call durations have increased, C9-1-1 has not met their KPI target for answering 95% of the Police 9-1-1 calls within 15 seconds in either 2015 or 2016 and will likely have similar challenges for 2017. Audit made four recommendations to further enhance the controls that facilitate the achievement of call handling KPIs to mitigate the risk of delays or problems in the call handling process.

Utilities and Environmental Protection

Landfill Disposal & Processing Services (DPS), a division within the Waste & Recycling Services (WRS) Business Unit, manages the operations of The City’s three active landfills. The objective of the audit was to evaluate the effectiveness of landfill processes and associated controls to ensure transactions are accurate, complete and monitored. DPS have implemented processes that are designed to ensure that vehicles are inspected, and their weight recorded with associated payment made as required. Key data is entered for each transaction, and processes have been established to process payments received and follow up on payments owed. However, our audit work identified that supporting IT systems and adherence to internal procedures required improvement to mitigate the risk of inaccurate or inappropriate transactions. We also identified opportunities to improve the efficiency and effectiveness of the operation of landfill processes. Seven recommendations were raised to further enhance DPS’s operations to maximize the efficiency and effectiveness of landfill processes.

Adding Value: Landfill

“I was very impressed with [the audit

team’s] capabilities and willingness to

understand our business.” – Leader,

Operational Performance, Waste and

Recycling

Adding Value: 9-1-1 Call

Centre

“It was a pleasure working

with all members of the Audit

team. They were

professional, prepared and

helpful throughout the

process.” – Commander,

Calgary 9-1-1

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Utility Billing The Water Utility (the Business Units of Water Services and Water Resources) is responsible for managing the quality and delivery of Calgary’s water supply. Responsibility for billing water consumers has been contracted to ENMAX, although The City has overall accountability for complete and accurate water billing. The objective of this audit was to provide assurance on the completeness and accuracy of the utility billing process. The audit determined the Water Utility’s current controls and processes provide only partial effectiveness in supporting the business objective that the water billing conducted by ENMAX is complete and accurate. Controls conducted by the Finance Department, which include review of daily and monthly water revenue and annual rate change process, are designed and operating effectively. However, these process are not designed to ensure accuracy or completeness of billing on an individual customer accounts basis. Over the past two years, the Water Utility has developed new technological tools and processes relating to billing that provide improvements, however further enhancements and additional controls to increase billing confidence on an individual customer accounts basis were recommended by our audit. We raised three recommendations to support the Water Utility in their accountability to Calgarians and provide The City with assurance that water billing is accurate and complete.

Other Subsidiaries and Partners

New Central Library The New Central Library (NCL) project is a significant build with a planned budget of $245M and expected completion by Q4 2018. The objective of this audit was to provide timely assurance that the NCL project is on track to meet business objectives of time, cost and quality. The project management team have designed and implemented project controls to effectively support project objectives of completing the project within the approved budget, meeting approved quality requirements, and identifying and responding to risks. Schedule management represents the current highest uncertainty to the achievement of the project’s objectives. Quality inspections identified material and fabrication defects. The subsequent impact and resolution assessment has delayed the project’s estimated date for obtaining the occupancy permit. The project management team deliver project status reports to the project’s Steering Committee monthly. However, project status reports do not identify the status of the project’s activities relative to the project’s master schedule. We raised two recommendations to mitigate the risk to the project schedule.

Planning and Development

POSSE The Public One Stop Service (POSSE) system is a business process management tool, used predominately for land management. It is The City’s definitive source of parcel data. The audit objective was to provide assurance on the data integrity and sustainability of the POSSE system.

Data integrity testing focused on controls to ensure the accuracy and completeness of key elements of parcel data and controls over the accuracy of address, licensing and permit data transferred to the assessment Business Unit to support property tax assessments. Based on our testing, the majority of controls to ensure the integrity of parcel data were effective and sample testing of parcel data did not identify any errors. However, responsibility for resolving ownership data exceptions identified during data transfer was assigned to a single IT resource, rather than a business user. The audit raised two recommendations to improve the integrity of POSSE ownership data and to decrease reliance on a single IT resource.

Sustainability testing focused on controls to ensure the ongoing ability to support POSSE including interface and customization documentation, effective vendor management and system availability. Testing of sustainability controls identified areas in which further improvements should be made to enhance control effectiveness and improve overall process efficiency. Eight recommendations were raised that focused on improving interface documentation, formalizing processes to manage vendor performance and contract compliance, and improving the effectiveness of the existing Helpdesk, change management and interface failure monitoring processes.

Adding Value: New Central

Library

“After 2 audits on the Library,

I find that the observations

and recommendations have

been useful and improved

our performance to deliver

good results.” – Civic

Partnership Consultant,

Calgary Neighbourhoods

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Transportation Transit Fare Revenue Calgary Transit collects more than $180M in fare revenue annually. The objective of this audit was to assess the effectiveness of controls over the safe keeping of fare revenue. The audit focused on controls providing assurance over the completeness of fare revenue collected from vendor sales of tickets and passes, cash collected in Ticket Vending Machines (TVMs), and cash and single ticket fares collected on buses. The audit also assessed Calgary Transit’s monitoring of fare revenue received. While effective controls supported the collection of fare revenue from vendors, we raised one recommendation to further increase the efficiency of this process. We raised three recommendations to improve the effectiveness of cash collection and processing. Calgary Transit monitors revenue received daily, weekly and monthly. Ridership was estimated based on historical surveys, as well as revenue information. Enhanced information started to become available during 2017 as automated counters were introduced to selected buses and CTrain cars. We raised one recommendation supporting enhanced ridership monitoring.

1.3 Advisory Services

The CAO provides advisory services on an ad hoc or project basis as requested by Administration. The intent of our advisory services is to provide an independent view and best practice insight on current, new or emerging risks and opportunities facing The City. During 2017, the CAO provided these services to a number of areas including:

The City’s Infrastructure Calgary Steering Committee as an advisory member;

The City’s Corporate Project Management Framework Steering Committee as an advisory member;

A City Business Unit by providing advice on draft delegations of authority; and

A City Business Unit by providing advice on internal controls to mitigate the risk of conflict of interest.

As an independent group, without affiliation to a particular Business Unit or Directorate, The CAO is uniquely positioned to provide value add advice. We do this by combining our knowledge of best practice on risks, controls and governance frameworks along with our deep understanding of City strategies, culture and organization to provide practical and cost effective advisory services.

Adding Value: Transit

Fare Revenue

“I appreciate the

collaborative approach to

the recommendations.

Our business is better off

for the work that audit

has done.” – Director,

Calgary Transit

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

17

2

31 17 City Auditor's Office

2 Triage Partners(HR/Corporate Security)

3 On Hold

1 Management

10

28

10 Substantiated

28 Unsubstantiated

1.4 Investigative Services

During 2017, the Whistle-blower Program (WBP), which is independently operated through the CAO, received Whistle-blower reports from employees and Calgarians pertaining to concerns regarding City employees and/or operations at a volume level consistent with prior years. Report activity is positively regarded as an indication that awareness of the WBP, and employee confidence to report concerns, is widespread across the organization.

Procedural enhancements and efficiencies applied in 2016 continued to support more timely assessment and response to reported concerns during 2017, resulting in:

81% decrease in outstanding WBP files from prioryears

38% decrease in open files carried forward to2017 compared to prior year

2017 also presented the WBP an opportunity to reflect on ten years of operation since its implementation by Council policy. This retrospective look of the program’s operations and activities confirmed that:

The WBP is operating effectively Key phases/outputs of the WBP process are

aligned and trending with recognized best practices The WBP provides added value to The City and to Calgarians

As illustrated below, WBP activity during the 10-year period ending June 30, 2017 has resulted in:

Whistle-blower Program Activity (2017)

New Reports

83

In-Progress Investigations (at Dec. 31)

23

Closed Investigations

38

6

41

36

6 Undetermined

41 Employee

36 Non-employee

•707 reports received

•885 allegations raised

Intake and Assessment

•330 reports approved for further investigation (47%)

•549 allegations investigated (62%)

Referral or Investigation •23.4% of allegations

investigated and determined to be substantiated

Conclusion and Reporting

•250 opportunities for improvement and/or corrective action identified in 321 investigations concluded

Recommendations and Closing

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Some key highlights of the 10-year review include:

Effective Decision Tree. Utilization of a comprehensive Decision Tree is an instrumental reference tool in the WBP assessment process, driving a consistently applied objective approach to each concern reported and ensuring prompt attention on priority concerns.

Strong Employee Utilization. Over the last 10 years employees reported the majority of concerns received. This is a positive trend indicating a speak-up culture exists, confidence in our safe reporting channel, and is a reflection of City values. Further, we noted that reports received from employees were assessed and approved for further investigation at a higher percentage and were more frequently substantiated. This result aligns with NAVEX

Global1 data and supports that employees with greater working knowledge of the WBP, complemented by training

and awareness of The City’s Code of Conduct, are more likely to report concerns associated with suspected acts of waste and/or wrongdoing.

Source # of Reports % Investigated Substantiation Rate (concluded investigations)

Corrective Actions (concluded investigations)

Employee 364 (51.5%) 57.4% (209 of 364) 30.6% (64) 166 (66.4%)

Non-employee 277 (39.2%) 33.6% (93 of 277) 22.6% (21) 67 (26.8%)

Undetermined 66 (9.3%) 42.4% (28 of 66) 21.4% (6) 17 (6.8%)

Trending Categorization. The categorization and classification of each concern reported to the WBP has provided the ability to identify and benchmark trends to issues raised, and their origins, allowing Administration to focus on opportunities for corrective action. As reported to Audit Committee in July 2017, the WBP utilized NAVEX Global data as a benchmark to compare its categorization, which identified strong alignment to the top reporting categories HR, Diversity and respectful Workplace; Business Integrity; and Misuse, Misappropriation of Assets. This alignment reinforces the value and effectiveness of the Whistle-blower Policy encouraging and supporting employees to safely report a broad range of wrongdoing concerns.

Dedicated Investigators. Originally supported by audit staff, WBP resources now include dedicated investigators who support the WBP with diverse professional accreditation and broad work experience attained from a variety of investigative roles within private and public environments. This work experience includes nearly 30 years directly related to managing and/or working within confidential and anonymous reporting programs. Dedicated

resources enable the WBP to conduct investigations with less reliance on Administration resources, and ensures standardization and consistency in investigation approach and reporting.

Improvements and rigour applied to the WBP process and approach to investigations over the last 10 years have been instrumental in improving case closure timelines, as illustrated.

In addition to the operational efficiencies and improvements applied during the past 10 years, the WBP has also enhanced its methods of communicating WBP activity to its various stakeholders: a number of informative messages are provided in communications with program users throughout the processing of a reported concern;

Calgarians are provided with procedural information via the WBP website; and Administration and members of

1 The NAVEX Global 2017 Ethics & Compliance Hotline & Incident Management Benchmarking Report comprises data (2012-2016) from over 927,000

individual hotline reports disclosed by more than 2,000 organizations representing 26 industries and 38.5 million employees globally.

075

150225300375450525600675750

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Average Days to Close

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Audit Committee are updated through a number of communication and reporting channels. These communications serve to increasing awareness of the WBP process and what to expect when a Whistle-blower report is submitted. The sharing of process and aggregate data does not compromise confidentiality and is regarded as a positive approach to building greater understanding of the WBP and value provided.

Improved Internal Communication. Communication on WBP activities includes quarterly meetings with General Managers and City Manager (through the Whistle-blower Oversight Group), which supports timely and appropriate discussions on results of investigations, corrective actions, trends and related analysis of WBP activity. When appropriate, ad-hoc meetings are held to discuss specific concerns.

Transparent Public Reporting. Reporting on WBP activity has evolved and matured from a single annual report of statistics and summaries of substantiated investigations to quarterly reporting statistical activity to Audit Committee, consolidated quarterly corrective action reporting on our website, as well as a comprehensive summary provided as a key component of the CAO Annual Report. Additional information regarding recommendations can be found at Section 2.2 below.

Submitting a Whistle-blower Concern?

Where possible:

Verify that your allegation is related to waste and/or wrongdoing as defined in the Whistle-blower Policy. If uncertain, contact the WBP and speak with an investigator;

Ensure that your allegation is clearly communicated;

Verify that your concern is related to a City employee or operation;

Provide specific and factual detail of the event including dates, times, locations, people involved;

Provide available supporting documentation and other evidence;

Identify individual(s) and/or Business Unit implicated in your allegation;

Avoid reporting concerns based on hearsay, speculation, opinions or conclusions;

Stay involved. All concerns reported online allow for ongoing anonymous and confidential communication with WBP investigators to whom additional clarification and information can be provided; and

New in 2017: When reporting online through the independently operated reporting tool, consider selecting the option to leave your email in order to receive timely activity notifications related to your report. Information provided with this option will not be shared with the WBP.

www.calgary.ca/whistle

Item #6.2

http://www.calgary.ca/whistle

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

2.1 Audit Recommendations The CAO takes a risk-based approach to the planning and execution of audits. Each audit focuses on key risks to the achievement of Administration’s objectives, which supports meeting Council Priorities. During planning, we work with staff to gain a thorough understanding of the area, project or process being audited and key risks. With Administration’s input, risks are ranked high, medium or low based on the impact and likelihood should the risk event occur. The fieldwork plan is designed to test successful risk mitigation.

In 2017, the majority of action plans (69 in total) to address recommendations raised (48 in total) related to high and medium risks, which supports our approach and demonstrates that the audits have an impact.

Through our expanded use of software we began tracking the level of risk by recommendation raised. By providing levels of prioritization, Administration is able to identify recommendations that require prompt action.

2.0

Our Impact

26

37

6

Action Plans by Risk Level

High

Medium

Low

3

3

2

6

4

887

5 5

3

9

4

2

0

1

2

3

4

5

6

7

8

9

10

Chief FinancialOfficer's

Department

CommunityServices

Deputy CityManager'

Office

Law &LegislativeServices

Planning &Development

Transportation Utilities &Environmental

Protection

2017 Action Plan Risk Level By Department

High

Medium

Low

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

The CAO provides independent assurance regarding the effectiveness of governance, risk management, and internal control. The COSO Internal Control Framework is a widely accepted framework that outlines the principles and components necessary for an organization to effectively manage its risks by implementing internal controls. The CAO categorizes recommendations into the five fundamental COSO components to identify potential trends and provide Administration with additional insight into the effectiveness of internal controls. Over the last three years the CAO has consistently raised recommendations focused on enhancing operating control effectiveness.

5%

11%

50%

9%

25%

16%

14%

30%

13%

27%

19%

20%

55%

4%

1%

0% 10% 20% 30% 40% 50% 60%

Monitoring

Information & Communication

Control Activities

Risk Assessment

Control Environment

Monitoring


Control Activities

Risk Assessment

Control Environment

Monitoring


Control Activities

Risk Assessment

Control Environment

201

52

01

62

01

7

Recommendation by COSO Element

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

2.2 Investigation Recommendations The WBP investigative process has been designed to look beyond the specific action under investigation, with a focus to identify the root cause for the concern raised. Recommendations for improvement are not limited to correcting substantiated allegations and can be localized to a specific work area or more broadly applicable to the entire organization. Identifying root causes and remediation of identified deficiencies is widely recognized as essential to operating an effective employee reporting program, and is aligned with policy. This approach applied to each allegation investigated during the 10-year period ending June 30, 2017 has identified opportunities for improvement or corrective action in 77.9% of all concluded investigations. In 2017, a formalized process, scheduled to begin in 2018, was added to follow-up with Administration to confirm implementation of corrective actions self-identified by Management or recommended by the WBP directly resulting from investigation of allegations raised to the WBP. During 2017, the more prevalent root causes identified by investigation were associated with reinforcement of policies and procedures related to theft of time, acceptable use of City technology resources, and inefficient use, or misuse, of City resources, representing a different series of issues than those most prevalent in 2016, as shown below.

Reporting Category Classification 2017 2016

Misuse, Misappropriation of Assets Theft of Time 6 -

Misuse, Misappropriation of Assets Acceptable Use of City Technology Resources 6 -

Misuse, Misappropriation of Assets Inefficient Use, or Misuse of City Resources 3 1

Business Integrity Conflict of Interest 2 4

HR, Diversity and Respectful Workplace Recruiting & Employment - 6

HR, Diversity and Respectful Workplace Respectful Workplace - 9

Other Other 5 8

22 28

With the support of the City Manager and beginning in 2017, a summary of each investigation resulting in corrective action is now posted to the WBP website (www.calgary.ca/whistle) on a quarterly basis. Summary information provided excludes personal or identifying information in support of the WBP’s adherence to WB protection and confidentiality. The publishing of allegations and the corresponding investigative findings and corrective actions is a positive practice in support of transparency of investigation actions, accountability and commitment for appropriate response and action. Each summarized corrective action provided on the WBP website may incorporate multiple specific actions.

Item #6.2

http://www.calgary.ca/whistle

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

3.1 Key Performance Indicators As part of our commitment to continuous improvement, we track our added value based on the following key performance indicators of efficiency, effectiveness, quality delivery, and staff proficiency. We are pleased to report that overall we exceeded four targets set for 2017.

3.0

Our

Commitment

to Value Add

Measure Area

Performance Indicator

Target 2017 2016 Comments

Efficiency

On Track to Annual Plan

100% 92% 98% Slightly below target reflecting vacant auditor roles during Q3 and Q4, partially alleviated by contract audit support.

Efficiency

Project Budget Variance

+/-10% -17% -1% Decrease in average hours reflects CAO’s ability to adapt and narrow the scope of audits to provide focused assurance.

Effectiveness

Recommendation Agreement

95% 100% 98% All recommendations were agreed to which reflects knowledge and buy-in on risk mitigation strategy.

Effectiveness

Timely Implementation of Recommendations

N/A 61% 71% Results may be attributed to increased demand on resources and/or optimistic implementation dates.

Quality

Client Satisfaction

75% 98% 94% Eleven client surveys were received during 2017 covering ten audits with a response rate of 70%.

Staff

Training Plan Achieved

80% 98% 93% Professional training included a cost effective mix of internal/external and self-study activities.

Staff

Average Years of Service

3.50 4.26 4.13 An investigator and a data analyst joined the team in Q2. Two auditors left the team in Q2/Q3, another auditor joined the team in Q4.

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

3.2 Professional Work Standards Professional work standards across all audit, advisory and Whistle-blower investigations are key to adding value in day to day CAO work. The CAO conducts its audit activities in adherence with the Institute of Internal Auditors’ (IIA) International Standards for the Professional Practice of Internal Auditing (Standards). The Standards require that an internal quality program is established and maintained to monitor adherence to Standards, and that an external quality assessment be conducted at least every five years. An external assessment occurred in March, 2017, by the Institute of Internal Auditors (IIA) to provide independent assurance to Council and Administration on the professional practice and quality of the CAO. The CAO received an overall opinion of generally conforms (the highest possible rating) to Standards. Our internal quality program was conducted throughout the year, and included the completion of:

Quarterly audit file peer reviews (by an auditor not involved in the audit);

Quarterly KPI monitoring;

Post-audit lessons learned exercises and client surveys; and

Periodic review and update of key audit processes, which in 2017 included updates to internal templates to continually improve efficiencies and effectiveness of audit processes.

The internal quality activity did not identify any instances of non-conformance to Standards, and any identified potential process improvements have been incorporated into on-going updates of procedures and practices. The activity also allows the CAO to confirm the organizational independence of its operation.

Whistle-blower investigations are carried out in alignment with best practices and the codes of conduct of the Association of Certified Fraud Examiners and Association of Certified Forensic Investigators. Quality reviews are conducted on all completed investigations. The foundation of the CAO is the professional skills and knowledge of the staff. To run effective audits, advisory projects and investigations, a range of complementary designations enhances the team’s effectiveness. All staff conducting audits, advisory and Whistle-blower investigations have at least one (or are studying towards gaining) of the Certified Internal Auditor, Certified Information Systems Auditor, Certified Fraud Examiner or Certified Forensic Investigator designations. To further enhance certifications and to keep current on best practices, all staff participate in on-going professional training. The CAO supports 40 hours of training per year, however many staff engage in additional continuous learning on their own personal time. Throughout 2017, certain staff members from the CAO have also contributed to their external peer community in the following ways:

Member of the Canadian national board of the Institute of Internal Auditors;

Sub-committee Chair of the Association of Local Government Auditors; and

Member of the Association of Local Government Auditors’ judging panel for the annual Knighton Awards (exceptional performance audit reports).

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

3.3 Budget

The CAO strives to provide the highest level of independent and objective assurance, advisory and investigative services within Council–approved budget. Our approved 2017 annual budget includes costs associated with completing audit, advisory and investigative services.

Operating Budget The CAO maintains funding within its budget to enable the office to hire subject matter experts to evaluate specialized risk areas or provide specific knowledge. During 2017 the CAO utilized contract audit resources to assist with the Transit Fare Revenue, Green Line LRT and Treasury Management audits.

($’000’s) Annual Budget Actual Variance*

Salary 2,619 2,304 315

Contracts 135 201 (66)

Training 65 71 (6)

Other 131 114 17

Total 2,950 2,690 260

* Variance due to staff vacancies and delays in filling vacant Whistle-blower, Data Analytics and Senior Auditor positions.

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

4.0 Appendices

Appendix A – Audit Activity Status as at December 31, 2017

2017/2018 Approved Audit Plan

# 2016 Carry Forward Audits Status

1 Landfill An operational audit assessing the effectiveness of processes established to meet business objectives.

Complete: Reported March

2 POSSE System An IT audit focusing on the data integrity and sustainability of the business application.

Complete: Reported March

3 New Central Library An operational audit to provide assurance the project is on track and will meet business objectives of time, cost and quality.

Complete: Reported January

4 Community Associations An operational audit on Calgary Neighbourhoods’ support of Community Associations.

Complete: Reported June

5 Human Resources – Succession Planning

An operational audit assessing the effectiveness of succession planning strategies conducted across the organization.

Complete: Reported June

# 2017 Audits Status

1 Transit Fare Revenue An operational audit assessing the effectiveness of controls over the safe keeping of fare revenue.

Complete: Reported April

2 IT Follow-up A follow-up audit focused on management actions in response to previous CAO audit recommendations raised over the last 5 years.

Complete: Reported July

3 911 Call Centre An operational audit evaluating the efficiency of tools and resources employed in the emergency call handling processes.

Complete: Reported

September

4 Green Line LRT An operational audit on the effective utilization of citizen engagement to support the objectives of the capital project. This is the first in a series of audits on Green Line LRT to be conducted over the lifespan of the project.

Reporting

5 Utility Billing A follow-up audit focused on management actions in response to control improvement recommendations raised in a 2012 CAO advisory activity.

Complete: Reported

September

6 Corporate Facilities/Asset Management

An operational audit which continues an original CAO audit conducted in 2013.

Complete: Reported December

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

2017/2018 Approved Audit Plan

# 2017 Audits Status

7 Treasury Management An operational audit of treasury (cash flow) management.

Fieldwork

8 Procurement A follow-up audit which will focus on management actions to address recommendations raised in previous CAO audits (from 2009 to current).

Reporting

9 Cyber Security Incident Response

An IT audit assessing the effectiveness of response processes established to support and protect critical data from cyber-attacks.

Fieldwork

10 New Central Library Project An operational audit on the readiness of Calgary Public Library and The City of Calgary to assume hand-off from the Calgary Municipal Land Corporation of the New Calgary Central Library.

Reporting

11 Corporate Credit Card (Data Analytics)

A compliance audit utilizing data analytics to assess the effectiveness of related Corporate Credit Card compliance and fraud prevention controls.

Reporting

# 2018 Audits Initiated in 2017

1 Employee Expenses A compliance audit of employee expenses utilizing data analytics

Fieldwork

2 2017 Election Day A management request (City Clerk’s Office) to conduct a root cause analysis review of the issues which occurred on the 2017 election day and to evaluate proposed strategies to improve the election day process.

Planning

Item #6.2

of 21

ISC: UNRESTRICTED

AC2018-0019

Attachment

Appendix B – Audit Recommendation Follow-up

There were 62 outstanding recommendation action plans at 2017 year end. Of these, 74% were not yet due and classified as pending audit review, 26% were in-progress and are being tracked to a revised implementation date.

2017 Recommendation Action Plan Turnover

Status Opening-

January 1, 2017 Revised Date

Required Reported in

2017 Closed-Risk

Mitigated Closed-Risk

Accepted

Ending December 31,

2017

Pending 61 (19) 69 (63) (2) 46

In-Progress 11 19 (13) (1) 16

Total 72 0 69 (76) (3) 62

Follow-up results continued to be positive this year. Of the 69 recommendation action plans reported in 2017 (56 in 2016), 23 (33%) were closed (29% in 2016), 6 (26%) of which were closed in advance of the implementation date in the audit report (20% in 2016). The remaining 46 were either pending (38) or in-progress (8) at year-end. Of particular note all of the action plans from the following 2017 audits were implemented in 2017:

Ten action plans from the POSSE audit (AC2017-0253); and

Both action plans from the New Central Library audit (AC2017-0054).

As well, all action plans from the Landfill (7) and HR Succession Planning (1) audits that were due for follow-up in 2017 were implemented. Additional results are included in the charts below: The overall number of overdue action plans has increased slightly from 11 to 16 in 2017. There were no action plans that were more than two years past their original commitment date. In 2017, we received 19 (14 in 2016) requests to revise action plan implementation dates, all of which were first time revisions. The 16 in-progress action plans at year end are all first time revisions.

0

5

10

15

20

25

< 1 Year > 1 Year > 2 Years

# o

f R

eco

mm

en

dati

on

s

Overdue Action Plans

2016 2017

0 1 2 3 4 5

AC2017-0341 Transportation

AC2017-0590 Chief Financial Officer's

AC2017-0590 Law & Legislative Services

AC2016-0754 Utilities & Environmental Protection



AC2015-0892 Law & Legislative Services


# of Recommendations

Outstanding Recommendations at Year-End # of Revised Date Requests

1st revised date

2nd revised date

3rd revised date

Item #6.2

city auditor’s office 2017 annual report

Documents