management of the system-wide data review … · management of the system‐wide data review...

www.audit.act.gov.au

ACT AUDITOR–GENER AL’S REPORT

MANAGEMENT OF THE SYSTEM-WIDE DATA REVIEW IMPLEMENTATION PROGRAM

REPORT NO. 5 / 2019

© Australian Capital Territory, Canberra 2019

ISSN 2204-700X (Print) ISSN 2204-7018 (Online)

This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without written permission from the Territory Records Office, Shared Services, Chief Minister, Treasury and Economic Development Directorate, ACT Government, GPO Box 158 Canberra City ACT 2601.

ACT Audit Office

The roles and responsibilities of the Auditor-General are set out in the Auditor-General Act 1996.

The Auditor-General is an Officer of the ACT Legislative Assembly.

The ACT Audit Office undertakes audits on financial statements of Government agencies, and the Territory’s consolidated financial statements.

The Office also conducts performance audits, to examine whether a Government agency is carrying out its activities effectively and efficiently and in compliance with relevant legislation.

The Office acts independently of the Government and reports the results of its audits directly to the ACT Legislative Assembly.

Accessibility Statement

The ACT Audit Office is committed to making its information accessible to as many people as possible. If you have difficulty reading a standard printed document, and would like to receive this publication in an alternative format, please telephone the Office on (02) 6207 0833.

If English is not your first language and you require the assistance of a Translating and Interpreting Service, please telephone Canberra Connect on 13 22 81.

If you are deaf or hearing impaired and require assistance, please telephone the National Relay Service on 13 36 77.

Audit TeamErika Hudleston Karen Ruse

The support of Rachel Gilbertson and David Kelly is appreciated.

Produced for the ACT Audit Office by Publishing Services, Shared Services, Chief Minister, Treasury and Economic Development Directorate, ACT Government

Publication No. 19/0622

ACT Government Homepage address is: http://www.act.gov.au

PA 19/01

The Speaker ACT Legislative Assembly Civic Square, London Circuit CANBERRA ACT 2601

Dear Madam Speaker

I am pleased to forward to you a Performance Audit Report titled ‘Management of the System‐Wide Data Review implementation program’ for tabling in the Legislative Assembly pursuant to Subsection 17(5) of the Auditor‐General Act 1996.

Yours sincerely

Mr Michael Harris Auditor‐General 30 May 2019

The ACT Audit Office acknowledges the Ngunnawal people as traditional custodians of the ACT and pays respect to the elders; past, present and future. The Office acknowledges and respects their continuing culture and the contribution they make to the life of this city and this region.

Appendix B: Team-based plan checklist

Page 108 Access Canberra business planning and monitoring

PA 18/03


Dear Madam Speaker

I am pleased to forward to you a Performance Audit Report titled ‘Access Canberra business planning and monitoring’ for tabling in the Legislative Assembly pursuant to Subsection 17(5) of the Auditor-General Act 1996.

Yours sincerely

Mr Michael Harris Auditor-General 23 April 2019

Appendix B: Additional Community Services Directorate response to draft proposed report

Page 86 Recognition and implementation of obligations under the Human Rights Act 2004

are not happy with something at Bimberi.

The Commissioners also visit Bimberi to look at documentation such as incident reports.

The ACT Human Rights Commission also has powers to undertake a Commission Initiated Consideration of matters related to the provision of services at Bimberi.

Inspector of Correctional Services

The Inspector of Correctional Services Act 2017 was established to enable the independent and transparent systematic review of correctional centres and correctional services. The Act is inclusive of operations at Bimberi Youth Justice Centre with commencement of the role of the Inspector to occur prior to the end of 2019.

PA 18/07


Dear Madam Speaker

I am pleased to forward to you a Performance Audit Report titled ‘Recognition and implementation of obligations under the Human Rights Act 2004’ for tabling in the ACT Legislative Assembly pursuant to Subsection 17(5) of the Auditor‐General Act 1996.

Yours sincerely

Auditor‐General 22 February 2019

The ACT Audit Office acknowledges the Ngunnawal people as traditional custodians of the ACT and pays respect to the elders; past, present and future. The Office acknowledges and respects their continuing culture and the contribution they make to the life of this city and this region.

Management of the System‐Wide Data Review implementation program Page i

CONTENTS

Summary ................................................................................................................................... 1

Conclusions ............................................................................................................................... 1

Key findings .............................................................................................................................. 2

Recommendations .................................................................................................................... 6

Agency response ....................................................................................................................... 7

1 Introduction ..................................................................................................................... 9

ACT Health data ........................................................................................................................ 9

ACT Health System‐Wide Data Review outcomes report ...................................................... 11

Program management ............................................................................................................ 14

Audit objective and scope ...................................................................................................... 15

Audit criteria, approach and method ..................................................................................... 16

2 Governance for the System‐Wide Data Review implementation program .......................19

Summary ................................................................................................................................. 19

Governance of the System‐Wide Data Review implementation program ............................. 21

Program management roles and responsibilities ................................................................... 31

Risk management ................................................................................................................... 35

3 Planning for the System‐Wide Data Review implementation Program .............................39

Summary ................................................................................................................................. 39

Program planning ................................................................................................................... 41

Program monitoring and reporting ........................................................................................ 46

Program financial planning and reporting .............................................................................. 51

Program review and evaluation ............................................................................................. 52

Management of the System‐Wide Data Review implementation program Page 1

SUMMARY

In the modern health care system, accurate health data is essential for the delivery of efficient and

high quality care to the ACT community and to help ensure that health funding is appropriately

allocated.

Between February 2017 and August 2018, the ACT Health Directorate undertook the System‐Wide

Data Review; a review commissioned by the ACT Minister for Health and Wellbeing to identify the

root causes of systemic issues of data accuracy and performance reporting in ACT Health

performance reports. On 21 August 2018 the Minister presented to the Legislative Assembly the

final report from the System‐Wide Data Review titled Leading Data Reform: The Way Forward –

Outcomes of the ACT Health System‐Wide Data Review (August 2018). The report, and a suite of

supporting documents, identified a program of activity to be implemented over a three‐year period

to address issues of data accuracy and performance reporting.

The ACT Health Directorate has embarked on phase two of the System‐Wide Data Review

implementation program, in the first year of the three‐year program. Conducting a performance

audit at this early stage of the implementation means that the audit’s findings and

recommendations may inform the ACT Health Directorate’s program management arrangements

for the future phases of the implementation and, at the same time, inform the Territory’s wider

community of the ACT Health Directorate’s approach to managing this important program.

The performance audit examines the effectiveness of the ACT Health Directorate’s program

management arrangements supporting the implementation of recommendations, actions and

initiatives identified in the ACT Health System‐Wide Data Review using two criteria:

Criterion 1: Are effective governance arrangements in place to provide oversight of the

implementation of the System‐Wide Data Review program of activities?

Criterion 2: Is there effective planning for the implementation of the System‐Wide Data

Review program of activities?

Conclusions

GOVERNANCE FOR THE SYSTEM‐WIDE DATA REVIEW IMPLEMENTATION PROGRAM

Governance arrangements to support the System‐Wide Data Review program are under

development and are not yet mature enough to effectively support its implementation.

There is no documented program governance plan and program roles and responsibilities are not

adequately defined. A series of committees and working groups that have been identified as

supporting the implementation of the System‐Wide Data Review are not yet effective; there is a

lack of clarity with respect to roles and purpose and the groups are not consistently meeting and

operating. Risk management arrangements to support the implementation of the program are in

the early stages of development.

Summary

Page 2 Management of the System‐Wide Data Review implementation program

The Data Repository Population Project, a major component of the System‐Wide Data Review

program, has well‐defined project management arrangements in place that provide transparency

and rigour to monitoring the progress and implementation of the new data repository. The

governance and management of the System‐Wide Data Review implementation program as a

whole could benefit from the adoption of similar arrangements.

PLANNING FOR THE SYSTEM‐WIDE DATA REVIEW IMPLEMENTATION PROGRAM

Planning for the implementation of the System‐Wide Data Review program is not effective. There

is neither transparency nor clarity about how and when the three‐year program will be delivered

and if the program will be completed as envisaged in the Leading Data Reform: The Way Forward

– Outcomes of the ACT Health System‐Wide Data Review.

The ACT Health Directorate has implemented a short‐term planning approach to the program (six

months at a time) and has resourced implementation activities as ‘business as usual’. There is little

information or specificity for program activities planned beyond June 2019 and inadequate

documentation of the critical path(s) of the program and its activities, including key dependencies

and milestones or when activities are estimated to be completed.

Processes for reporting on the progress of implementation are under development and do not yet

provide adequate information on activities’ estimated completion dates, key dependencies

between and across projects, the resources necessary to complete the program or the realisation

of benefits.

Key findings

GOVERNANCE FOR THE SYSTEM‐WIDE DATA REVIEW IMPLEMENTATION PROGRAM Paragraph

There is no program governance plan for the System‐Wide Data Review implementation program that identifies program governance structures, dependencies, assumptions and constraints. Nevertheless, the ACT Health Directorate developed a draft Data Management and Governance Framework as an early deliverable of the System‐Wide Data Review implementation program. The draft Data Management and Governance Framework does not identify the governance arrangements for the System‐Wide Data Review implementation program, but rather relates to the ACT Health Directorate’s broader data management and governance arrangements such as data management committees as well as roles and responsibilities for collecting, managing, maintaining, using and sharing data.

2.17

A series of data governance committees and working groups have been established to support data management practices across the ACT Health Directorate. The purpose and roles of these groups are outlined in the draft Data Governance

2.25

Summary


Committees; Supporting the Data Management and Governance Framework document. The System‐Wide Data Review Reference Group is the key committee with oversight responsibility of data governance and management in the ACT Health Directorate. The System‐Wide Data Review Reference Group is supported by the Data Standards Working Group and the Data Repository Working Group. The establishment of these groups is anticipated to improve data governance and management arrangements across the ACT Health Directorate.

There is a lack of clarity associated with the purpose of the System‐Wide Data Review Reference Group. Various documents, including the Phase One Implementation Plan and the draft Data Governance Committees document, describe an ongoing, longer‐term role for the System‐Wide Data Review Reference Group with respect to data governance and management. However, the terms of reference for the System‐Wide Data Review Reference Group, which were finalised in February 2019, identify that the tenure of the group is time‐limited, with a key focus on the System‐Wide Data Review program implementation. The System‐Wide Data Review Reference Group has met twice since its establishment in November 2018 (29 November 2018 and 28 February 2019), which means that, if its purpose is to focus on System‐Wide Data Review program implementation, then it has had limited opportunity to oversee the management and implementation of the program thus far.

2.38

Two working groups are intended to support the System‐Wide Data Review Reference Group; the Data Standards Working Group and the Data Repository Working Group. While the Data Repository Working Group is established and has met regularly since October 2018, the Data Standards Working Group is yet to meet and commence operating. The terms of reference for both groups remain in draft at the time of audit fieldwork in April 2019. While the Data Repository Working Group has had the opportunity to contribute to, and influence, the implementation of the System‐Wide Data Review program since October 2018, the Data Standards Working Group is yet to do so.

2.50

Two teams in the Commissioning and Performance Division’s Health Insights Branch are operationally responsible for the management of the System‐Wide Data Review implementation program; the Data Governance and Management team and the Data Development team. A Program Manager for System‐Wide Data Review program implementation has been employed since May 2018 (the most recent Program Manager commenced in the role in November 2018). The role and responsibilities of the Program Manager are not documented. Other roles and responsibilities for the implementation of the System‐Wide Data Review program are not adequately documented, although information on roles and responsibilities may be surmised from a range of other documents. The absence of clear and complete documentation of program management roles and responsibilities can lead to misunderstandings about responsibilities and inconsistencies in the way in which the roles are carried out.

2.58

Documentation and governance arrangements associated with the Data Repository Population Project demonstrate well‐established processes and a thorough and mature approach to project management. There is an opportunity for the ACT Health Directorate to recognise the maturity of the governance and management

2.64

Summary


arrangements associated with the Data Repository Population Project and apply these to the System‐Wide Data Review program more broadly, in order to bring improved transparency and rigour to the overall implementation program.

Risk management of the System‐Wide Data Review implementation was initially undertaken through a System‐Wide Data Review Implementation Plan (Phase 1) ‐ Risks Register (the Phase 1 Risk Register). The Phase 1 Risk Register was developed after the program of work commenced, in October 2018, and included six risks for 14 activities across four Domains. The risks were broad, generic and lacked underlying detail. In February 2019, a revised draft Risk Register – ACT Health System Wide Data Review Implementation Plan (revised draft Risk Register) was prepared. The revised draft risk register is more comprehensive and contains information on risk sources and consequences, risk treatments, risk owners and risk status. The revised draft risk register identifies ten risks related to 12 activities across four System‐Wide Data Review Domains. However, the revised draft risk register is not tailored as a strategic or program level risk assessment and does not identify risks associated with the management of the System‐Wide Data Review program as a whole.

2.74

PLANNING FOR THE SYSTEM‐WIDE DATA REVIEW IMPLEMENTATION PROGRAM Paragraph

The ACT Health Directorate has published the System‐Wide Data Review Implementation Plan, Phase One, to December 2018 (Phase One Implementation Plan) and the Phase Two Implementation Plan March 2019 System‐Wide Data Review (Phase Two Implementation Plan). These documents list a series of activities and outputs that the ACT Health Directorate planned to deliver ‘up to December 2018’ and ‘up to the first half of 2019’ respectively. The two implementation plans do not clearly explain what the outputs represent in terms of delivering the overall System‐Wide Data Review implementation program. There is no explanation about the ACT Health Directorate’s planning arrangements for delivering the outputs, such as the reasons for working on particular outputs in the current phase, what business areas are responsible for delivering the outputs and the resources needed to deliver the outputs. It is not clear from the Phase Two Implementation Plan how well the implementation program is progressing, if the implementation program is on schedule, whether there are any impacts to the broader implementation program associated with the Phase One outputs that are still underway and if there have been any benefits achieved from the outputs that have been completed.

3.22

The ACT Health Directorate has developed a Microsoft Excel workbook, the SWDR Program, to assist in internal planning for the System‐Wide Data Review Program. The SWDR Program comprises two spreadsheets; Program Overview and Program of Activities. The spreadsheets include information on 95 activities to be implemented as part of the System‐Wide Data Review implementation program; 30 activities are allocated to the System‐Wide Data Review, 38 activities allocated to Phase One, 11 activities to Phase Two and 16 activities to future phases. The spreadsheets contain high‐level information on the activities’ Project Leads, the activities’ key stakeholders, as well as information on the activities’ current status and the percentage of progress to completion. The spreadsheets do not include a timeline

3.27

Summary


beyond June 2019 (except for the designation of some activities to ‘Future Phases’), information on the critical path(s) of the overall System‐Wide Data Review program and its activities, key dependencies and milestones or when activities are estimated to be completed. By not planning for the longer term, i.e. beyond June 2019, the ACT Health Directorate increases the risks of a poorly executed System‐Wide Data Review implementation program and not realising the benefits anticipated from the three‐year program of work.

The ACT Health Directorate monitors and reports on the implementation of the System‐Wide Data Review program using Project Plan Briefs (a ‘project’ is made up of two or more implementation program activities) and Data Repository Population Project Status Reports. Reporting through Project Plan Briefs commenced in November 2018. In January 2019, there were eleven ‘projects’ reported in eleven corresponding Project Plan Briefs. Reporting through the Project Plan Briefs, including the quality and comprehensiveness of reporting, is under‐developed. Reporting on the implementation of the Data Repository Population Project is influenced by separate governance arrangements for the project and reflects more robust arrangements. The Data Repository Population Project Status Reports include information on the project’s:

tasks’ start and finish dates;

current status and progress to completion;

deliverables to be achieved; and

key open issues and risks.

3.40

The ACT Health Directorate records and tracks overall progress on the implementation of System‐Wide Data Review activities using the Program Overview and Program of Activities spreadsheets; the spreadsheets are populated using information from the Project Plan Briefs. The current approach integrates subjective estimates of activities’ completion obtained from individual Project Leads and then sums these and divides them by the total number of activities to provide a quantitative estimate of overall program progress. This approach to program monitoring treats all projects equally, and does not take into account differences in size, scope or resource demands of the projects. The ACT Health Directorate is not systematically recording at a program level:

activities’ estimated completion dates;

key dependencies between and across projects and the implications of slippage to Phase One activities’ progress; or

the resources necessary to complete the program.

3.49

The resources required for the System‐Wide Data Review implementation program do not have a separate, identifiable funding allocation and associated budget. The System‐Wide Data Review activities are funded from existing resources, that is from ‘business as usual’ (i.e. time, people, budget). Accordingly, the ACT Health Directorate does not routinely track implementation program costs against a program budget. By not planning for program costs and monitoring these against a program budget, the ACT Health Directorate cannot effectively manage future costs

3.55

Summary


or put in place mitigation strategies to manage resource risks that could affect the delivery of the implementation program and its anticipated benefits.

The ACT Health Directorate has not documented its procedures or processes for the review and evaluation of the System‐Wide Data Review implementation program in any program governance or program planning documents. While the ACT Health Directorate has identified the benefits from the System‐Wide Data Review implementation in Leading Data Reform: The Way Forward – Outcomes of the ACT Health System‐Wide Data Review (August 2018), Project Initiation Document Data Repository Population Project ‐ Phase 1, and the draft Performance, Reporting and Data Management Strategy 2019‐2021, benefits are not regularly recorded or monitored at a program level.

3.66

Recommendations

RECOMMENDATION 1 SYSTEM‐WIDE DATA REVIEW PROGRAM GOVERNANCE

The ACT Health Directorate should:

a) document the governance arrangements for the System‐Wide Data Review implementation

program, including roles and responsibilities for program management and oversight;

b) embed oversight and monitoring arrangements for the program, by finalising and endorsing

terms of reference for the System‐Wide Data Review Reference Group and supporting

Working Groups and ensuring these groups meet regularly to review implementation

progress; and

c) develop and implement risk management arrangements that assess and monitor risks to the

implementation of the program.

RECOMMENDATION 2 SYSTEM‐WIDE DATA REVIEW PROGRAM PLANNING


a) develop and implement program management planning practices that enables:

i) scheduling and monitoring of activities based on estimated completion dates;

ii) a consistent approach to prioritisation of activities and the progress made;

iii) identification of dependencies and key milestones;

iv) tracking and reporting of costs to inform planning for future phases; and

b) include in future published System‐Wide Data Review Implementation Plans information

about the ACT Health Directorate’s program planning arrangements, overall implementation

progress and the realisation of benefits.

Summary


Agency response

In accordance with the subsection 18(2) of the Auditor‐General Act 1996, the ACT Health

Directorate was provided with:

a draft proposed report for comment. All comments are considered and required

changes reflected in the final proposed report; and

a final proposed report for further comment.

The ACT Health Directorate provided comments for inclusion in this Summary Chapter.

ACT Health Directorate response

The availability of accurate data is essential to ensure the efficient delivery of high‐quality health

services, both at the operational and system‐wide levels. ACT Health is committed to continuous

improvement of data management, reporting and analysis and to the implementation of the

program of work arising from the System‐Wide Data Review.

ACT Health found this to be a highly engaging, constructive and timely performance audit process,

and welcomes the findings outlined in this report. The audit was conducted at an early stage in the

implementation of the System‐Wide Data Review recommendations, and the findings of the Audit

Office will add significant value to the management of the program of work moving forward.


1 INTRODUCTION

ACT Health data

1.1 The ACT Health Directorate collects and uses a variety of data sourced from ACT Health

systems to:

assist its internal processes for planning and prioritising resources;

fulfil performance reporting obligations that form a part of the Commonwealth

government’s activity based funding arrangements with the states and territories; and

inform the ACT public and health consumers about the performance and state of the

ACT Health system and its services.

1.2 The importance of accurate data for the ACT Health Directorate’s strategic planning and the

delivery of health services was identified by the ACT Minister for Health and Wellbeing in

May 2018:

… Data is a vital resource for managing territory‐wide services and planning and making sure the ACT’s health service performance is continually improving. Our community expects that ACT Health have good data collection and management practices, and Canberrans can have confidence that the results of the system‐wide data review will achieve that.

… Quality and accessible data is absolutely vital for making sure health consumers get the right care they need in the right setting.

1.3 Issues with the accuracy of ACT Health data and performance reporting were identified on

several occasions throughout 2015, 2016 and into early 2017. These issues were described

in previous ACT Audit Office reports including Integrity of Data in the Health Directorate

(Report No 5/2015) and, more recently, ACT Health’s management of allegations of

misconduct and complaints about inappropriate workplace behaviour (Report No 9/2018).

System‐Wide Data Review of ACT Health data

1.4 The ACT Minister for Health and Wellbeing announced the commencement of the

System‐Wide Data Review of ACT Health Data and Reporting (the System‐Wide Data Review)

on 14 February 2017. The review was commissioned by the Minister to identify the root

causes of systemic issues of data accuracy and performance reporting in ACT Health

performance reports.

1: Introduction


1.5 According to the terms of reference for the System‐Wide Data Review, the purpose of the

review was as follows:

The system‐wide review (the Review) will be completed by 31 March 2018 and undertaken in a timely, transparent and effective manner and shall:

1. investigate the extent, and where possible, the root cause of the current data issues

2. establish revised governance processes and protocols for data management, reportingand analysis

3. develop a framework for the provision of essential data reports derived directly fromsource systems as an interim process and re‐building of the ACT data warehouse

4. implement the framework outcomes

5. provide a detailed road‐map to address existing recommendations from the Auditor‐General and ACT Health external advisers

6. provide advice on the publication of data for consumers that facilitates improvedunderstanding of ACT Health information, performance, quality and safety, includingoptions for real‐time provision of information.

Restructuring of ACT Health

1.6 On 23 March 2018, the Minister for Health and Wellbeing and Minister for Mental Health

jointly announced a new organisational structure for ACT Health, separating operational

health services from policy and planning functions:

From 1 October 2018, ACT Health will be separated into two organisations, focussed on:

‐ the delivery of quality health services to our growing community, and

‐ the strategic policy and planning stewardship of the health system.

This will bring the ACT into line with every other Australian jurisdiction.

…

The health service delivery organisation will focus on delivering person‐centred care, while the health policy and planning organisation will oversee the operation of the health system and set its direction, as outlined by the ACT Government, specifically the Minister for Health and Wellbeing and the Minister for Mental Health.

…

Two new executive positions will be created to lead the new organisations and the current Director‐General position will no longer exist from 1 October 2018.

1.7 On 1 October 2018, ACT Health became two distinct organisations:

Canberra Health Services ‐ responsible for ‘the delivery of quality, safe and consistent

public health services to Canberrans’; and

the ACT Health Directorate ‐ responsible for ‘providing strategic policy advice to

Ministers, development plans for the future health needs of the community,

developing plans for the prevention of disease, modelling demand for and supply of

health services and leading workforce and clinical training strategies’.

1: Introduction


ACT Health System‐Wide Data Review outcomes report

1.8 On 21 August 2018, the Minister for Health and Wellbeing presented to the Legislative

Assembly the final report from the System‐Wide Data Review titled Leading Data Reform:

The Way Forward – Outcomes of the ACT Health System‐Wide Data Review (August 2018).

The report presented nine key themes (named Domains) that the review focused on,

together with key findings, recommendations and a program of activities. A description of

each Domain and the recommendations are summarised in Table 1‐1.

Table 1‐1 System‐Wide Data Review Domains and recommendations

Domain Domain Description Recommendation

D1 – Data management

The design and architecture of the systems and processes, which store and prepare ACT Health data for analysis and reporting.

Build a new data repository, which will collect, store, extract and transform quality data to deliver better insights to the community.

D2 – Data governance

The structures and rules in place to provide direction, oversight and accountability to the use of ACT Health data and the reporting of information.

Embed new governance structures to delineate roles and responsibilities in relation to data across ACT Health, ensuring accountability and transparency is a priority.

D3 – Data quality The accuracy, reliability, validity and timeliness of ACT Health data

Continually improve accuracy of data through robust data quality assurance activities.

D4 – Metadata management

Documenting ACT Health data so that there is a common understanding of how it is defined and how it should be interpreted.

Improve the understanding of how data is defined and how it should be interpreted through the documentation of data definitions, data models and data flows.

D5 – Data security and privacy

The secure storage and access to sensitive information, as well as the measures put in place to ensure that the information is maintained in accordance with all privacy requirements.

Maintain security and privacy of the data held by the ACT Health.

D6 – Workforce Organising teams working with data and enabling successful data management practices through training and communication.

Continually improve internal workforce capabilities around data to build a strong data and analytics team, and train the health workforce to leverage the use of data for decision‐making.

1: Introduction


Domain Domain Description Recommendation

D7 – Communication

The channels through which ACT Health staff can get information and assistance.

Improve communication about data across ACT Health, mobilising a cultural shift around data management, with the intent of creating a new relationship between data, the community and stakeholders.

D8 – Change management

The rules, procedures and policies put in place to ensure that changes to ACT Health data or reporting of information are communicated, approved and implemented in a consistent and structured manner.

Embed change management practices as business needs evolve and ensure these enhancements are appropriately governed.

D9 – Information and insights

The wide range of reporting and analytics capabilities for ACT Health.

It includes mandatory reporting to external bodies, as well as the delivery of insightful analytics and reporting capable of driving strategic and operational decision‐making.

Be transparent within and outside ACT Health, by meeting regulatory requirements as well as deliver insights and intelligence about the healthcare services delivered throughout ACT.

Source: ACT Audit Office, based on Leading Data Reform: the Way forward ‐ Outcomes of System‐Wide Data Review (August 2018)

1.9 Several supporting documents were also published with the final report, and made available

on the ACT Health Directorate’s website:

ACT Government Response to Leading Data Reform – The Way Forward: Outcomes of

the ACT Health System‐Wide Data Review;

Implementation Plan, Phase one, to December 2018 System‐Wide Data Review;

Terms of Reference of the System‐Wide Data Review; Leading Data Reform: The Way

Forward – Outcomes of the ACT Health System‐Wide Data Review;

Recommendations from External Reviews; Leading Data Reform: The Way Forward –

Outcomes of the ACT Health System‐Wide Data Review;

Essential and Priority External Reports; Leading Data Reform: The Way Forward –

Outcomes of the ACT Health System‐Wide Data Review;

Root Cause Analysis; Leading Data Reform: The Way Forward – Outcomes of the ACT

Health System‐Wide Data Review; and

State of consumer data during the System‐Wide Data Review.

1: Introduction


ACT Government’s response

1.10 The ACT Government Response to Leading Data Reform – The Way Forward: Outcomes of

the ACT Health System‐Wide Data Review stated:

To ensure that the outcomes of the System‐Wide Data Review are delivered and planned in a systematic way, ACT Health has developed a comprehensive Implementation Plan. The government acknowledges there will continue to be a rollout of activity over the three year reform roadmap, with regular updating of progress and implementation activities over this period, providing opportunities for ongoing learning and refinement.

System‐Wide Data Review implementation program

1.11 The final report from the System‐Wide Data Review, Leading Data Reform: The Way

Forward ‐ Outcomes of the ACT Health System‐Wide Data Review (August 2018), refers to

‘significant program planning to address the terms of reference and the 175

recommendations from external reviews’. It also refers to the:

Development of a program of activities for each domain that describes the end‐to‐end solution to addressing the terms of reference, the recommendations from the external reviews and additional activities required to move ACT Health towards innovation and best practice. This approach provided for a consistent methodology to develop a program of activities ensuring completeness and transparency so that all inputs were addressed. (Emphasis added)

1.12 The Implementation Plan, Phase One, to December 2018 System‐Wide Data Review (Phase

One Implementation Plan) also describes a program of activities to be implemented:

The Review identified a program of activities grouped by nine informatics domains, representing an end‐to‐end solution that addressed the recommendations arising from the Review.

...

This Implementation Plan represents Phase One of an ongoing program of work over three years. The projects identified in this Plan have been informed by the program of activities identified through the Review, and are scheduled to commence in the period to December 2018. ACT Health will publish updates to the Plan every six months, which will include progress updates on projects underway or completed, and will identify projects scheduled to commence in the coming six‐month period. (Emphasis added)

1.13 The Phase One Implementation Plan also notes ‘ACT Health will continue this significant

program of work to realise and address the full findings of the Review’.

1.14 The ACT Health Directorate has described the System–Wide Data Review implementation

as a program of activities, including in the Leading Data Reform: Outcomes of the ACT Health

System‐Wide Data Review (August 2018) and the System‐Wide Data Review

Implementation Plan, Phase One, to December 2018. This creates an expectation that the

ACT Health Directorate has in place the processes and documents that characterise the

discipline of program management.

1: Introduction


Program management

1.15 For the purpose of this audit, the definition of a program is that contained in The Standard

for Program Management Fourth Edition (2017) (the Standard). The Standard is produced

by the Project Management Institute, a not‐for‐profit, United States‐based professional

membership association that produces, amongst other things, globally recognised

standards. The Standard provides information on the principles of program management:

It provides generally accepted definitions of programs and program management and concepts important to their success – program management performance domains, the program life cycle, and important program management principles, practices, and activities.

1.16 The Standard defines a program as:

Related projects, subsidiary programs, and program activities managed in a coordinated manner to obtain benefits not available from managing them individually.

1.17 In relation to the purpose and benefits of program management, the Standard states:

Organizations employ program management to improve their abilities to deliver benefits. In noncommercial organizations, benefits may be delivered in the form of social or societal value (for example, improved health, safety or security). …

Program management enables organizations to more effectively pursue their strategic goals through the coordinated pursuit of projects, subsidiary programs, and other program‐related activities. Program management seeks to optimize the management of related component projects and programs to improve the generation of business value.

1.18 There are many potential benefits from implementing an effective program management

approach, including:

the development and implementation of consistent practices, defined roles and

responsibilities and authority across the entire program;

coordinating, scheduling, prioritising and sharing of resources across the program;

and

the potential to prevent a narrow activities based approach that could lead to historic

errors and problems being repeated.

Reasons for conducting the performance audit

1.19 The System‐Wide Data Review implementation program has a three‐year timeframe (2018

to 2021). Managing the implementation of the outcomes and recommendations of the

System‐Wide Data Review as a program should provide a coordinated approach to

undertaking the many activities and projects over the duration of the program.

1: Introduction


1.20 The audit has provided the ACT Audit Office with an opportunity to comment on the ACT

Health Directorate’s program management arrangements six months into a three‐year

program of activities. The audit’s findings and recommendations may inform the ACT Health

Directorate’s program management arrangements for the future phases of the

implementation and, at the same time, inform the Territory’s wider community of the ACT

Health Directorate’s approach to managing this important program.

Audit objective and scope

Audit objective

1.21 The objective of this audit is to provide an independent opinion to the Legislative Assembly

on the effectiveness of program management arrangements supporting the

implementation of recommendations, actions and initiatives identified in the ACT Health

System‐Wide Data Review.

Audit scope

1.22 The audit examined the planning, governance and administrative arrangements put in place

by the ACT Health Directorate to effect the implementation of recommendations, actions

and initiatives identified in the System‐Wide Data Review.

1.23 The audit focused on the ACT Health Directorate’s activities following the finalisation of the

System‐Wide Data Review in August 2018 (although it is noted that some of the

recommendations, actions and initiatives identified in the System‐Wide Data Review were

planned and potentially commenced or finalised prior to this).

Out of scope

1.24 The audit did not:

review the System‐Wide Data Review or its recommendations, or re‐perform aspects

of the System‐Wide Data Review, including the root cause analysis;

consider the accuracy of ACT Health data;

investigate the technical characteristics of new information technology systems to

capture, store, transform and share ACT Health data; and

assess the quality of outputs delivered during Phase One of the System‐Wide Data

Review Implementation Plan.

1.25 Accordingly, the audit does not provide assurance:

on the effectiveness of the delivery of the Implementation Plan, Phase One, to

December 2018 System‐Wide Data Review (the Phase One Implementation Plan);

1: Introduction


that the ACT Health Directorate will successfully implement the System‐Wide Data

Review implementation program; and

with respect to whether the recommendations, actions and initiatives identified in the

System‐Wide Data Review, and being implemented by the ACT Health Directorate

(and Canberra Health Services), will address the issues in the ACT Health Directorate’s

performance information and reporting.

Audit criteria, approach and method

Audit criteria

1.26 To form a conclusion against the objective, the following audit criteria were used:

Criterion 1: Are effective governance arrangements in place to provide oversight of

the implementation of the System‐Wide Data Review program of activities?

Criterion 2: Is there effective planning for the implementation of the System‐Wide

Data Review program of activities?

1.27 For Criterion 1 the audit considered the following areas of focus:

Is there a documented and approved System‐Wide Data Review implementation

governance plan in place?

Are roles and responsibilities for activities, outputs and outcomes defined,

documented, agreed and ascribed to an appropriate level within the organisation?

Are there committee arrangements in place to oversee the program and its

implementation to assure delivery of outputs and outcomes?

Are monitoring and reporting processes in place for the System‐Wide Data Review

implementation that are documented, regularised and mapped to critical

implementation risks?

Are there risk management arrangements in place for System‐Wide Data Review

implementation:

are risks to the System‐Wide Data Review implementation program identified

and documented, including through a comprehensive risk management plan and

risk register?

is there regular monitoring of risks, risk levels and risk treatments?

do the governance committees for the implementation regularly review and

agree the risk management plan?

is there escalation and management attention and activity for high level risks?

1: Introduction


1.28 For Criterion 2 the audit considered the following areas of focus:

Is there a comprehensive, documented and approved program management plan for

the delivery of the System‐Wide Data Review implementation program of activities,

which clearly sets out:

outcomes, objectives and outputs for all phases of the implementation;

a consistent approach to prioritisation of individual activities/projects;

identification of critical paths and key dependencies;

an implementation schedule, including key milestones and timelines (scheduling);

an agreed approach to developing a business case for those activities that require

additional resources or involve new investment; and

the costs and tracking of costs of implementing the System‐Wide Data Review?

Is there an agreed and consistent review and evaluation methodology to:

measure and report progress on individual activities as well as overall progress;

promote timely delivery and accountability;

identify benefits and dis‐benefits; and

identify priority activities throughout the implementation program?

Audit approach and method

1.29 The audit approach and method consisted of:

reviewing relevant documentation related to the ACT Health Directorate’s

governance, planning and program management arrangements (including risk

management) for the System‐Wide Data Review implementation;

interviews and discussion with key ACT Health Directorate staff; and

an engagement quality control review.

1.30 The audit has referred to The Standard for Program Management Fourth Edition (2017) (the

Standard) as a guide to the systems and processes involved in the discipline of program

management. The Standard provides guidance on the principles of program management,

generally accepted definitions of programs and program management as well as concepts

important to their success, including program management principles practices and

activities.1 The Standard brings together a considerable body of contemporary knowledge

on program management to guide program managers and the implementation of programs.

1 The Standard for Program Management Fourth Edition (2017), Programme Management Institute Inc., Newtown Square, Pennsylvania, USA.

1: Introduction


1.31 The audit was performed in accordance with ASAE 3500 – Performance Engagements. The

audit adopted the policy and practice statements outlined in the Audit Office’s Performance

Audit Methods and Practices (PAMPr) which is designed to comply with the requirements

of the Auditor‐General Act 1996 and ASAE 3500‐ Performance Engagements.

1.32 In the conduct of this performance audit the ACT Audit Office complied with the

independence and other relevant ethical requirements related to assurance engagements.


2 GOVERNANCE FOR THE SYSTEM‐WIDE DATA

REVIEW IMPLEMENTATION PROGRAM

2.1 This chapter examines the ACT Health Directorate’s governance arrangements for the

System‐Wide Data Review implementation program. This includes consideration of

whether roles and responsibilities have been appropriately defined and articulated, and

whether appropriate program governance documentation has been prepared, including risk

management documentation.

Summary

Conclusions

Governance arrangements to support the System‐Wide Data Review program are under

development and are not yet mature enough to effectively support its implementation.

There is no documented program governance plan and program roles and responsibilities are not

adequately defined. A series of committees and working groups that have been identified as

supporting the implementation of the System‐Wide Data Review are not yet effective; there is a

lack of clarity with respect to roles and purpose and the groups are not consistently meeting and

operating. Risk management arrangements to support the implementation of the program are in

the early stages of development.

The Data Repository Population Project, a major component of the System‐Wide Data Review

program, has well‐defined project management arrangements in place that provide transparency

and rigour to monitoring the progress and implementation of the new data repository. The

governance and management of the System‐Wide Data Review implementation program as a

whole could benefit from the adoption of similar arrangements.

Key findings Paragraph

There is no program governance plan for the System‐Wide Data Review implementation program that identifies program governance structures, dependencies, assumptions and constraints. Nevertheless, the ACT Health Directorate developed a draft Data Management and Governance Framework as an early deliverable of the System‐Wide Data Review implementation program. The draft Data Management and Governance Framework does not identify the governance arrangements for the System‐Wide Data Review implementation program, but rather relates to the ACT Health Directorate’s broader data management and governance arrangements such as data management committees as well as roles and responsibilities for collecting, managing, maintaining, using and sharing data.

2.17

2: Governance for the System‐Wide Data Review implementation program


A series of data governance committees and working groups have been established to support data management practices across the ACT Health Directorate. The purpose and roles of these groups are outlined in the draft Data Governance Committees; Supporting the Data Management and Governance Framework document. The System‐Wide Data Review Reference Group is the key committee with oversight responsibility of data governance and management in the ACT Health Directorate. The System‐Wide Data Review Reference Group is supported by the Data Standards Working Group and the Data Repository Working Group. The establishment of these groups is anticipated to improve data governance and management arrangements across the ACT Health Directorate.

2.25

There is a lack of clarity associated with the purpose of the System‐Wide Data Review Reference Group. Various documents, including the Phase One Implementation Plan and the draft Data Governance Committees document, describe an ongoing, longer‐term role for the System‐Wide Data Review Reference Group with respect to data governance and management. However, the terms of reference for the System‐Wide Data Review Reference Group, which were finalised in February 2019, identify that the tenure of the group is time‐limited, with a key focus on the System‐Wide Data Review program implementation. The System‐Wide Data Review Reference Group has met twice since its establishment in November 2018 (29 November 2018 and 28 February 2019), which means that, if its purpose is to focus on System‐Wide Data Review program implementation, then it has had limited opportunity to oversee the management and implementation of the program thus far.

2.38

Two working groups are intended to support the System‐Wide Data Review Reference Group; the Data Standards Working Group and the Data Repository Working Group. While the Data Repository Working Group is established and has met regularly since October 2018, the Data Standards Working Group is yet to meet and commence operating. The terms of reference for both groups remain in draft at the time of audit fieldwork in April 2019. While the Data Repository Working Group has had the opportunity to contribute to, and influence, the implementation of the System‐Wide Data Review program since October 2018, the Data Standards Working Group is yet to do so.

2.50

Two teams in the Commissioning and Performance Division’s Health Insights Branch are operationally responsible for the management of the System‐Wide Data Review implementation program; the Data Governance and Management team and the Data Development team. A Program Manager for System‐Wide Data Review program implementation has been employed since May 2018 (the most recent Program Manager commenced in the role in November 2018). The role and responsibilities of the Program Manager are not documented. Other roles and responsibilities for the implementation of the System‐Wide Data Review program are not adequately documented, although information on roles and responsibilities may be surmised from a range of other documents. The absence of clear and complete documentation of program management roles and responsibilities can lead to misunderstandings about responsibilities and inconsistencies in the way in which the roles are carried out.

2.58



Documentation and governance arrangements associated with the Data Repository Population Project demonstrate well‐established processes and a thorough and mature approach to project management. There is an opportunity for the ACT Health Directorate to recognise the maturity of the governance and management arrangements associated with the Data Repository Population Project and apply these to the System‐Wide Data Review program more broadly, in order to bring improved transparency and rigour to the overall implementation program.

2.64

Risk management of the System‐Wide Data Review implementation was initially undertaken through a System‐Wide Data Review Implementation Plan (Phase 1) ‐ Risks Register (the Phase 1 Risk Register). The Phase 1 Risk Register was developed after the program of work commenced, in October 2018, and included six risks for 14 activities across four Domains. The risks were broad, generic and lacked underlying detail. In February 2019, a revised draft Risk Register – ACT Health System Wide Data Review Implementation Plan (revised draft Risk Register) was prepared. The revised draft risk register is more comprehensive and contains information on risk sources and consequences, risk treatments, risk owners and risk status. The revised draft risk register identifies ten risks related to 12 activities across four System‐Wide Data Review Domains. However, the revised draft risk register is not tailored as a strategic or program level risk assessment and does not identify risks associated with the management of the System‐Wide Data Review program as a whole.

2.74

Governance of the System‐Wide Data Review implementation

program

Program governance

2.2 Program governance is defined in The Standard for Program Management Fourth

Edition (2017) (the Standard) as:

The performance domain that enables and performs program decision making, establishes practices to support the program and maintains program oversight.

…

Program governance comprises the framework, functions and processes by which a program is monitored, managed and supported in order to meet organisational strategic and operational goals. The focus of program governance is the delivery of program benefits by establishing the systems and methods by which a program and its strategy are defined, authorised, monitored and supported by its sponsoring organisation. … Program governance is performed through the actions of a review and decision making group that is charged with endorsing or approving recommendations regarding a program under its authority.

…

Program governance may also refer to the framework, functions and processes by which a program team monitors and manages the components that are being performed to support the program. Governance of components is often achieved through the actions of the program



manager and project team responsible to the integrated outcomes of the program. Such a responsibility may also be called component governance.

2.3 The governance arrangements for the System‐Wide Data Review implementation program

should therefore address how the implementation program is controlled and operated and

the accountability mechanisms by which it, and staff managing the program, are held to

account.

2.4 The governance arrangements for the System‐Wide Data Review implementation program

should encompass processes and procedures to control the System‐Wide Data Review

implementation program and the overall decision‐making. This decision‐making framework

comprises roles and responsibilities, approval processes, work deliverables, and the

workflow that aligns the overall decision making steps. Program governance also includes

those processes that allow the ACT Health Directorate to maintain oversight and control of

the overall program.

2.5 The Audit Office sought to understand what governance arrangements were in place for

System‐Wide Data Review program implementation, including whether:

the governance arrangements for the program were fully documented, e.g. through a

program governance plan;

program roles and responsibilities had been established and documented, including

appropriate oversight by committees; and

a program risk management plan and risk register had been established.

Program governance plan

2.6 The Standard for Program Management Fourth Edition (2017) states:

To facilitate the design and implementation of effective governance, many organisations prepare documented descriptions of each program’s governance frameworks, functions and processes. Such descriptions are summarized in a program governance plan, which may be a stand‐alone document or subsection of the program management plan.

The purpose of the program governance plan is to describe the systems and methods to be used to monitor, manage and support a given program, and the responsibilities of specific roles for ensuring the timely and effective use of those systems and methods. This plan is referenced throughout the program’s duration to ensure that the program is conforming to established governance expectations and agreements. …

2.7 There is no documented, approved program governance plan for the System‐Wide Data

Review implementation program.

2.8 The ACT Health Directorate advised that in practice, program management and governance

arrangements have been implemented ‘implicitly’, with management and governance

arrangements achieved through direct and ongoing involvement of two key ACT Health

Directorate executives. The ACT Health Directorate also advised that ‘the current

governance oversight for the implementation was in a state of evolution; it was not a text

book oversight approach but information has flowed in both directions, i.e. up and down,



in order to keep across issues and highlighting risks as they emerge’ and that ‘up to now the

governance oversight [for the System‐Wide Data Review] has been fit for purpose, given

staffing and resource limitations’.

2.9 The ACT Health Directorate advised that:

While there is no formal program‐level governance plan, the following two governance plans support the overall program:

[t]he Data Governance and Management Framework will provide a governance framework for the recommendations and agreed outcomes from the System‐Wide Data Review, including for a significant proportion of program activities … [and]

the largest and highest risk project (i.e. the build of the data repository) has a governance plan in place …

2.10 The Data Management and Governance Framework is discussed in the following section,

while governance arrangements for the Data Repository Project are discussed in paragraphs

2.59 to 2.64.

Data Management and Governance Framework

2.11 The ACT Health Directorate identified a suite of documents that were in the process of being

developed to inform program governance for the System‐Wide Data Review

implementation. These documents, which remain in draft at the conclusion of audit

fieldwork in April 2019, include:

Data Management and Governance Framework; Outlining the structures and rules in

place for data use and the reporting of information across ACT Health (Data

Management and Governance Framework); and

Data Governance Committees; Supporting the Data Management and Governance

Framework (last updated 11 January 2019).

2.12 The Phase One Implementation Plan also describes some actions in relation to establishing

governance arrangements as ‘outputs up to December 2018’ for one of the System‐Wide

Data Review’s nine Domains. In this respect two of the outputs for Domain D2‐Data

governance are:

Appropriate data governance committees, including a tier 1 committee, established with the defined terms of reference and accountabilities that will facilitate improved data management and quality from collection to reporting.

A finalised Data Management and Governance Framework that clearly articulates the roles and responsibilities of data management, which will be widely communicated and embedded into day to day operations at all levels, through an education program.



2.13 The scope and purpose of the draft Data Management and Governance Framework is

described as follows:

This document outlines ACT Health’s approach to data governance to ensure there are well‐defined and best practice structures, policies and procedures in place for management of data and information across the ACT Health System.

It also defines key components of the ACT Health Data Management and Governance Framework to provide guidance to ACT Health staff and broader stakeholders about their roles and responsibilities for collecting, managing, maintaining, using and/or sharing data.

2.14 The draft Data Management and Governance Framework is an early deliverable of the

System‐Wide Data Review implementation program. It describes the ACT Health

Directorate’s approach to data governance across the ACT Health system and provides

guidance for ACT Health Directorate staff and broader stakeholders about roles and

responsibilities for collecting, managing, using and sharing data. The draft Data

Management and Governance Framework also provides information on five core principles,

which are intended to ‘provide a view of what ACT Health aims to achieve by implementing

the ACT Health Data Management and Governance Framework’: Principle 1 (data is a

valuable asset), Principle 2 (data meets business needs), Principle 3 (data is accessible and

useable), Principle 4 (data is actively managed) and Principle 5 (data is of a high quality).

2.15 The draft Data Management and Governance Framework describes, at a conceptual, high‐

level, the data management and data governance structures in place for the ownership and

accountability of data across the ACT Health Directorate. It also provides high‐level guidance

on ways in which the framework is intended to be implemented and operationalised.

2.16 The draft Data Management and Governance Framework relates to the ACT Health

Directorate’s broader data management and governance arrangements. It does not

mention or otherwise reference the System‐Wide Data Review implementation program.

Furthermore, the draft Data Management and Governance Framework does not include key

information that would be relevant to a System‐Wide Data Review implementation

program governance plan such as:

descriptions of the systems and methods to monitor, manage and support the

broader System‐Wide Data Review implementation program;

governance structures for the System‐Wide Data Review implementation program,

including schedules for meetings, ‘decision‐point reviews’ and ‘program health

checks’; and

System‐Wide Data Review implementation program governance dependencies,

assumptions and constraints.

2.17 There is no program governance plan for the System‐Wide Data Review implementation

program that identifies program governance structures, dependencies, assumptions and

constraints. Nevertheless, the ACT Health Directorate developed a draft Data Management

and Governance Framework as an early deliverable of the System‐Wide Data Review

implementation program. The draft Data Management and Governance Framework does

not identify the governance arrangements for the System‐Wide Data Review



implementation program, but rather relates to the ACT Health Directorate’s broader data

management and governance arrangements such as data management committees as well

as roles and responsibilities for collecting, managing, maintaining, using and sharing data.

Data governance committees

2.18 A series of data governance committees and working groups have been established. The

purpose of these groups is outlined in the draft Data Governance Committees; Supporting

the Data Management and Governance Framework (last updated 11 January 2019) (Data

Governance Committees document), the purpose of which is described as follows:

This document aligns data governance committees with the overall ACT Health approach to governance committees. It provides detail of the data governance committee structure to be embed[ded] across ACT Health. This document is a ‘living document’ and will be updated as changes occur.

2.19 The draft Data Governance Committees document includes information about the

governance arrangements for the System‐Wide Data Review implementation program:

The [System‐Wide Data Review Reference Group] will have oversight of the full span of data management and governance domains that have been adopted by ACT Health as a result of the SWDR [System‐Wide Data Review] …

…

The System‐Wide Data Review Reference Group (SWDR RG) is the Tier 3 committee for data within ACT Health. Tier 3 committees provide advisory roles and report to the relevant Tier 2 committee, in this case the Health System Finance and Performance Committee. Tier 3 committees are commissioned by a Tier 2 committee to ensure ACT Health continues to meet agreed data outcomes, and the purpose and membership of these committees may be reviewed and amended periodically.

2.20 The draft Data Governance Committees document states:

Data management and governance activities that require dedicated oversight will be governed by specific data working groups. These groups are not a part of the formal ACT Health Governance Structures. These working groups are commissioned by the Tier 3 System‐Wide Data Review Reference Group to ensure ACT Health continues to meet agreed outcomes, and the purpose and membership of these working groups may be reviewed and amended periodically.

These data working groups will report to the Tier 3 System‐Wide Data Review Reference Group, but may have a direct link to other Tier 2 and 3 committees.

2.21 The draft Data Governance Committees document identified that two working groups

would report to the System‐Wide Data Review Reference Group:

the Data Standards Working Group; and

the Data Repository Working Group.

2.22 The Data Standards Working Group’s focus is expected to be on metadata management

while the Data Repository Working Group is expected to focus on data management.



2.23 The draft Data Governance Committees document also stated, in respect of the

System‐Wide Data Review Reference Group, the Data Standards Working Group and the

Data Repository Working Group, that:

… All three committees will need to work collaboratively with the Tier 2 Diagnostic and Clinical Services Systems Steering Committee and the Tier 2 Change Control Board on specific issues or activities, to ensure a holistic approach to data‐related activities across ACT Health, Canberra Health Services, and Calvary Public Hospital Bruce.

2.24 Figure 2‐1 shows the hierarchy of the ACT Health Directorate data governance committees,

including the System‐Wide Data Review Reference Group, the Data Standards Working

Group and the Data Repository Working Group, with governance responsibilities for the

System‐Wide Data Review implementation program.

Figure 2‐1 ACT Health Data Governance committees

Source: ACT Health Directorate’s draft Data Governance Committees; Supporting the Data Management and Governance Framework (Last updated 11 January 2019)

2.25 A series of data governance committees and working groups have been established to

support data management practices across the ACT Health Directorate. The purpose and

roles of these groups are outlined in the draft Data Governance Committees; Supporting the

Data Management and Governance Framework document. The System‐Wide Data Review

Reference Group is the key committee with oversight responsibility of data governance and

management in the ACT Health Directorate. The System‐Wide Data Review Reference

Group is supported by the Data Standards Working Group and the Data Repository Working

Group. The establishment of these groups is anticipated to improve data governance and

management arrangements across the ACT Health Directorate.

Directorate Leadership Committee

Tier 1Tier 3

Tier 2

Corporate Committee

Health System Finance and Performance

Committee

System‐Wide Data Review Reference Group

Technology Committee

Diagnostic and Clinical Service Systems Steering

CommitteeChange Control Board

Data Standards Working Group

Data Repository Working Group

Data W

orking

Groups



System‐Wide Data Review Reference Group

System‐Wide Data Review Reference Group meetings

2.26 The System‐Wide Data Review Reference Group met for the first time on

29 November 2018. At the time of the conclusion of audit fieldwork in April 2019, the

System‐Wide Data Review Reference Group had met twice; 29 November 2018 and

28 February 2019.

2.27 The ACT Health Directorate advised that prior to November 2018, ‘oversight to the

implementation had been provided by day to day reporting lines [of two executive ACT

Health Directorate staff]’.

2.28 At its first meeting on 29 November 2018, the System‐Wide Data Review Reference Group

reviewed the draft terms of reference for the System‐Wide Data Review Reference Group,

including details of the Reference Group’s membership. The terms of reference were noted

as ‘finalised’ at the System‐Wide Data Review Reference Group’s second meeting on

28 February 2019.

2.29 The ACT Health Directorate advised that ‘there were no implications associated with the

draft terms of reference for System‐Wide Data Review Reference Group [because] in

practice, there was a higher level of governance committees in transition and in a formative

state’. The ACT Health Directorate also noted that ’the System‐Wide Data Review Reference

Group is endorsed at management level’. Notwithstanding the ACT Health Directorate’s

advice, taking six months to finalise the terms of reference for a key governance mechanism

for the program represents a missed opportunity to have robust and formalised governance

arrangements from the outset of the implementation program.

System‐Wide Data Review Reference Group terms of reference

2.30 As discussed at paragraph 2.19, the draft Data Governance Committees document states

that the System‐Wide Data Review Reference Group is the Tier 3 committee for data within

the ACT Health Directorate.

2.31 The terms of reference for the System‐Wide Data Review Reference Group note that the

Reference Group’s role is:

To provide governance, oversight and guidance for the development and implementation of the outcomes of the System‐Wide Data Review (SWDR), including reporting on the progress of the SWDR Implementation Plan.

This group is time‐limited to the duration of developing and implementing the activities presented in the System‐Wide Data Review Outcomes Report.

2.32 The ACT Health Directorate’s draft documentation describes two roles for the System‐Wide

Data Review Reference Group; one relates to a broader and ongoing role as a Tier 3

committee and the other relates to a time‐limited role overseeing the System‐Wide Data

Review implementation program.




It is intended that the SWDR [System‐Wide Data Review] Reference Group will oversee the SWDR program of activities for the three‐year period that was agreed in the SWDR Outcomes Report. Closer to the end of that three‐year period, ACTHD [ACT Health Directorate] anticipates consideration will need to be made regarding possible ongoing governance and roles for the SWDR Reference Group, the Data Repository Working Group, and the Data Standards Working Group.

2.34 The terms of reference for the System‐Wide Data Review Reference Group detail the

Reference Group’s reporting mechanisms, its functions, membership and meeting

frequency, the Reference Group’s quorum number, how agenda requests will be dealt with

and how often the terms of reference should be reviewed.

2.35 The System‐Wide Data Review Reference Group’s functions, as listed in the terms of

reference, are to:

Oversee and monitor [System‐Wide Data Review] outcome activities.

Provide advice and support to the planning and implementation of System‐Wide Data Review outcome activities.

Approve project documentation.

Oversee and approve the [System‐Wide Data Review] implementation plans.

Ensure strategic and business alignment of [System‐Wide Data Review] activities.

Monitor risks, issues and schedules of projects.

Oversee and monitor the [System‐Wide Data Review] communications, ensuring alignment with the [System‐Wide Data Review] communication strategy;

Commission and direct relevant working groups required to meet specific [System‐Wide Data Review] needs, including the Data Standards Working Group and the Data Repository Working Group.

2.36 The terms of reference lists eleven roles that make up the core membership of the

System‐Wide Data Review Reference Group, plus one attendee/observer, and notes that

‘[o]ther individuals will be invited to attend on an ad‐hoc basis to provide expert opinion or

to inform specific issues’. The core members and observer are:

Executive Group Manager, Commissioning and Performance, ACT Health (Chair)

Executive Branch Manager, Health Insights, ACT Health (Deputy Chair)

Director, Health Insights, Commissioning and Performance, ACT Health

Deputy Executive Director, Research Office, Canberra Health Services

Executive Director, Cancer, Ambulatory and Community Health Support, Canberra Health Services

Executive Director, Surgery and Oral Health, Canberra Health Services

Program Director, Recurrent Health Services Program, ACT Health

Chief Financial Officer, Canberra Health Services

Executive Branch Manager, ICT Operations, ACT Health

Chief Finance Officer and Director of Corporate Services, Calvary Public Hospital Bruce



Executive Director, Health Care Consumers’ Association

Attendees/Observers

Assistant Director, Clinical Insights, Commissioning and Performance, ACT Health

2.37 The terms of reference for the System‐Wide Data Review Reference Group clearly specifies

the Reference Group’s role, functions and membership including representation from the

ACT Health Directorate, Canberra Health Services, Calvary Public Hospital Bruce and the

Health Care Consumers’ Association. There is, however, inconsistency between the terms

of reference for the System‐Wide Data Review Reference Group and the draft Data

Governance Committees document as to whether the System‐Wide Data Review Reference

Group exists for the purpose and duration of the three‐year implementation program or if

it will have a broader and longer term role as the ACT Health Directorate’s Tier 3 Committee

for data governance. The ACT Health Directorate advised that consideration will be needed

as to whether the System‐Wide Data Review Reference Group will have an ongoing

governance role beyond three years.

2.38 There is a lack of clarity associated with the purpose of the System‐Wide Data Review

Reference Group. Various documents, including the Phase One Implementation Plan and

the draft Data Governance Committees document, describe an ongoing, longer‐term role

for the System‐Wide Data Review Reference Group with respect to data governance and

management. However, the terms of reference for the System‐Wide Data Review Reference

Group, which were finalised in February 2019, identify that the tenure of the group is

time‐limited, with a key focus on the System‐Wide Data Review program implementation.

The System‐Wide Data Review Reference Group has met twice since its establishment in

November 2018 (29 November 2018 and 28 February 2019), which means that, if its

purpose is to focus on System‐Wide Data Review program implementation, then it has had

limited opportunity to oversee the management and implementation of the program thus

far.

Data Standards Working Group and the Data Repository Working Group

2.39 As discussed in paragraph 2.20 to 2.25, two working groups are intended to support the

System‐Wide Data Review Reference Group; the Data Standards Working Group and the

Data Repository Working Group. While the Data Repository Working Group is well

established, the ACT Health Directorate advised that it is working to establish the Data

Standards Working Group, with a first meeting anticipated in April 2019 (a January 2019

Project Plan Brief that incorporates information about the establishment of the Data

Standards Working Group identifies that the group was planned to commence in

March 2019). Both working groups have draft terms of reference, which are yet to be

finalised.

Data Standards Working Group draft terms of reference

2.40 The draft terms of reference for the Data Standards Working Group describe the working

group’s role as ‘to provide governance, oversight and guidance in the development of

metadata for ACT Health and Canberra Health Services’. The ACT Health Directorate



describes metadata management in Leading Data Reform: The Way Forward ‐ Outcomes of

the ACT Health System‐Wide Data Review (August 2018):

… metadata management relates to the documentation of data so that there is a common understanding of how it is defined and how it should be interpreted. This information includes data definitions, data models and data flows.

Data Repository Working Group draft terms of reference

2.41 The draft terms of reference for the Data Repository Working Group describe its role as:

To inform the approach and provide advice and guidance:

for data management activities undertaken by ACT Health, including the developmentand delivery of the data repository; and

to ensure that appropriate data and relevant technologies are available andeffectively used to support and inform insights, analysis and reporting (dataanalytics).

2.42 The draft terms of reference for the Data Repository Working Group states that it reports

to:

… the Tier 3 System‐Wide Data Review Reference Group.

This working group works collaboratively with the Data Standards Working Group, the Tier 3 Change Control Board, and the Tier 3 Diagnostic and Clinical Service Systems Steering Committee to ensure a holistic approach to data‐related activities across ACT Health Canberra Health Services.

2.43 The draft terms of reference detail the Data Repository Working Group’s reporting

mechanisms, its functions, membership and meeting frequency, as well as how agenda

requests will be dealt with and how often the terms of reference should be reviewed. The

draft terms of reference identify a series of core members of the group, comprised primarily

of ACT Health Directorate executives, as well as other observers.

2.44 While the draft terms of reference for the Data Repository Working Group states that the

Chair is the Executive Branch Manager, Health Insights Branch, the ACT Health Directorate

Chief Information Officer has also fulfilled this role, as evidenced in the Data Repository

Working Group minutes for meetings on 18 and 24 October 2018 and 4 and

13 February 2019.

Data Repository Working Group meetings

2.45 According to the draft terms of reference for the Data Repository Working Group, the group

meets weekly. A review of Data Repository Working Group Minutes shows two meetings

were held in October 2018, two meetings in December 2018, three meetings in

January 2019 and two in February 2019 (on 4 and 13 February).

2.46 Prior to October 2018 and the establishment of the Data Repository Working Group, two

Data Repository Regular Catch Up meetings were held on 30 August 2018 and

18 September 2018. Attendees were key project staff within the Digital Solutions Division.



2.47 Data Repository Working Group meetings are held for an hour and discussion items have

included the data repository project program of activities under way, the project’s status

and upcoming timelines. There is evidence that the Data Repository Working Group made

decisions during these meetings and actions to be taken were noted. It is not evident from

the Data Repository Working Group Minutes dated 18 and 24 October 2018, 10 December

2018 and 4 and 13 February 2019 that all core members listed in the draft terms of

reference for the Data Repository Working Group attended the meetings; the draft terms

of reference do not include a quorum number.

2.48 The Data Repository Working Group (and previously as the Data Repository Regular Catch

Up) has met regularly, either weekly or less frequently from 30 August 2018, to make

decisions and agree actions to be taken for the Data Repository Population Project.

2.49 At its meeting on 28 February 2019, the System‐Wide Data Review Reference Group

decided that a Project Control Group is to be established to oversee the Data Repository

Population Project and report to the System‐Wide Data Review Reference Group. The

Project Control Group will include representatives from Canberra Health Services and

Calvary Public Hospital Bruce.

2.50 Two working groups are intended to support the System‐Wide Data Review Reference

Group; the Data Standards Working Group and the Data Repository Working Group. While

the Data Repository Working Group is established and has met regularly since October 2018,

the Data Standards Working Group is yet to meet and commence operating. The terms of

reference for both groups remain in draft at the time of audit fieldwork in April 2019. While

the Data Repository Working Group has had the opportunity to contribute to, and influence,

the implementation of the System‐Wide Data Review program since October 2018, the Data

Standards Working Group is yet to do so.

Program management roles and responsibilities

2.51 The Standard for Program Management Fourth Edition (2017) states that a program

governance plan should also describe:

… the structure and composition of the group of governance participants and [define] the roles and responsibilities of key stakeholders. The plan defines who will have accountability and authority with respect to key decision making categories and responsibility boundaries.

2.52 The ACT Health Directorate has described the System‐Wide Data Review implementation

as a ‘dispersed work program’ involving the Digital Solutions Division (responsible for

Information and Communication Technology related program activities) and the

Commissioning and Performance Division (responsible for the other implementation

activities).

2.53 Figure 2‐2 shows the ACT Health Directorate’s organisational structure. Both the Digital

Solutions Division and Commissioning and Performance Division are within the ACT Health

Directorate’s Corporate Services.



Figure 2‐2 ACT Health Directorate’s organisational structure

Source: ACT Health Directorate website – https://www.health.act.gov.au/about‐our‐health‐system/organisation‐structures

System‐Wide Data Review implementation operational teams

2.54 Two teams in the Commissioning and Performance Division’s Health Insights Branch are

operationally responsible for the management of the System‐Wide Data Review

implementation program:

the Data Governance and Management team is responsible for program management

and fulfils the role of secretariat for the System‐Wide Data Review Reference Group

as well as other responsibilities, including a number of projects under the

System‐Wide Data Review implementation program; and

the Data Development team is responsible for the Data Standards Working Group

(not yet established) as well as other responsibilities, including a number of projects

under the System‐Wide Data Review implementation program.

2.55 A Program Manager for the System‐Wide Data Review implementation has been employed

since May 2018 (the current Program Manager commenced in the role in November 2018).

2.56 The System‐Wide Data Review implementation Program Manager does not have a position

or role description and information about other key roles for the implementation program

are dispersed in documents such as the draft terms of reference for the System‐Wide Data

Review Reference Group, the System‐Wide Data Review Program of Activities spreadsheet

and the draft Data Roles and Responsibilities for datasets transitioning to the new ACT

Health Data Repository.



2.57 Notwithstanding the lack of formally documented roles and responsibilities, information on

roles and responsibilities was sought during audit interviews with ACT Health Directorate

executives and managers and from other implementation‐related documents (refer to

paragraph 2.56). A summary of program implementation roles and responsibilities is shown

in Table 2‐1.

Table 2‐1 System‐Wide Data Review implementation program roles and responsible ACT Health Directorate staff

Responsible ACT Health Directorate staff member

Roles in System‐Wide Data Review implementation program

Next reporting level in the ACT Health Directorate

Deputy Director‐General Corporate Services

System‐Wide Data Review implementation Program Executive Sponsor

Director‐General, the ACT Health Directorate

Executive Group Manager, Commissioning and Performance Division

Chair, System‐Wide Data Review Reference Group


Acting Executive Branch Manager, Health Insights Branch, Commissioning and Performance Division

Deputy Chair, System‐Wide Data Review Reference Group

Executive Group Manager, Commissioning and Performance Division

Chief Information Officer, Digital Solutions Division

Chair of the Data Repository Working Group


Director, Health Insights Branch, Commissioning and Performance Division

Project Lead in Phase One for domains D2‐Data Governance, D3‐Data Quality, D4‐Metadata Management, D5‐Data Security and Privacy, D7‐Communications and D9‐Information and Insights

Acting Executive Branch Manager, Health Insights Branch, Commissioning and Performance Division

Director, Information Management Hub, Technology Operations, Digital Solutions Division

Project Lead in Phase One for D1‐Data Management including the Data Repository Population Project

Chief Information Officer, Digital Solutions Division

Assistant Director, Data Governance and Management Team, Health Insights Branch, Commissioning and Performance Division

System‐Wide Data Review implementation Program Manager

Director, Health Insights Branch, Commissioning and Performance Division

Source: Audit Office, based on interviews with ACT Health Directorate staff and other ACT Health Directorate documents.

2.58 Two teams in the Commissioning and Performance Division’s Health Insights Branch are

operationally responsible for the management of the System‐Wide Data Review

implementation program; the Data Governance and Management team and the Data

Development team. A Program Manager for System‐Wide Data Review program

implementation has been employed since May 2018 (the most recent Program Manager



commenced in the role in November 2018). The role and responsibilities of the Program

Manager are not documented. Other roles and responsibilities for the implementation of

the System‐Wide Data Review program are not adequately documented, although

information on roles and responsibilities may be surmised from a range of other documents.

The absence of clear and complete documentation of program management roles and

responsibilities can lead to misunderstandings about responsibilities and inconsistencies in

the way in which the roles are carried out.

Data Repository Population Project

2.59 The Data Repository Population Project is a key project of the System‐Wide Data Review

implementation program relating to the Domain D1 – Data management recommendation

to ‘build a new data repository, which will collect, store, extract and transform quality data

to deliver better insights to the community’.

2.60 The ACT Health Directorate’s Digital Solutions Division is responsible for the technical build

and the initial population of the data repository, while working in close collaboration with

the ACT Health Directorate’s Commissioning and Performance Division.

2.61 The Digital Solutions Division has developed a Project Delivery Framework (Version 1.0,

15 March 2019) the purpose of which is:

… to bring together, into a central document, the Approaches that support the Digital Solutions Division (DSD) staff to deliver technology solutions projects. This set of Approaches and documentation is to provide all stakeholders with a single, authoritative and up‐to‐date source of advice on the project management practices to be used.

2.62 The planning, management and governance documentation for the Data Repository

Population Project includes:

Project Initiation Document Data Repository Population Project – Phase 1 (version 0.8,

21 February 2019) – the purpose of the document is to define the project, give the

direction and scope of the project and guidance on timings for ‘Gate Reviews, which

allows for validation of current project activities’. This is a consolidated and

comprehensive document that brings together a range of project management

principles and features, including information on the scope of the project, the project

approach, the project plan, the budget, project constraints, project dependencies,

project human resources, project benefits, project governance information, the

project delivery Gantt chart and project controls;

Project Scope Statement Data Repository Population (16 January 2019) – the purpose

of this document is to define the boundaries of the project;

Data Repository Project Status Report for the Tier 2 committee, Diagnostic and Clinical

Service Systems Steering Committee – this document contains information about the

project’s sponsors, resources, scope, performance indicators, percentage of

completion, the current stage milestones, the progress update, the key activities for

the next reporting period, the risk heat map and open issues assessment;



Data Repository Population Project Status Report for the Data Repository Working

Group – this document contains information on the status of the project in relation to

time, cost, scope, risk and quality, project activities percentage to completion,

deliverables in the reporting period and the next reporting period, key open issues

and risks and a Gantt chart of tasks; and

Draft Roles and Responsibilities for datasets transitioning to the new ACT Health Data

Repository – this document details activities, timings, responsible staff, stakeholders,

reviewers and approvals necessary for ‘inputs pre‐repository’, for the ‘repository’ and

‘outputs post repository’.

2.63 The governance arrangements for the Data Repository Population Project has involved

oversight by the Data Repository Working Group, the Tier 2 committee Diagnostic and

Clinical Service Systems Steering Committee and the System‐Wide Data Review Reference

Group (since its establishment in November 2018). The System‐Wide Data Review

Reference Group decided at its second meeting on 28 February 2019 that a Project Control

Group (to report to the System‐Wide Data Review Reference Group) would be established

for the ‘roll out of the new data repository’. This decision demonstrates the continual review

of governance arrangements for the Data Repository Population Project.

2.64 Documentation and governance arrangements associated with the Data Repository

Population Project demonstrate well‐established processes and a thorough and mature

approach to project management. There is an opportunity for the ACT Health Directorate

to recognise the maturity of the governance and management arrangements associated

with the Data Repository Population Project and apply these to the System‐Wide Data

Review program more broadly, in order to bring improved transparency and rigour to the

overall implementation program.

Risk management

2.65 The Standard for Program Management Fourth Edition (2017) (the Standard) identifies that

risk monitoring should be undertaken to determine whether:

program assumptions are still valid;

assessed risk has changed from its prior state, with analysis of trend;

proper risk management policies and procedures are being followed; and

cost and schedule contingency reserves are modified in line with the risks of the

program.

2.66 The Standard also states:

[Program risk identification] determines which risks could affect the program, documents their characteristics, and prepares for their successful management.

…



Risk identification is an iterative activity. As the program progresses, new risks may evolve or become known.

2.67 Risk management for the System‐Wide Data Review implementation was initially

undertaken through a System‐Wide Data Review Implementation Plan (Phase 1) ‐ Risks

Register (the Phase 1 Risk Register). The Phase 1 Risk Register was developed in

October 2018 as a one page spreadsheet that identified six risks for 14 activities in four

domains; D2 ‐ Data governance, D4 – Metadata management, D5 – Data security and

privacy and D9 – Information and insights. The risks were linked to two ACT Health

Directorate generic risk categories with associated definitions:

People – ‘Risks which can impact objectives & responsibilities relating to people –

employees, public, etc.’

Business systems and processes – ‘Risks associated with our responsibility to

implement and maintain efficient effective systems that comply with regulations and

legislation, the management, application and implementation of processes and

business systems relating to the efficient & effective operation [of] business’.

2.68 The risks in the Phase 1 Risk Register were poorly described and were not supported by

statements associated with the source of the risk and its consequences. There were no

identified risk mitigation/treatment strategies or resources required for effective risk

treatments. One risk appeared to be a statement of an intended future action:

Due to technical security focus of this project, propose that project ownership is handed over to DSD [Digital Solutions Division], with governance support as needed to be provided by the Data Management and Governance team.

2.69 The initial Phase 1 Risk Register was not tailored as a strategic or program level risk

assessment. It did not describe risks associated with the management of the

implementation program as a whole, risk sources and consequences, risk

mitigation/treatment strategies or the responsible risk owner.

2.70 In late February 2019, the ACT Health Directorate advised:

The Program Risk Register is currently being revised in conjunction with the ACTHD [ACT Health Directorate] internal risk team to ensure alignment with the ACTHD [ACT Health Directorate] approach to risk management.

2.71 On 28 February 2019 a revised draft Risk Register – ACT Health System Wide Data Review

Implementation Plan (revised draft Risk Register) was prepared. The revised draft Risk

Register (February 2019) is more comprehensive than the initial Phase 1 Risk Register

originally prepared in October 2018. The expanded document contains detailed statements

describing ten risks relating to:

six activities in domain D2 – Data governance;

one activity in domain D3 – Data quality;

two activities in domain D7 – Communication; and

three activities in domain D9 – Information and insights.



2.72 The revised draft Risk Register also includes information on:

risk sources and consequences;

an assessment of the current level of risk and a target level for the risk;

the risk treatments, percentage of progress and further risk treatment to be taken;

risk owners (with one exception all risk treatments were the responsibility of

Commissioning and Performance Division); and

the risk status.

2.73 The revised draft Risk Register, while more comprehensive than the initial Phase 1 Risk

Register, does not identify risks associated with the management of the System‐Wide Data

Review program. It is not tailored as a strategic or program level risk assessment.

2.74 Risk management of the System‐Wide Data Review implementation was initially

undertaken through a System‐Wide Data Review Implementation Plan (Phase 1) ‐ Risks

Register (the Phase 1 Risk Register). The Phase 1 Risk Register was developed after the

program of work commenced, in October 2018, and included six risks for 14 activities across

four Domains. The risks were broad, generic and lacked underlying detail. In February 2019,

a revised draft Risk Register – ACT Health System Wide Data Review Implementation Plan

(revised draft Risk Register) was prepared. The revised draft risk register is more

comprehensive and contains information on risk sources and consequences, risk treatments,

risk owners and risk status. The revised draft risk register identifies ten risks related to 12

activities across four System‐Wide Data Review Domains. However, the revised draft risk

register is not tailored as a strategic or program level risk assessment and does not identify

risks associated with the management of the System‐Wide Data Review program as a whole.

RECOMMENDATION 1 SYSTEM‐WIDE DATA REVIEW PROGRAM GOVERNANCE


a) document the governance arrangements for the System‐Wide Data Review

implementation program, including roles and responsibilities for program management

and oversight;

b) embed oversight and monitoring arrangements for the program, by finalising and

endorsing terms of reference for the System‐Wide Data Review Reference Group and

supporting Working Groups and ensuring these groups meet regularly to review

implementation progress; and

c) develop and implement risk management arrangements that assess and monitor risks

to the implementation of the program.


3 PLANNING FOR THE SYSTEM‐WIDE DATA

REVIEW IMPLEMENTATION PROGRAM

3.1 This chapter examines whether the ACT Health Directorate has effectively planned for the

System‐Wide Data Review implementation program. This includes consideration of

whether there are effective implementation plans in place, as well as arrangements for the

ongoing monitoring and review of the implementation of the program.

Summary

Conclusions

Planning for the implementation of the System‐Wide Data Review program is not effective. There

is neither transparency nor clarity about how and when the three‐year program will be delivered

and if the program will be completed as envisaged in the Leading Data Reform: The Way Forward

– Outcomes of the ACT Health System‐Wide Data Review.

The ACT Health Directorate has implemented a short‐term planning approach to the program (six

months at a time) and has resourced implementation activities as ‘business as usual’. There is

little information or specificity for program activities planned beyond June 2019 and inadequate

documentation of the critical path(s) of the program and its activities, including key dependencies

and milestones or when activities are estimated to be completed.

Processes for reporting on the progress of implementation are under development and do not yet

provide adequate information on activities’ estimated completion dates, key dependencies

between and across projects, the resources necessary to complete the program or the realisation

of benefits.

Key findings Paragraph

The ACT Health Directorate has published the System‐Wide Data Review

Implementation Plan, Phase One, to December 2018 (Phase One Implementation

Plan) and the Phase Two Implementation Plan March 2019 System‐Wide Data

Review (Phase Two Implementation Plan). These documents list a series of activities

and outputs that the ACT Health Directorate planned to deliver ‘up to

December 2018’ and ‘up to the first half of 2019’ respectively. The two

implementation plans do not clearly explain what the outputs represent in terms of

delivering the overall System‐Wide Data Review implementation program. There is

no explanation about the ACT Health Directorate’s planning arrangements for

delivering the outputs, such as the reasons for working on particular outputs in the

current phase, what business areas are responsible for delivering the outputs and

3.22

3: Planning for the System‐Wide Data Review implementation Program


the resources needed to deliver the outputs. It is not clear from the Phase Two

Implementation Plan how well the implementation program is progressing, if the

implementation program is on schedule, whether there are any impacts to the

broader implementation program associated with the Phase One outputs that are

still underway and if there have been any benefits achieved from the outputs that

have been completed.

The ACT Health Directorate has developed a Microsoft Excel workbook, the SWDR Program, to assist in internal planning for the System‐Wide Data Review Program. The SWDR Program comprises two spreadsheets; Program Overview and Program of Activities. The spreadsheets include information on 95 activities to be implemented as part of the System‐Wide Data Review implementation program; 30 activities are allocated to the System‐Wide Data Review, 38 activities allocated to Phase One, 11 activities to Phase Two and 16 activities to future phases. The spreadsheets contain high‐level information on the activities’ Project Leads, the activities’ key stakeholders, as well as information on the activities’ current status and the percentage of progress to completion. The spreadsheets do not include a timeline beyond June 2019 (except for the designation of some activities to ‘Future Phases’), information on the critical path(s) of the overall System‐Wide Data Review program and its activities, key dependencies and milestones or when activities are estimated to be completed. By not planning for the longer term, i.e. beyond June 2019, the ACT Health Directorate increases the risks of a poorly executed System‐Wide Data Review implementation program and not realising the benefits anticipated from the three‐year program of work.

3.27

The ACT Health Directorate monitors and reports on the implementation of the System‐Wide Data Review program using Project Plan Briefs (a ‘project’ is made up of two or more implementation program activities) and Data Repository Population Project Status Reports. Reporting through Project Plan Briefs commenced in November 2018. In January 2019, there were eleven ‘projects’ reported in eleven corresponding Project Plan Briefs. Reporting through the Project Plan Briefs, including the quality and comprehensiveness of reporting, is under‐developed. Reporting on the implementation of the Data Repository Population Project is influenced by separate governance arrangements for the project and reflects more robust arrangements. The Data Repository Population Project Status Reports include information on the project’s:





3.40

The ACT Health Directorate records and tracks overall progress on the implementation of System‐Wide Data Review activities using the Program Overview and Program of Activities spreadsheets; the spreadsheets are populated using information from the Project Plan Briefs. The current approach integrates subjective estimates of activities’ completion obtained from individual Project Leads and then sums these and divides them by the total number of activities to provide a quantitative estimate of overall program progress. This approach to program

3.49



monitoring treats all projects equally, and does not take into account differences in size, scope or resource demands of the projects. The ACT Health Directorate is not systematically recording at a program level:


key dependencies between and across projects and the implications of slippage to Phase One activities’ progress; or


The resources required for the System‐Wide Data Review implementation program do not have a separate, identifiable funding allocation and associated budget. The System‐Wide Data Review activities are funded from existing resources, that is from ‘business as usual’ (i.e. time, people, budget). Accordingly, the ACT Health Directorate does not routinely track implementation program costs against a program budget. By not planning for program costs and monitoring these against a program budget, the ACT Health Directorate cannot effectively manage future costs or put in place mitigation strategies to manage resource risks that could affect the delivery of the implementation program and its anticipated benefits.

3.55

The ACT Health Directorate has not documented its procedures or processes for the review and evaluation of the System‐Wide Data Review implementation program in any program governance or program planning documents. While the ACT Health Directorate has identified the benefits from the System‐Wide Data Review implementation in Leading Data Reform: The Way Forward – Outcomes of the ACT Health System‐Wide Data Review (August 2018), Project Initiation Document Data Repository Population Project ‐ Phase 1, and the draft Performance, Reporting and Data Management Strategy 2019‐2021, benefits are not regularly recorded or monitored at a program level.

3.66

Program planning

3.2 In reviewing program planning arrangements for the System‐Wide Data Review

implementation program, the Audit Office sought information on whether and how the ACT

Health Directorate had documented its planning for the program, including information on:

the program of activities and outputs, including their relative priorities;

scheduling of activities and outputs, including timelines and milestones; and

critical paths and key dependencies for the program of activities and outputs.

3.3 All of these elements are usually incorporated into a program management plan.

3.4 The Standard for Program Management Fourth Edition (2017) (the Standard) discusses

program management plan development during the program planning phase. The Standard

states:

The program management plan is the document that integrates the program’s subsidiary plans and establishes the management controls and overall plan for integrating and managing the



program’s individual components. These controls measure performance against the program management plan using information that is collected and consolidated from the constituent projects. Its main purpose is to ensure the program is continually aligned with the strategic priorities of the organization in order to deliver the expected benefits. The program management plan is developed based on the organization’s strategic plan, business case, program charter, roadmap, and any other outputs from program formulation.

3.5 There is no program management plan for the System‐Wide Data Review implementation

program. The ACT Health Directorate’s planning for the System‐Wide Data Review

implementation program is embodied in:

publically available ‘six‐month’ implementation plans; and

an internal‐use Microsoft Excel workbook, the SWDR Program, which comprises two

spreadsheets; Program Overview and Program of Activities.

System‐Wide Data Review implementation plans

3.6 The ACT Health Directorate has published two ‘six‐month’ implementation plans:

the Implementation Plan, Phase One, to December 2018 System‐Wide Data Review

(the Phase One Implementation Plan) for activities to December 2018; and

the Phase Two Implementation Plan March 2019 System‐Wide Data Review (Phase

Two Implementation Plan) for ‘ongoing activities commenced during Phase One as

well as activities commencing during the first half of 2019’.

Implementation Plan, Phase One, to December 2018 System‐Wide Data Review

3.7 The ACT Health Directorate published the Phase One Implementation Plan on

21 August 2018 as part of the suite of documents prepared and published with the final

report from the System‐Wide Data Review Leading Data Reform: The Way Forward –

Outcomes of the ACT Health System‐Wide Data Review (refer to paragraph 1.9). The

document is publically available on the ACT Health Directorate’s website.

3.8 According to the Phase One Implementation Plan:

This Implementation Plan represents Phase One of an ongoing program of work over three years. The projects identified in this Plan have been informed by the program of activities identified through the Review, and are scheduled to commence in the period to December 2018.

3.9 The Phase One Implementation Plan also states:

ACT Health will publish updates to this Plan every six months, which will include progress updates on projects underway or completed, and will identify projects scheduled to commence in the coming six‐month period.

3.10 The Phase One Implementation Plan presents information on ‘implementation’ for each of

the domains identified in the System‐Wide Data Review. For each of the domains the

document includes:

the recommendation associated with the domain;



a description of the key findings associated with the domain;

a description of the activities associated with the domain; and

‘outputs up to December 2018’.

3.11 The ‘outputs up to December 2018’ may be described as a series of actions and initiatives.

Depending on how the ‘outputs up to December 2018’ are interpreted up to 48 different

actions and initiatives were identified for delivery up to December 2018. They are a mix of

specific and general actions, some of which may be interpreted as an end in itself, while

others may be interpreted as the commencement of a series of actions. For example, in

relation to Domain D3 – Data quality, two outputs have been identified for December 2018:

Draft coding strategy will be finalised with a supporting implementation plan.

Draft Data Quality Framework and Strategy developed in line with ACT Government Office of the Chief Digital Officer.

3.12 These outputs describe actions to be undertaken by December 2018, but indicate that there

is ongoing further work for their implementation. However, there is no further information

in the document as to actions to be undertaken going forward, either in relation to these

specific actions or other new or additional actions to be undertaken in relation to the

Domain. In this respect, there is no clear pathway or series of actions, with associated

timeframes, for the single activity ‘D3.6.1 Embed the clinical coding strategy’ associated

with Domain D3 – Data quality.

3.13 More generally, the Phase One Implementation Plan does not clearly explain what the

outputs in Phase One represent in terms of delivering the overall System‐Wide Data Review

implementation program; there is no explanation about the ACT Health Directorate’s

planning arrangements for delivering the outputs, such as the reasons for working on these

particular outputs to December 2018, what business areas are responsible for delivering

the outputs and the resources needed to deliver the outputs.

Phase Two Implementation Plan March 2019 System‐Wide Data Review

3.14 The ACT Health Directorate has prepared and published on its website the Phase Two

Implementation Plan in April 2019. The production of this document is late, noting that the

first Phase One Implementation Plan had an end date of December 2018.

3.15 The Phase Two Implementation Plan states:

This Implementation Plan continues on from the System‐Wide Data Review Phase One Implementation Plan and represents the next stage of the three‐year program of work, informed by the activities identified through the Review. The projects identified in this Plan consist of ongoing activities commenced during Phase One as well as activities commencing during the first half of 2019.

3.16 The Phase Two Implementation Plan also states:

Rolling six‐monthly updates to the Implementation Plan will continue to be published, providing status updates for previous phases and outlining the program of work for the next six‐month period.



3.17 The Phase Two Implementation Plan is presented in the same format as the Phase One

Implementation Plan and includes information on:

Phase One outputs that are either completed, commenced or not commenced. The

commenced outputs are described as either ‘for completion June 2019’, ‘for

completion 2019’ or ‘ongoing’. The ‘not commenced’ outputs are described as ‘for

completion June 2019’, ‘for completion 2019’ or ‘for completion 2020’; and

Phase Two outputs that are either commenced or not commenced.

3.18 Depending on how the Phase Two outputs are interpreted up to 19 different new actions

have been identified for Phase Two.

3.19 Similar to previous comments on the Phase One Implementation Plan (refer to paragraphs

3.7 to 3.13), there is no information in the document as to actions to be undertaken going

forward, either in relation to these specific outputs or other new or additional outputs to

be undertaken in relation to the domains.

3.20 The Phase Two Implementation Plan also includes a section on ‘Ongoing External

Governance’ relating to the ACT Health Directorate’s progress in addressing 175

recommendations from previous reviews of health data during 2012‐16.

3.21 It is not clear from the Phase Two Implementation Plan how well the implementation

program is progressing, if the implementation program is on schedule, whether there are

any impacts to the broader implementation program associated with the Phase One

outputs that are still underway and if there have been any benefits achieved from the

outputs that have been completed.

3.22 The ACT Health Directorate has published the System‐Wide Data Review Implementation

Plan, Phase One, to December 2018 (Phase One Implementation Plan) and the Phase Two

Implementation Plan March 2019 System‐Wide Data Review (Phase Two Implementation

Plan). These documents list a series of activities and outputs that the ACT Health Directorate

planned to deliver ‘up to December 2018’ and ‘up to the first half of 2019’ respectively. The

two implementation plans do not clearly explain what the outputs represent in terms of

delivering the overall System‐Wide Data Review implementation program. There is no

explanation about the ACT Health Directorate’s planning arrangements for delivering the

outputs, such as the reasons for working on particular outputs in the current phase, what

business areas are responsible for delivering the outputs and the resources needed to

deliver the outputs. It is not clear from the Phase Two Implementation Plan how well the

implementation program is progressing, if the implementation program is on schedule,

whether there are any impacts to the broader implementation program associated with the

Phase One outputs that are still underway and if there have been any benefits achieved

from the outputs that have been completed.



System‐Wide Data Review Program spreadsheets

3.23 The ACT Health Directorate has developed, for internal‐use, a Microsoft Excel workbook,

the SWDR Program, to assist in planning the System‐Wide Data Review Program. The SWDR

Program comprises two spreadsheets; Program Overview and Program of Activities.

Program Overview spreadsheet

3.24 The Program Overview spreadsheet is an Excel spreadsheet that consolidates information

on activities associated with the System‐Wide Data Review implementation program,

including:

the System‐Wide Data Review Domains and Domain activities (the total number of

activities is shown as 95; the ACT Health Directorate stated that the 95 activities were

identified in Leading Data Reform: The Way Forward – Outcomes of the ACT Health

System‐Wide Data Review (August 2018);

the number of activities allocated to specific implementation phases i.e. the

System‐Wide Data Review Phase has 30 activities, Phase One has 38 activities, Phase

Two has 11 activities and Future Phases has 16 activities; and

the progress of each activity to a percentage of completion and the individual

activity’s status as completed, commenced or not yet commenced.

3.25 The Program Overview spreadsheet does not include dates for when activities are estimated

to be completed, a program timeline, program milestones, the critical paths and key

dependencies at a program level.

Program of Activities spreadsheet

3.26 The Program of Activities spreadsheet provides detail on each of the 95 activities, including

the activity’s owner/lead, key stakeholders, the activity’s percentage towards completion

and its status. As at 25 February 2019, this spreadsheet included a timeline to June 2019

and noted that the ‘Implementation Plan Phase One commenced’ between 20 and

27 August 2018 and the ‘Implementation Plan Phase Two commenced’ between

31 December 2018 and 7 January 2019. It is not clear from this spreadsheet when individual

activities are estimated to be completed. This spreadsheet does not include a timeline for

the three‐year implementation program nor does it include activities’ critical paths, key

dependencies or milestones. The Program Overview and Program of Activities spreadsheets

are discussed further at paragraphs 3.41 to 3.49.

3.27 The ACT Health Directorate has developed a Microsoft Excel workbook, the SWDR Program,

to assist in internal planning for the System‐Wide Data Review Program. The SWDR Program

comprises two spreadsheets; Program Overview and Program of Activities. The

spreadsheets include information on 95 activities to be implemented as part of the

System‐Wide Data Review implementation program; 30 activities are allocated to the

System‐Wide Data Review, 38 activities allocated to Phase One, 11 activities to Phase Two



and 16 activities to future phases. The spreadsheets contain high‐level information on the

activities’ Project Leads, the activities’ key stakeholders, as well as information on the

activities’ current status and the percentage of progress to completion. The spreadsheets

do not include a timeline beyond June 2019 (except for the designation of some activities

to ‘Future Phases’), information on the critical path(s) of the overall System‐Wide Data

Review program and its activities, key dependencies and milestones or when activities are

estimated to be completed. By not planning for the longer term, i.e. beyond June 2019, the

ACT Health Directorate increases the risks of a poorly executed System‐Wide Data Review

implementation program and not realising the benefits anticipated from the three‐year

program of work.

Program monitoring and reporting

3.28 The Audit Office sought information on what arrangements the ACT Health Directorate had

in place for the ongoing monitoring and reporting of progress on the System‐Wide Data

Review implementation, including whether there was:

documented monitoring and reporting processes; and

evidence of regular reporting.

3.29 The ACT Health Directorate has not documented its procedures or processes for monitoring

and reporting on the System‐Wide Data Review implementation program in any program

governance or program planning documents.

3.30 Notwithstanding the lack of documented procedures or processes for monitoring and

reporting on the System‐Wide Data Review implementation program:

the System‐Wide Data Review implementation Program Manager receives monthly

Project Plan Briefs from the Project Leads to inform the Program Overview and

Program Activities spreadsheets; and

the System‐Wide Data Review Reference Group receives an implementation progress

report (the Program Overview and Program Activities spreadsheets) and considers

individual Project Plan Briefs by exception.

Monitoring and reporting documents

3.31 The ACT Health Directorate monitors and reports on the activities and projects of the

System‐Wide Data Review implementation program using:

Project Plan Briefs;

Data Repository Population Project Status Reports; and

the Program Overview and Program of Activities spreadsheets.



Project Plan Briefs

3.32 System‐Wide Data Review implementation program activities are monitored and reported

on using Project Plan Briefs. Project Leads, accountable for their relevant activities and

timeframes for delivering activities, are required to complete the Project Plan Briefs

monthly.

3.33 The Project Plan Briefs include information on:

the current status of the relevant Domain’s activities, grouped as a project;

the activities’ percentage of progress to completion and the project finish date; and

project Risks, Assumptions, Issues and Dependencies analysis.

3.34 The Project Plan Briefs are aggregated together to inform the Program Overview and

Program of Activities spreadsheets.


monthly reporting via the Project Plan Briefs commenced in November 2018 with the

establishment of the System‐Wide Data Review Reference Group (project reporting

did not occur in December 2018 due to resource availability);

there are no individual project plans that inform the Project Plan Briefs, with the

exception of the Data Repository Population Project; and

prior to November 2018, projects’ progress was monitored informally via regular

management meetings within the Commissioning and Performance Division and the

Data Repository Working Group.

3.36 The ACT Health Directorate also advised that the most effective way for the Program

Manager to obtain completed Project Plan Briefs is to schedule time with the Project Lead

to complete the reporting collaboratively.

3.37 In January 2019, eleven project briefs were completed by three Project Leads. Ten projects’

progress were reported in ten Project Plan Briefs, while the Data Repository Population

Project was reported in a different format, through an undated Word document. Eight of

the projects were the responsibility of one Project Lead; this Project Lead also had joint

responsibility with another Project Lead for a ninth project. This Project Lead and one other

Project Lead had individual responsibility for the two remaining projects.



Data Repository Population Project Status Reports

3.38 The Data Repository Population Project is a project in Domain D1 – Data management. Data

Repository Population Project Status Reports are used to report on the Data Repository

Population Project’s progress to the Data Repository Working Group. The Data Repository

Population Project Status Reports included:

a project status summary with an indication of Green, Amber and Red coding against

the project’s time, cost, scope, risk and quality indicators;

progress diagrams for activities presenting the percentage completion and, in some

cases, a status statement;

deliverables within the reporting period;

deliverables for the next reporting period;

key open issues and risks; and

a detailed Gantt chart including individual tasks’ start and finish dates, the percentage

of completion and project milestones.

3.39 The ACT Health Directorate’s reporting on the implementation of the Data Repository

Population Project demonstrates a more rigorous approach to monitoring and reporting on

progress reflecting the more effective project management arrangements associated with

the Data Repository Population Project (refer to paragraphs 2.59 to 2.64).

3.40 The ACT Health Directorate monitors and reports on the implementation of the

System‐Wide Data Review program using Project Plan Briefs (a ‘project’ is made up of two

or more implementation program activities) and Data Repository Population Project Status

Reports. Reporting through Project Plan Briefs commenced in November 2018. In

January 2019, there were eleven ‘projects’ reported in eleven corresponding Project Plan

Briefs. Reporting through the Project Plan Briefs, including the quality and

comprehensiveness of reporting, is under‐developed. Reporting on the implementation of

the Data Repository Population Project is influenced by separate governance arrangements

for the project and reflects more robust arrangements. The Data Repository Population

Project Status Reports include information on the project’s:





System‐Wide Data Review Program spreadsheets

3.41 The ACT Health Directorate tracks the overall progress of the System‐Wide Data Review

implementation program (made up of 95 activities identified by domain in Leading Data

Reform: The Way Forward – Outcomes of the ACT Health System‐Wide Data Review) using



the Program Overview and Program of Activities spreadsheets; the spreadsheets are

populated using information from the Project Plan Briefs and the Data Repository

Population Project Status Reports.

3.42 Table 3‐1 shows the ACT Health Directorate’s assessment of overall program progress from

November 2018 to February 2019 (represented as a percentage). The ACT Health

Directorate advised that overall program progress was calculated by summing the

estimated percentage completion for all activities, divided by 95 (the number of activities).

Table 3‐1 System‐Wide Data Review program of activities progress to completion

Month Program Progress Percentage

November 2018 41.5

January 2019 48.7

February 2019 48.6

Source: ACT Health Directorate Program Overview Microsoft Excel Spreadsheet

3.43 The Program Overview and Program of Activities spreadsheets (dated 25 February 2019)

contained information on the numbers of activities completed, commenced or not yet

commenced by phase up to January 2019. This information is shown in Table 3‐2.

Table 3‐2 Status of System‐Wide Data Review program of activities to January 2019

Phase Activities Completed

Activities Commenced

Activities Not Yet Commenced

Total Activities

System‐Wide Data Review

30 0 0 30

Phase one 7 27 4 38

Phase Two 0 1 10 11

Future Phases 0 0 16 16

Total 37 28 30 95

Source: ACT Health Directorate Program Overview Microsoft Excel Spreadsheet


the Program Overview and Program of Activities spreadsheets are ‘living documents’

that is updated based on information provided through Project Plan Briefs;

the Program Manager’s involvement in monthly reporting, in collaboration with

Project Leads, helps to ensure a ‘consistent, independent assessment’ for progress

calculations; and

activities are prioritised for a six monthly implementation phase in the lead up to the

phase by consulting with key stakeholders such as the Minister for Health and

Wellbeing, ACT Health Directorate senior executives, clinical and business areas in

Canberra Health Services and the Health Care Consumers Association; however, as

Phase Two is only commencing, activity reprioritisation has not occurred to date.



3.45 Furthermore, the ACT Health Directorate advised that in respect of assurance that activities’

progress is measured consistently at a program level, while a large and complex work

program, ‘… it has been led by a small number of people who are across everything and

know what is going on’.

3.46 During the audit, the ACT Health Directorate updated the Program of Activities spreadsheet

to include a timeline to June 2019. While some of the activities have narrative information

about the tasks needed to complete the activities, there are no specific dates recorded as

to when activities will be completed.

3.47 A review of the Program Overview and Program of Activities spreadsheets shows that:

the information in the spreadsheets is essentially a ‘roll up’ of activities to track

individual progress and then, from this, estimate overall program completion;

there are no phases beyond Phase Two (June 2019);

there are no specific dates as to when activities will be completed;

key dependencies between and across projects are not shown and the implications of

activities allocated to Phase One that may have slipped or that have not been

completed as originally planned are unclear; and

there is no indication of the resource implications of individual projects; all projects

are treated the same way for reporting purposes and there is nothing to distinguish

between resource intensive and resource minimal activities.

3.48 With respect to dependencies between activities at a program level, the ACT Health

Directorate stated that:

… based on advice from the ACTHD’s internal risk team in February 2019, the Project RAID [Risks, Assumptions, Issues and Dependencies] analysis section of each project plan brief was expanded to better capture risks, assumptions, issues and dependencies. During the February [2019] reporting round, the Program Manager has spent time with each Project Lead to better capture project‐level dependencies. This information will then be used to inform program‐level dependencies.

3.49 The ACT Health Directorate records and tracks overall progress on the implementation of

System‐Wide Data Review activities using the Program Overview and Program of Activities

spreadsheets; the spreadsheets are populated using information from the Project Plan

Briefs. The current approach integrates subjective estimates of activities’ completion

obtained from individual Project Leads and then sums these and divides them by the total

number of activities to provide a quantitative estimate of overall program progress. This

approach to program monitoring treats all projects equally, and does not take into account

differences in size, scope or resource demands of the projects. The ACT Health Directorate

is not systematically recording at a program level:


key dependencies between and across projects and the implications of slippage to

Phase One activities’ progress; or




Program financial planning and reporting

3.50 The Audit Office sought information on what arrangements the ACT Health Directorate had

in place for the financial management of the System‐Wide Data Review implementation,

including:

monitoring and reporting on implementation program costs; and

resource planning, including budgets, for the implementation program.

3.51 The financial implications of the System‐Wide Data Review implementation program have

not been separately planned for through a separate, identifiable funding allocation and

associated budget. With the exception of the Data Repository Project, all other

System‐Wide Data Review activities are funded from existing resources, that is from

‘business as usual’ (i.e. time, people, budget).

3.52 The ACT Health Directorate advised that a business case was developed for funding of the

initial infrastructure and build of the data repository, a key component of the System‐Wide

Data Review and a major activity of Phase One of the System‐Wide Data Review

implementation program. The ACT Health Directorate advised that the data repository

business case was considered and approved by Cabinet.

3.53 The ACT Health Directorate advised that there is no regular/routine tracking of

implementation costs for the System‐Wide Data Review program as a whole, although this

information could be generated when required.

3.54 By not monitoring and planning program costs the ACT Health Directorate cannot plan a

budget for the implementation program, effectively manage future costs or put in place

mitigation strategies to manage resource risks that could affect the delivery of the

implementation program and its anticipated benefits.

3.55 The resources required for the System‐Wide Data Review implementation program do not

have a separate, identifiable funding allocation and associated budget. The System‐Wide

Data Review activities are funded from existing resources, that is from ‘business as usual’

(i.e. time, people, budget). Accordingly, the ACT Health Directorate does not routinely track

implementation program costs against a program budget. By not planning for program costs

and monitoring these against a program budget, the ACT Health Directorate cannot

effectively manage future costs or put in place mitigation strategies to manage resource

risks that could affect the delivery of the implementation program and its anticipated

benefits.



Program review and evaluation

3.56 The Audit Office sought to understand what arrangements or plans were in place for the

review and evaluation of the System‐Wide Data Review implementation program. In doing

so, the Audit Office sought to understand:

the arrangements for the review and evaluation of the System‐Wide Data Review

implementation program and its activities; and

how the benefits associated with the System‐Wide Data Review implementation

program were identified, monitored and evaluated.

Planning for review and evaluation

3.57 The ACT Health Directorate has not documented its procedures or processes for the review

and evaluation of the System‐Wide Data Review implementation program in any program

governance or program planning documents.

Benefits identification and reporting

3.58 The Standard for Program Management Fourth Edition (2017) (the Standard) identifies

Program Benefits Management as a key element of program management:

The purpose of Program Benefits Management is to focus program stakeholders … on the outcomes and benefits to be provided by the various activities conducted during the program’s duration.

3.59 The Standard identifies the following elements to a Program Benefits Management

approach:

identifying and assessing the value and impact of benefits from the program;

monitoring the interdependencies between the various activities and outputs

associated with the implementation of the program, and how the activities and

outputs are contributing to the overall program’s outcomes;

analysing the potential impact of changes to the program and its implementation on

the expected benefits and outcomes;

aligning the expected benefits with the organisation’s strategic goals and objectives;

and

assigning responsibility and accountability for the realisation of program benefits and

ensuring that benefits can be sustained.



3.60 A documented Program Benefits Management approach has not been developed and

implemented for the System‐Wide Data Review implementation program. Key elements of

a Program Benefits Management approach have not been implemented including:

benefits identification, including a benefits register;

benefits analysis and planning, including a benefits management plan; and

monitoring and reporting on program benefits.

3.61 Notwithstanding the lack of a documented Program Benefits Management approach, the

Leading Data Reform: The Way Forward – Outcomes of the ACT Health System‐Wide Data

Review August 2018 describes benefits associated with the program. Under the section

titled ‘What will the future look like for Canberrans?’ the Leading Data Reform: The Way

Forward – Outcomes of the ACT Health System‐Wide Data Review August 2018 states:

ACT Health’s objective is to transform healthcare outcomes through data, new knowledge and enhanced technology. Health consumers, health professional, researches and the broader health system will benefit from the delivery of the Performance, Reporting and Data Management Strategy.

3.62 The Leading Data Reform: The Way Forward – Outcomes of the ACT Health System‐Wide

Data Review August 2018 identifies a range of high‐level benefits for:

consumers;

health workforce;

ACT health system; and

teaching, training and research.

3.63 The Leading Data Reform: The Way Forward – Outcomes of the ACT Health System‐Wide

Data Review August 2018 also includes information on the ‘impact’ of the activities that

have been identified for each of the nine domains. These ‘impacts’ are phrased in a way

that indicates that they are expected to be positive benefits.

Data Repository Population Project

3.64 The Project Initiation Document Data Repository Population Project ‐ Phase 1 (version 0.8

dated 21 February 201[9]) includes three benefits of:

greater confidence in the reliability and accuracy of ACT Health Data;

greater level of consistency in ACT Health local and national data submissions; and

access to near real‐time service and performance metrics, improving patient

outcomes through timely access to data and information.

3.65 The ACT Health Directorate advised that implementation program benefits would be

considered as part of the implementation evaluation approach.



3.66 The ACT Health Directorate has not documented its procedures or processes for the review

and evaluation of the System‐Wide Data Review implementation program in any program

governance or program planning documents. While the ACT Health Directorate has

identified the benefits from the System‐Wide Data Review implementation in Leading Data

Reform: The Way Forward – Outcomes of the ACT Health System‐Wide Data Review

(August 2018), Project Initiation Document Data Repository Population Project ‐ Phase 1,

and the draft Performance, Reporting and Data Management Strategy 2019‐2021, benefits

are not regularly recorded or monitored at a program level.

RECOMMENDATION 2 SYSTEM‐WIDE DATA REVIEW PROGRAM PLANNING


a) develop and implement program management planning practices that enables:

i) scheduling and monitoring of activities based on estimated completion dates;

ii) a consistent approach to prioritisation of activities and the progress made;

iii) identification of dependencies and key milestones;

iv) tracking and reporting of costs to inform planning for future phases; and

b) include in future published System‐Wide Data Review Implementation Plans

information about the ACT Health Directorate’s program planning arrangements,

overall implementation progress and the realisation of benefits.


Audit reports

These and earlier reports can be obtained from the ACT Audit Office’s website at

http://www.audit.act.gov.au.

Reports Published in 2018‐19

Report No. 04 – 2019



2017‐18 Financial Audits Computer Information Systems

Access Canberra Business Planning and Monitoring

Recognition and Implementation of Obligations under the Human Rights ACT 2004

Report No. 01 – 2019 Total Facilities Management Procurement

Report No. 12 – 2018 2017‐18 Financial Audits – Financial Results and Audit Findings

Report No. 11 – 2018 2017‐18 Financial Audits ‐ Overview

Report No. 10 – 2018 Annual Report 2017‐18

Report No. 09 – 2018 ACT Health’s management of allegation of misconduct and complaints about inappropriate workplace behaviour

Reports Published in 2017‐18

Report No. 08 – 2018 Assembly of rural land west of Canberra

Report No. 07 – 2018 Five ACT public schools’ engagement with Aboriginal and Torres Strait Islander students, families and community

Report No. 06 – 2018 Physical Security

Report No. 05 – 2018 ACT clubs’ community contributions

Report No. 04 – 2018 2016‐17 Financial Audits – Computer Information Systems

Report No. 03 – 2018 Tender for the sale of Block 30 (formerly Block 20) Section 34 Dickson

Report No. 02 – 2018 ACT Government strategic and accountability indicators

Report No. 01 – 2018 Acceptance of Stormwater Assets





2016‐17 Financial Audits – Financial Results and Audit Findings

2016‐17 Financial Audits – Overview

Annual Report 2016‐17

Selected ACT Government agencies’ management of Public Art

management of the system-wide data review … · management of the system‐wide data review...

Documents