Improving Access to Psychological Therapies (IAPT)“A practical guide to the data set” Thursday 3rd May 2012

Welcome and Introductions

Netta HollingsProgramme Manager for Mental Health and Community

Introduction

• Team introductions

• Today is a practical ‘end to end’ consideration of data set processes

• Data set mandated from 1st April 2012 – data collection is hopefully underway!

• First submission window opens 7th May (for Aprils data) and closes on 25th May Window opens next Monday ! Today is the opportunity to gain more

understanding about what is required, ask questions and go away ready to start making submissions.

The ‘end to end’ process

Analysis and reporting

Data quality and other feedback reports made available

Data validation and processing

Data extracts downloaded

Data collected in local system

Data extracted and validated

Data uploaded to IAPT Intermediate database (IDB)

IDB uploaded to Bureau Service Portal (BSP)

Current status• System has been available for voluntary submission since

December 2011

• 30+ sites have made a submission in readiness for mandation

• Pre deadline processing outputs are available

• Post deadline will not be available when submission window closes on 25th May. – Assurance work still to complete– Updates will be given as the development work

progresses– Continue to submit as normal and make use of data

quality reports and pre deadline extracts to refine processes

The Bureau Service Portal

Alan Scott and Gary Sargent NHS Connecting for Health Systems and Service Delivery

Open Exeter web pageThe Open Exeter page where all the documents livehttp://www.connectingforhealth.nhs.uk/systemsandservices/ssd/prodserv/vaprodopenexe

Open Exeter web page

The Open Exeter page where all the documents livehttp://www.connectingforhealth.nhs.uk/systemsandservices/ssd/prodserv/vaprodopenexe

Caldicott Guardian Form

The Caldicott Guardian form which all sites will need to have in place before we can allocate users to the systemhttp://www.connectingforhealth.nhs.uk/systemsandservices/ssd/prodserv/caldicottcert.pdf

Data User Certificate

The DUC form (Data User Certificate) which the Caldicott Guardian signs to allow accesshttp://www.connectingforhealth.nhs.uk/systemsandservices/ssd/prodserv/iaptduc.pdf

The Bureau Service Portal

https://nww.openexeter.nhs.uk/nhsia/index.jsp

Building a Submission

Tim HoldsworthInformation Products Testing Manager

The IDB contains the IAPT data set

• IAPT data set submitted to the Bureau Service Portal (BSP) as the IAPT Intermediate database (IDB):– A Microsoft Access database– No alternative method for submission exists– Contains 4 empty data tables - no additional functionality

such as a data entry interface

• Defined validation rules on submission leading to warnings or failures.

• Requires skills in data manipulation & an understanding of Microsoft Access to populate the database in the required manner.

• Request the database from the Health and Social Care Information Centre Contact Centre at enquiries@ic.nhs.uk or via the BSP following registration.

The Intermediate database (IDB)

• Support for all versions of Access since 2002.• Tables: Person, Referral, Appointment &

Disability. • Required / mandatory / optional fields.• Linkage:

– LPTID links tables– Service Request ID links referrals to activity

(distinct from MHMDS)• Defined structure which cannot be changed –

removal of tables / amending structure will lead to a rejection.

Document

Content of the IDB

• Inclusion rules:– Referrals either opened in the RP or already open

at the start of the RP– Referrals closed before the RP with a follow-up

appointment (type ‘06’)– Appointments linked to those referrals– Patients associated with those referrals– Disabilities associated with those patients

• Referrals don’t necessarily have appointments & not all patients have disabilities.

For more details see ‘IAPT IDB data inclusion rules’ document available at www.ic.nhs.uk/iapt.

RP = Reporting Period

Data collation & submission

• Local data collection & validation (invest):– NHS management management– Postcode management– Mapping to secondary use standard

• Data extraction from local system(s) for the required RP (month).

• Import into the IDB in the required format.• Submit to the Bureau Service Portal (BSP).

RP = Reporting Period

Data validation

• Undertaken on each submission leading to either:– A failure message: indicating a significant issue(s)

has been identified causing the file to be rejected. Issue needs to be rectified & the full data set resubmitted.

– Warning messages: indicate issues have been identified that won’t stop the file processing. Where possible/necessary these issues should be rectified & the data set resubmitted.

More detail on the validations is available in the ‘Summary of validations undertaken upon receipt of data set by Bureau Service Portal’ document available at www.ic.nhs.uk/iapt.

Example validations

Warnings

• NHS number status indicator code is null when a NHS number is provided.

• General medical practice code is not a valid ODS code.

Failures• Local patient ID is null in any of the 4 IDB tables.• Any date is not in a valid format.• NHS number fails Mod 11 check.

Data summary report warnings

Data summary report diagnostics

Data submission scenarios

• Only one single unique organisation (provider/site) code per IDB.

• Each provider/site code used must be associated with its own BSP account.

• Provider/site code & commissioner code are in the person & referral tables respectively.

• Explanation in the User Guidance for making submissions in the case of 1/multiple providers & 1/multiple commissioners. Your choice.

Known Issues

Tim HoldsworthInformation Products Testing Manager

Examples

• Importing local system data into the IDB via Microsoft Excel causes truncation of values – retain in CSV format.

• Unhelpful warnings e.g.– Missing diagnosis when not yet assessed– Missing scores when haven’t yet attended an

appointment• Several system specific issues (e.g. around NHS

numbers & postcodes).

Consult our FAQ document available at: www.ic.nhs.uk/iapt

Adobe Acrobat 7.0 Document

IDB summary

• Only method of submission.• Requires data manipulation/database skills.• Careful transfer of data from local systems – not

via Excel.• Can submit data for more than one RP –

inclusion rules will extract required information.• Warnings are meant to inform

Break

11.40 – 11.55

Stages of Processing

Sarah McDiarmid Senior Information Analyst

Basic submission principles (recap)

• Monthly submissions of data via the BSP.• Submission types:

– Primary (P): current RP & compulsory – Refresh (R): previous RP & optional (last chance) – Primary & Refresh (P & R): 2 files at once (can’t

queue)– Not for Submission (NFS): testing purposes, user

defined RP (not necessarily current or previous)• One Submission Period covers pre-deadline & post-

deadline processing of submissions received during the current Submission Window.

RP = Reporting Period

Pre-deadline processing

• Occurs while the Submission Window is still open & is applied to every IDB (including NFS) uploaded to the BSP. Includes:– On submission validation.– Extraction of data for the chosen upload type in

accordance with submission requirements & data inclusion rules.

– Derivation of additional data items e.g. age at start & end of the RP, PCT of GP practice & postcode district.

– For every submission: a Data Summary Report containing warnings, validation failures, diagnostics & aggregate counts to help providers assess the quality of their submission.

– For every successful submission: flattening/normalising of the IDB into 5 CSV files to produce a provider’s ‘pre-deadline (test) extract’.

Post-deadline processing

• Occurs after the Submission Window has closed - during the final week of the Submission Period.

• Applied to only the ‘last good’ P & R (if submitted) IDBs that have passed validation by the submission deadline. NFS files are not post-deadline processed.

• All data which meets the pre-deadline inclusion criteria will flow to post-deadline (unlike MHMDS).

• At the end of post-deadline processing:

– Providers can download their post-deadline (final) extracts

– Commissioners & the HSCIC can download their extracts

• Extracts contain post-deadline derivations e.g. IAPT record number & First/Last scores.

Outputs from post-deadline processing

Pseudonymised data set

(non identifiable)

Patient identifiable data set

IAPT commissioners

HSCIC

IAPT providers

• IAPT providers only receive data related to the patients they have treated.

• IAPT commissioners only receive data related to the patients for whom they have commissioned a service.

• The HSCIC receives data for all patients seen by all providers.

Provider outputs from pre & post deadline processing

IAPT IDB - mdb•Person table•Referral table•Appointment table•Disability table•(PbR table)

Linked on LPTID

Provider pre-deadline

(test) extract - zip•Header csv•Patient csv•Referrals csv•Appointments csv•Disabilities csv

Linked on LPTID &

IAPT record no

Provider post-deadline

(final) extract - zip•Header csv•Patient csv•Referrals csv•Appointments csv•Disabilities csv

Linked on LPTID &

IAPT record no

Data Summary Report - txt

•Summary•Warnings•Validation failures•Diagnostics•Aggregate counts

Pre

-dea

dli

ne

pro

cess

ing

Po

st-d

ead

lin

e p

roce

ssin

g

Pre & post deadline derivations

Pre-deadline processing• Various derivations utilising data only present within

the current submission & reference data e.g. age at referral received date.

Post-deadline processing• Various derivations utilising data from one or more

submissions (across RPs) e.g. – GAD first and last scores.– IAPT person ID (used to track patients across providers &

submissions). Not included in provider extracts as LPTID & Person ID cannot appear together.

04/06/12

Submission window opens

22/06/12

Pre-deadline processing Providers receive a pre-deadline (test) extract &

data summary report for every successful primary & refresh submission

Submission window closes

Post-deadline processing

June Submission Period• Month 2 (May '12) Primary (P) submissions• Month 1 (April '12) Refresh (R) submissions

~ 1 week

Post-deadline (final) extracts are available

Providers, commissioners & the HSCIC receive their primary & refresh post-

deadline (final) extracts for the submission period

New Submission

window opens

02/07/12

Submission & processing timescales

Reporting period Window opens

Window closes

Closed for processing w/c

Processed data set available

April (P) 07/05/12 25/05/12 28/05/12 04/06/12

May (P), April (R) 04/06/12 22/06/12 25/06/12 02/07/12

June (P), May (R) 02/07/12 27/07/12 30/07/12 06/08/12

• Window opens on a Monday for 3-4 weeks.• Window closes on a Friday for a week of post-deadline

processing.• Post-deadline extracts available the following Monday &

the same day the new window opens.

Question and Answers

Website: http://www.ic.nhs.uk/iapt

Queries: enquiries@ic.nhs.ukPlease include IAPT in the subject line

Call us: 0845 300 6016

Lunch

13.00 – 13.45

Bureau Service Portal Demonstration

Navin Bose and Matthew ParkinsonInformation Analyst & Higher Information Analyst

Navigation

• Login screen

• IAPT home page– Submission Period details

– Days left until the submission deadline

– IDB template

• Upload screen

• Submission history screen

• Post-deadline extracts screen

• Known issues with the screens

Homepage

Fixed period processing

• Data is automatically processed for a fixed period of time e.g. an IDB submitted as a primary file in May will be processed as April data even if the data relates to another RP.

• Submissions cannot be made for the current Submission Period after the submission window has closed.

• 2 opportunities (months) to submit data for a RP – primary & refresh.

Upload screen - overview

• Submit & assess the quality of your IDB. • Shows the status of all IDB files that have been

uploaded in the current Submission Window.• All IDBs assigned unique BSP ID – 2 IDs are

assigned for combined P & R files.• Can download the Data Summary Reports (as text

files) for all submissions in the current Submission Period.

• Can download pre-deadline (test) extracts for only for your latest good P and R submissions in the current Submission Period (replacement).

• Empties when post-deadline processing is run.

Upload screen - process

1. Click on upload button from IAPT homepage to enter screen.

2. Select ‘submission upload type’ (primary, refresh, primary & refresh or NFS) & browse to select IDB to be uploaded.

3. Monitor progress of submission in the ‘processing status’ & ‘output status’ fields:

– Processed & complete with warnings (check warnings in DS report)

– Failed (view Validation Failures in DS report)

– Failed & referred to support (review file & re-submit. If occurs again then contact helpdesk).

4. Email notification when the file has finished processing. If successful the submission will turn yellow indicating ‘last good (P or R) file’.

5. Download extract & view Data Summary Report to check quality/expectations, rectify any issues & re-submit if required.

Upload screen – upload type

Upload screen – NFS upload

Upload screen – select IDB

Upload screen - processing status

Upload screen – DS report

Upload screen – download extract

Upload screen – download extract

Tips on submitting

• Making that deadline...– Mix-ups: choose submission type carefully

– NFS: avoid for the current RPs & near the deadline

– Combined P & R files: preferable as can’t queue submissions

– Pace submissions over submission period– Commitment to process any file you submit before midnight

☺

• General tips / note:– Close IDB before submitting – ZIPs are better – Keep a copy: IDBs (to re-submit), BSP IDs & extracts.– Mergers: avoid duplication of data, keep Exeter informed

about your plans.

MHMDS submissions during the Q3P/Q2R Submission Period

0

10

20

30

40

50

60

7008

-Feb

09-F

eb10

-Feb

11-F

eb12

-Feb

13-F

eb14

-Feb

15-F

eb16

-Feb

17-F

eb18

-Feb

19-F

eb20

-Feb

21-F

eb22

-Feb

23-F

eb24

-Feb

25-F

eb26

-Feb

27-F

eb28

-Feb

29-F

eb01

-Mar

02-M

ar03

-Mar

04-M

ar05

-Mar

06-M

ar07

-Mar

08-M

ar09

-Mar

Num

ber o

f sub

mis

sion

s

Not for Submission Primary Primary and Refresh Refresh

Week 3Week 1 Week 2 Week 4 Week 5

MHMDS receipt of ‘last good files’ during the Q3P/Q2R Submission Period

0

2

4

6

8

10

12

1408

-Feb

09-F

eb10

-Feb

11-F

eb12

-Feb

13-F

eb14

-Feb

15-F

eb16

-Feb

17-F

eb18

-Feb

19-F

eb20

-Feb

21-F

eb22

-Feb

23-F

eb24

-Feb

25-F

eb26

-Feb

27-F

eb28

-Feb

29-F

eb01

-Mar

02-M

ar03

-Mar

04-M

ar05

-Mar

06-M

ar07

-Mar

08-M

ar09

-Mar

Num

ber o

f las

t goo

d fil

es

Primary Refresh

'Good' primary files = 65'Good' refresh files = 57

After successful processing…

• View the ‘Summary Report’:– Helps you to assess the quality of your submission.– Does the ‘Submission Type’ reflect the intended month

(primary / refresh)?– Is the ‘Total Record Count’ reasonable?– Review the diagnostics e.g.

• % of valid postcodes. Direct impact on data quality for your organisation.

• % of valid commissioner codes. Direct impact on flow of data to commissioners.

Submission history screen

• Shows your last good submissions from previous Submission Periods & their Data Summary Reports.

• Does not store your pre-deadline extracts for these submissions so ensure you download these while you can.

• Your latest last good file(s) (from the Upload screen) will appear following post-deadline processing.

• Post-deadline record count = number of records in your post-deadline patient CSV = number of records in your pre-deadline patient CSV.

• Known issues around field names & population of the downloaded by/on fields – scheduled to be fixed soon.

Submission history screen

Post-deadline extracts screenPost-deadline extracts screen

• Go here for your final P and R extracts. • Stores extracts for the previous 3 RPs only so

preferably download & store extracts as they appear.

June (P)May (R)May (P)April (R)April (P)

May August(open window)

Extracts stored in the Post Deadline Extracts screen

June July

Post-deadline extracts screen

IAPT Reporting

Dave Cracknell and Jo Simpson

Our approach

• Code of Practice for Official Statistics• Developmental• Transparency• Collaborative• Publish and polish…

Improving Access to PsychologicalTherapies (IAPT) Data Quality

• Will be based on the extract that the HSCIC receives from Connecting for Health

• Will be produced on both Provisional and Final data

• Will consist of a data quality Excel workbook at both England and submitter level

IAPT Data Quality cont…

• Excel workbook will consist of VODIM report:Valid

Other

Default

Invalid

Missing

• For certain selected items:Person gender, Ethnicity, Source of referral, Appointment type,

PHQ-9 and others

IAPT Data Quality cont…

Will be similar to the trust level data quality produced from MHMDS:

www.ic.nhs.uk/pubs/mhmds

IAPT data quality cont...

Example VODIM rules:

DQ Measure Description Data Item(s) Denominator Valid Other Default Invalid Missing

Ethnic Category The Patient's Ethnic Category is tested against the Ethnic Category code list2. (Only the first character is tested for the Valid and Other categories).

Ethnicity All person records

Records where the first character of the Patient's Ethnic Category equals A-H, J-N, P, R or S.

Records where the first character of the Patient's Ethnic Category equals to Z.

Records with a Patient's Ethnic Category 99.

Records where the first character of the Patient's Ethnic Category does not equal A-H, J-N, P, R, S, Z and where the Patient's Ethnic Category is not equal to 99

Records with no Patient's Ethnic Category

Birth Date 1 Birth Date is tested using the Age at Reporting Period End Date

Age_End_Reporting_Period

All person records

Records where the Age at Reporting Period End Date is between 16 and 120.

Records where the Age at Reporting Period End Date is between 0 and 15 years.

N/A Records where the Age at Reporting Period End Date is greater than 120.

Records with no Age at Reporting Period End Date.

Person Gender Current

Person Gender Current is tested against the Person Gender code list2

Gender All person records

Records with a Person Gender Current code 1 or 2.

Records with a Person Gender Current code 9.

Records with a Person Gender Current code 0.

Records with a Person Gender Current code not 0, 1, 2 or 9

Records with no Person Gender Current code

IAPT Quarterly Reporting

• Department of Health and the HSCIC are working together to produce a specification

• Examples of reports include:o Duration of Treatmento Breakdown of all ‘Attended or Did Not Attend’ codeso Paired Observationo medication status o Paired observation outcome scores

IAPT Quarterly Reporting cont…

Timescale for reporting:

•First set of reports scheduled to be published 1st November 2012 consisting of:

Quarterly reports for April to June Final 2012VODIM June 2012 FinalVODIM July 2012 Provisional

•27th November VODIM reports for July 2012 Final and August 2012 Provisional

•14th December VODIM reports for August Final 2012 and September Provisional 2012

IAPT Quarterly Reporting cont…

IAPT Key Performance Indicators (KPI’s)

•The IAPT KPI Omnibus collection will continue•Collection will be retired subject to successful implementation of the dataset•Latest publication can be found at:

http://www.ic.nhs.uk/pubs/psychologicaltherapies1112

IAPT Annual reporting

• Planned for 2013/14• Will be similar to the MHMDS bulletin:

http://www.ic.nhs.uk/pubs/mhbmhmds11

IAPT PbR – an overview of data flow

Nick BridgesSenior Business Anlayst

Payment by results pilot

• Use same technology and process as ‘normal’ data set

• Compile an IDB – which includes one extra table (‘pbr’)

• Submit via BSP• No validations undertaken on PbR data

– You must ensure it meets requirements• Receive no data quality feedback via the BSP• Receive no PbR data extract via BSP• Amended IDB will be issued to pilot sites (with the

extra 5th table)

Timescales

• Submission begins from opening of first submission window on 7th May

• Collection for 9 months• 33 data items

– Patient experience questionnaire responses– W&SAS scores – Long term conditions

Data processing

• Data items are included in PbR table to allow linkage back to main data set

• Will be appending the pseudonymised patient identifier to the PbR table to allow linkage between PbR table and main data set and so allow use of the full data set during analysis

• Data quality and PbR analysis reports to be developed

Structure of the PbR table

Data linkage Items

1 LPTID Local patient ID2 DOB Date of birth3 POSTCODE Postcode

4 NHSNO NHS Number5 SERVICEID Service request ID6 APPOINTMENT Appointment date

PEQ End of Assessment

7 PBR1 PEQ EOA - Did staff listen to you and treat your concerns seriously?

8 PBR2 PEQ EOA - Do you feel that the service has helped you to better understand your difficulties and start getting the help you need?

9 PBR3 PEQ EOA - Were the appointment arrangements made in a way that is convenient for you?

10 PBR4 PEQ EOA - Were you satisfied with the time you waited for your first contact and this first appointment?

11 PBR5 PEQ EOA - On reflection, do you feel that you will now get the care that matters to you?

12 PBR6 PEQ EOA - Where you given information about options for choosing a treatment that is appropriate for your problems?

13 PBR7 PEQ EOA – Do you have a preference for any of the treatments among the options available?

14 PBR8 PEQ EOA - Have you been offered your preference?

PEQ Mid to End of Treatment

15 PBR9 PEQ MET - Did staff listen to you and treat your concerns seriously

16 PBR10 PEQ MET - Do you feel that the service has helped you to better understand and address your difficulties?

17 PBR11 PEQ MET - Did you feel involved in making choices about your treatment and care?

18 PBR12 PEQ MET - On reflection, did you get the help that mattered to you?

19 PBR13 PEQ MET - Did you have confidence in your therapist and his / her skills and techniques? 

20 PBR14 PEQ MET - Where you given information about options for choosing a treatment appropriate for your problems?

21 PBR15 PEQ MET - Did you have a preference of treatment among the options presented to you?

22 PBR16 PEQ MET - Were you offered your preference?

Work and Social Adjustment Scale

23 PBR17 WASAS - Work24 PBR18 WASAS - Home management25 PBR19 WASAS - Social leisure activities27 PBR21 WASAS - Family and relationships

Long Term Conditions28 PBR22 LTC - Coronary Heart Disease (CHD) 29 PBR23 LTC - Chronic Obstructive Pulmonary Disease (COPD) 30 PBR24 LTC - Non Insulin Dependent Diabetes Mellitus (NIDDM)31 PBR25 LTC -Insulin Dependent Diabetes Mellitus (IDDM) 32 PBR26 LTC - Chronic Muscular Skeletal 33 PBR27 LTC - Other

More discussion later for PbR pilot sites

Next Steps

Nick BridgesSenior Business Anlayst

Next steps

• Consider what has been said today• Do you know what you need to do and how you

need to do it?• Make use of available guidance• Let us know if you still have queries or

uncertainties

Further guidance includes

• User guide

• Validation rules

• Inclusion rules

• ISB documentation

• Data handbook

• Data set readiness checklist

• Documents issued by system suppliers

IAPT Websites

• www.ic.nhs.uk/iapt

• http://www.isb.nhs.uk/documents/isb-1520/amd-29-2011/index_html

• http://www.iapt.nhs.uk/

Readiness checklist…….

• Review the guidance• Register your interest in receiving updates • Communicate data set requirements and the

implementation timeframe to system suppliers• Identify local resources required to produce

monthly extracts, manage data quality issues, prepare the data set for submission, make the submission, resolve any data quality issues and undertake any resubmissions required.

Readiness checklist…….

• Ensure those tasked with the collection, processing and submission of the data set are fully aware of the requirement and their responsibilities.

• Understand that a series of validations are undertaken following the submission of the data set – which can lead to either a failure of the submission or a warning message.

• Apply for an Organisation Code from the Organisation Data Service (ODS) if not already obtained.

Readiness checklist…….

• Apply for N3 connectivity, if this is not already in place• Obtain and gain familiarity with the IAPT intermediate

database. • Register with the Connecting for Health Systems and

Service Delivery Team to gain access to the Bureau Service Portal

Readiness checklist…….

• The data set contains patient identifiable data, such as NHS number, date of birth and postcode. Local processes for data extraction and storage must adhere to all required Information Governance standards to prevent the inappropriate disclosure of identifiable data.

• Understand the data set submission timetable.

Question and Answers

Website: http://www.ic.nhs.uk/iapt

Queries: enquiries@ic.nhs.ukPlease include IAPT in the subject line

Call us: 0845 300 6016

Payment By Results

IC Leeds 3rd May 2012

3rd May 2012

Collecting & Submitting Data

• PC-MIS Late (from 10th April)• No Other Issues?• PEQ Confidentiality

– PC-MIS in next release– IAPTus after 21st May– Others?

• IDB 5th Table• Window Opens 7th May

3rd May 2012

MH Clustering (1)

• Two Needs– Feed into Economic Modelling (Assess if a Correlation

between Cluster & Resources used in IAPT)• Sample, 180 cases each in June & October• Using Full MHCT Only• Data Needs:

– Provider Code– Cluster– High or Low Intensity Treatment– Type of Treatment (e.g. CBT)– IAPT Scale Scores (e.g. PHQ9, GAD7, etc.)

3rd May 2012

MH Clustering (2)

– Validate IAPT Tailored MHCT• Needs to cluster all patients (not a sample)• Some full MHCT, some Tailored MHCT• Data Needs

– Scale Scores– Assigned Cluster– TBD

• Volunteers required

3rd May 2012

Other Data Requirements

• Costing– Staff Costs– Overheads

• Local Data– Local Prevalence– Local Equalities– Local Demographics

• Pricing (Commissioners Contract Prices)• Workshop 31 May to Define