1 tma dq program overview 28 august 2007. 2 dq management control program - purpose to provide an...
TRANSCRIPT
2
DQ Management Control Program - Purpose
• To provide an overview on the Data Quality Management Control (DQMC) Program in the MHS for FY2007 to MHS Data Quality Management personnel
Implementation 1 Dec 00 (FY01)Implementation 1 Dec 00 (FY01)
3
DQ Management Control Program
• Department of Veterans Affairs (VA) FY 2005 Performance and Accountability Report, 15 November 2005
“VA’s ability to accomplish its mission is dependent on thequality of its data. Each day, VA employees use data tomake decisions that affect America’s veterans. In deliveringmedical care, processing benefits, and providing burialservices, data accuracy and reliability are paramount.Each program office has initiated specific actions to improvedata quality to better support business planning andday-to-day decision-making. In addition, the Office of theInspector General (OIG) has conducted audits to determinethe accuracy of our data. We consider OIG reviews tobe independent and objective…describes in detail the actions
each VA administration has taken to improve its data quality.”
http://www.va.gov/budget/Report/2005/AssessmentDataQuality.pdf
4
Overview• This Morning….
External AuditsCurrent Status/Updates POCs and Resources
Program and Metrics
Background
Data Quality Concepts
5
Data Quality Concepts
• Attributes…
Accuracy
CompletenessConsistency
Timeliness
6
Why Worry about Data Quality?One reason is external scrutiny...
IG GAO
AuditAgencies
Congress
7
Problems with Information Technology
• Typically, Data Quality is formulated as an IT problem..– Some of our problems with data quality can be
attributed to problems with Information Technology (IT).
– Examples:• Errors in transmission of data.• Errors in processing data.• Unsynchronized databases.
• But…– The most difficult problems we face with data quality
are not directly attributable to IT, nor readily fixed by IT solutions.
8
Non-Technological Problems
• Just a few examples of non-IT problems causing problems with data quality...– Lack of standardized business rules and
policies. – Inconsistent choices of codes, weights, and
algorithms.– Lack of adequate training and education.– Lack of adequate local data quality assurance.– Failure to set and enforce tough performance
expectations about data quality.
9
Why is this Important?
• Consequences of an incorrect diagnosis or an overly simplistic formulation of the problem…
– Treat only part of the problem.
– Don’t treat serious problems that need fixing.
– Responsibility, accountability, resources in the wrong places.
10
Complex Paradigm for a Complex Problem
Business
Leadership
Human
IT
Business HumanBusiness Human
ITIT
Business
IT
Human
11
Establish Leadership Emphasis
• Extremely important in establishing good data quality…– Data quality must matter to
commanders at all levels.• Establish and enforce performance
expectations.• Must emphasize Data as Fundamental
Business Bookkeeping.
12
Locked in a Vicious Circle
The Data are No Good
The Data are Not Valued
No Investmentin Quality
Poor QualityData Sent
13
Business/IM Relationship
• We need to know our business before we can automate our business…– Program Management Office
• Enrollment & Patient Appointing. • Managerial Cost Accounting.• Uniform Business Office and Third Party Collections.
– Information Management• Defines Integrated Set of Requirements.
– Standardization.• Business Rules and Practices.• Design and Use of Systems and Data.
14
Training and Education
• Our people have to understand both the business and the technology…– Data Quality Training Course.
• Aimed at data quality managers and others.
– Education: Quality data requires more than training data-entry personnel
• MEPRS Application & Data Improvement (MADI)• Working Information Systems to Determine Optimal
Management (WISDOM)• Physicians and Coding.
15
Solutions-Feedback
• To effectively fix data problems, Commanders and users need reasonable feedback…– Metrics: Fast feedback to Commanders
about the quality of their data.– Rapid availability of data for use.
16
Information Technology
– Complete redesign of Corporate Information Systems.
• Synchronization of databases.• Consistent engineering of data
paths from source to central repository.• Quality control built into data feeds.
– Redesign of user interfaces.– Elimination of Human Interfaces between
Systems.
17
DQ Management Control Program - Background
Department of Defense
INSTRUCTION
Regulatory GuidanceDODI 6040.40
Military Health System Data Quality Management Control Procedures
18
DQ Management Control Program Background
• DoDIG identified material management control weakness for MHS - Directed development of data quality assurance and management control program.
• DoD Inspector General report concerning the FY98 retirement liability estimate.
• GAO Medicare subvention demonstration report.
• ASD(HA) concurred with DoDIG material management control weakness findings.
• ASD(HA) designated TMA Resource Management Steering Committee to oversee the development of an MHS DQ Management Control Program.
19
DQ Management Control Program Background (Cont’d)
TMA Executive Director directed formation of a DQ integrated process team (IPT). Development of DQMC involved multiple working groups to include major system representatives. DoD comptroller, DoDIG, and GAO provided oversight in its development. Program has been staffed through the Services with substantial input from the field (Region 11 MTFs). Policy memorandum signed on 29 Nov 00 for implementation on 1 Dec 00. Subsequently updated by policy memorandums dated 9 May 01, 17 Oct 01, 5 Sep 02 and DoDI 6040.40 dated 26 Nov 02.
20
DQ Management Control Components of Program
Data Quality Manager and Data Quality Assurance Team. DQMC Review List - Internal tool to assists MTFs monthly in identifying and correcting financial and clinical workload data problems. This list is prepared by the Data Quality Manager and Data Quality Assurance Team. Commander’s Monthly Data Quality Statement – Specific information from the DQMC Review List that the commander approves for forwarding to Service DQ managers and the TMA Management Control Program Manager.
21
Critical Success Factors
Active Leadership Involvement
Data Quality Managers
Data Reconciliation and Audits
Use of Data/Metrics by the MTFs
Timely Coding/End-Of-Day Processing
Rapid Feedback
File/Table BuildData Base Management
DQ Management Control Components of Program
22
DQ Management Control Components of the Program
• Commander’s Statements are received from Services on the 10th of the month for the preceding month.
• A summary “Barber Pole” along with supporting charts are constructed for briefs to both the Resource Management Steering Committee on a monthly basis and TMA Senior Leadership and Service DSGs Quarterly.
• These charts along with an updated “Hard Spots List” are distributed to the Service DQMC POCs for their monthly meeting at TMA-MC&FS.
• Service and TMA-Wide issues are discussed and documented at these meetings.
• Mr. David Fisher, Director of Management Controls & Financial Studies, Office of the CFO, TMA POC.
23
DQ Management Control ProgramService Metrics FY2007
• The following General Comments, “Barber Pole”, and Charts outline the summarized results of the Data Quality Commander’s Statements submitted by each Service for Mar 07 (Data Month Jan FY2007).
• Metric Standards (colors) are as follows:
• Green - 95-100
• Yellow - 80-94
• Red - 0-79
24
DQ Management Control Program : :How are we doing this year?How are we doing this year?
See attached Excel Worksheets-Below is an exampleSee attached Excel Worksheets-Below is an example
25
Jul Aug Sep Oct Nov Dec Jan Comments: # of Tickets, #1 Issue, #2 Issue, #3 Issue Overall
AHLTA 61.9 55.4 48.6 56.7 50.7 59.1 60.8 2,385 Tickets created in Jan 07; Patient Merge, Encounters, Outages.CHCSI 18.8 15.8 15.8 14.2 13.9 14.1 13.3 1,741 Tickets created in Jan 07; Lab, Sys Admin, and Outages.DMHRSi 26.2 26.6 14.7 22.4 30.6 28.1 25.7 918 Tickets created in Jan 07;Account reset, New accounts, Timecards.M2 3.1 7.2 9.7 11.3 10.3 10.6 13.2 348 Tickets created in Jan 07; Reset, Data Issues, SW Request.TPOCS 47.7 47.9 50 53.5 57.8 64 57.1 126 Tickets created in Jan 07.CCE 99.2 93.1 97.5 95.8 96.5 98.3 98.6 219 Tickets created in Jan 07.EASIV 16.7 58 69.1 42.4 45.4 69 72 257 Tickets created in Jan 07; Application Error, Password Reset, Financial Data
MHS Help Desk Tier 3 Escalation
0102030405060708090
100
Jul Aug Sep Oct Nov Dec Jan
Month 2006 - 2007
Pe
rcen
tag
e
AHLTACHCSIDMHRSiM2TPOCSCCEEASIVB
etter
All CCE Trouble-Tickets go to the Vendor
DQ Management Control ProgramWorking Group Metrics
26
DoDIG Results (Apr 01-Oct 01)
Report Signed 29 Aug 02
DQ Management Control Inspections/External Audits
27
DQ Management Control Program DoDIG General Comments (01)
Overall indifference toward the program.
Lack of communication and timely feedback to identify shortfalls in performance.
Inadequate preparation and training of the DQ Team for completion of the Control Review List.
Lack of monitoring to ensure proper implementation of the program.
28
DQ Management Control Program DoDIG MTF Specific Comments
Responses to the Commander’s Statements and Control Review Lists are unreliable. Need audit/- validation of responses in the Commander’s Statement/Review List.
Lack of audit trail - no supporting documentation.
Lack of accountability.
Lack of training.
Inadequate dedication of resources to data quality.
29
DQ Management Control ProgramConcerns
Prevents DHP and OSD from receiving a “Clean Audit Opinion.”
Puts future DHP funding in jeopardy.
Puts Prospective Payment in jeopardy.
Third Party Collections (TPC) placed in jeopardy.
Jeopardizes remaining Managed Care Support contract BPAs/REAs.
Jeopardizes MTF Accrual Financing Reimbursement.
30
DQ Management Control
Management Plan (Jul 01)
DoDIG (Service Directs) - Memo signed 17 Oct 01
Clarify DQMC Review List Questions. - Service initiative began work on 13 Jul 01 - Completed
Training Web-Based Solution - Enhance current DQ/DQMC Web- Site i.e. FAQs etc. - Completed
Current Quarterly Data Quality Training Class - Completed
Compliance Monitoring (Service IG, IC, Audit Agency, ARS Bridge Confirmation) - Memo signed 17 Oct 01
MTF Visibility and Feedback - Memo signed 17 Oct 01
31
DQ Management Control ProgramManagement Plan (Cont’d)
Leverage Service Data Analysis Capability (Army PASBA etc.) - Working/Discussed at DQMC Meeting
“Low Hanging” Enhancements (e.g. CHCS Workload Accounting Enhancements, Automated Reporting) - Funded
Investigate Coding Improvement initiatives - Working
Continue EAS-IV Implementation Oversight - Working
“Spread the Word” - Brief DQMC Program at Various TMA and Service-level events.
DQMC Program incorporated into a Department of Defense Instruction (DoDI) – 26 Nov 02
33
DQ Management Control Program
Findings CY02
• Unavailability of records (47%).• Specific encounter not found in 9% of the records.• Coded incorrectly, 27%; 70% over coded, 30% under
coded.• Coded correctly, 17%.
Advancemed – Similar results • Availability of records (9%).
Iowa Foundation - 50 sites, 11,254 cases
34
DQ Management Control Program : :
External MHS Targeted Coding AuditExternal MHS Targeted Coding Audit
35
DQ Management Control Program : :
External MHS Coding AuditExternal MHS Coding Audit
36
DQ Management Control Program Modalities 07 Plan
• AHLTA Implementation/Prospective Payment System
• Training (490 at DQ Course FY03 – YTD)•810+ briefed at 2003-2005 MEPRS Confs.•Taught Navy PAD, UBO/UBU Confs.•Mini MADI/M2 Primer Course
• Annual Data Quality Commanders’ Statement Update• AHLTA, CCE, and DMHRSi added• Clarification of Coding Audit Criteria• Clarification of DD Form 2569 (OHI Questionnaire)• Electronic Health Record/Digital Signature• Inpatient Coding Question Expansion• Addition of AHLTA Penetration Question
• Internal Compliance, External Evaluation•Memo to Assistant Secretaries for M&RA (IG/Audit Service)•MHS External Coding Audit Contract
37
DQ Management Control ProgramService Data Quality Managers
Army -Mr. Tim BaconTelephone: (210) 295-8725 DSN [email protected]
Navy -Ms. Jane CunninghamTelephone: (202) 762-0551 DSN [email protected]
USAF -TSgt Jody CallenderTelephone: (703) 681-6504 DSN [email protected]
http://tricare.osd.mil/ocfo/mcfs/dqmcp/metrics_reports.cfm
39
DQ Management Control ProgramBenefits
• Availability of records provide the communications link between providers; continuity of care.
• Record/coding provides evidence of treatment, supports budget, reimbursement, billing.
• Record/coding supports training and education.
• Record/coding facilitates quality assurance processes.
• Record provides the legal defense for patients, providers, MHS.
41
DQ Management Control ProgramHow Can You Help
Brief medical staff on command data.
Executive Steering Committee
Department and Division Heads
Develop Dashboards.
Provide feedback to staff.
Be well-read/knowledgeable in data quality.
Reporting
Analysis
Network and share information.
43
MEPRS Data Quality Tools
TMA MEPRS Program OfficeManagement Control and Financial Studies
Division
MEWACSS2M3
44
Objectives:o Use MEPRS Early Warning and
Control System (MEWACS) to reconcile data and identify data quality issues
o Understand purpose of Six Sigma MEPRS Management Metrics (S2M3) and how to use the tool to gauge discrepancies.
45
MEPRS Early Warning and MEPRS Early Warning and Control System Control System
(MEWACS)(MEWACS)
51
Clicking on the outlier month will take you to MTF
Data Profiles
63
Six Sigma MEPRS Six Sigma MEPRS Management Metrics Management Metrics
(S2M3)(S2M3)
65
DMIS ID MTF NameRx Dispensing
Costs
Available FTE per Daily Occupied
Bed
Ratio of Support Personnel to
Provider FTEs
Rx Workload per Rx FTE
Lab Workload per Lab FTE
Rad Workload per Lab FTE
Inpatient Costs per RWP
Ambulatory Costs per APG
Medical Centers
0108 WILLIAM BEAUMONT AMC-FT. BLISS 1.14 0.50 0.46 1.22 1.54 0.34 1.15 -0.050124 NMC PORTSMOUTH 0.73 0.46 1.26 0.66 -0.32 -0.43 1.01 1.420029 NMC SAN DIEGO 0.04 0.78 0.08 1.29 1.19 -0.33 1.32 0.410125 MADIGAN AMC-FT. LEWIS -0.10 0.57 -0.30 0.71 0.63 0.60 1.13 0.920089 WOMACK AMC-FT. BRAGG 1.10 0.39 -2.31 1.64 0.07 0.90 1.11 1.100109 BROOKE AMC-FT. SAM HOUSTON -0.38 0.88 0.12 -0.05 1.57 0.74 0.22 -0.270067 NNMC BETHESDA 1.21 0.69 0.92 -1.35 0.64 1.52 -1.18 0.080052 TRIPLER AMC-FT SHAFTER 0.80 0.03 -0.04 0.13 -0.22 0.38 -0.83 1.130037 WALTER REED AMC-WASHINGTON DC -0.32 0.35 0.32 -1.60 0.09 1.31 -1.27 -0.840117 59th MED WING-LACKLAND -1.38 -0.11 0.86 -0.83 -0.83 -0.06 0.00 -0.170047 EISENHOWER AMC-FT. GORDON -0.05 0.43 -1.97 -0.71 -1.18 -0.81 -0.11 0.230095 74th MED GRP-WRIGHT-PATTERSON 0.48 -0.93 0.25 -0.46 -1.13 -1.11 -1.02 -1.350073 81st MED GRP-KEESLER -1.38 -1.38 0.29 -0.01 -0.62 -1.84 -0.31 -2.120014 60th MED GRP-TRAVIS -1.88 -2.67 0.07 -0.64 -1.43 -1.20 -1.22 -0.52
Standardized Executive Summary by Peer Group*FY05/FY06 S2M3
Six Sigma MEPRS Management and Control Metrics
*Note:The S2M3 Executive Summary has been standardized to reflect uniform positive/negative Z-score values across all metrics, thus positive Z-scores indicate more desirable values and negative Z-scores are less desirable values. For example, in the Pharmacy Dispensing Cost metric a negative Z-score denotes a dispensing cost that falls below the peer group average. Since in that case a negative Z-score is more desirable than a positive value, the Z-score sign reflected in the S2M3 Executive Summary has been changed to positive. The Z-score signs have been reversed for the following metrics in this Executive Summary:
• Rx Dispensing Costs• Available FTE per Daily Occupied Day• Ratio of Support Personnel to Provider FTEs• Inpatient Costs per RWP• Ambulatory Costs per APG
66
FY05/FY06 Cost of Pharmacy Dispensing: Medical Centers
DMIS ID DMIS ID Name Raw WorkRx $ Less
Supply CostDisp Cost per
ScriptZ Score Statistic Raw Work
Rx $ Less Supply Cost
Dispensing Cost per Script
0067 NNMC BETHESDA 405,284 1,647,269$ 4.06$ -1.50 Better Mean: 603,702 4,731,155$ 7.83$ 0052 TRIPLER AMC-FT SHAFTER 473,295 2,502,350$ 5.29$ -1.01 Median: 570,153 4,820,801$ 8.07$ 0108 WILLIAM BEAUMONT AMC-FT. BLISS 375,291 1,986,596$ 5.29$ -1.01 St. Dev: 190,124 1,928,156$ 2.52$ 0047 EISENHOWER AMC-FT. GORDON 570,153 3,658,725$ 6.42$ -0.560095 74th MED GRP-WRIGHT-PATTERSON 744,925 4,820,801$ 6.47$ -0.54 Value nearest peer group mean0124 NMC PORTSMOUTH 859,223 5,627,524$ 6.55$ -0.51 MTFs within 1 Std. Deviation from the peer group mean0029 NMC SAN DIEGO 772,496 6,231,173$ 8.07$ 0.09 2 Std. Deviations above/below the peer group mean0037 WALTER REED AMC-WASHINGTON DC 536,539 4,392,374$ 8.19$ 0.14 3 Std. Deviations above/below the peer group mean0125 MADIGAN AMC-FT. LEWIS 876,977 7,189,536$ 8.20$ 0.150109 BROOKE AMC-FT. SAM HOUSTON 799,719 6,892,603$ 8.62$ 0.310073 81st MED GRP-KEESLER 334,148 3,721,626$ 11.14$ 1.310117 59th MED WING-LACKLAND 634,692 7,071,504$ 11.14$ 1.310014 60th MED GRP-TRAVIS 465,380 5,762,936$ 12.38$ 1.81 Worse
FY05/FY06 Cost of Pharmacy Dispensing Summary Statistics
67
0
1
2
3
4
5
-3.00 -2.00 -1.00 0.00 1.00 2.00 3.00
St. Deviations from the Mean
MT
F F
req
uen
cy
Dispensing Cost per Script
$-
$2.00
$4.00
$6.00
$8.00
$10.00
$12.00
$14.00
$16.00
$18.00
0067 0052 0108 0047 0095 0124 0029 0037 0125 0109 0073 0117 0014
S2M3 Graph 1
S2M3 Graph 2
69
Questions?
TMA MEPRS Program OfficeManagement Control and Financial Studies
Division
MEWACSS2M3