fall 2013 process monitoring group project
DESCRIPTION
Fall 2013 Process Monitoring project tracking Audit cycle times for a company before and after a process change.TRANSCRIPT
Audit Report Cycle
Time Duration Study
Process Monitoring - Pharmaceuticals 5629 (101) – Fall 2013
Neil Norton, Chuck Kemmerer, Karen Otrupchak
• To identify potential root cause for
Audit to Final Audit Report cycle
time to approach the 42-day limit
• To identify sources of variation
• To determine potential actions to
create fixes - reduce cycle time
Objectives
Process Flow
Edit report adding comment and corrections
requested
Print, sign, and scan report
Post report and observations into Audit tracking
Issue report to supplier
Open audit Template
Document audit from notes and observations
Route report to management/audit team for
review and comment
Inputs (X’s)• Responsibility: Auditing Group/Department • Audit Date: Date the audit started• Lead Auditor: Code letters given for each lead auditor• Audit Team Size: Number of team members • Observations: Numbers of Critical, Major, and Minor
observations found• Supplier Type: Type of facility being audited (API, Commodity,
Device, lab, etc.)• Change Control: Routine vs. non-Routine• Audit Process: “Stage” of the Process. (process
changed/streamlined for Routine Audits April2013 • Audit Duration: Number of days spent at a site• Report Issue Date: Date final report was issued
Output (Y)• Report Cycle Time: Time from Audit start date to Report Issued
(Limits - be less than 42 days)
Inputs and Outputs
Data Collected*
* Data compiled from 105 audits over a duration of 12 months
Audit
Responsibility
Lead
Auditor
Audit
Team
Size
Audit
Date
Audit
Duratio
n
(days)
Report
Issue
Date
Report
Cycle
Time
(Days)
Critical
Obs
Major
Obs
Minor
Obs
Supplier
TYPE
Change
Control
History
Global Audit EU B 1 09/05/12 2 09/21/12 16 0 0 3 API Routine
Global Audit
Americas E 109/11/12
1 10/08/12 27 0 2 4EXC
Routine
Global Audit
Americas E 209/12/12
2 10/03/12 21 0 0 0Lab
Routine
Global Audit EU B 3 09/18/12 1 09/27/12 9 0 6 8 TPM Routine
Global Audit EU D 7 09/19/12 2 10/19/12 30 1 4 4 Device Routine
Global Audit EU D 1 09/20/12 1 09/27/12 7 0 1 0 EXC Routine
Global Audit
Americas A 109/20/12
1 10/25/12 35 0 0 0Device
Routine
Global Audit
Americas E 209/26/12
2 10/19/12 23 0 1 0TPM
Routine
Global Audit
Americas G 309/28/12
2 10/30/12 32 0 4 4TPM
Routine
…
…
…
I-Chart - Report Cycle Times
JMP
Note: Special Test (6) - 4 out of 5 points more than 1 standard deviation from center line (on one side of CL) at points: 25, 27
Dot Plot – Report Cycle Times
Minitab
Lead Auditor
0
10
20
30
40
50
60
70
80
90
100
110
Cou
nt
E A B C D F G H J K L
Lead Audi tor
0
10
20
30
40
50
60
70
80
90
100
Cum
Per
cent
Plots
Pareto Plot
0
5
10
15
20
25
30
Num
ber
of
Aud
its 20
1716
10
26
6 6
1 1 1 1
A B C D E F G H J K L
Lead Audi tor
Bar Chart
JMP
Note: Auditor E conductedthe most number of audits
Lead Auditor (ANOVA)
JMP
0
5
10
15
20
25
30
35
40
45R
epo
rt C
ycle
Tim
e (
Days
)
A B C D E F G HJKL
Lead Audi tor
Rsquare
Adj Rsquare
Root M ean Square Error
Mean of Response
Observations (or Sum Wgts)
0.372156
0.305364
8.098001
21.19048
105
Summary of Fit
Lead Audi tor
Error
C. Total
Source
10
94
104
DF
3653.8941
6164.2964
9818.1905
Sum of
Squares
365.389
65.578
Mean Square
5.5719
F Ratio
<.0001*
Prob > F
Analysis of Variance
A
B
C
D
E
F
G
H
J
K
L
Lev el
20
17
16
10
26
6
6
1
1
1
1
Number
23.6500
22.7059
18.6875
25.8000
13.5769
29.6667
31.5000
12.0000
12.0000
23.0000
42.0000
Mean
1.8108
1.9641
2.0245
2.5608
1.5881
3.3060
3.3060
8.0980
8.0980
8.0980
8.0980
Std Error
20.05
18.81
14.67
20.72
10.42
23.10
24.94
-4.08
-4.08
6.92
25.92
Lower 95%
27.245
26.606
22.707
30.885
16.730
36.231
38.064
28.079
28.079
39.079
58.079
Upper 95%
Std Error uses a pooled estim ate of error variance
Means for Oneway Anova
Oneway Anova
Oneway Analysis of Report Cycle Time (Days) By Lead Auditor
Note: Auditor E had the shortest overall mean turnaround time for report
Bar Chart: Counts of total
observations
Minitab
Total Observations (ANOVA)
Minitab
One-way ANOVA: Report Cycle Time (Days) versus Total Observations
Source DF SS MS F P
Total Observations 14 1246.7 89.1 0.94 0.525
Error 90 8571.5 95.2
Total 104 9818.2
S = 9.759 R-Sq = 12.70% R-Sq(adj) = 0.00%
Change in Process
Open audit Template
Document audit from notes and observations
Route report to management/audit team for
review and comment
Open audit Template
Routine
Audit?
Routine audit report
template
Non-Routine audit
report template
Document audit from notes and observations
Change in Process
Minitab
2-Sample t-test by audit stage
Two-Sample T-Test and CI: Report Cycle Time (Days), Audit Process
Two-sample T for Report Cycle Time (Days)
Audit
Process N Mean StDev SE Mean
1 35 23.5 10.7 1.8
2 40 20.15 9.05 1.4
Difference = mu (1) - mu (2)
Estimate for difference: 3.34
95% CI for difference: (-1.26, 7.93)
T-Test of difference = 0 (vs not =): T-Value = 1.45
P-Value = 0.152 DF = 67
Based on this data, the change in routine audits had no statistically significant effect
Initial IVP: Cycle time vs. Audit
Responsibility
Audit Responsibility- After Process
Change
Minitab
Note: one “outlier” removed fromGlobal Audit Americas
Routine Audit Report Cycle Time
Audit
Responsibilit
y
# of Audits Mean
Cycle Time
(days)
Std. Dev SE Mean
Americas 21 15.71 7.85 1.7
PAA 5 19.20 7.33 2.0
EU 14 27.19 6.34 2.8
Comparison DF P-Value Stat. Sig.
Diff
Americas
vs. EU
29 0.000 YES
Americas
vs. PAA
7 0.328 NO
EU vs. PAA 8 0.050 YES
T-Test Results
Comparison of Data- Audit Responsibility after change
Conclusions
Auditor E has conducted the most audits and has
the lowest average turn-around time
There was no factor that could be linked to high
cycle time
Initial analysis of the change in audit process
didn’t appear to show any statistically significant
change in cycle time
Further analysis showed that certain groups
weren’t following the process change.
There was a statistically significant improvement
in cycle time for those groups using the new form
versus those that weren’t