5 of 5 quality metrics & science by obaid ali
TRANSCRIPT
Saturday, October the 10, 2015
Quality Metrics & Science
Habib University, Karachi
Obaid Ali & Roohi B. ObaidCivil Service Officers at Drugs Regulatory Authority of Pakistan
1 Math, Statistics and Computer in Quality Science
2Cultural Scores and Site Performance for key Quality Metrics
5 Quality is everyone's responsibility
6Closing words - Simple, clear and loud
3 Essential Attributes for Culture of Quality
4Roots and Principle of Quality Culture
Where everyone willingly takes responsibility for patient protection and
quality of product
Where patient is centre of everything
Econometrics
Pharmacometrics
Chemometrics
Science
Rational
Data Integrity – Deviant Behaviors
Not recording activities
contemporaneously
Backdating Fabricating data
Copying existing data as
new data
Re-running samples
Discarding data
Alicia M. Mozzachio, July 2014, FDLI, Washington DC
Trends help indeed to understand and predict
Lets have a visit to some slides of US-FDA
Time
Proc
ess P
aram
eter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
today
Uncertainty
RISK: For a given severity of risk event, what are the chances (probability) of exceeding the USL in the next period of time?
G. Claycamp, FDA, Sept. 2005
Tomorrow ?
Proc
ess P
aram
eter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
Uncertainty
Tomorrow ?
G. Claycamp, FDA, Sept. 2005
RISK: Control options are scenarios for risk management. Note that this scenario shows the best estimate is below the USL.Time today
Time
Proc
ess P
aram
eter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
today
Uncertainty
Take a cut at a moment in time: Risk has a distribution.
Tomorrow ?
G. Claycamp, FDA, Sept. 2005
Quality Metrics
Indicator of operational reliability Quality Culture
Quality Metrics
Lot Acceptanc
e Rate Product Quality
Complaint Rate
Right first time Rate
Invalidated OOS Rate
APR or PQR on
time Rate
CAPA Rate
Ability to initiate
voluntary CAPA
Lots rejected/total lots in a year
Lot Acceptance Rate
Lot Acceptance Rate
4/355=
Deviated lots/ total lots in a year
Right First Time Rate
38/355=
Right First Time Rate
APR triggered CAPA/ total number of APRs
Corrective & Preventive Action (CAPA) Rate
Corrective & Preventive Action (CAPA) Rate
3/30=
Process capability or performance index for each critical quality attribute that
triggers a CAPA
Ability to initiate voluntary CAPA
Ability to initiate voluntary CAPA
Cp value
The number of batches for which complaint received/ total number of
batches (very specific to the product)
Product Quality Complaint Rate
Product Quality Complaint Rate
6/30=
Invalidated initial OOS tests / total number of tests in a year
Invalidated OOS Rate
Invalidated OOS Rate
200/9900=
The number of APRS within 30 days of annual due date / the total number of products
produced during the year
APR or PQR on time Rate
APR or PQR on time Rate
05/40=
Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Stronger culture is correlated with fewer action limit excursions in steriles
1/4 ISPE
Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Stronger culture scores are associated with higher lot acceptance
2/4 ISPE
Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Sites with higher quality culture scores have lower deviations recurrence
3/4 ISPE
Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Sites with higher quality culture scores have higher CAPA effectiveness
4/4 ISPE
Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Leadership Emphasis
Message Credibility
Peer Involvement
Employee Ownership
Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Maintaining a leadership emphasis on quality
What they say & What they do
1/4
Even when executives have the best intentions, there are often gaps b/w
What they say & What they doAs a result, employees get mixed messages about
whether quality is truly important. Company leaders must first buy into quality improvement initiatives and clearly demonstrate their own personal commitment to this effort
to employees
Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Ensuring Message Credibility
One Message
2/4
Quality messaging should be tailored in such a way that it resonates with every employee.
Different messages may need to be developed to resonate appropriately with employees at
different sites.
The key is to regularly test these messages with employees, solicit their feedback, and refresh
these messages over time if necessary
Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Encouraging Peer Involvement
Participation
3/4
A variety of different tactics can be used to create positive social pressure that encourages employees to participate in (& even generate) quality initiatives. These can include friendly
“quality competitions”.The key is to test these tactics and solicit
feedback to see which are most effective with employees.
Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Increasing Ownership & Empowerment
Engage in Decisions
4/4
Employees should be free to apply their own judgment to situations that fall outside the
common directive. Providing the right level of guidance is crucial to this effort
but they must know how to act when they encounter specific quality issues.
Visible Signals of Quality Culture
Right tone at the top
Focus on the right value
Constant display of putting patient
above everything
Constant display of concern for quality
Consistently walks the talk
With great people skill
& strong value
More than financial rewards
Roots of Strong Quality Culture
Introduction of Improvement & Innovation for quality driven productivity
Adaptation & Investment in Technology to ensure state of control over system
Promotion of change & continuous improvement in Quality System
Roots of Strong Quality Culture
Prompt adaptation of best practice, low staff rotation
Proactive & transparent behavior
Strong relationship based on dialogue, science & mutual respect
Principle of Quality Culture
Product & Process Understanding Sensible Business Management
Sufficient depth of science to explain why deviations have not impacted on quality of
product
Too fast growth & operating at limit of capacity triggers often
issues & non-compliance
How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Small problems ultimately snowball into larger ones, and management becomes aware only when there is a crisis
How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Nearly always reactive, but there is willingness to change. Patchwork corrections are the norm
How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
More proactive. Increasingly detects emerging adverse trends, surfaces major issues, and makes lasting manufacturing and
system improvements
How mature is your quality system
1 2 3 4Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Routinely acts preventively as described in level 3. Fully institutionalizes and reinforces (rewards) a vigilant culture that makes meaningful manufacturing and systemic improvements
Systems Thinking
System is the product of interacting parts
Improving the parts taken separately will not improve the system
Quality is everyone’s responsibility
Selected Quotes, W. Edwards Deming
Lack of knowledge …
that is the problem
If you do not know how to ask
the right question, you
discover nothing
If you can’t describe what you are doing as a process, you don’t know what you are
doing
Rational behavior requires theory.
Reactive behavior requires only reflex
action
Whenever there is fear, you will
get wrong figures
Learning is not compulsory … neither is survival
Strong Quality Culture Efficient Quality System Consistent manufacturing
It’s the culture that decides the path
Poor Quality Culture Unreliable Quality System Defects and Recall
Soft gelatin capsule Paracetamol tablet
Pyrimethamine tragedy Metronidazole label
Dis
inte
grat
ion
failu
re
Dis
solu
tion
failu
re
Mat
eria
l mix
up
Labe
l Mix
up
One of the important Hormonal Product
Contract manufacturing
Changed in color
Stability was not done
Quality Agreement
Batch of 1 million tablets 22 batches Underweight
Caught after 2 weeks
System ???or culture ???
Antibiotic & steroid cream
Analytical method
unable to quantify
Manipulated method used
for years
Ended up with
withdrawal of product
from market
Lets Look Around …
09 Oct, 2015, Tab Paracetamol - USA
Tablet Paracetamol
Wrongly labeled
325 mg instead of 500 mg
175 mg additional than claimed
What do you think what should be the action?
End up with recall
Aug 2015Puerto Rico &
Taiwan
Wood fragments found in
toothpaste
3.9 M tubes recalled
Attix Canada April 2015
Attix Canada April 2015
Health Canada …. FDA closed an API
repackager in Toronto
recall hundreds of bulk APIs … cross-
contaminated by penicillin
failed to use separate facilities, or
equipment like hoods and air handlers
cross-contamination … employees move
freely b/w the different areas
Attix stance: We have carefully cleaned
everything before use
Regulatory Stance: Cleaning is not a
substitute for segregating products
Cross-contamination with sensitizing agents
can initiate life-threatening allergic
reactions
No safe level of penicillin
contamination has been determined to be
a tolerable risk
Attix Canada April 2015
Philippines-Aug 2014
GMP non-compliance in
ItalyTetrahydrozoli
ne HCl RM
Used in manufacturing of ophthalmic
solution in China
Phillippinesinstructed
discontinuation of
distribution, sale & use
Many lots recalled
Do you calculate a cost of poor quality
at your site?
ISPE/PDA Survey Sep 2011
Have you evaluated the cost of improving quality
vs. the cost of failure (recalls, rejections, low yield, downtime etc)?
ISPE/PDA Survey Sep 2011
Lets make it simple, clear & loud
Me, my kids & parents use same
drug which I manufacture
No batch has ever failed
Drug manufactured in non-cGMPenvironment
It does not mean that there is necessarily something wrong with the drug
But “intention in question”
Please remember
Lack of adverse event reports, or “something is wrong” is NOT the same
as evidence of no harm….
Closing words
Innocence
• Quality mainly outsourced to regulators
Awareness
• Quality unit will identify the issue
Understanding
• Quality system & Metrics reveals reality & drives action
Competence
• Quality mind set in all functions. Design quality in & anticipate issues
Excellence
• Competitive advantage through prevention of quality issues
Journey of uncertainty to certainty matters
Link
age Science &
compliance
Cha
lleng
e Traditional validation & continuous verification Mai
ntai
n Process robustness & drives reliability
Product & Process Criticality
Culture of Quality beyond complianceNew technology
Even a great quality system will fail, if the
Quality Cultureis not equal standard
The question isn’t whether we will have standards …
The question is how useful can we make standards