model risk and corporate governance of ......corporate governance of models-our story!model...
TRANSCRIPT
MODEL RISK AND CORPORATE GOVERNANCE OF MODELS WITH SAS
Copyr i gh t © 2014 , SAS Ins t i tu t e Inc . A l l r ights reser ve d .
BOAZ GALINSON -VPHEAD OF GROUP CREDIT RISK MODELING &MEASUREMENT
Stockholm User Group Nov. 2015
Boaz Galinson is Head of the Group Credit Risk Modeling and Measurement in the Risk Management Division at Bank Leumi. Mr. Galinson is a VP at Leumi with 19 years of experience in risk management for Leumi and for the Supervision Unit of the Central Bank of Israel.
Boaz leads a team of experts in quantifying and implementing advanced credit risk methodologies in daily credit decisions and credit policy and optimizing risk & Return. During his career, he has also served as a market risk manager
Boaz Galinson
has also served as a market risk manager.
Boaz speaks frequently at professional conferences and risk conferences and lectures in the Executive MBA programs (including Kellogg) in Tel -Aviv University.
He holds MBA degree in Finance and B.Sc degree in Mathematics from TLV University, Israel.
OutlineWhat is a model? What is model risk?
What can go wrong? What should we look at?
Corporate governance of models
What’s next? Mitigating model risk with SASWhat s next? Mitigating model risk with SAS
Wrap up and Q&A
This presentation presents the point of view of its author, which may differ from Leumi’s. It supports an oral presentation and is incomplete without it. The information is supplied in summary form and is therefore not necessarily complete.
Disclaimer
4
What is a model?
A model is an abstraction of reality using assumptionsand past history to foresee future behavior.
Th d l i i t t t l t i t i d i i
5
The model is an important tool to assist in decision-making.
5
Model Risk- The risk of a significant difference arising between the value received from the model and reality.
The more you lean on a model for decision-making, the
What is Model Risk?
higher the cost of error.
Making misguided decisions as a result of a flawed model leads to losses.
6
Model Risk - Examples
7
-no model can be used for large exposures (may be also relevant to LDP).Following the tsunami of regulation, some of the existing models have lost their predictive power.
OutlineWhat is a model? What is model risk?
What can go wrong? What should we look at?
Corporate governance of models
What’s next? Mitigating the model risk with SAS
8
What s next? Mitigating the model risk with SAS
Wrap up and Q&A
What can go wrong? What should we look at?Reasons for actualization of model risk
Human resources:1.Lack of specialist knowledge2.High turnover3. Overloaded
Over conservative/Too lenient
Compromising at development’s stage due to no synchronization of systems and lack of access to data
Assumptions (statistical/business) do not match reality
OfQ f
99
Over-reliance on modelUse of wrong modelQuality of dataIncorrect parameters
Inability to see things from the developer’s point of view.
Model is not appropriate for business needs
Changes: environmental,business, economic, regulatory, strategic
Leaving out a significant variable
Model too complex1. Operational
challenge2. Business intuition
disappears
No validation performed
Lack of data1. No data2. Data exists but not
within bank systems3. Little data
Lifetime of model Identification of
business problem Formulation of
business question
Location of relevant data
Periodic validation
Update model orchange model
documentation
Legend:
10
Building data panel for analysis
Model development
Internal validation
Preliminaryvalidation -impartial
Transfer of model to
production documentationvalidator
developer
both
Model Model inappropriate inappropriate for business for business
needsneeds
Validation not
Incorrectparameters
Identification of business problem Formulation of
business question
Update model orchange model
documentation
staff
performed
Lack of data and data
availability
Location of relevant data
Building data panel for analysis
Model development
Internal validation
impartial
Transfer of model to
production
validation
documentation
OutlineWhat is a model? What is model risk?
What can go wrong? What should we look at?
Corporate governance of models.
What’s next? Mitigating the model risk with SAS.
12
What s next? Mitigating the model risk with SAS.
Wrap up and Q&A
Corporate Governance of Models- Our Story!Model Corporate Governance Committee, headed by the CRO thatmonitors the credit models on the group level and approves significant models for use by business officers.
13
Monitoring the models:
Ensure that the forecasting ability of the model is still good.
If the quality deteriorates, steps need to be taken such as calibrating the model , refreshing its coefficients, updating the data history, rebuilding the model, or alternatively, stopping to use it.
Role of the Corporate Governance of Models CommitteeTo periodically review the group model quality.To approve and monitor the group work plan in models including setting handling priority and resources allocation.To grant approval for transferring significant models for production.To receive reporting for transferring non significant models for production.To discuss and make decisions where there exists disagreement
14
To discuss and make decisions where there exists disagreement between the developer and the validator.To act regarding models that have deteriorated.To report to the BoD on significant models.To examine relevant changes of regulations.
Frequency of discussions – every 2 months at least
OutlineWhat is a model? What is model risk?
What can go wrong? What should we look at?
Corporate governance of models.
What’s next? Mitigating the model risk with SAS.
15
What s next? Mitigating the model risk with SAS.
Wrap up and Q&A
Mitigating the model risk with SASArchitecture Benefits
Automated validation
Enhanced Model Monitoring
Model Monitoring
Model Manager
Dashboard for Models
Automated documentation
Enhancing Corporate governance
Quick Model Deployment
16
Dynamic ABT
SAS VA
SAS Detail Data StoreSAS Detail Data Store--
Credit data mart
Flexibility in model development to
users
Automated Documentation
Single Version of the Truth Single Version of the Truth Synchronized, Consistent, Competence
Revealing new relationships
PD model-MortgagesParent modelלהלן דוגמאApplication modelSecondary modelMortgage UnitResponsibilityno Main status (under development yes/no?)
UpdatedSecondary status (refreshed/ validated/updated)
A-StarModel importance
*The traffic light is set according to a combination of statistical measurements such as ROC and Capture
current example
17
75%/72% *ROC /development /today70%/68%*Capture Response/development/ today
Traffic light from previous committeeCurrent traffic light
Response to examine model quality.SAS Model Monitoring includes many more of these gauges.
Needs to be taken out of production
Needs discussing and updating
Needs monitoring + if A-Star/significant and has deteriorated – needs discussing
The model works well
What’s Next? Model Monitoring - Dashboard
18
What’s Next?SAS VA The ability to view and discover relationships that you hadn’t thought of in advance.
SAS VA provides you with a way to find previously unknown relationships in your data and to spot trends in data using an intuitive user interface.
BIG Data & Credit Risk Models
19
g g y ,capture (on the transaction level for example rather than on the borrower level) including unstructured data (relevant data from recorded conversations with the borrower for example ) moving into data that is outside the organization (Social media data on the borrower for example).
OutlineWhat is a model? What is model risk?
What can go wrong? What should we look at?
Corporate governance of models.
What’s next? Mitigating the model risk with SAS.
20
What s next? Mitigating the model risk with SAS.
Wrap up and Q&A
Wrap upModels are heavily embedded in decision making and processes. Hence, risk has been shifted to models. If the model is flawed, than the cost of error may be huge.
Developing strong corporate governance of models is crucial.
21
Advance tools for models (Dynamic ABT, SAS Model Monitoring & Manager, Dashboards and SAS VA) are essential to develop, validate, monitor, deploy and reduce model risk.
What’s Next?- Incorporating BIG DATA in risk models.
