basel ii-12-mar-07

© © Christian Marlier, March 2007Christian Marlier, March 2007

Basel II : Risk Model Validation

Quiet Revolution in the Supervisory World

State Bank of Pakistan

KARACHI, March 12th, 2007Christian Marlier

Senior Director

Head of CRO Executive Office


Basel II : Impact on Authorities Scope of Validation Supervisory & Regulatory Authorities : Impact & Roles Harmonisation

Risk Model Basel II Compliance General principles Organisational Compliance Technical Compliance Implementation Compliance

Supervisory Review : Points of attention Policy & Organisation Models & Process People IT & Data

Conclusion

Agenda

N° N° 33© © Christian Marlier, March 2007Christian Marlier, March 2007

The higher the pressure put on trust The more important transparency and accountability become

“PROVE ME”

“TELL ME”

“TRUST ME”High

Low

Low High

Basel I to Basel II

TR

US

T

TRANSPARENCY

“SHOW ME”

From “Trust Me” to “ Prove & Evidence Me World”


TrainingCapital CalculationPillar I Validation

Co-ordiantion of InformationPlanning & Co-ordination

ofSupervisory activities

Coherency Between

local accounting & tax framework

LiquiditySpreadCapital

BASEL IIImpact on Authorities

Strengthen soundness & stability

of international banking system

Promote Stronger Risk Mgt Practices

MinimumRequired Capital

SupervisoryReview

Market Discipline

Supervisors

Consolidated / Home – Host

Monetary Authority – Central Bank


Basel II impact on authoritiesHome & Foreign View

Home State Bank

Home State Bank

Home Private Bank

Home Private Bank

Foreign Private Bank

Foreign Private Bank

Home Supervisor

Mother Private Bank

Mother Private Bank

Validation

Host / Home

HostSupervisor

LiquiditySpreadCapital

MonetaryAuthorities

CentralBank

SovereignRatings


Consolidating Supervisor / Home and Host Supervisor

• Supervision at the level of each legal entity but effective supervision = at group level (Basel 2 - risk management centralised)

Consolidated supervisor - responsibility at 2 levels: stand alone and consolidated (extra burden for the financial institution)

• Supervisors should act in a coherent way with harmonised regulation but :

Strengthening the role of the consolidated supervisor = political impact & Host supervisors have (legally) different responsibilities:

• National supervisor reports to national political bodies, must act in the interest of the country

• Financial responsibility towards their government (guarantees, etc.) • In case of crisis, the national supervisor will try to limit the impact on his country• Countries where foreign financial institutions play a major role are the most

sensitive to this call for harmonisation


CEBS - guidelines for cooperation between consolidating and host supervisors

• Consolidating supervisor conducts SREP (Supervisory Review and Evaluation Process) for the group as a whole

• Host supervisors conducts this review at the subsidiary level: • perform the SREP at local level,• assess the internal governance at local level, within the framework of the group• review and challenge the bank’s ICAAP (adequate capital), taking into account

the group’s level• review the local plan to close the gaps• and communicate its conclusions to the consolidating supervisor

• Home / Consolidating supervisor approve Application File after intensive review and dialogue and taking into account the feedback from host supervisors.

Consolidating Supervisor / Home and Host Supervisor


Role of Home and Host Supervisor Dialogue “Square Box”

ConsolidatingSupervisor

MotherCompany

SupervisoryCooperation

Act as onecompany

Bank’sSubsidiary

Bank’sSubsidiaryHost

Supervisor

HostSupervisor

College ofsupervisors

Local solvency reporting

CRD local implementation

Current legal requirements

Consolidated Solvency Reporting

Supervisory review

Single entry pointValidation & permission





Conclusion Industry View on “ Supervisors” Expectations

Agenda


Risk Model Basel II Compliance

First Pillar

Art. 349 to 500: Credit Risk - Internal Ratings-Based Approach

Art. 620 to 639: Operational Risk

GeneralPrinciples

GeneralPrinciples

• Ownership of senior management in the structures

• Independence of risk modeller

• Integrity of the risk / rating process

Benchmarking• Comparison to best

practices• Risk measures• Pricing• Process

Backtesting• Ex- Ante & Ex- Post• Model inputs & Model

Result• Stress Testing

Methodology Review• Model Appropriateness

for business use• Robustness• Transparency &

Complexity• Consistency with

existing models

OrganisationalWho ?

ImplementationHow ?

Technical What ?

Basel II Risk Model - Compliance / Validation

Second Pillar

Art. 396 to 399 : Stress Tests

Art. 416 – 418 : Definition of default

Compliance with Minimum Criteria


RiskCommittees

ExternalThird

Parties

Internal &

ExternalAudit

SupervisoryAuthorithies

Basel II

11 2244

33

5566

77

88

Business Lines & CRO

Needs

Risk Model Basel II Validation

Multiple Stakeholders Organisationa

l Compliance

Organisational

Compliance


• Assessing the predictive ability & use• historical experience, forward looking, discriminating power, reassessment when divergence

from expected results,

• Primary responsibility = the bank NOT supervisors

• Iterative process

• changing market and operating conditions

• on-going process

• No single method• statistical tools• other methods (back testing, benchmarking, …internalisation of ECAI’s)• combination of methods

• Qualitative and quantitative• not only a mathematical exercise• must cover structures, procedures, controls, ...

• Subject to independent review• validation ≠ audit

Risk Model Basel II Validation AIG subgroup : 6 principes

Technical Compliance



Types of models Target

• Scorecards (Model for Credit Application) Retail• Pools model Retail• Behaviour models Retail / Micro & Small SME • Econometric / Statistical models Medium Enterprises• Generic / Market Models Public Corporate• Look a like Models Sovereign / Financials• Expert / Holistic Models Corporate• Simulation models Project Finance• Manual Models Start Up

• Basel Probability of default Models (PD)• Exposure at Default (EAD) Models• Loss given Default (LGD) Models

• Build – Internal Models• Re-use – Out of The Shelf Models• Buy – Out of the shelf Models

DATA

Risk Model Basel II Validation Technical Compliance



Create the

development sample

Choose the

rating criteria (variables)

Model

Development

PD calibration

Model validation




Create the

development sample

Choose the


Model

Development

PD calibration

Model validation



Important to maintain coherency between field and modeller : art. 379, 386 - 391

Automatic Model Obligor Rating

(AMOR)

Expert

Overruling

FINAL

OBLIGOR

RATING

( FOR)

Automatic Model Obligor Rating

(AMOR)

Expert

Overruling

FINAL

OBLIGOR

RATING

( FOR)

Rating Anomaly Managers

• Support the analysis of differences between model rating and experts opinion

• Technical issues (balance-sheet / nace /...)• Logical issues (financial analysis)• Undisclosed Credit Event

• Organise, support and tests of new models

• Support and Validate - override


Create the

development sample

Choose the


Model

Development

PD calibration

Model validation



Definition of Default

Basel IIdefault

Recovery default

Timeline

Back to non-default level

Riskclass

End-of- year

Timeline

Performingloans

Pass

SpecialMention

Uncertain

Doubtful

Outstanding RecoveryF ( Product Behaviour)EAD

LGD Recovery< LGD Recovery LGD

PD Recovery> PD Recovery PD

RecoveryBasel IIImpact

Outstanding RecoveryF ( Product Behaviour)EAD

LGD Recovery< LGD Recovery LGD

PD Recovery> PD Recovery PD

RecoveryBasel IIImpact


Create the

development sample

Choose the


Model

Development

PD calibration

Model validation



Definition of Default

Transitions from Recovery level to the the Basel II level


Create the

development sample

Choose the


Generic Model

Build or Buy ?

PD calibration

Model validation

1.1. How was it developed? How was it developed?

2.2. How applicable is it to: How applicable is it to:

a) The specific product?a) The specific product?

c) The local market?c) The local market?

3.3. How can this be proven?How can this be proven?

4.4. What is the quality of documentation?What is the quality of documentation?

5.5. ““Generic” models built on different product dataGeneric” models built on different product data

6.6. “ “Generic” models built on different country dataGeneric” models built on different country data

7.7. New credit bureau “generic models”New credit bureau “generic models”

8.8. How to adapt a generic to local situation ? How to adapt a generic to local situation ?




Model power

measures

Model integrity

Validation samples

Validation over time1.1. Very useful – a generally accepted measureVery useful – a generally accepted measure

2.2. If a validation sample is used, the Gini should be If a validation sample is used, the Gini should be based on this – not the development samplebased on this – not the development sample

3.3. No absolute rules for “satisfactory” model strength No absolute rules for “satisfactory” model strength – but < 25% is poor, > 75% is good!– but < 25% is poor, > 75% is good!

4.4. Can be influenced by the default definition Can be influenced by the default definition

Create the

development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation

ROC or CAP statistics, or ROC or CAP statistics, or GiniGini coefficients. coefficients.

B

ARandom Model

Developed Model

Perfect Model

All Clients

Defaults

Powerstat (P-Stat) = A

A+B




Model power

measures

Model integrity

Validation samples

Validation over timeCreate the

development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



Does the model rank order? Does the model rank order?

Has the model used sensible characteristics? Has the model used sensible characteristics?

Has an appropriate model methodology been Has an appropriate model methodology been used?used?

Has validation down to characteristic Has validation down to characteristic analysis been undertaken – analysis been undertaken – e.g. marginal e.g. marginal chi-squared analysischi-squared analysis? ?

Has it been tested on different good / bad Has it been tested on different good / bad definitions?definitions?

Has it subsequently been used in practice by Has it subsequently been used in practice by the bank?the bank?


Model power

measures

Model integrity

Validation samples


development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



0

50

100

150

200

250

300

350

400

450

500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Rating Master Scale

No of Firms

Rating 1 Moody's Rating 2

Validation with Ram Expert Judgement

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

RAM Judgement

Mid

Cap

Rat

ing

115 files

20 Rating Anomaly Managers (Ram)

Correlation : 80 % with range 65 % and 95 % depending on Ram’s

115 files

20 Rating Anomaly Managers (Ram)

Correlation : 80 % with range 65 % and 95 % depending on Ram’s


Model power

measures

Model integrity

Validation samples


development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



44.8%

9.3%

6.1%

10.9%

8.6%

5.4% 5.0%

2.3% 1.8% 1.9%1.0% 0.9% 0.4% 0.5% 1.2%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Classe de rating

Distribution before Calibration

1.4%2.1%

5.8%

15.2%

22.0%

25.6%

18.5%

4.6%

2.8%

1.2%0.4% 0.4%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

3 4 5 6 7 8 9 10 11 12 13 14

Classe de rating

Distribution after Calibration


Model power

measures

Model integrity

Validation samples


development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



0%

2%

4%

6%

8%

10%

12%

14%

1716151413121110987654321

0%

2%

4%

6%

8%

10%

12%

14%frequency model

% 'defaults'

Masterscale Mapping


Model power

measures

Model integrity

Validation samples


development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



Two types: Two types:

““Hold out samples” – from same pool of Hold out samples” – from same pool of exposures as the development sample – exposures as the development sample – but a random 25%. but a random 25%.

All modelling performed on 75%All modelling performed on 75%

Validation on 25%Validation on 25%

Validation of characteristic analysisValidation of characteristic analysis

Validation on PD calibrationValidation on PD calibration

Validation on model powerValidation on model power

““Out-of-time samples” – from different pool of Out-of-time samples” – from different pool of exposures as the development sampleexposures as the development sample


Model power

measures

Model integrity

Validation samples


development sample

Rating criteria

(variables)

Model

Development

PD calibration

Model validation



A model A model should be re-validatedshould be re-validated each and every each and every month after implementation and reviewed yearlymonth after implementation and reviewed yearly

Similar characteristic analysisSimilar characteristic analysis

1.1. Similar score to PD calibrationSimilar score to PD calibration

2.2. Similar model strengthSimilar model strength

3.3. Soft of Hard review ?Soft of Hard review ?

Distribution of Accounts

0.00000

0.02000

0.04000

0.06000

0.08000

0.10000

0.12000

Score

Score

Pro

po

rtio

n

Recent Sample

Development Sample


Credit Limit

CreditProcess

Pricing &

Performance

Market RiskMonitoring

Key uses

Revenues : Customer Pricing & Performance Analysis

0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

6.0%

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Rating

Spread (% of Exposure)

Operational Risk Profile Risk Awareness

Number of losses

Ave

rag

e lo

ss

Year 1 Year 2 Year 3

Banking 3

Banking 2

Banking 4

Banking 1

Banking 5

Risk Model Basel II Validation Implemantation

Compliance

Implemantation

Compliance

750 MM

500 MM

0AAA AA A BBB BB B CCC CC C

Nominal Amount

Theoretical Risk Curve

Implemented DelegationPower

Credit Delegation Power Market Risk


Stress tests in assessment of Risk model adequacy• Identify possible events or future changes in economic

conditions• Regular perform

Historical Scenario’s

• 1994 Bond Crash• 1997 THB Crisis

• 2001 September 11 Attacks

Hypothetical Scenario’s• Confidence Crisis• Real Estate Crisis

Tail Risk

stress

test

Body Risk

stress

test

Two approaches for stress testing

Event driven

approach

Portfolio

driven

approach

Event

Risk parameters

Portfolio loss

Portfolio loss

Risk parameters

Event

Risk Model Basel II Validation Implemantation

Compliance

Implemantation

Compliance





Conclusion

Agenda


Supervisory ReviewPoints of attention

Policy & Organisation

People

Data & ITModels & Porcesses

Basel II Compliance

Consultants Internalisation Risk Appetite Application file Communication & Guidance …

Scarcity of Ressources Turn Over Management Knowledge Quant Syndrom ...

Outliers Detection ECAI Internalisation Overruling process Involvement of local team Change culture /spreadsheet bank ?

…

Central Documentation Repository Collateral management Reconciliation of Data OR Tight schedule with limited tests…





Conclusion

Agenda


Conclusion

Management empowerment is crucial for validation Model validation not an exact science : a model might assess relative quality of the

counterparty but it cannot capture all elements as it is based on portfolio analysis : this means : on average over time

Expert judgement is of critical importance : for modelling and for communication Data issues center around quantity not quality Regional difference in culture and modelling e.g. equity model versus debt model Use test of critical importance Cherry Picking / Materiality Issues Supervisory Teams : Quant and Process Specialist Interviews on the Spot Benchmarking with internal models or industry sampling ( Securitisation ECAI experience

look-a-like)


Christian Marlier

• Head of CRO Executive Office, Counsellor to Chief Risk Officer, Member of Central Risk Management Team

Former Director of Corporate and Institutional Banking Business Support. Business Support ensures that counterparty risk assessment (reputation, credit, operational and strategic) are timely managed from the inception of a customer to its daily monitoring

Former Director of Credit Risk Management (Analytics, Reporting, Policy & Portfolio Management), Fortis Representative for Working Group On Capital Adequacy for Basel II Implementation (IIF)

Extensive Financial Markets Experience ( Option Chief Trader, Corporate Chief Sales, Global Head of Market Research, Fortis Representative for Market Risk IIF Task Force - Basel I Market Amendment)

Degrees in Applied Mathematics and Business Administration from the Catholic University of Louvain (Belgium) and the University of Western Ontario (Canada).

Registered EC Basel II Expert: Senior lecturer for EC sponsored programs : SME Financing (Europe -ESBG), Czez Basel II (Czech Republic – BBA – National Czech Bank), Basel II & Risk Management (ESBG – China Banking Regulatory Commission)

Senior Lecturer FEBELFIN & International Risk Confererences Publications : Treasury Risk Management 1996; Accounting, financial and fiscal aspects

of Derivatives, 1996 Personal : Fencing’instructor

(32-2) 565.56.00 3, Montagne du Parc B-1000 Brussels, [email protected]

basel ii-12-mar-07

Documents

host supervisor supervisionat

group level basel

role of home

group review

national supervisor

local level

intensive review

high low low high basel