down the blind alley (pdf)

52
The Future of Risk Management / Where Will Risk Management Go ..? 1 ISSA Interntional Conference Baltimore October 2011 Original title: The Future of Risk Management. This one appeared to be a little bit more alluring, if at all. Note that this presentation is Work In Progress for a major part of the content. Please contribute.

Upload: jurgen-van-der-vlugt

Post on 10-Jul-2015

889 views

Category:

Documents


2 download

DESCRIPTION

The Down the Blind Alley presentation in PDF, with properly formatted front page and notes pages.

TRANSCRIPT

Page 1: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

1 ISSA Interntional Conference Baltimore October 2011

Original title: The Future of Risk Management. This one appeared to be a little bit more

alluring, if at all.

Note that this presentation is Work In Progress for a major part of the content. Please

contribute.

Page 2: Down the Blind Alley (PDF)
Page 3: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

3 ISSA Interntional Conference Baltimore October 2011

You may or may not see the last bullets of this presentation. Nevertheless I hope to convey

some content.:

Ir = Engineer (MSc, IT), drs = Masters (MBA, finance), RE = Chartered IS auditor (comp

CPA), CISA, CRISC I take it are known

With KPMG: IT audit (Windows NT, Year2000)

With ABN AMRO: Global “IT” Audit; relations mgt, auditing programs/projects, and auditing

outsourcing deals (plus some BCM and governance/compliance stuff), but also Information

component in Security (physical, forensics, integrated sec)

With Noordbeek (boutique consultancy): Again, Information risk audit at various clients (size:

small to DoD), focus on control frameworks, governance, some certifications

With Achmea: IT audit and governance reviews. Hey, my job will end per 31/12/2011 so if you

have an opening…

With NOREA (Dutch charter of IS auditors): Professional Practices Committee, Standards

Committee, Professional Education Committee, Working Group Advisory Services Regulation

With ISSA: (Global) Ethics Committee

Speaker at various conferences, author of list of articles, columns on professional practices

and methodology

[ DISCLAIMER: From here on, when I speak of „you‟, I mean „I‟, too.]

Interrupting questions are welcome – although I may defer answering them to later in the

presentation.

[ DISCLAIMER: Any of this presentation does not ncessarily concur with any official opinion

of my employer. Possibly, quite the contrary. Their bad. ]

Page 4: Down the Blind Alley (PDF)

[ DISCLAIMER: I nor any close relatives, friends, colleagues, or business relations, have

business interests within arms‟ length than would benefit from this presentation. ]

The Future of Risk Management / Where Will Risk Management Go ..?

3 ISSA Interntional Conference Baltimore October 2011

Page 5: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

4 ISSA Interntional Conference Baltimore October 2011

Information security is mainly about safeguarding the information assets of an organization.

Those assets are a mainstay of total assets.

As we deal more with data, we‟re in the Operational part of organization-wide risk

management.

But there‟s also part of our work that deals with realization of the value embedded in

information. We don‟t do too much with it, usually, as it would make us enter the realm of

business. We should do more about it! But that‟s not the focus of this presentation.

In this presentation, we deal more with risk management per se.

Page 6: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

5 ISSA Interntional Conference Baltimore October 2011

We don‟t learn from history. If at all, we learn from history that we don‟t learn from history.

In come risk managers that want data on allsorts that has happened in the past. Just like

auditors, on a highway into the future with limited sight away from the rearview mirror.

Like the Greek god Janus, we stand in the present and can look both ways. Back into the

dark, with a grimace. Forward into the future, smiling into the bright light. When you see a

light at the end of a tunnel, it may be a train speeding towards you.

The past didn‟t have a train speeding towards us, as we‟re still kind of alive. That bright light

may be so for others, when we have been run over.

So why do we value historic data so much? Why do we want metrics? Do we learn from

history?

Short answer: We don‟t, and even when we want it, we can‟t…

Oh and fraud has a nasty habit of being of all times, we‟ll never be done. Which is a plus, job

contract wise. But higher-ups may disagree when they see no progress.

With all I say after this, remember to CYA, it‟s the least (and maybe most) you can do. Do the

simple (?) stuff and let the organization regain control over risks, that you can achieve.

Plus hopefully learn from what‟s next and help (me) develop better stuff.

Page 7: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

6 ISSA Interntional Conference Baltimore October 2011

We don‟t learn from history. Or history presents itself as something new every time.

Or we don‟t recognise correctly what history turns up in a new guise this time.

Page 8: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

7 ISSA Interntional Conference Baltimore October 2011

This is what we came up with. Operational Risk Management. Structures, models, processes.

Indicators. Worst of all: „Controls‟… I haven‟t put in all Boards and Meetings that come along

with setting up and running All Of The Above. And I haven‟t even put Governance and

Compliance things in the picture. That would create an even bigger overhead bulge.

Imagine being in the primary organization process. Would you really like to work hard to carry

all the overhead? Would you still show initiative and resource to „help‟ those leaning over your

shoulder from all sides ..?

Page 9: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

8 ISSA Interntional Conference Baltimore October 2011

Which one of the onlookers is you ..?

Although we know the feeling of doing the drilling and not getting anywhere near the root

cause of a problem.

Page 10: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

9 ISSA Interntional Conference Baltimore October 2011

Didn‟t we all meddle along in operational risk management, without a proper framework to

work with ..? Or did we do serious work already?

Anyway, over in Europe, in a pittoresk little town (hardly city…) called Basel, the Bank of

International Settlements (bank of central banks), issued guidance on risk management. After

some bickering over details, it was turned into European law. Other regions moved in the

same direction.

Page 11: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

10 ISSA Interntional Conference Baltimore October 2011

Your name. Oh great.

But SOx didn‟t give too much guidance hence it turned into an auditors‟ bonanza.

[ Disclaimer: I lean more to the Orioles, Blue Jays and Cubs. Yeah, blame the Europeans for

not understanding the game. ‘You’ do well in curling… ]

And I need not mention the many, many other regulations that have been poured out over us.

Page 12: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

11 ISSA Interntional Conference Baltimore October 2011

Guidance is nice, unless it‟s bad guidance or poorly understood guidance or … guidance can go wrong in a number of ways.

In case of Basel:

• Whereas Basel II was intended to remedy major incidents with root causes in operational hiccups, 95% of text was devoted to financial instrument details. Less than 5% was devoted to operational risk management;

• In particular the operational risk part, was intended as guidance (to standard setters);

• The ops risk guidance was flawed in its approach:

•Cause, effects are loosely defined,

•Definitions overlap (no orthagonality in classification),

•Events are defined as 1 cause, 1 effect,

•No feedback loops (effect being cause of next failed link of the chain/mesh),

•Focus on learning from history and improving from there.

• Then, the guidance was taken as Directive (CYA). Banks did NOT already themselves have an urge to prevent preventable losses, only moved now they were forced to

• I.e., they did the least possible to be able to bluff their way into complaince

• By, e.g., building ops loss tracking databases:

•Tresholds without the „requirement‟ to aggregate incomplete picture,

•Self-reporting of losses by managers and executives, in the peak of the performance bonus days. Yeah, that‟ll work,

•Of self-reporting through (ad hoc, local) accounting rules incomplete, biased picture

•With too little guidance on classification inconsistent filing

•Which leads to useless data, not information.

• Oh and did we mention that there was little guidance (!) on what positive to do with the results ..? (re: no urge to improve)

Page 13: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

12 ISSA Interntional Conference Baltimore October 2011

Results are: Formal, paper compliance to the letter, but no (better) operational risk

management…

Seems like Basel II was more of an incident in itself, fire fighting staved off the ill (!) effects …

But it started me to think on how one should do operational risk management.

[First skirmishes led to a perceived need to change the bank‟s approach to ops risk mgt.

Couldn‟t get that through, and as I didn‟t want to be part of something so faulty, I first left the

audit department, then left the bank…]

Page 14: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

13 ISSA Interntional Conference Baltimore October 2011

Usually, according to „best‟ practice.

Chance is some frequency. Impact is some (dollar) amount.

Scales are translations according to some, hopefully uniformly defined and used, definitions.

Note that the scales are interval scales (http://en.wikipedia.org/wiki/Level_of_measurement;

regular intervals) with elements of a ratio scale (has a zero)

Risks are prioritsed according to their severity.

Maybe using Color in fancy heat maps. Placate some higher-ups, at their level of intelligence.

Which they may perceive as your level of intelligence, and/or perceive as your perception of

their intelligence.

The „best‟ practice risk management may not be good enough

Page 15: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

14 ISSA Interntional Conference Baltimore October 2011

The colors turn into a black-and-white picture that may be a little bleak, since

Page 16: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

15 ISSA Interntional Conference Baltimore October 2011

Problems are easily scetched, but models tend to over-simplify.

• Turning qualitative and wildly biased guesstimates into interval or ratio scales ? Didn‟t you

unlearn that in high school ..?

• Frequency per what? Per 1,000 transactions, per minute, every second, every year, or what

..? If the chance is 1 / 1,00 (i.e., 1%) per day, you‟re pretty darn sure to be hit a couple of

times every year – on average, and can expect to be hit two, three, even four times per

week very regularly.

• What sort of frequency distribution do you use ..? Normal, bell shaped, right ..? Very, very

wrong. Hardly anything has that distribution. Consider all the flight-of-fancy characteristics of

the normal distribution. You simply don‟t know the distribution.

• OK, for impact, we sometimes have some data. But how typical is it …? A sample of one

…?? (Because all but certainly, next time‟s different.) Is it complete, believable ..?

• How bad is a „score‟ of 16? Is it worse than 15,5 ..? Or 15,999? Statisticians use decimal

points to prove they have a sense of humor. You use numbers to show you don‟t

understand them. [Apologies for putting that slightly undiplomatic!]

• The vast majority of all this is guesswork. Don‟t claim precision or science when they‟re

NOT. You DON‟T falsify or seriously (…) verify whether your assumptions are true, or

reasonable.

• And, let‟s not forget you don‟t know whether your data is sufficiently complete … In

particular, the turkey before Thanksgiving problem. Or, last time I looked, I was still alive.

And I have tens of thousands of data points that demonstrate that every morning, I am alive.

So … I am immortal …?

Page 17: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

16 ISSA Interntional Conference Baltimore October 2011

Even if you were to establish some sort of correct model …:

The frequency (of occurence) distribution is a distribution in its own right. A high probability of

a low number of occurences, and the other way around. Note that the average doesn‟t say

very much, nor does the median, or „variance‟ …

The impact distribution may not be linear but rises.

The result (product) will probably be an exponential thing. The tail is very, very fat. While on

frequency alone, we usually disregard it…

Page 18: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

17 ISSA Interntional Conference Baltimore October 2011

There‟s your problem: You don‟t know any of these factors. You guess all the way.

Page 19: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

18 ISSA Interntional Conference Baltimore October 2011

It‟s a balancing act. Yes, young man, you too can be an astronaut, or even better, a risk

manager!

Page 20: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

19 ISSA Interntional Conference Baltimore October 2011

[ Assuming there is such a thing as a frequency versus impact graph ;-]

On the left, there‟s operational losses. Simple little errors and omissions that lead to small

losses (mainly costs of repair and restore). They occur frequently enough to amount to

something, so analysis may lead to simple coutermeasures (controls, procedures) to prevent,

or detect and restore, the defects systematically and efficiently. Job done.

This is the realm of Operational Risk Management as it is usually carried out in transactional

services.

On the right, we see the low frequency of very, very bad things happening. They don‟t occur

often, but even if there is a high probability that they haven‟t happened yet, they will or they

wouldn‟t exist as a threat. Many of these things fall off the radar. With Black Swan

consequences… When not if one of these incidents happens, the organization‟s survivability

is under threat.

The unpredictable (?) nature of these incidents means we have to be as vigilant as possible

to see them coming – usually, they‟re not a complete surprise, early warnings exist – and

then do all we can to limit the damage. This is the Business Continuity Management sector of

risk management. Be Prepared…

We (information) security are stuck in the middle. Incidents happen. Not as often as to be

routine (or you will have things under control via standard procedures), but often enough and

with enough damages incurred to sum up to something sizeable.

Having developed over the axes of separate „programs‟, ORM, Security, and BCM, have

been known to get involved in turf wars. As we have a continuum, who will determine

methodologies, who will control budgets and power ?

ORM will declare that all of the above should be under their supreme command. Security and

BCM are just variants under their same header. So does BCM say, from the other side. And

we are stuck in the middle.

Page 21: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

20 ISSA Interntional Conference Baltimore October 2011

Three lines of defense… sounds serious, but is a bit eager beaver. There‟s no defense like in

being armed and shooting going on in the second and third lines!

Three levels of being in control, is more what it is. Or three lines of abstraction away from

material problems.

Taken the other way around, it‟s more about three lines of defending the regulator from

getting a clue.

Personally, as an auditor …

• I dislike the development of Risk Management as a defense against auditors;

• I dislike the abstraction layers and al the formal organizational procedures, hierarchy,

meetings, discussion platforms, communities of practice, TPS Reports, etc. etc, that come

with these structures.

My heart may be too much with actual content to care about formalities. We all want to be

effective and solve problems, or do you not want that but want to just conform like a

robot…?

Page 22: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

21 ISSA Interntional Conference Baltimore October 2011

All the detail. This is just within one Line of Defense, and is still way incomplete in depicting

all meetings, gatherings, discussion platforms, etc.

Page 23: Down the Blind Alley (PDF)

OK, one from Despair.com, heartily recommended.

Page 24: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

23 ISSA Interntional Conference Baltimore October 2011

Both are good reads, though not necessarily easy reads. The UK examples of organizations,

etc., you can easily replace with similar ones from any of your own country/ies. Organized

Uncertainty in particular spells out the boom of Risk Management as an abstract discipline

with chain reaction avalanche growth.

Page 25: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

24 ISSA Interntional Conference Baltimore October 2011

OK, getting back to the details.

We analyse from 1 cause, 1 effect, all the way into lumping all threats into one CIA rating,

and then fan back out again with all sorts of controls and countermeasures.

[Not to mention the methodological / communications comady of errors due to even slight

definition differences, in particular re the latter two terms.]

But then we lose a lot of relevance. Which we may sometimes re-input leading to hybrid

models that are ill understood, contradictory, etc.

Let alone that 90%+ of our day-to-day problems come from psychological and

(organizational-)sociological difficulties with Man.

Those have been around since the savannah days. Oh, those were the days!

Those problems of time immemorial, haven‟t been solved. That is why the Classics are

classics.

So,

A. We solve them in a decade, for once and forever – and pull off what the greats and the

giants of all times couldn‟t pull off even when they didn‟t have serious deadlines and

budgets to consider,

B. We learn to live with them. Which means, we, technies par excellence, will have to know

„all‟ about psychology and sociology. (And maintain our technical edge.) And change our

mindsets. No more silver bullets, but actual management of risks by shaving off the rough

edges and leaving the rest to meddle along with. Uhm, I mean, accept.

Page 26: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

25 ISSA Interntional Conference Baltimore October 2011

25

We oversimplify our models!

Re Albert: modeling is an analysis tool, to weed out noise. Not more, or you end up with

something that may have too little predictive value; the error rate will dominate.

Yes, that‟s very, very bad. Because we blinder (blinker) ourselves, and the ones that we

advise. We tunnel our vision and filter too much.

That‟s why Black Swans happen.

Even worse, Gödel‟s incompleteness theorem (proven for mathematics, seems valid

elsewhere, where models are extremely more inaccurate) states that we cannot include

everything (relevant) in our models. So, the unexpected will happen, and things not even in

your model (not conceivable) will happen.

Contrary to that, risk managers also have been found guilty of after melting, restoring the

exact ice cube from the water. Next time, it‟ll be a different ice cube that melts. Hindsight is

easy and the model will fit. Going forward, it will not.

In particular, the turkey before Thanksgiving problem.

Page 27: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

26 ISSA Interntional Conference Baltimore October 2011

Even worse, Gödel‟s incompleteness theorem (proven for mathematics, seems valid

elsewhere, where models are extremely more inaccurate) states that we cannot include

everything (relevant) in our models. So, the unexpected will happen, and things not even in

your model (not conceivable) will happen.

So, the best we can do is handle what we do know – once we do know those things which is

different from guesswork.

Note that once we do know things sure enough, they may not be labeled „risks‟ anymore.

The uncertainty is shrunk to insignificance, and if we have proper controls in place, we‟re left

with the remainder risk.

And of that remainder risk, a now larger part is unmanageable…

Do your job well, and your organization ends up worse than before. You‟re on the road to

CxO.

How can the future be so hard to predict when all of my worst fears keep coming true?

Page 28: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

27 ISSA Interntional Conference Baltimore October 2011

Your outlook:

• A bumpy road (the easy road leads to…, you know….);

• Mist, fog;

• Any number of threats jumping out of the woodwork. Are you a) on your way to a good

hunting spot when a white tail jumps out of the woods, or b) Altavista and Page&Brin jump

out of the woods …? Unfortunately, odds are it‟s b)

We just can‟t predict the future …! In particular, the turkey before Thanksgiving problem.

Now this is methodologically correct, but not a viable model …?

Page 29: Down the Blind Alley (PDF)

Another one from Despair.com How useful that site and its products for us in the InfoSec

world.

Page 30: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

29 ISSA Interntional Conference Baltimore October 2011

Hey, those look like bullets, disguised. Yes, but they‟re yours. I wouldn‟t use any of those. And, every single line is self-deception.

1. Nothing is perfect. But not everything is as flawed as your models.

2. The assumptions are not reasonable. They‟re biased guesses that a monkey would do better (no bias!).

3. If the assumptions don‟t matter, why state them? And, they do matter or you have no functioning model (however flawed).

4. Conservative, compared to what ..? And they would better be right, for your models to have some realism. Conservatism may/will lead to the wrong conclusions.

5. Your assumptions are vastly more easily proven wrong then they are proven (!) to be right. Same, even for plausibility!

6. So, if everybody else jumps in the water, you follow ..? CYA may not be good enough…

7. Beware of the false prophet. Is the decision-maker better off by being mislead …!?

8. Oh yes they are because they‟ll lead you astray, until you know which parts work. Why not strip the rest, then ..? Or use a horoscope, that soothes peoples‟ axieties, too.

9. Garbage in, garbage out. And your best may not be good enough even if the data were accurate. „Completeness‟, anyone?

10.Yes. But be sure to make the rights ones, and to brutally scrutinize their validity, and determine the impact of changes in assumptions. Do you, ever (even identify your assumptions) ..?

11.Why ..? They‟re not babies. They‟re tools.

12.The harm is you, and your clients, are led astray by emperor‟s new clothes. Why pilot a plane from JFK to Atlanta and try to land using a map of Meg‟s Field …? Would you buy or drive a car when all parts are custom designed but e.g. the brakes not seriously tested ..? Analogies abound.

Page 31: Down the Blind Alley (PDF)

Oh, Despair, how right you are.

Page 32: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

31 ISSA Interntional Conference Baltimore October 2011

31

As for the future, we are.

Half of the companies you read about in the papers today, will not exist in 20 years time.

They all have great strategic planning…

How long will the DVD last ..? Did anyone at Altavista see students Page and Brin program

op their laptops? (a handful of years later only, Altavista had gone fro hero to zero)

Page 33: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

32 ISSA Interntional Conference Baltimore October 2011

Don‟t try to be Superman at work. Reserve that for your significant other.

Page 34: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

33 ISSA Interntional Conference Baltimore October 2011

Don‟t worry. Even if in a support role, we can be of much value.

Otherwise, the future InfoSec folks will look to us, like we look onto past trainwrecks.

The problems we face, fall into two categories:

•Perennial ones, that require risk management;

•Solvable ones, for which everyone must stop to ask for Structure, but we must just solve

them, like engineers tackle a problem.

For the perennials: Remember Einstein‟s quote: “There are two thing infinite: Human stupidity

and the universe. And I‟m not sure about the universe…”

(Repeat) Note that once you control the solvable problems, they are not risks in a sense that

they should be managed, apart from remainder risks.

A bit more on the solvable ones, first.

Page 35: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

34 ISSA Interntional Conference Baltimore October 2011

As a start, get the simple things right. Nu half measures that are ineffective or have negative

side-effects that are worse.

And don‟t over-promise. Call the bluff of those that do (e.g., dare vendors to put their (!)

money where their mouth is with respect to their silver bullet‟s effectiveness and efficiency).

And do analyse not only incidents, but also the tactics and strategy behind attacks.

(Conscious attackers) Be aware that the Others may learn fast. Faster than you ..?

To sum up, don‟t drop all your work and starve in analysis paralysis. Keep on doing what you

do but don‟t make it pretty and fancy by putting bad risk modeling icing on the cake.

Page 36: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

35 ISSA Interntional Conference Baltimore October 2011

Down to detail.

This includes being picky on issues like authorizations. That nothing has happened yet (at

your organization!) doesn‟t mean that one day, you‟ll be vindicated. If you do not take care,

then you‟ll be blamed, „for sure‟.

Page 37: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

36 ISSA Interntional Conference Baltimore October 2011

As for the perennials: Count on never ending stories.

Come on, people! We‟re engineers! We should know all about control loops. Why don‟t we

apply them in practice and let MBA types tell us all about management control cycles that are

just watered-down versions of the above …?

We need to

1.Devise our own control frameworks,

2.Point out the errors and inapplicability of „theirs‟.

We need to focus on „trigger‟ signals. E.g., if and only if I see evidence that a manager has

actually assessed a log analysis report and has taken action on risky deviations, do I know

that someone drafted a log analysis report, and hence logging was done in a way that allows

log analysis, and risky deviations are picked out. You don‟t need to check each and every

activity, if the last one in line tells you the health of the system.

Well, this is a hypothetical example but you get my drift.

If you have the time to restore the output, output quality measurements will suffice.

When you don‟t, preventative (and detective/corrective) controls are required.

By the way, the nesting of control loops would make an ideal three lines of defense model…

Unfortunately, just like Prince II compliance, Pino [Prince in name only] and nePino [not even

Pino], we find hardly any real three lines in practice, but much Tino and neTino.

Be aware that many laws and regulations are actually devised to have a „one size fits all‟

principle-based appearance (!) that results in even more abstract control loops. We don‟t

need a top-down approach; we need a bottom-up approach for this.

Page 38: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

37 ISSA Interntional Conference Baltimore October 2011

And please don‟t fight yesterday‟s war. That is so passé.

But do learn from military strategy/tactics developments over the centuries. There you have

the one (kind of) organization that has persisted, or not.

Page 39: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

38 ISSA Interntional Conference Baltimore October 2011

Train like you‟ll fight, then you‟ll fight like you train. Being Prepared isn‟t half bad.

Even if you‟re the Canadian air force. [Sorry, Canucks. The beaver is a proud and noble

animal, etc., I know…]

But be prepared for the things you don‟t see coming. Be open-minded.

Fear and calculate for the worst case.

Don‟t focus on chance %, focus on avoiding the negative impacts (and be prepared to take a

stand that they will be high!)

Don‟t „Me, Too‟ or pass the buck, take your own stand. When passing the buck, the Greater

Fool Theory will catch up with you: If you have a problem, pass it on to an even greater fool

than you and it will end up with the greatest fool – if you don‟t know who that is, it‟s probably

you.

(Don‟t fight forecasters, just play pranks with them)

Now this ties in with the direction that Risk Management is going. Or, should be going.

Page 40: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

39 ISSA Interntional Conference Baltimore October 2011

Hmmm, what‟s that below the author name …!?

Or, is he the only one who has read the thing so far …? I would have guessed he knows

what‟s in the book already; why read it then …?

Page 41: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

40 ISSA Interntional Conference Baltimore October 2011

Risk Management as a meme may be at its peak. Nice. Now get down to do something other

than holiday activities. You‟re (or rather, „they‟ are) paid to effectively deliver something.

What not to do: Keep on climbing. Remember the Tradition (de)motivator …?

[Repeat]

Don‟t focus on chance %, focus on avoiding the negative (!) impacts (and be prepared to take

a stand that they will be high!)

Don‟t „Me, Too‟ or pass the buck, take your own stand. When passing the buck, the Greater

Fool Theory will catch up with you.

And don‟t, don’t use flawed logic or flawed quasi-mathematics. Those cures are far worse

than the disease.

Page 42: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

41 ISSA Interntional Conference Baltimore October 2011

Official Risk Management will disappear as a separate mega-function; will revert to only coordinating the work of risk officers in the first line. The latter, will be information security.

The „Program‟ aspect of Risk Management will wane. (Re Michael Porter)

What Risk Management i.e. we !! will have to do:

• Team up with physical security, learn from them:

• Be Prepared

• Prevent, and detect and remediate in balance

• Assume the worst. Be sure to be able to mitigate the (negative) consequences. Be sure to see exponential relations.

• Incidents will always happen. Get over that. Note that we‟ll have a job forever…! Run away from the company that think you have „solved‟ their problems.

• Focus on qualitative risk assessments. Quantities are a fraud. Qualititative risks are more easily communicated.

• Do ruthless scenario analysis and stress testing. Frappez, frappez toujours! [Attrib Napoleon: Strike, strike always when you can, strike hard]

• Distinguish between reliance on information flows and IT versus threats and vulnerabilities

• Be alert. Learn from the military: a G2 or S2 (intel) officer in a generals‟s staff functions as aid to operational and tactical mangement re information gathering. G3/S3, the general himself (herself??) decides ..! Continuous sitreps. (Then, Audit can function as an airmobile brigade on hire; with you, not against you!)

• [Be alert. The world needs more lerts.]

• Don‟t be bureaucratic about department borders, silos, or about neatly divided 3, 4, 5 lines of defense.

• But do the simple things bottom-up, first things first, and build structures on that. Evolve.

Page 43: Down the Blind Alley (PDF)

And that’s where many laws and regulations, and many risk management

departments, fail today. The top-down smoothly deductive design in isolated

departments, leads to analysis paralysis with results that don’t fit on / in practice. The

ideal may call for square pegs, but they don’t fit in round holes.

The problem of squaring the circle is provable impossibly solved. When this problem is

translated to risk management, it would be “just one of the many issues for which the solution

is postponed for a while; first let‟s do a pilot.” The problem doesn’t go away by ignoring or

denying it!

All worst fears come true, because

A. They just do, you better count on that

B. We better not remain stuck in analysis paralysis

C. Or we deny the worst problems and live happy go lucky till we don‟t.

This translates to information security, too: Don‟t wait till others stop whining. Solve problems

first, then do marketing.

[Marketing being translating what you have achieved, into regulationspeak to demonstrate

compliance.]

Act now, talk later!

Will do is nothing Doing is something Have done is everything.

Page 44: Down the Blind Alley (PDF)

Ah, life may not be just that simple and we may indeed ourselves need categorization, if only

to be sure we are doing „all‟ the right things.

“Factors may be:

•Irregularities in human performance;

•Machine and/or system break-downs;

•Failures to maintain standard operating procedures;

•Inadequate assessment of impact of external forces (market, economy, political

environment);

•Inefficient use of resources (funds, personnel, equipment, technology, knowledge);

•Lack of appropriate controls of business functional complexity.”

As an example. The factors overlap. And they may be factored down to root causes, but work

forward, in a mesh of effects and feedback loops. Which might be solved with e.g. Markov

chain analysis, but there we have the huge sensitivities for slight variations in input

parameters again…

Though, it must be said, the above list has enough perennials to work with …

So, stifle and paralyze the model freaks with their own methods. How effective was their

budget spend ..?

The Future of Risk Management / Where Will Risk Management Go ..?

43 ISSA Interntional Conference Baltimore October 2011

Page 45: Down the Blind Alley (PDF)

A second major line of business for us: stress testing. Since perennials tend to return on a

larger scale too, but in an unpredictable way – we don‟t learn enough from history to be able

to pick up the right early warnings and feed those through the right models.

The financial industry has moved from basic indicators to stress testing using scenarios.

Reason: Systemic risks in the sector. Currently done only for financial industry /

interdependencies of financial instruments.

The Dodd-Frank Act includes regulations on “crisis management” in the financial industry.

Whereas Basel‟s BIS (and BCBS) focused on minimum buffer capital requirements to

counter, mostly!, financial crises, the Financial Standards Board now also includes data

standards and collection. But still, it focuses on systemic financial risks.

The scenarios include macro-economic shocks as cause for riples in the financial industry, by

the way.

We don‟t have such institutions in our sector, do we?

Have you considered Advanced Persistent Threats as uncertainties about the systemic

vulnerabilities of the IT industry, with its global connectedness and dependencies?

You should test for „systemic‟ risks re information processing (of all kinds and processes)

within your own organization, and industry-wide … Do the war games!

And do all sorts of other (systemic or not) stress tests. How to include macro-IT-shocks as

cause for ripples in our industry? What would happen if suddenly a major systemic

vulnerability would be found in the TCP/IP stack ..? How do we get a grip on the

unpredictable nature of the next major blow to the (sec) industry? I don‟t know. Nobody‟s

perfect.

The Future of Risk Management / Where Will Risk Management Go ..?

44 ISSA Interntional Conference Baltimore October 2011

Page 46: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

45 ISSA Interntional Conference Baltimore October 2011

Now will Risk Management as a sector be allowed to move into that direction ..? Or how far

are we already into a blind alley ..?

You know what happened when Alexander was told about the Gordian Knot that tied up a

cart, and tied up many minds on how to untangle it.

Bam! He put a sword to it and hey presto no more problem.

Unfortunately, it takes an Alex The Great to pull such a thing off.

Or a huge number of politicians that for once forget to cover their behinds with ever more

rules. Said A the Great had the advantage of being supreme ruler, or course, so he wasn‟t

forced to compliance to petty rules and procedures.

Nevertheless, laws and regulations are close, very close to being the Gordian knot. And let

me tell you: The more tightly knit, the less effective

Page 47: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

46 ISSA Interntional Conference Baltimore October 2011

If you set standards high enough, they‟re ever more easy to go underneath.

[What game are we in …? Not so sure…]

The solution is NOT to raise the bar even further.

Regulators commonly do.

We may need an Alexander the Great.

This means

1. We need to train more on the pole vault, which is not so easy and takes numerous leaps

of faith. Or we go limbo in the back yard.

2. The regulatory and risk management industry needs to move to high quality standards,

i.e., smarter standards. They‟ll probably be more principle-based, but smarter. Not

describing too much apparently random detail, but catching the health of the whole

system of controls.

3. I.e., the regulatory industry needs to focus on the bottom-up approach, not the top-down

structures on a case-by-case basis…

4. Guidance will be of the essence. Not guidance that is taken as unthinking route to

compliance, but guidance the other way around, allowing the flexibility we need.

Page 48: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

47 ISSA Interntional Conference Baltimore October 2011

Apologies to regulators, but …

Where have we lost the self-regulation …?

How can we gain control over regulations? By providing lawmakers with our own,

demonstrably impartial independent and hopefully proven effective standards …

Page 49: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

48 ISSA Interntional Conference Baltimore October 2011

… Darn! Forgot to delete the last few bullets.

Well, to sum up: See slide.

Nice crammy slide, this one. And yes, I‟m of an age when „slide‟ meant slide or sheet. What,

when the Desktop is no longer a proxy for your pysical desktop when you wave a tablet in the

air …?

Page 50: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

49 ISSA Interntional Conference Baltimore October 2011

All presented, is work in progress. By default, and here in particular.

All help is appreciated. [ Comments, pointers, etc. etc., to jvdvlugt åt xs4all døt nl. Please

include a descriptive subject line or I might unduly offload your message. ]

Page 51: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

50 ISSA Interntional Conference Baltimore October 2011

Oh, trust me; the ropes are all managed by Risk Management, in line with best practice, risk

appetites and predominant with efficiency concerns in mind.

Remember John Glenn‟s words.

Page 52: Down the Blind Alley (PDF)

The Future of Risk Management / Where Will Risk Management Go ..?

51 ISSA Interntional Conference Baltimore October 2011

Few! You‟ve made it through and sat it out.

Now, are there any questions …?

Some closing remarks, after the presentation, including your input and what I learned at the

Conference: It seems that the two-pronged approach to „operational‟ infosec (do the simple

stuff right, and defend against the impact of the difficult stuff) would best be applied at tactical

and strategic levels, too. Tactical: Take care to be on board in projects. And don‟t say No to

every business initiative, stand ready with secure solutions. Strategic: Have reports about

attacks prevented ready. And demonstrate cool control over problem solving when something

serious happens.

Hmmmm, this sounds like an article in the ISSA Journal in the making…