Make Your Actuarial Make Your Actuarial Spreadsheets FlySpreadsheets Fly

Designing and Programming Spreadsheets for Designing and Programming Spreadsheets for Risk MeasurementRisk Measurement

William C. ScheelWilliam C. ScheelDFA Technologies, LLCDFA Technologies, LLC

Pete WickPete WickMilliman USAMilliman USA

2002 Risk & Capital 2002 Risk & Capital Management SeminarManagement Seminar

Spreadsheet-Centric Model Spreadsheet-Centric Model DesignDesign

• AdvantagesAdvantages– Flexible model design; open platformFlexible model design; open platform– Visual Basic for ApplicationsVisual Basic for Applications– Accessible to both non-programmers and Accessible to both non-programmers and

programmersprogrammers– Rapid application development and deploymentRapid application development and deployment

• DisadvantagesDisadvantages– Inadequate to task of complex modelsInadequate to task of complex models– Seemingly slow computationSeemingly slow computation

The Fast and Slow if ItThe Fast and Slow if It

• When does VBA code run faster than When does VBA code run faster than cell-based logic? An example using cell-based logic? An example using the Burville simulation methodthe Burville simulation method

• Conclusions from Burville.xls example:Conclusions from Burville.xls example:– Complex cell logic should be in codeComplex cell logic should be in code– Data sheets with values work fastData sheets with values work fast– Some cell-based arrays with formulas Some cell-based arrays with formulas

such as lookups run very wellsuch as lookups run very well

John Burville’s Simulation John Burville’s Simulation MethodMethod• Intervalize a probability distribution; create a Intervalize a probability distribution; create a

1,000 element vector with x values loaded in 1,000 element vector with x values loaded in proportion to interval probabilities, f(x)proportion to interval probabilities, f(x)

• Staged Simulation involves:Staged Simulation involves:– Generate a uniform value, u, between 1 and 1,000Generate a uniform value, u, between 1 and 1,000– Dereference element u of the array to obtain the inverse Dereference element u of the array to obtain the inverse

valuevalue– Use a placeholder for any interval where finer granularity Use a placeholder for any interval where finer granularity

is needed. Load another array of length 100 with x is needed. Load another array of length 100 with x values in proportion to the sub-interval probabilities. values in proportion to the sub-interval probabilities.

– If the placeholder is observed on a draw, use another If the placeholder is observed on a draw, use another uniform selection to sample from the sub-arrayuniform selection to sample from the sub-array

– Use as many levels as needed to achieve granularity. Use as many levels as needed to achieve granularity. E.g., tail areas of the distributionE.g., tail areas of the distribution

Excel VBA ProgrammingExcel VBA Programming

• The VBA EditorThe VBA Editor• Viewing the object modelViewing the object model• Dynamic range specification and memory-Dynamic range specification and memory-

to-cell data transferto-cell data transfer• Traversing objects in codeTraversing objects in code• IDE Tools.Options Editor TabIDE Tools.Options Editor Tab

– Auto Syntax Check OFFAuto Syntax Check OFF– Require Variable Declaration ONRequire Variable Declaration ON– Auto Indent; Tabs Width 2Auto Indent; Tabs Width 2

• Intellisense code completionIntellisense code completion

Why Generate a Worksheet of Why Generate a Worksheet of Claims?Claims?• Excel is a good database mechanism with value-Excel is a good database mechanism with value-

based workbooksbased workbooks• Random numbers can be samples of random Random numbers can be samples of random

numbersnumbers• Convolutions, event-driven logic can be done with Convolutions, event-driven logic can be done with

claims worksheetsclaims worksheets• Conclusions from Generate.xlsConclusions from Generate.xls

– Range-based sampling; a programming trickRange-based sampling; a programming trick– Programmatic Lookup in F(x) significantly faster then Programmatic Lookup in F(x) significantly faster then

inverse function in array formulainverse function in array formula– Create bootstrap samples from ranges of valuesCreate bootstrap samples from ranges of values

Rapid Convolutions…Rapid Convolutions…

• Yes, Virginia, Santa Claus may not be Yes, Virginia, Santa Claus may not be able to always use Fast Fourier able to always use Fast Fourier Transforms (FFT)Transforms (FFT)

• Event monitoring.Event monitoring.

• Messaging and computational designMessaging and computational design

Rapid Convolutions…Rapid Convolutions…

• Conclusions from SimpleConvolution.xls.Conclusions from SimpleConvolution.xls.– Construct worksheets with sample values Construct worksheets with sample values

(claims) that are generated prior to the (claims) that are generated prior to the convolutionconvolution

– Use randomly constructed range references Use randomly constructed range references to quickly obtain blocks of random variablesto quickly obtain blocks of random variables

– Use worksheet functions, if possible, to do Use worksheet functions, if possible, to do convolution mathconvolution math

Rapid BootstrappingRapid Bootstrapping

• More conclusions from Generate.xlsMore conclusions from Generate.xls– Abstract data to be abstracted to an array of Abstract data to be abstracted to an array of

randomly selected rangesrandomly selected ranges– Use VBA code to calculate bootstrap samples Use VBA code to calculate bootstrap samples

with with rangerange calculate applied to the array calculate applied to the array– Use cell formulas that reference the bootstrap Use cell formulas that reference the bootstrap

sample range to calculate statistics being sample range to calculate statistics being bootstrappedbootstrapped

– After a <bootstrap range>.Calculate, do the After a <bootstrap range>.Calculate, do the <bootstrap statistic range>.Calculate<bootstrap statistic range>.Calculate

– This only can be done using VBA programmatic This only can be done using VBA programmatic techniquestechniques

Designing Spreadsheet-Centric Designing Spreadsheet-Centric ModelsModels

• Do what Excel+VBA does best…Rapid Do what Excel+VBA does best…Rapid Application Development (RAD)Application Development (RAD)

• Always think componentsAlways think components• No model is beyond the scope of Excel!No model is beyond the scope of Excel!• User interface…80-90 percent of the effortUser interface…80-90 percent of the effort• Think virtual general ledgerThink virtual general ledger• Statistical and graphical renderings may Statistical and graphical renderings may

be done with “object” worksheets and VBAbe done with “object” worksheets and VBA

Rapid Application Rapid Application DevelopmentDevelopment

• RAD modeling is a VBA programmer RAD modeling is a VBA programmer with Excelwith Excel

• Actuary is an interpreter of Actuary is an interpreter of informationinformation

• Actuary is a forecaster using modelsActuary is a forecaster using models

• Actuary should do RAD in ExcelActuary should do RAD in Excel

Excel as a Data EngineExcel as a Data Engine

• Value-base worksheetsValue-base worksheets

• Data Sort/Autofilter vs ADO (Active Data Sort/Autofilter vs ADO (Active Data Objects class). Both can be Data Objects class). Both can be done programmaticallydone programmatically

• Conclusions from Vgl.xlsConclusions from Vgl.xls– ADO recordsets made from ranges ADO recordsets made from ranges – Virtual general ledgers from range Virtual general ledgers from range

objectsobjects

Excel as a Component Excel as a Component EngineEngine

• Oh, by-the-way. A worksheet and Oh, by-the-way. A worksheet and just about everything else in Excel just about everything else in Excel can be a component.can be a component.

• Demonstration of drag-and-drop Demonstration of drag-and-drop modeling to illustrate:modeling to illustrate:– Worksheet componentsWorksheet components– Multiple process interaction (Visio and Multiple process interaction (Visio and

Excel)Excel)

Excel on Multiple ComputersExcel on Multiple Computers

• When a spreadsheet isn’t fast When a spreadsheet isn’t fast enough, you always can use multiple enough, you always can use multiple computers.computers.

• An introduction to Microsoft MSMQ An introduction to Microsoft MSMQ and its use with Excel.and its use with Excel.

• Pass handles to objects using ByRef Pass handles to objects using ByRef arguments. But, always store them arguments. But, always store them in variables of type “Object.” Do in variables of type “Object.” Do NOT use early bound variable types.NOT use early bound variable types.

My Favorite Excel Tricks…My Favorite Excel Tricks…

• CurrentRegion (for cell block CurrentRegion (for cell block identification)identification)

• Resize (for ranges)Resize (for ranges)• Variant array assignments for transferring Variant array assignments for transferring

content between cells and memorycontent between cells and memory• Collections and “super” collections Collections and “super” collections

(collections of collections)(collections of collections)• Public methods in worksheets, workbooks, Public methods in worksheets, workbooks,

and formsand forms

My Favorite Excel Tricks…My Favorite Excel Tricks…

• VBA calculation of specific ranges or worksheets VBA calculation of specific ranges or worksheets is is NOTNOT workbook calculate=F9…<ws>.Calculate workbook calculate=F9…<ws>.Calculate or <r>.Calculate often provide needed control or <r>.Calculate often provide needed control over calculation orderover calculation order

• <range>.copy <range destination cell><range>.copy <range destination cell>• With <object>….End WithWith <object>….End With• With Application.WorksheetFunction With Application.WorksheetFunction

… .AVERAGE(<range/array>) ... .PERCENTILE … .AVERAGE(<range/array>) ... .PERCENTILE … .<Whatever> … End With… .<Whatever> … End With

• Region item specification using <single cell Region item specification using <single cell range>.CurrentRegionrange>.CurrentRegion

My Favorite Excel Tricks…My Favorite Excel Tricks…

• Use worksheets as property bags for Use worksheets as property bags for persisting informationpersisting information

• Microsoft Scripting Runtime class for file Microsoft Scripting Runtime class for file system objects (Reference=Microsoft system objects (Reference=Microsoft Scripting Runtime)Scripting Runtime)

• DFATech methods Get/PutRegionItem used DFATech methods Get/PutRegionItem used for persisting in-memory arrays to region for persisting in-memory arrays to region items in worksheetsitems in worksheets

• DFATech methods for persisting ranges to DFATech methods for persisting ranges to XML filesXML files

My Favorite Excel Tricks…My Favorite Excel Tricks…

• Data binding between worksheet Data binding between worksheet data and controls (e.g., lists) is easy data and controls (e.g., lists) is easy programmatically when you programmatically when you remember this remember this assignment: .RowSource = ws.name assignment: .RowSource = ws.name & “!” & r.address& “!” & r.address

My Favorite Excel Tricks…My Favorite Excel Tricks…

• Use worksheets as property bags for Use worksheets as property bags for persisting informationpersisting information

• DFATech methods Get/PutRegionItem used DFATech methods Get/PutRegionItem used for persisting in-memory arrays to region for persisting in-memory arrays to region items in worksheetsitems in worksheets

• DFATech methods for persisting ranges to DFATech methods for persisting ranges to XML filesXML files

• Microsoft Scripting Runtime class for file Microsoft Scripting Runtime class for file system objects (Reference=Microsoft system objects (Reference=Microsoft Scripting Runtime)Scripting Runtime)

Spreadsheet-Centric Spreadsheet-Centric ProgrammingProgramming

• Worksheet cells and functions serve as a Worksheet cells and functions serve as a rich enhancement to a programmers toolsrich enhancement to a programmers tools

• Intrinsic Excel functions work well in Intrinsic Excel functions work well in programsprograms

• User-designed methods in sheets, forms User-designed methods in sheets, forms can be called from modules, classes and can be called from modules, classes and elsewhereelsewhere

• ““Program behind” workbook and Program behind” workbook and worksheet eventsworksheet events

ObjectsObjects

• The structure of an object is a class.The structure of an object is a class.– Private variables for properties.Private variables for properties.– Methods for program actionsMethods for program actions– Events for doing something in response to something Events for doing something in response to something

happening affecting the objecthappening affecting the object

• An instantiation of an object from a class is like a An instantiation of an object from a class is like a copy of the structurecopy of the structure

• A A worksheetworksheet can be a class! Copies of can be a class! Copies of worksheetworksheet templates are objects. LOBs are easily cloned.templates are objects. LOBs are easily cloned.

• Microsoft Visio Drag-and-drop exampleMicrosoft Visio Drag-and-drop example

How Do You Learn the How Do You Learn the Tricks?Tricks?• Must know Excel’s Class Model. The Must know Excel’s Class Model. The

object browserobject browser• Must know elements of object Must know elements of object

programmingprogramming• Must know IDE (integrated Must know IDE (integrated

development environment)development environment)• Must have exceptionally high Must have exceptionally high

frustration tolerancefrustration tolerance

How To Learn Model RADHow To Learn Model RAD

• Build a reinsurance convolution Build a reinsurance convolution model…it is a good place to identify model…it is a good place to identify the hurdles. Do it in VBA code as a the hurdles. Do it in VBA code as a learning exercise.learning exercise.

• Range referencing is the hard partRange referencing is the hard part

What We Learned...What We Learned...

• Value-based spreadsheets load and can Value-based spreadsheets load and can be managed quicklybe managed quickly

• Do not parse individual cells; assign a Do not parse individual cells; assign a range of them to in-memory variant range of them to in-memory variant arrays and do the parsing there. Do this arrays and do the parsing there. Do this in reverse when loading cells with data.in reverse when loading cells with data.

• In-memory data can be persisted to In-memory data can be persisted to spreadsheets or ADO recordsets very spreadsheets or ADO recordsets very fastfast

What We Learned…What We Learned…

• You can rapidly sample from pre-You can rapidly sample from pre-generated claims information using generated claims information using rangesranges

• VLOOKUP can be used on intervalized VLOOKUP can be used on intervalized cumulative probability distributions in cumulative probability distributions in array formulasarray formulas with astonishing speed with astonishing speed

• Slice’n’dice with VBA code on in-Slice’n’dice with VBA code on in-memory datamemory data

What We Learned…What We Learned…

• Proxy variables in strings can be Proxy variables in strings can be easily converted programmatically to easily converted programmatically to formulas; a type of Excel-scripting formulas; a type of Excel-scripting without using explicit cell referenceswithout using explicit cell references

Advanced Techniques…Advanced Techniques…

• Use the Windows messaging paradigm Use the Windows messaging paradigm in VBA code to communicate among in VBA code to communicate among Excel objectsExcel objects

• Consider the use of Consider the use of WorksheetWorksheet templates (not just Workbook templates (not just Workbook templates)templates)

• Use multiple computers strung together Use multiple computers strung together with MSMQ (Microsoft Message Queue)with MSMQ (Microsoft Message Queue)…it works on all MS operating systems.…it works on all MS operating systems.

Advanced Techniques…Advanced Techniques…

• Constantly remind yourself that an Constantly remind yourself that an Excel worksheet is just an event-Excel worksheet is just an event-savvy objectsavvy object

• Collections (and dictionaries, which Collections (and dictionaries, which are like keyed or indexed objects) are like keyed or indexed objects) are among the best programming are among the best programming tools. Collections of collections tools. Collections of collections (super collections) also are useful.(super collections) also are useful.

Advanced Techniques…Advanced Techniques…

• Excel classes are always useful. They are Excel classes are always useful. They are essential for WithEvents programming.essential for WithEvents programming.

• Forms+Modules; Forms+Modules; Forms+Classes+CollectionsForms+Classes+Collections

• Single computer, cross-process Single computer, cross-process communication (say, Visio-to-Excel) can be communication (say, Visio-to-Excel) can be done with object references passed ByRef done with object references passed ByRef between them! This cannot be done between them! This cannot be done across machine boundaries.across machine boundaries.

Microsoft Office and .NET?Microsoft Office and .NET?

• Office products such as Excel will Office products such as Excel will increasingly consume web services. Model increasingly consume web services. Model buildingbuilding becomes even easier and more becomes even easier and more RAD-oriented.RAD-oriented.

• .NET .NET web service web service• DFA will change systematically as new DFA will change systematically as new

components come on-linecomponents come on-line• Vended web services such as scenario Vended web services such as scenario

generators, specialized risk measurement generators, specialized risk measurement techniques, data (e.g., payment patterns)techniques, data (e.g., payment patterns)

Microsoft Office and .NET?Microsoft Office and .NET?

• ““Data” web services (web queries) that plug-into-your-Data” web services (web queries) that plug-into-your-model will come on-line. E.g., investment and claims model will come on-line. E.g., investment and claims scenarios, payment patternsscenarios, payment patterns

• June 17, 2002, EWeek, “June 17, 2002, EWeek, “Microsoft to Give XML Bigger Microsoft to Give XML Bigger Role in Office” Role in Office” By By  Peter GalliPeter Galli

•Microsoft Corp., which faces mounting competition and Microsoft Corp., which faces mounting competition and price pressure focused on its Office desktop productivity price pressure focused on its Office desktop productivity suite, is set to release the first beta of an Office suite, is set to release the first beta of an Office upgrade….Due later this year, the beta will feature far upgrade….Due later this year, the beta will feature far greater use of greater use of XML and Web servicesXML and Web services for reporting, for reporting, analyzing, importing and exporting information—analyzing, importing and exporting information—particularly in Outlook and Excel, Steven Sinofsky, senior particularly in Outlook and Excel, Steven Sinofsky, senior vice president for Office, in Redmond, Wash., told eWeek.vice president for Office, in Redmond, Wash., told eWeek.

Q&A: DFA Modeling: Should Q&A: DFA Modeling: Should You Be a “Programmer” too?You Be a “Programmer” too?

• When turn-key modeling isn’t enough!When turn-key modeling isn’t enough!

• Knee-jerk acceptance of Knee-jerk acceptance of “do it all”“do it all” in in Excel cell-based formulasExcel cell-based formulas

• Not just a spreadsheet-centric modeling Not just a spreadsheet-centric modeling issue…”being a programmer” and issue…”being a programmer” and architect/builder is generic issue for all architect/builder is generic issue for all computer-based modelscomputer-based models

ReferencesReferences

• Excel VBA Programming: M. C. Excel VBA Programming: M. C. Martin, et al, Martin, et al, Mastering Excel 2000Mastering Excel 2000; J. ; J. Green, Green, Excel 2000 VBAExcel 2000 VBA

• Bootstrapping: B. Efron and R. J. Bootstrapping: B. Efron and R. J. Tibshirani, Tibshirani, An Introduction to the An Introduction to the BootstrapBootstrap

• Got an Excel question? Got an Excel question? scheel@mindspring.comscheel@mindspring.com