compact, may 2004, issue no. 17. - member | soa · 2012. 1. 19. · compact electronic newsletter...

CompAct Electronic Newsletter • Issue No. 17 • May 2004 • Published in Schaumburg, IL by the Society of Actuaries

Inside

Chairperson’s Corner 3by David Snell

Surprising Results from Curriculum Survey 4by Randall Kaye

Computer Science Sessions in Anaheim &San Antonio 5by Phil Gold

Standards: XTbML Tables Beyond Mortality 7by Dean Slyter

P versus NP, or How Long Will This Take? 10by Carol Marler

Using Excel to Prototype RelationalDatabase Applications 12by Dave Snell

CompAct

Articles Needed for the CompActElectronic NewsletterYour help and participation is needed and welcomed. All arti-cles will include a byline to give you full credit for your effort.CompAct is pleased to publish articles in a second languageif a translation is provided by the author. For those of you in-terested in working on the CompAct, several associate edi-tors are needed to handle various specialty areas such asmeetings, seminars, symposia, continuing education meet-ings, new research and studies by SOA committees and soon. If you would like to submit an article or be an associateeditor, please call Dave Snell, editor, at 636.736.7345.

The CompAct is published as follows:

Publication Date Submission DeadlineSeptember 15 July 15 March 15 January 15

Preferred FormatIn order to efficiently handle articles, please use the following format when submitting material:

Please e-mail your articles as attachments in either MSWord (.doc) or Simple Text (.txt) files. We are able to con-vert most PC-compatible software packages. Headlines aretyped upper and lower case. Please use a 10 point TimesNew Roman font for the body text. Carriage returns are putin only at the end of paragraphs. The right-hand margin isnot justified.

If you must submit articles in another manner, please callErica Barraca, 847.706.3549 at the Society of Actuaries forassistance.

Please send electronic copies of the articles to:

Dave SnellComputer Science Section Editor1370 Timberlake Manor ParkwayChesterfield, MO 63017-6309phone: 636.736.7345fax: [email protected]

Thank you for your help.

Computer Science Section NewsletterIssue Number 17 • May 2004

Published quarterly by the Computer ScienceSection of the Society of Actuaries

475 N. Martingale Road, Suite 600Schaumburg, IL 60173phone: 847.706.3500fax: 847.706.3599

World Wide Web: www.soa.org

This newsletter is free to section members.

Dave SnellComputer Science Section EditorRGA Reinsurance Company, Inc. 1370 Timberlake Manor ParkwayChesterfield, MO 63017-6309phone: 636.736.7345 fax: [email protected]

Computer Science SectionSection CouncilDavid L. Snell, ChairpersonBrian S. Septon, Vice-ChairpersonHal S. Tepfer, Treasurer/SecretaryWilliam R. Aguayo, Council Member/Web LiaisonCharles S. Fuhrer, Council MemberPhilip Gold, Council MemberKok Bin Liew, Council MemberFrank G. Reynolds, Council MemberMichael K. Rigby, Council MemberCharles S. Linn, Immediate Past Chairperson

Lois Chinnock, Sections [email protected]

Clay Baznik, Publications [email protected]

Erica Barraca, Graphic [email protected]

Facts and opinions contained in these pagesare the responsibility of the persons whoexpress them and should not be attributed tothe Society of Actuaries, its committees, the Computer Science Section or the employers of the authors. Errors in fact,if brought to our attention, will be promptlycorrected.

Copyright© 2004 Society of Actuaries.All rights reserved.Printed in the United States of America.

Chairperson’s Cornerby David Snell

Welcome to this issue of CompAct.Personal thanks are due to myimmediate predecessor, Charlie

Linn, who restarted the newsletter last fall,after a long hiatus. We plan to make this a more frequent way of communicating withyou. Of course, the communication processshould occur in both directions, so let usknow what you think about the articles, andeven better, contribute some articles for thenext issue.

A few decades ago, when I was a fledglingactuarial student, I read about an SOA studythat asked people on the street to explain theterm “Actuary.” Some of us laughed at someof the answers. I thought the most creativeresponse was that an Actuary was “a placewhere you bury dead actors.”

Back then, of course, the insurance industryknew of actuaries as the ultimate gurus offinancial calculations and as natural leadersof their companies. Many, if not most, insur-ance companies were managed by an actu-ary. I remember hearing then that if youliked mathematics, and you liked money(i.e., having some), this was the professionof choice.

Last year the SOA conducted another study,this time asking financial services profession-als, and the results were markedly different.We are still viewed as very good with mathe-matics, but now we have a lot of competitionon that front. Communication skills and busi-ness savvy are also important and in thoseareas, the CFA and MBA are consideredstronger than the FSA. The role of actuariesas company leaders has diminished.

One thought is that the actuarial academictraining curricula may be a bit out of touchwith current needs. We decided to investi-gate this from a computer science perspec-tive, and Randall Kaye summarizes thethought-provoking survey results for you inthis issue of CompAct.

Also in this issue, we have an article byDean Slyter about XTbML, a standard we,along with ACORD (an industry standardsorganization), are promoting to facilitatethe sharing of tabular data between differ-ent programs, operating systems and evenhardware platforms. Cindy Jeness andJacques Rioux introduced us to XTbML inthe last issue of CompAct, and Dean’s arti-cle continues with an example beyond justmortality tables.

We even have an article about a mathe-matics contest with a $1,000,000 prize (mathematics and money, revisited). Carol Marler describes this for us.

You can read about the exciting ComputerScience Section sessions planned for thisyear’s Spring and Annual SOA Meetings,and about how to use Excel, a tool as indis-pensable to actuaries now as the calculatorwas decades ago, in new ways, to proto-type relational databases.

We think we have an assortment of articlesyou might find interesting. I assure you,we have buried no dead actors in thisissue. :

Dave Snell is chairperson of theComputer ScienceSection. He is the technology evangelistfor RGA ReinsuranceCompany, and has alot of fun designing customized solutions for RGA’s Asia-Pacificoperations. His cur-rent hobby is studyingMandarin Chinese.Dave can be reached at [email protected]

CompAct • 3

4 • CompAct

Randall A. Kaye ASA,

FLMI/M is a principal

with arc360, a consult-

ing firm providing

systems for the life

insurance industry.

While working with

illustration and admin

systems, he also helps

companies with their

universal life product

features and policy-

holder taxation inter-

pretations. He can be

reached at RAKaye@

compuserve.com.

Surprising Results from Curriculum Surveyby Randy Kaye

In performing their work, Computer ScienceSection members utilize a wide variety of techni-cal skills. Many useful “computer science”-relat-ed subjects are not covered by the SOA exams,though they may be considered. The ComputerScience Section recommends that colleges anduniversities develop curricula for actuarial sci-ence programs that reflect the results of this sur-vey and which enable college and university stu-dents interested in pursuing an actuarial careerto select appropriate courses.

The Computer Science Section membershipwas surveyed to help us understand which top-ics are important for actuarial staff to have a working knowledge of, at either the “earlycareer” or “mature career” stage. Over 250members completed the survey.

The results below are based on a scoring sys-tem from 0 = Unimportant, to, 5 = Must Have,then converted to a percentage. A score of 3.5or more is significant, since it would imply alevel of importance somewhere between:

1. 70 percent of the respondents felt that it was 5 = Must Have, and,

2. 100 percent of the respondents felt that it was 3.5 = Important.

A score of 3.5 or more was achieved in the following areas:

While the importance of system-related topicssuch as spreadsheets, programming lan-guages, database concepts, documentationand project management were expected, thesurprising result from this survey is that therespondents felt so strongly that numericalalgorithms (especially basic calculus, interpola-tion, extrapolation, estimation and errors) arestill considered a Must Have in importance.

Yet the traditional actuarial skills of calculus,numerical analysis and graduation methodshave been removed from the E&E syllabus overthe last decade or, if not removed, mentionedas “prerequisites”. This survey shows that themembership still judges these skills asextremely important, and the ComputerScience Section council encourages the E&ECommittee to consider reinstating numericalanalysis and graduation on both the E&E syl-labus and examinations.

Colleges and universities are also encouragedto review these results and interpret them as a“call to arms”, or at least, reinstatement, forsome of the more traditional actuarial skills.

Complete survey results can be found in theComputer Science Section of the SOA Web siteat www.soa.org/ccm/content/areas-of-prac-tice/special-interest-sections/computer-sci-ence/compact-newsletter/ :

Score Early Career

4.7 Mathematical applications, especially spreadsheets (including macros).

3.5 Programming languages, especially procedural languages (C, Fortran, Basic).

3.5Statistical analysis, especially regression, distributions, standard deviation andMonte Carlo methods.

3.8Numerical algorithms, especially basic calculus, interpolation, extrapolation, estimation and errors.

3.5Database Concepts, such as hierarchical & relational, queries, unions and intersections.

3.9Documentation, such as program comments, system documentation and user documentation.

Score Mature Career

4.0 Mathematical applications, especially spreadsheets (including macros).

3.8Project management & planning, such as steering committee, project sponsor, critical path, dependencies and Gantt charts.

3.6Financial aspects of information systems, such as total cost of ownership, business risk and return on investment.

CompAct • 5

• continued on page 6 •

Computer Science Sessions in Anaheim &San Antonioby Phil Gold

Now’s your chance to sign up for some special sessions at the upcoming spring meetings.

We are sponsoring two sessions in Anaheim in May:

Hot TechnologiesWednesday May 19, 10:30 a.m.Moderator: Charles S. Linn.Panel: Neil Raden, Jit Sinha, Jerry Goedicke

This session covers a variety of hot technologies, including:• Wireless business solutions• Data management for analytical applications• Grid computing

Attendee benefits include:• All the new buzzwords• Ideas for revolutionizing your own work environment

Nontraditional Computer Science Roles for ActuariesThursday, May 20, 8:30 a.m.Moderator: Brian Reid.Panel: Lyle S. Semchyshyn, Nancy Walczak, Emily B. Kraft

This forum highlights:• The career paths of the panelists• The experience of other session attendees.

Attendee benefits include ideas for job opportunities in less conventional areas ofactuarial employment.

We are co-sponsoring a panel discussion with the Health Section:

Data Warehousing Wednesday, May 19, at 4:00 p.m.

Phil Gold is a partner

in GGY Inc., the devel-

opers of the AXIS

Actuarial System. Phil

holds three passports,

writes on music and

stereo, and has at one

time or another been

a shepherd, a teacher,

a tank gunner, a

programmer, a VP

of underwriting, an

actuary, and an object

of derision. Phil lives

in Toronto and is

deteriorating rapidly.

He can be reached

at [email protected].

6 • CompAct

• Computer Sessions continued from page 5 •

We are sponsoring two sessions in San Antonio in June:

Actuaries versus IT – The Perpetual Dilemma Monday, June 14, 10:30 a.m.Moderator: Michael Hartfield.Panel: Frank Reynolds, Susie Lee, Emil B. Kraft

Actuaries and IT people rarely see eye-to-eye. Neither understands the other’s perspective, withfrustrating results for both sides. Yet, with some tolerance and understanding, both can profitfrom the other’s approach.

Actuaries expect IT to:• Adjust priorities to quickly implement important changes• Help fix program bugs and technical difficulties• Design programs for easy modification with little verification• Provide error-free systems that are already tested

IT expects actuaries to: • Write specs from which include all reasonably foreseeable modifications• Write specs IT can program • Rigorously test all system calculations

Business Intelligence for ActuariesTuesday, June 15, 10:30 a.m.Moderator: Michael K. RigbyLecturer: Michael Levine

This session covers:• Context and general background on business intelligence (BI) and online

analytical processing (OLAP)• How to build a successful application• How to integrate with other applications

Attendee benefits include:• A survey of products and approaches• Examples of actuarial/financial applications

We are also co-sponsoring a workshop with the Financial Reporting Section:

Applications of Stochastic Modeling in Financial Reporting Tuesday, June 15, 2:00 p.m. :

CompAct • 7


If you have been a member of theComputer Science Section for a whileyou know about the Table Manager

tool. You may have used it to access astorehouse of mortality, morbidity andlapse tables. You probably know this sec-tion worked with ACORD, the life insur-ance standards organization, to general-ize the table manager format into XML(eXtensible Markup Language). This newstandard is called XTbML (sometimespronounced as “x-table”).

Moving the table format to ACORD XML issort of like sending our child into theworld. The little being we nurtured tohandle our mortality/morbidity tables isbeing asked to carry all kinds of data.Insurance companies are trying to useXTbML to carry premiums, cash values,commission rates, modal factors, divi-dend rates and interest rates. And theseare just the uses I know about.

Beginning with this issue of CompAct, I’mgoing to highlight business use cases whereXTbML can define the tabled data.

My first example is from the life insurancepractice area. If you’re willing to e-mail mea table data business case from your prac-tice area, I would be happy to represent itusing XTbML in a future CompAct newsletter.

Business Use Case: Tabular Cash ValuesThis table provides cash values for a 20pay whole life insurance plan. After pay-ing 20 annual premiums, no additionalpremiums are paid. The cash values arebased on issue age and duration (duringthe premium paying period) or justattained age (after the premium payingperiod). Duration 1 is the first year fromthe policy’s issue date and the cash valuesin this table are end of duration cash values. If you have a whole insurancecontract, you will find that it contains atable of cash values. Below you’ll findversions examples of these cash valuetables truncated by issue age or attainedage to save space.

Now let’s look at how this table might berepresented in XTbML. Keep in mind thatXTbML is a self-defined machine-readable

Standards: XTbML Tables Beyond MortalityBy Dean Slyter

Premium Payment Period Cash Values

Duration

IssueAge 2

0.390.7

0.97

314.8115.6316.43

429.8131.1532.47

545.3847.2549.12

661.5663.9766.39

778.3581.3384.35

895.7999.36

103

9113.92118.11122.38

10132.77137.6

142.54

Duration

12172.78178.98185.32

IssueAge 13

194.01200.93208.01

14216.12223.78213.62

15239.13247.56256.17

16263.09272.3

281.69

17288.02298.03308.22

18313.98324.8

335.83

19340.98352.66364.57

20369.11381.7

394.56

AttainedAge

Paid-Up Period Cash Values

55381.7

56394.56

57407.75

58421.26

59435.09

60449.23

61463.64

62478.26

63493.03

64507.91

1000

11152.38157.88163.51

Dean Slyter is actuary

and account manager,

LIDS Consulting in

Woodridge, Ill.

He can be reached

at [email protected].

• XTbML continued from page 7 •

8 • CompAct

<?xml version="1.0" encoding="UTF-8" ?> - <XTbML id="Table1" Version="XTbML2.9.91"

xmlns="http://ACORD.org/Standards/Life/2"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://ACORD.org/Standards/Life/2 XTbML2.9.91.xsd">

- <ContentClassification><ContentType tc="45">Cash Value</ContentType><TableName>Tabular Cash Value</TableName>

</ContentClassification>- <Table>

- <MetaData><ScalingFactor>0</ScalingFactor> <DataType tc="5">Currency</DataType><CurrencyTypeCode tc="840">USD</CurrencyTypeCode> <TableDescription>Issue Age -

Duration</TableDescription>- <AxisDef id="A_1">

<ScaleType tc="3">Age</ScaleType><AxisName>Issue Age</AxisName> <MinScaleValue>35</MinScaleValue> <MaxScaleValue>45</MaxScaleValue><Increment>1</Increment>

</AxisDef>- <AxisDef id="A_2">

<ScaleType tc="4">Service</ScaleType> -  <AxisName>Duration</AxisName> <MinScaleValue>1</MinScaleValue><MaxScaleValue>20</MaxScaleValue><Increment>1</Increment>

</AxisDef></MetaData>

- <Values>- <Axis AxisDefID="A_1" T="35">

- <Axis AxisDefID="A_2" T="1">

format for transferring and storing tabled data.If a machine can read this format, so can you.I’ll give you a few XML reading hints to help youread it:

1. XML provides opening (e.g.: <Table>) and closing (e.g.: </Table>) for each data element. A lot of child data can be represented between the opening and closing of a data element.

2. The content classification defines the type of values contained in the table. It uses a typecode (tc) lookup table where typecode ‘45’ is cash value.

3. The cash values are represented in two stacked tables, first the pre-mium paying period table, then the paid-up period table.

4. A low duration of 21 has been defined for the paid-up period table to clarify the first duration where it should be used. The data element <Continuous> indicates that the duration 21 cash value is used until a higher duration cash value is given.

Cash value tables are used for product pric-ing, financial reporting, sales and illustrationand policy administration. If each such sys-tem could import and export data in theXTbML format, data transfer between any twosystems would be possible.

Please send your tabled data business casesand XTbML questions to me [email protected]. :

CompAct• 9

<V>0</V> </Axis>

- <Axis AxisDefID="A_2" T="2"><V>0.39</V>

</Axis>- <Axis AxisDefID="A_2" T="3">

<V>14.81</V> </Axis>



<V>45.38</V> </Axis>



<V>78.35</V> </Axis>

- 


- <Axis AxisDefID="A_2" T="1"><V>0</V>


<V>0.7</V></Axis>



<V>31.15</V> </Axis>



<V>63.97</V></Axis>


</Axis>

</Axis>

</Values></Table>- <Table>

- <MetaData><ScalingFactor>0</ScalingFactor> <DataType tc="5">Currency</DataType> <CurrencyTypeCode

tc="840">USD</CurrencyTypeCode><TableDescription>Attained

Age</TableDescription>- <AxisDef id="A_3">

<ScaleType tc="3">Age</ScaleType>

<AxisName>Attained Age</AxisName><MinScaleValue>55</MinScaleValue>

<MaxScaleValue>99</MaxScaleValue><Increment>1</Increment>

</AxisDef>- <AxisDef id="A_4">

<ScaleType tc="4">Service</ScaleType> -  <AxisName>Duration</AxisName><MinScaleValue>21</MinScaleValue> <Increment>1</Increment><Continuous>1</Continuous>

</AxisDef></MetaData>

- <Values>- <Axis AxisDefID="A_3" T="55">


</Axis></Axis>

- <Axis AxisDefID="A_3" T="56">- <Axis AxisDefID="A_4" T="21">

<V>394.56</V> </Axis>



</Axis></Axis>

- <Axis AxisDefID="A_3" T="58">- <Axis AxisDefID="A_4" T="21">

<V>421.26</V> </Axis>

</Axis>

</Values></Table>

</XTbML>

10 • CompA ct

P versus NP,or How long will this take?by Carol Marler

For a new and creative solution to this ques-tion, you could win a prize of $1,000.000.

On May 24, 2000, the ClayMathematics Institute offeredprizes of $1 million each for seven

unsolved problems in mathematics.Landon Clay (whose major at Harvard wasnot in mathematics, but in English) want-ed to get more media attention for the fieldof mathematics. His personal fortune wasused to found the Clay MathematicsInstitute (www.claymath.org).

The idea for these "millennium problems"came from an address given by mathe-matician David Hilbert in 1900, in whichhe listed the most significant mathemat-ics problems at that time. One problemon his list was Fermat's Last Theorem,which was solved recently.

Although the solution to Fermat's LastTheorem made extensive use of comput-ers, the theorem itself does not refer inany way to computers. Very few prob-lems in theoretical math, even today, fallunder the heading of computer science.Most computer science questions arepractical, not theoretical. But lookingback in history, some theory was devel-oped even before any modern computerswere built.

The theoretical basis of computer sciencewas developed by a mathematiciannamed A.M. Turing. His model of a com-puter was a black box of sorts with n dif-ferent internal states, and an input “tape”

consisting of zeros and ones. The opera-tion of the black box was defined by atransition matrix in which the currentstate and the current value on the inputtape determined the next state.

The amazing thing about this model isthat any real-world computer programcould be mimicked with a suitable set ofstates and a transition matrix. The theo-reticians can then analyze whole familiesof programs. The P versus NP millenniumproblem considers a grouping of pro-grams based on a formula for how long(at most) it takes to process an inputstring of length n.

Programs whose time formula is a poly-nomial in n are denoted as P, for polyno-mial time. For example, consider aprogram that sorts N values in ascendingorder. A simple way to do this is to loopthrough the list, comparing each pair ofnumbers (N-1 comparisons), and switch-ing them if the second number is smallerthan the first. This loop would then berepeated N-1 times, until all the numbershad been sifted through to their properposition in the sorted list. If each com-pare and switching took k units of time,the formula is k*(N-1)*(N-1) plus somesmall amount of time to set things up,etc. The time required is clearly a seconddegree polynomial. Improvements arepossible, such as noting that each itera-tion of the outer loop reduces by one thenumber of comparisons required, sincethe last i+1 entries are now determined.

Incidentally, changing a list of n zeros andones into N values is also known to be apolynomial process. But some processessimply cannot be done in polynomialtime. For example, consider the “travel-ing salesman” problem. The salesman (orwoman, but let's use male pronouns forsimplicity) has a list of Ndestinations thathe wants to visit. There is an N*N matrix ofthe distances from each destination to anyother. The salesman wants to minimize

Carol Marler is a

former editor of the

Speculative Fiction

contest. She is

currently unaffiliated.

She can be reached

at 704-948-0545.

CompAct• 11

time on the road, and this can be pro-grammed by listing all possible routes, cal-culating the length of each route, and thentaking the shortest. If there are only five orso cities, this can even be accomplished byhand. But it is apparent that as N increas-es, the program will require N! distance cal-culations. And N! for large N growsexponentially. You don't want to kick off acalculation for 50 destinations if you expectthe answer in a reasonable length of time.

The next step in analyzing this kind ofproblem is to generalize the computermodel. One very useful variation, and onethat is rather actuarial in nature, is to re-place the deterministic computer with anon-deterministic one. Instead of definingthe resulting state, the transition matrixproduces a probability distribution ofstates. Nobody has yet built a non-deter-ministic computer, but if we had one, thetraveling salesman problem could besolved on it in polynomial time.

In fact, the traveling salesman problem isjust one of a whole family of problems forwhich the individual calculations are simpleenough, but the number of possibilities tobe tested grow exponentially. These prob-lems are denoted as NP-complete.

In 1971, Stephen Cook proved that if anyNP-complete problem could be solved on adeterministic computer in polynomial time,then all of them could be similarly solved.The millennium problem then asks whetherthese NP-complete problems are a subsetof P, meaning that they could be solved de-terministically with the right kind of algo-rithm, or are they inherently not solvable inthat way?

In the real world, the Cook analysis leavesjust two possibilities. First, if a problem isknown to be NP-complete, and a softwaredeveloper doesn't want to waste time look-ing for a solution that has already been

sought so many times without success, the problem can be discarded and the soft-ware can be built for some other probleminstead. A great many problems of impor-tance to business and industry have beendropped for this reason. Some other prob-lems of less practical value also fall into thiscategory—you may want to Google “NP-complete” just to see how many times theminesweeper game pops up. Second, ifsome genius inventor in a garage can findthe answer to any particular NP-completeproblem, he will soon be richer than BillGates (even without the millennium prize).

However, if you think it is important to havethe ability to do secure online transactions,you had better hope that the genius is notsuccessful, since the most commonly usedsecurity method is based on a known NP-complete process—factoring a very largenumber into its prime factors. If NP-com-plete turns out to be a subset of P, themethodology will no longer be secure.

If, in fact, NP-complete is not a subset of P,the millennium prize will go to the personwho can prove that fact. A great many at-tempts have been made to do this, andnone have yet been successful. It could beanother century before this problem getssolved. Some even theorize that it is truebut cannot be proven.

One intermediate possibility also exists.Even if NP-complete is a subset of P, thepower of the required polynomial mayprove to be so large that no practical solu-tion is available. A 50-degree polynomial isnot much of an improvement on an expo-nential formula if the traveling salesmanhas no more than 50 destinations.

Do you and your PC have the insight andCPU cycles to attack the problem? If so,see http://www.claymath.org/millenni-um/ to get entry information. :

Nobodyhas yet built a

non-determin-istic computer,

but if we had one,the traveling

salesman problem could be

solved on it inpolynomial

time.

Using Excel to Prototype Relational DatabaseApplicationsby Dave Snell

Excel is often dismissed by infor-mation technology (IT) profession-als because it is viewed by them as

a toy—suitable only for single-user appli-cations. Actuaries know the value ofspreadsheets in rate calculations andwhat-if scenario testing; but most actuar-ies have never realized that their conven-ient little spreadsheet tool is also capableof lifting some rather heavy weights in thedatabase environment.

In my opinion, Microsoft Excel is a high-ly functional programming languagethat happens to come with an awesomebuilt-in grid control. It is the one of themost cost-effective tools I know of formaking high-quality small applicationsthat can be used on almost any corpo-rate desktop. It can also be great for pro-totyping larger database applications.

I propose to show in this article that youcan develop and test relational databaseprojects from your spreadsheets. Youcan do it rapidly, and with less angst orcost than you might feel by goingthrough the learning curve of the bigdatabase tools, like Microsoft’s SQLServer or Oracle. I’ll assume you havesome familiarity with Structured QueryLanguage (SQL), which we won’t coverhere. I also assume you’ve some expo-sure to Visual Basic for Applications(VBA) and that you may have evenheard of Active Data Objects (ADO); butthis can be minimal. Once you follow theinstructions for entering the code below,you can embellish it at your leisure. Thisarticle is to help get you started usingSQL from within Excel.

Using SQL with Excel sourcesExcel has the capability to contain multipletables and ranges, and it can be used asa source for SQL queries.

An easy way to accomplish this is via ADO.First, you must add a reference to ADO inyour project in the VBA mode, which youcan reach quickly from the spreadsheetmode via Alt-F11 (hold down the Alt keyand press the F11 key). Once in VBAmode, click on Tools, References.

There are several ADO libraries. I recom-mend using the lowest numbered one thatmeets your needs so that your workbookwill be more transportable. In this case, Ichose version 2.5, which probably camewith Excel 97.

Next, you are going to need a little VBA helpin the background to let you effectively usethis conduit to SQL capability. Click Insert,then Module to get a working area. Thencopy and paste the code in from this article.

The first item to copy is a Sub to readfrom an Excel database via Active DataObjects (ADO).

12 • CompAct

Sub CopyFromXLDatabaseADO( _sDBRangeBook As String, _sSQLRangeName As String, _sDestinationSheet As String, _Optional bolClearSheet As Boolean = True, _Optional bolShowFieldNames As Boolean = True, _Optional sDestRange As String = "A1")

'written by Dave Snell'This Sub allows you to perform SQL queries against'tables (named ranges) in an Excel workbook

Dim cn As New ADODB.ConnectionDim rs As New ADODB.RecordsetDim fld As ADODB.FieldDim lngOffset As LongDim lngStartTime As LongDim cmd As New ADODB.CommandDim s As StringDim sSQL As StringDim I As Integer

lngStartTime = Timer

Sheets(sDestinationSheet).ActivateIf bolClearSheet Then

Cells.SelectSelection.ClearContents

End If

'populate with new information from the Access database extractWith cn

'PROVIDER STRING follows (change for different source type e.g., Oracle).Provider = "Microsoft.Jet.OLEDB.4.0" 'If you use a 'different'version of 'the Jet engine, change the 4.0 to whatever you use..Properties("Extended Properties").Value = "Excel 8.0" 'If you'use Excel '2000, this is correct; Excel 2002 is 9.0, etc..Open ActiveWorkbook.Path & "\" & sDBRangeBook

End WithWith cmd

Set .ActiveConnection = cn'build a SQL query from the text on the SQL worksheet

With Sheets("SQL")s = " "sSQL = ""While Len(s) > 0s = .Cells(.Range(Range(sSQLRangeName).Address).Row + I, _

.Range(Range(sSQLRangeName).Address).Column).ValuesSQL = sSQL & " " & sI = I + 1

WendEnd With.CommandType = adCmdText.CommandText = sSQLSet rs = .Execute

End IfEnd With

If Not rs.EOF ThenWith ActiveSheet.Range(sDestRange)

If bolShowFieldNames Then'write field names as column headersFor Each fld In rs.Fields

.Offset(0, lngOffset).Value = fld.NamelngOffset = lngOffset + 1

Next fldEnd If

End With'copy the rest of the records one row below the headers


CompAct • 13

ActiveSheet.Range(sDestRange).Offset(1, 0).CopyFromRecordset rsActiveSheet.UsedRange.EntireColumn.AutoFit

ElseMsgBox "Error: No records read", vbCritical

End If

Range(sDestRange).SelectMsgBox "Done!" & vbCrLf & "Elapsed time = " & Timer - lngStartTime & " seconds"

ExitRoutine:rs.Closecn.CloseSet cn = NothingSet rs = NothingSet cmd = Nothing

End Sub 'CopyFromXLDatabaseADO

This Sub should be usable for many situations.Note however that for readability of the sam-ple code, I removed a lot of error checking.You will want to add error checking in variousportions of the Sub. Since this approach usesADO for the data access, you can also use itto query Access databases, Oracle databases,etc. with a mere change of the Providerstring. In practice, I have written a longer

version of this macro that determines thesource type from parameters I set andchanges the provider string accordingly (afuture article). Rather than get complicatedhere though, let’s just go ahead and try anapplication showing how you might make useof the supplied Sub.

Let’s create a very simple calling routine:

14 • CompAct

This tells the Sub CopyFromXLDatabase tolook in the current workbook (we could alsorefer to any other workbook) for the RangeName “SQL_Example.” Then, it reads downthe column from cell SQL_Example and buildsthe SQL query until it encounters a cell thatis blank. The final set of results is then copiedto the Excel spreadsheet “Example_Output.”

All of this looks OK, but obviously nothing isgoing to happen unless we actually have arange named “SQL_Example” somewhere inthe workbook. Let’s assume we have a sim-ple workbook with just three worksheets in it.I’ll name my three sheets Example_Source,SQL and Example_output.

In Example_Source we’ll place our tables.Assume we have two tables of data as shownin Figure 1 on page 15.

Note that I have the Applicants table high-lighted. Likewise, I named the Apps table(cells G4 through I25) “Apps.” The easy wayto accomplish this naming process is to high-

light the desired area with your mouse, thentype the desired Range name in the input boxjust above cell A1. In this screen shot you cansee Applicants in this input box.

Next, we look at the SQL sheet, Figure 2 onpage 15.

Cell A4 on this sheet is the one I namedSQL_Example. I also made sure I have anoth-er sheet for my output. This one is namedExample_Output.

Everything is in place. Now, let’s run themacro (Figure 3, page 15).

Clicking on Tools, Macro then Macros (or justpressing the shortcut key Alt-F8), we see a listof the macros available (in this case just the onetest macro).

Run the macro Test_CopyFromXLDatabaseand voila! You have the SQL query outputright away on the Example_Output sheet.(Figure 4, page 16).

SubTest_CopyFromXLDatabase()CopyFromXLDatabaseADOThisWorkbook.Name, "SQL_Example", "Example_Output"

End Sub 'Test_CopyFromXLDatabase

• Using Excel continued from page 13 •

CompAct• 15


Figure 1

Bigger and BetterLet’s face it. This is a very simple example. It’shardly worth writing a VBA Sub to accomplishthis task. You could have obtained similar resultswith Excel’s Data, Filter, Advanced Filter featureor with its built-in QueryBuilder (accessed viaData, Import External Data, New DatabaseQuery) capability.

I wrote (and use) the VBA approach for a coupleof reasons. First, it allows me to get very flex-ible with my SQL statements—beyond the lim-its of the built-in Excel tools. In a report I wrote

Note the Range Name “SQL_Example”assigned to cell A4 of this sheet.

Figure 2

Figure 3

for one of our foreign offices in Asia, I use aSQL query more than 100 lines long thatinvolves a 16-table join with dozens of specialcriteria. Second, I like the fact that once Idebug my routine on a small scale against anExcel or Access database, I can just changethe provider string in my Sub and use exactlythe same SQL and VBA code to scale this upto a huge Oracle database.

In my presentation on relational databases atthe SOA Annual Meeting in Orlando last year,I showed a video of how much faster a bigchainsaw can cut a big black walnut log than anelectric circular saw or a hand-held scroll sawcan. In the beginning of the video though, Ishow that if you use the chainsaw the same

way you use a handsaw, by rubbing it backand forth against the wood, it gives very dis-appointing results. My point here is that thebigger database tools, like Oracle and SQLServer, have a serious learning curve associ-ated with their efficient usage. The actuarydoes not have to become a database admin-istrator (DBA) in order to intelligently proto-type a large database project. Excel, supple-mented with VBA and ADO, provides a veryconvenient tool for trying out your ideas, andalso gives you a scalable solution you canlater turn over to your IT associates as aworking set of specifications for the big toolimplementation.

Happy prototyping! :

16 • CompAct

• Using Excel continued from page 17 •

Figure 4

475 N. Martingale Suite 600 • Schaumburg, IL • 60173 • www.soa.org

compact, may 2004, issue no. 17. - member | soa · 2012. 1. 19. · compact electronic newsletter...

Documents