"~I

I \ ~

I \ ..

I ~

t

1

THE USE OF SAS/IML SOFTWARE IN CALCULATION OF CERTAIN CONSUMER PRICE INDEX NUMBERS

Martin Gregory - Central Statistics Office, Ireland

Introduction

The Consumer Price Index (CPI) is designed to measure the change in the level of Ule prices paid by private households for consumer goods and services. It is compiled by the Central Statistics Office (CSO) on a quarterly basis in respect of the middle Tuesday of the months of February, May, August and November each year. An existing COBOL based system validates Ule price quotations collected each quarter and produces tIle overall index and the sub-indices for the 10 major commodity groups which comprise the published results.

In addition to these basic results, index numbers for various groupings ofitems·are required for national use and also by the Statistical Office of the European COllllUunities (Eurostat). These extra series number in the hundreds and were traditioually computed manually or iu receut years usiug a·SAS PROC MATRIX based program based on the output ofthe COBOL system. The subject of this paper is the SAS/IML Software based system wlUch I developed to provide a flexible front end to the existing CO BO L CPI processiug system to allow the production of iudices for any combination of items.

2 CPI Methodology

The Consumer Price Index measur,es, in index form, changes over tinle in the. cost of purchas­ing a constaut represeutative "basket" of consumer goods and services. The fixed quantity of each item in this basket is proportional to the amount wltich the average household purchases. Identical products are priced in the same shops on eacll occasiou so that cllanges in the cost of this constant basket reflects only price changes. The fixed quantities or index weigltts are updated every seven years or so from the results of largescale national Household Budget Surveys. As a consequence there are several sets of index number series, spanning the periods between Housebold Budget Surveys .

. Consumer goods and services covered by the curreut index are classified into 10 main commodity groups (e.g. Food), 159 item groups (e.g. Beef), and 728 items as shown in Table 1. The individual items are priced in a fixed panel ofrepresentative retail and consumer service outlets throughout the country. In all about 35,000 individual pri~e quotations are collected each quarter.

The iudex is calculated as a Laspeyres type iudex with fixed weights, namely:

where:

Ie = b_qoPe X 100 2.: qoPo

• qo and Po are tl~ quautity (fixed) aud uational average price, respectively,of an item in the base (0) quarter;

• Pc is the .national average price of the item in the current (c) quarter;

• 2.: represents summation over all the index varieties.

38

~"

Table 1. cpr Commodity Groups Commodity Group Item Group Items F~od 50--iz6-

Alcoholic Drink Tobacco Clothing and Footwear Fuel and Light Housing Durable Household Goods Other Goods Transport Services and Related Expenditure

3 3

24 7 5

13 20 13 21

13 4

68 10 24 70

116 92

199

A modified version of this formula is used for calculation purposes. For the first quarter after the base we can write this as:

I:qp~ I:~~ ~v I - 0 °po x 100= ___ 1'_" X 100 = _L, __ l X 100 1 - I: qoPo I: Vo I: Vo

where

• Vo is the base value weight; (i.e. fixed base quarter quantities valued at base quarter prices)

• Vl is the first quarter value weight; (i.e. fixed base quarter quantities valued at price levels in the first quarter)

for each individual item. These item value weights are updated each quarter using thequar­terly percentage price changes based on matched sets of price quotations for identical items. The index for the current quarter (c) is calculated as:

I:Ve-l x ~ x 100 I:v Ie = I: po= ~ c x 100 Vo L,VO

This allows for the most efficient use of price quotations-a greater match of prices for identical items is possible between the current quarter and the last quarter than between the current quarter and the base quarter, which could be up to seven years earlier. The resulting updated item value weights can be combined additively to produce price index series for any group (G) of items as follows:

[9 _ I:9 Ve x 100 c - 1:!1 Vo

3 Requirements for Sub Index Numbers

From the individual updated item value weights we wish to:

• generate index numbers for any grouping of the items;

39

,.

t

r ',-,

• link tbese index numbers for tbe different CPI series (currently available for bases mid November 1968, 1975, 1982 as 100. Tbese are tbe years for WIticll Housebold Budget Survey results are available);

• display tbe index numbers to any specified quarter as base 100;

• display tbe index numbers to any specified year (aritbmetic mean oUbe four constituent quarters) as base 100;

The program must be independent of tbe number of items aud of the number of quarters to allow for possible variatiou for differeut index series.

4 Implementation

I decided to write a separate program for eacb of tile 4 differeut tasks. Because we have quarterly item value data for successive CPI series, alld because there are possible applications for otber price index surveys, I decided to write tbe programs in as general a fasItiou as possible.

Now, the obvious choice for generalisation is the SAS MacrQ language, but these systems were intended to IUn under VMS2 where the Macro language is not yet available. However, oue can still acllieve quite good results even 'with the limited Macro availability in SAS under VMS. Tltis was demoustrated by a number of papers at Seugi '88.

5 Calculating the Index Numbers

The progralll INDEX.SAS calculates index number series for any number of groupings of tbe basic index items. The input to the program is a SAS data set with an observation for each quarter. The variables are DATE, a SAS date variable, and one real numeric variable, Vnnn, for each item value weight. We call represent the items as a matrix V = (vI) with the rows (subscripts) being quarters and tbe columns (superscripts) items. If the subgroup of items we are interested in is represented by the set S = {nl' n2, n3, ... } then the columu vector

""'stA L.J)E • 100' 8.=---. X EjEs 11{

will coutaiu the index time series for tItis subgroup calculated to tbe base quarter as 100. In SAS /IML tItis is easy to acllieve:

subset ={nl n2 n3 ... } ;

s = ( 100 '" v[,subset][,+] ) I v[,subset] [1,+]

TillS is all extremely compact uotation. Specifically, there is uo refereuce to either the number of rows or coluullls and tItis satisfies the condition for indepeudeuce of the number of quarters or items listed iu sectiou 3 above. Tltis, essentially is the core of tile program. The remaiuder is concerned with suell things as choosiug the required items for the particular groupiug, choosing the particular base period or outputting the results.

40

i

In general, multiple sub-index groupings will be required at anyone time, for example, 104 ESA 3-digit bousebold consumption sub-indices are required by Eurostat for Purcbasing Power Parity index number updating. For tbis reason tbe program is designed to calculate index numbers for any number of item groupings at tbe one time. Tbe program is called (using %INCLUDE) after tbree macro variables bave been defined:

IN names tbe input SAS data set containing the item value weights;

OUT names tbe data set in wbicll tbe resulting index numbers will be stored, one variable per sub-index grouping;

REQUEST names a program file wbicb, for eacb sub-index grouping, defines tbe items making up tbe grouping, and provides a SAS variable name for tbe output data set .

. Tbis file is %INCLUDEd by INDEX.SAS.

The INDEX program bas tbree pbases: initialisation, index calculation and creation of tbe output data set.

In tbe initialisation pbase tbe quarterly item value weigbts data set is loaded into a matrix (VALUE), tbe matrix (CPISUB) from wbicb tbe output data set is to be created is set up as a columll vector containing t.be dates, and a row matrix (NAMES) to bold the variable names is created with 'DATE' as the first name. The final action is to define the module (SUBINDEX) which will calculate tbe coliInlll vector containing the index, append it to CPISUB, and append the variable name to the NAMES matrix.

Tbe calculation pbase consists of a single statement wbicb includes tbe REQUEST file. Tbis file defines tbe row matrix SUBSET wbich identifies tbe columns of tbe VALUE matrix wbicb belong to tbe subgroup, defines tbe 1 x 1 matrix N containing tbe variable name, and calls tbe SUBINDEX module.

Tbe final pbase creates tbe output data set from tbe matrix CPISUB.

6 Rebasing

Tbe second task is tbe rebasing of a set of index numbers and is accolliplisbed by tbe program REBASE.SAS. There are two types of rebasing necessary. Tbe first is to simply use anotber quarter as the base. Tbe·second type ofrebasing is where we want to use tbe allllual average. oftbe four quarters of one year as the base. Again the program is invoked using %INCLUDE after tbe following three macro variables have been given values:

IN is tbe input data set wbich will bave been created by a previous run of either tile IN­DEX.SAS, REBASE.SAS or LINK.SAS programs;

DATE specifies the period to be used as base;

OUT is tbe data set wbicll will contain the rebased series.

Tbe first. type. of rebasing is simply a rescaling of tbe index to a particular. quarter. We can represent a single index number series as a column vector, (m;). Tbe November 1975=100 series, for example, spans the 29 quarters from November 1975 to November 1982. If we wanted, say November 1980 = 100, we would need m21 = 100 and this could be achieved by the transformation:

41

t.

~: ;:

~;: ~.

l~

r~.

r ~. ,. K

~ ~ J 1~ ~ " " ~,' t"

" roo ~.

~.

~~

!: ~;: ;'

~ ~ I I, i \

\.

I \..

I ~

Tni ni = -- x 100

m21

If we are dealing with more than one series, we have a matrix rather than a column vector and this becomes:

. m~ ni = ---j- x 100 7n'31

The second type of rebasing is where we want to use tbe annual average of the four quarters of one year as the base. III this case the transformation is:

j

n~ = mi. x 100 , EiE19S0 mi/4

As the ilser would have to specify three parameters for this program (input data set, output data set, date) I didn't want to also include another value to specify whether the base was a single quarter or an annual average of four quarters. To get over tlus problem I decided to use format in which the date is specified to indicate which type of base is required. If the date is specified in the YYQ6. format this would imply a single quarter, if a four digit year is used, then an annual average would be used. Implementing tIus involved using a DATA step to assign an appropriate value to a macro variable TYPE as follows (&:DATE is the supplied date, &:IN the input data set): .

data _null._; if index (upcase ("lIdate")., "Q") then do;

date=input("lIdate",yyq6.); call symput(ntype","Q"); end;

else do; date=mdy(1,1,input("lIdate",4.»; cail syniput("type","A"); end;

call symput("date" ,put (date ,date7.»;

data base; set lIin; if "lItype" = "Q" then if date="lIdate"d then output; else if year(date) = year("lIdate"d) then output;

The data set WORK. BASE will have 1 observation if the base period is a single quarter and 4 observations if it's to be an annual average. In either case we can treat it identically in SAS/IML:

use base; read all var _num_ into base; base = base[:,2:ncol(base)];

The subscript reduction operator ":'" calculates the mean for each column so if BASE is a 4 x n matrix, after the operation is will be 1 X n. If it starts off as 1 x n then the operation has no effect.

To complete the rebasing we read from the input data set'into a n~atrix (CPI). As we don't want to 'rebase' the date we split the first colum11 off into another matrix (DATE).

42

;'

i-'

;}:

i " i" '~ ,

~; . . :;. E 1'1" "fk ~. ,'~.

:t ,'i"' ii.: 'l

r r ;! {."

;~' "" . . , { '>. L ~,:

;;;. i ~ ~, ~; f~ ";'1 .t'!J

;1 ~.

'f~ ~ .~.,

'Ii \ t: \ ~ ~ ?~ .~' :~~

~: " ,,-;;,

~: ~

use tin; read all var _num_ into cpi [colname=name]; date = cpi[, 1] ; cpi = cpi[,2:ncol(cpi)]j

For eaell column of CPI we want to divide by the appropriate .element of the row vector base. As the element-wise division is the only division operator in SAS/IML the easiest way to do this is to create a matrix of the same dimension as CPI by vertically concatenating BASE with itself. The rebase index can be calculated as follows:

base = repeat(base,nrow(cpi),1) rebase = date I I « cpi • 100 ) I base )

The ability to use a function like REPEAT rather than having to use a loop, makes the program simpler to read.

7 Linking Two or More Series

The third task we wish to perform is to link separate CPI series covering different time spans. This is accomplished by the LINK.SAS progralu which links two series. To combine lilore than two series an iterative process is used. Before calling the program the following macro variables must be defined:

INI is the data set containing series for the earlier time span, whiell will have been created by a previous run of either the INDEX.SAS, REBASE.SAS or LINK.SAS programSj

IN2 is the data set containing series for the later time span, whiell will also have been created by a previous run of either the INDEX.SAS, REBASE.SAS or LINK.SAS programs;

OUT is the data set which will contain the linked series.

The two series we wish to link must have a COlUmon period-the last quarter of the first series and the first quarter of the second series. If a1 is the earlier set of index numbers and b;. the later then we wish to transform B as follows:

where at. is the last quarter. We then verically concatenate A with C, omitting the. first row of C. To achieve this inSAS/IML without using loops I used a trick similar to that in the REBASE.SAS program: build up the transformation matrix from the tow vectors at. and b{:

c = b[2:nrow(b),2:ncol(b)] # repeat (a[nrow(a) ,2:ncol(a)] ,nrow(b).c1,1) I repeat(b[1,2:ncol(b)],nrow(b)-1,1) ;

I then appended this matrix to the first input data set. Note that linking is done at the level of the same sub-index grouping for .each series, not

at the individual item level. This meaus that the item composition of a grouping can vary for different CPI time spans-new items can·be introduced, former items can be dropped between successive ConsUliler Price Indices.

43

. ~

8 Printing the Results

The final task is the presentation of the resulting index numbers. The normal format for presentation of time series data used in the eso is one row per year with one column per quarter. I have not used SAS/IML to produce the output but have used instead a DATA _NULL_ step where more control is possible. Luckily for me, one of my colleagues has had to solve the problem previously and has produced a macro which will print a data set of time series, using the variable labels, in just such a format. I was able to use the idea behind this macro to write a program to obtain the desired result.

What one really wants to do is print all the values of the first variable, then of the second, aud so on, until all variables have been printed. The trick is to transpose the data set using the DATE as the ID variable. We start with a data set looking like the one in Table 2 and end up with one like that in Table 3 .

Table 2. Before Transpose DATE VARl YARn 1975Q4 x x

1982Q4 x x

Table 3. After Transpose _l982Q4

VARl Index 1 x x

YARn Index n x x

Now, a DATA _NULL_ step using an array starting with variable _l975Q4 and ending with _l982Q4, will allow us to produce the desired output format. A number of formatting details need to be looked after but I won't go into that here. One detail which is not obvious is how you get the start and end dates which you need both for definition of the array and for labelling the years. Before transposing I used a data step to read the first and last observations and assigned the values of the date variable to two macro variables which would be available in the next step (In SAS under MVS3 , you could dispense with this and use the VNAME function) .

9 Difficulties Encountered

The main difficulty I encountered related to the use of parameters in module definitions. The problem is that if you use a parameter in the definition then all matrices referenced within the module are local. . In particular, if you want a module to modify a matrix, that matrix must be passed as a parameter. To illustrate this point, consider the following SAS/IML program:

proe iml ; use epiexp82 read all var _num_ into value

44

start Bubindex(sub) ; value sub = value[,sub]

finish ; subset = {1 3 5} ; run subindex(subset)

Tbis program defines a module SUBINDEX wbicb is to create a subl1latrix of tbe matrix VALUE, cboosing tbe columns specified in a parameter passed to tbe module. Tbis program will generate an error because tbe matrix VAL UE does not exist in tbe referencing environment of tbe module. To make it available you must pass it as a parameter. Tbe modified lllodule definition and call:

start subindex(sub,val) valuesub = val[,sub] ;

finish ;

run subindex(subset,value)

will not produce an error but will create tbe matrix VALUESUB only witbin tbe referencing environment of tbe module. To make tbis available at tbe top level you need:

start subindex(sub,val,valsub) valsub = val[,sub] ;

finish ;

run subindex(subset,value,valuesub);

You can see, bowever, that tbe list of parameters is getting out of band-in tbe actual SUBINDEX module in INDEX we would bave to pass SUBSET, BASE, N, VALUE, CPISUB and NAMES.

Tbe 'problem' is solved by using no parameters at all in tbe module definition. Tbis means tbat all matrices are global. It would be nice if it were possible to declare matrices as global or local as required, as can be done for macro variables.

10 Future Developments

Tbe systelll I described is by no means a completed one. Tbe user interface is not particularly inspiring-long lists of item numbers bave to be entered and tbere is currently no cllecking tbat tbe correct item bas been elitered. To solve tbis I would like to put a SASj AF interface to tbe system wbicll would allow tbe user to select items by name and would tben generate tbe subset vector. Tbe menu systel11 could also allow users to run Ule rebaSing, linking or printing prograills and could validate parameter values entered.

Tbe otber major task is to autoillate the updatiilg of tbe input data. set of item value weights ",ben tbe main CPI processing system is finisbed.

Notes

1. SASand SASjIML are registered trademarks ofSAS Institute Inc.

45

\

\..

2. VMS is a registered trademark of Digital Equipment Corporation.

3. MVS is a registered trademark of Intemational Business Macbines Corporation.

Acknowledgements

I would like to tbank Donal Murpby for bis constructive criticism of an earlier draft of tbis paper, Adrian Redmond for bis idea for tbe print program, and Tom Keane for encouraging me to write tbepaper.

References

Irisb Statistical Bulletin, Volume LVIII No.1, Marcb 1983 contains an article describing tbe November 1982 based Consumer Price Index methodology.

46

i: ,

Y. ~ ;

/*************************************************************~*********/

/* Name: INDEX.SAS */ /* Function: calculate various subindices from CPI value weights */ /* matrix */ /* Author: Martin Gregory */ /* Date: 12 September 1988 */ /* Values: IN Input Data Set (in EOLAS format) */ /* OUT Output Data Set */ /* REQUEST Name of file containing subindex calls */ /***********************************************************************/

proc iml;

/* initialise: VALUE is cpi value weights CPISUB is where we collect the indices NAMES stores the index names */

use &in ; read all var num into value; cpisub = value[,l}; names {'date'}; value = value[,2:ncol(value)];

/* define subroutine to calculate each index and store it in CPISUB SUBSUM is the sum of weights for the sub-index SUBIND is the sub-index N is the variable name for the sub-index */

start subindex; sub sum value[,subset}[,+}; subind = (100 * subsum) / subsum[l]; cpisub = cpisub I I subind; names = names I I n;

finish;

/* Include th. file containing .the sub-index specifications and calls for the subroutine These will be of the form:

subset n

{a b c •••.. ] {'var');

'Uric &request. ;

column numbers of items in sub~et variable name on the output data set

/* Write the output data set. This will be in EOLAS format */

create &out from cpisub [colname = names]; append from cpisub; close &out;

quit; ru~;

47

*/

\

\, "

/***********~**************~*******~**********~******* ******************/

/* Name: REBASE.SAS */ /* Function: rebase subindex series to a specified quarter */ /* Author: Martin Gregory */ /* Date: 16 February 1989 */ /* Values: IN Input Data Set containing series produced by */ /* the INDEX, LINK or REBASE program, */ /* DATE Base period: Format depends on type: */ /* for quarterly use YYQ6., eg 1982Q4 */ /* for annual average use YYYY, .9 1982 */ /* OUT Output Data Set */ /********************************************~**************************/

data _null_; if index(upcase("&date"),"Q") then do;

date=inpUt("&date",yyq6.); call symput("type","Q"); end;

else do; date=m~y(1,1,input("&date",4.»; call symput("type", "A"); end;

call symput ("date" , put (date, date7. ) ) ; run;

data base; set ∈ if "&type" = "Q" then if date="&date"d then output; else if year(date) = year("&date"d) then output;

proc iml;

use &in read all var num into cpi [colname

use base ; read all var _num_ into base base baser:,);

date cpi[,l); cpi = cpi[,2:ncol(cpi)];

names)

base = repeat(base[,2:ncol(base»),nrow(cpi),1); rebase = date II «cpi * 100) / base) ;

/* .Create a dataset from the rebased later series */

create &out from rebase [colname = names) append from rebase close &out

quit run;

48

/***************************************************** *~****************/ /* Name: LINK.SAS */ /* Function: link subindeK series for two different base periods to */ /* the base of the earlier series */ /* Author: Martin Gregory */ /* Date: 15 February 1988 */ /* Values: INI Input Data Set for earlier base, produced by */ /* the INDEX program. */ /* IN2 Input Data Set for later base, also produced */ /* by the INDEX program. */ /* OUT Output Data Set */ /***********************************************************************/

proc iml;

/* Read the earlier series */ use &inl read all var _num_ into cpil [colname

1* Read the late~ series */ use &in2 read all var _num_ into cpi2

/* Rebase the later series

names]

Note that the definition of DIVISOR inust occur before the redefinition of CPI2 - it's the first line of the original CPI2 */

date2 = cpi2[2:nrow(cpi2),l] ; divisor = repeat(cpi2[l,2:ncol(cpi2)],nrow(cpi2)-1,1) cpi2 = cpi2[2:nrow(cpi2),2:ncol(cpi2)] ; mult = repeat(cpil[nrow(cpil),2:ncol(cpil)],nrow(cpi2),1); rebase date2 I I( (cpi2 # mult) / divisor) ;

/* Create ~ dataset from lhe rebased later series */ create cpilink from rebase [colname = names} ; append from rebase close cpil ink

quit run;

/* Back in SAS we append the rebased later series to the earlier one */

data &out set &inl cpilink end=end;

run;

49

l" ,

1',····1 •. ' '1

,

~

1***************************************************** ******************/ 1* Name: PRINT.SAS *1 1* Function: print results of a subindex calculation or linking. *1 1* Author: Marlin Gregory (from a program by Adrian Redmond) *1 1* Date: 15 February 1988 *1 1* Values: OUT Output Data Set from subindex or link step. *1 1* This should have already been specified in the *1 1* earlier step. *1 1* LABELS Name of file containing variable labels. The *1 1* file name should be enclosed in single quotes. *1 1***************************************************** ******************/

data temp ; %inc &labels; set &out end=end; format date yyq6.; qdate='Y'!!put(date,yyq6.) ; if n =1 then call symput( 'start',qdate); if end=l then call symput('end',qdate);

run;

proc transpose data=temp (drop=date) out=temp id qdate;

data _null retain begqtr lines; if n =1 then do;

begqtr=input(substr("&start",7,1),1.); begyear=input(substr("&start",2,4),4.); endyear=input(substr("&end",2,4),4.); lines=endyear-begyear+6; end;

file print 11=11 set temp; if ll<lines then put page; array index (*) &star~--&e;d; put _label_ II 'Year' @lO 'February' @20 ' May'

@30' August' @40 'November'l; year=input(substr("&start",2,4),4.); do i=l to dim(index);

quarter=mod(i+begqtr-2,4)+I; col=10*quarter; if quarter=1 or i=1 then put year 4. @; put @col index{i} 8.1 @; if quarter=4 then do; year=year+l; put; end; end;

put I; run;

50