social security policy branch – random sample surveys

Social Security Social Security

AssuranceAssurance

Tracking Process Error Tracking Process Error

using Sample Surveysusing Sample Surveys

ITSEW 2009 TallbergITSEW 2009 Tallberg

Social Security Policy Branch – Random Sample Surveys


Cost of Compliance Customer Attitude to Compliance Compliance Strategy Debt Recovery Strategy

Reparation sought through the courts

FaCSIA Fraud and Compliance Response

Those customers whose deliberate and organised non-compliance results in a significant attack on the integrity of the payment

Fraud Investigation/Prosecution Action/Criminal Conviction

Those who deliberately not comply

Targetted Compliance Activities e.g. Data Matching, Compliance Reviews

Those who do not take reasonable care to comply



Those who get it right

Small Debt Waiver: <$50

Adapted from Braithwaite Regulation Model

Those who try to comply but make honest mistakes/do not understand their requirements

Education Programmes e.g. Real Estate and Overseas Pensions publicity campaigns

Targetted Prevention e.g. Service Profiling


• Began in 98/99

• Continues as an ongoing project

• Methodology tested, improved and ratified by External Consulting

Actuaries

• Audited by the Australian National Audit Office (ANAO)

• Recommendations have been implemented

Random Sample Surveys Background


Service level agreement

• FaHCSIA and Centrelink have an SLA in place to manage the RSS

• SLA is a set of agreed processes and procedures to be followed by both

parties

• Living Document – can be modified to meet changing political or

departmental requirements


Sample Design

• Define Sample Population (Age Pension, Carer Payment)

• Make Exclusions where Necessary

• Define Strata – Population Groups defined by characteristics (eg Gender,

Rate, Marital Status)

• Consideration for Remote Customers (Clustering)

• Allocate Sample (operational requirements and strata)


Exclusions and Substituons

• In an Ideal sample ALL customers have a chance of selection

• RSS Population is not a closed system

• Need to balance randomness of the sample against burden to the

customers

• Exclusion Rules ensure that:

• customers that have an outstanding or recently completed review;

• customers that are seriously ill or out of the country;

Are not selected


Stratification

• The target population is divided into discrete, non-overlapping sub

populations

• This assists in ensuring a Representative Sample is selected

• Exploits the depth of administrative information held on populations

under review

• Sharpens the final estimates around variables associated with

compliance status

• Allows for unequal probability selection


RSS SAMPLE SPECIFICATION 2008-09 - DISABILITY SUPPORT PENSION

STRATUM ID POP Mar08 SAMPLE SKIPBY TRI MESTER

D01 22,140 <50yrs 128 172 43D02 47,143 50+yrs 187 252 62D03 33,402 <50yrs 158 212 53D04 84,984 50+yrs 252 338 84D05 35,302 <35yrs 162 218 54D06 54,720 35-49yrs 202 271 67D07 90,896 50+yrs 260 349 87D08 54,656 <35yrs 202 271 67D09 74,531 35-49yrs 236 316 79D10 93,920 50+yrs 265 355 88D11 52,644 198 266 66D12 42,384 178 239 59D13 46,629 165 283 55D14 46,736 165 283 55

3000TOTAL 780,088 2757

Note: sample is allocated using Caroll formula (that is proportional to the square root of the stratum population)The allocation of sample between nonremote and remote areas is proportional to respective customer populations

Nonremote

Max rate

Partnered

Female

Male

Unpartnered

Female

Male

Reduced rate

STRATIFICATION

PartneredUnpartnered

RemotePartnered

Unpartnered


Clustering - Background

• Centrelink’s most remote Areas (North & Central QLD,

North Australia, Western Australia) have smaller

customer populations over wider geographic areas

• Need to balance the requirement to sample in these

locations with the cost of travel and distance

• Clustering concentrates the selections to a smaller

location within the remote Area


Selection

• Design Delivered to Centrelink

• Random Selection of Customers within Allocation Boundaries by

Centrelink

• FaHCSIA views selection

• Centrelink proceeds after approval


Reviews

• Selected Customer attends an Interview in Centrelink Office (CSC or ASO)

• Full Re-Assessment of Entitlement conducted via Questionnaire (Income,

Assets, Residency, POI, Qualification, Accommodation)

• More Detailed information from Review filled out by Centrelink Officer after

review concluded and entered into RRRS

• Quality Assurance is performed by Centrelink on a sample of the results


Review Results

Assessment of Current and Prior Entitlement can have the following results

• Cancellation of Payment

• Variation (Upward or Downward) of Current Fortnightly Rate

• Debt Raised for an overpayment in an earlier period

• Arrears Paid for an underpayment in an earlier period

These Results are not mutually exclusive. A Cancellation/Variation can

happen in conjunction with a Debt and/or Arrears.

Supplementary Components (Eg RA, PHA, TAL) are also reviewed.


FaHCSIA Processing

• Centrelink Review Results Stored on ISIS/IRS/RRRS

• Data is Transmitted from Centrelink to FaHCSIA

• FaHCSIA tests the data integrity

• Validates a sample (10%) of the Reviews - (Centrelink QAs this sample)

• Tabulates and Analyses the Results


Results

• Results used by FaHCSIA to:

• Provide an estimate FaHCSIA Programme Outlay Accuracy

• Identify Emerging Risks

• Assess effectiveness of Compliance Budget measures

• Evidence Base for Compliance Policy

• Latent Debt Measure – Debt KPI

• Improve predictors for Service Profiling


Outlay Accuracy

• Modelled Estimate of the Mispayed Amount / Outlayed Amount subtracted

from 100%

• Does not discriminate between customer and Centrelink errors

• Weighted by $ Value – a $450 cancellation has a greater effect than a

$0.05 upward variation

• Only counts current episodes of mispayment ie. is not affected by debts for

prior periods of mispayment

• Sourced from Dataset 1


Payment Correctness• Modelled Estimate of the Number of Customers with an Administrative Error that affected

Payment / Total Population subtracted from 100%

• Only counts customers where the error was made by Centrelink which caused a dollar impact

• Does include dollar impact from debts (ie prior periods of mispayment)

• Does not count multiple errors – if any 1 of up to 4 Errors meets the criteria the customer is

counted once

• Not weighted by dollar value - Counts customers not amounts – a $35,000 debt contributes

the same as a $0.05 variation

• Sourced from Dataset 4


Latent Debt

• Measure of the level of undetected indebtedness in the benefit population

• Key Prevention Indicator for Debts is the Latent Measure of

Indebtedness to Outlays

• Make the distinction between “Current Year” Indebtedness and New

Debts


Figure 1.1 Current Year Indebtedness to Outlay (Latent measure) by Benefit, 2003-2007

12 month moving aggregate, 1 Standard Error bar

0.00%

0.20%

0.40%

0.60%

0.80%

1.00%

1.20%

1.40%

Ind

ebte

dn

ess:

ou

tlay

(%

)

AGE DSP New debts AGE New debts DSP CAR New debts CAR


Addressing emerging risktop 10 reasons for mispayment• Random Sample Results are further subdivided by reason for

mispayment

• These indicate the Risks to Outlays to be addressed by Compliance

and Prevention measures

• Mispayments in debt and variation can be for the same or different

reasons

• Risks are ranked by number (incidence) and value (magnitude)

• Charts observe the Higher risks to outlays


Age Pension - Top 10 Risks as at T1 08/09

$0 $2,000 $4,000 $6,000 $8,000 $10,000 $12,000 $14,000

Investment income from banks/buildingsocieties etc

Assets - value of investments

Assets - property/real estate

Income from income stream

Employment income

Other

Self employed - income from businessprecludes payment

Income from foreign pension

Investment income from shares

Assets - business (including Farm)

Rea

son

for

Var

iati

on

FN Variation Amt ($)

12 Months to T1 08/09

12 Months to T3 07/08


Trimester 1 Trimester 2 Trimester 3No of Reviews 1,666 1,667 1,664 4,997Fortnightly Amt 689,241.64 730,792.91 693,409.48 2,113,444.03Variation Amt 12,076.42 13,117.55 10,805.92 35,999.89Accuracy 98.25% 98.21% 98.44% 98.30%No Debts 623 626 537 1,786Debt Amt 309,569.68 333,658.37 325,313.97 968,542.02No of Reviews 499 498 499 1,496Fortnightly Amt 207,949.68 214,720.42 205,873.27 628,543.37Variation Amt 7,410.93 5,444.86 5,590.83 18,446.62Accuracy 96.44% 97.46% 97.28% 97.07%No Debts 133 134 133 400Debt Amt 82,086.46 104,244.20 127,261.13 313,591.79No of Reviews 84 83 83 250Fortnightly Amt 8,337.32 8,406.98 8,349.80 25,094.10Variation Amt 98.50 100.60 201.20 400.30Accuracy 98.82% 98.80% 97.59% 98.40%No Debts 0 0 1 1Debt Amt 0.00 0.00 359.12 359.12No of Reviews 919 919 920 2,758Fortnightly Amt 407,118.93 420,268.90 430,148.63 1,257,536.46Variation Amt 13,461.32 10,819.27 5,823.34 30,103.93Accuracy 96.69% 97.43% 98.65% 97.61%No Debts 190 171 148 509Debt Amt 283,469.87 197,850.06 87,941.00 569,260.93No of Reviews 0 791 716 1,507Fortnightly Amt 0.00 112,820.11 93,517.24 206,337.35Variation Amt 0.00 2,654.63 2,326.26 4,980.89Accuracy #DIV/0! 97.65% 97.51% 97.59%No Debts 0 17 8 25Debt Amt 0.00 13,484.20 2,902.52 16,386.72No of Reviews 0 498 0 498Fortnightly Amt 0.00 178,507.46 0.00 178,507.46Variation Amt 0.00 6,916.30 0.00 6,916.30Accuracy #DIV/0! 96.13% #DIV/0! 96.13%No Debts 0 161 0 161Debt Amt 0.00 63,009.87 0.00 63,009.87

Annual Total

Random Sample Surveys - Actual Outlay Accuracy

Carer Allowance

Disability Support Pension

Family Tax Benefit

Special Benefit

Benefit Reviewed Survey Period

Age Pension

Carer Payment


Trimester 1 Trimester 2 Trimester 3

No of Reviews 1,666 1,667 1,664 4,997Fortnightly Amt 689,241.64 730,792.91 693,409.48 2,113,444.03Variation Amt 12,132.41 13,073.02 11,186.50 36,391.93Accuracy 98.24% 98.21% 98.39% 98.28%No Debts 616 619 533 1,768Debt Amt 309,131.03 321,667.94 325,258.36 956,057.33No of Reviews 499 498 499 1,496Fortnightly Amt 207,949.68 214,720.42 205,873.27 628,543.37Variation Amt 7,410.93 5,404.19 5,545.27 18,360.39Accuracy 96.44% 97.48% 97.31% 97.08%No Debts 133 130 131 394Debt Amt 82,347.65 98,154.77 126,986.99 307,489.41No of Reviews 84 83 83 250Fortnightly Amt 8,337.32 8,406.98 8,349.80 25,094.10Variation Amt 98.50 100.60 201.20 400.30Accuracy 98.82% 98.80% 97.59% 98.40%No Debts 0 0 1 1Debt Amt 0.00 0.00 359.12 359.12No of Reviews 919 919 920 2,758Fortnightly Amt 407,118.93 420,268.90 430,148.63 1,257,536.46Variation Amt 13,927.48 10,838.34 5,823.34 30,589.16Accuracy 96.58% 97.42% 98.65% 97.57%No Debts 188 169 146 503Debt Amt 283,193.39 195,016.42 87,890.70 566,100.51No of Reviews 0 791 716 1,507Fortnightly Amt 0.00 112,820.11 93,517.24 206,337.35Variation Amt 0.00 2,486.65 2,263.36 4,750.01Accuracy #DIV/0! 97.80% 97.58% 97.70%No Debts 0 17 9 26Debt Amt 0.00 13,100.00 2,987.32 16,087.32No of Reviews 0 498 0 498Fortnightly Amt 0.00 178,507.46 0.00 178,507.46Variation Amt 0.00 8,747.94 0.00 8,747.94Accuracy #DIV/0! 95.10% #DIV/0! 95.10%No Debts 0 152 0 152Debt Amt 0.00 60,948.71 0.00 60,948.71

Family Tax Benefit

Special Benefit

Age Pension

Carer Payment

Carer Allowance


Random Sample Surveys - Restored Outlay AccuracyBenefit Reviewed Survey Period

Annual Total


Trimester 1 Trimester 2 Trimester 3

Actual 98.25% 98.21% 98.44% 98.30%Restored 98.24% 98.21% 98.39% 98.28%Actual 96.44% 97.46% 97.28% 97.07%Restored 96.44% 97.48% 97.31% 97.08%Actual 98.82% 98.80% 97.59% 98.40%Restored 98.82% 98.80% 97.59% 98.40%Actual 96.69% 97.43% 98.65% 97.61%Restored 96.58% 97.42% 98.65% 97.57%Actual #DIV/0! 97.65% 97.51% 97.59%Restored #DIV/0! 97.80% 97.58% 97.70%Actual #DIV/0! 96.13% #DIV/0! 96.13%Restored #DIV/0! 95.10% #DIV/0! 95.10%

Family Tax BenefitSpecial Benefit

Random Sample Surveys - Restored Outlay AccuracyBenefit Reviewed Survey Period

Annual Total

Age Pension

Carer Payment

Carer Allowance



Downward Variation

No Review Conducted

Present Rate to

Continue

Upward Variation

Downward Variation 1 7 3 2 13Present Rate to Continue 2 13 15 1 31Upward Variation 1 3 2 3 9total 4 23 20 6 53Downward Variation 1 1 3 0 5Present Rate to Continue 0 6 28 0 34Upward Variation 0 1 0 0 1total 1 8 31 0 40Present Rate to Continue 0 1 2 0 3total 0 1 2 0 3Present Rate to Continue 3 12 18 1 34Upward Variation 0 1 1 0 2total 3 13 19 1 36Cancellation 0 2 0 0 2Present Rate to Continue 0 11 18 0 29Upward Variation 0 0 1 0 1total 0 13 19 0 32Cancellation 0 0 1 0 1Downward Variation 1 0 2 2 5Present Rate to Continue 4 0 14 2 20Upward Variation 0 0 2 3 5total 5 0 19 7 31

Special Benefit

Random Sample Surveys - Replaced Case Result ComparisonBenefit

ReivewedActual Result

Carer Payment

Carer AllowanceDisability Support PensionFamily Tax Benefit

Restored Result total

Age Pension


1 2 3No of Reviews 24 15 14 53Fortnightly Amt 10,320.63 6,962.89 5,984.37 23,267.89Actual Var Amt 57.12 93.42 74.7 225.24Restored Var Amt 113.11 48.89 455.28 617.28Actual No. Debts 7 8 4 19Restored No. Debts 0 1 0 1Actual Debt Amt 438.65 12,011.77 55.61 12,506.03Restored Debt Amt 0 21.34 0 21.34No of Reviews 15 13 12 40Fortnightly Amt 6,372.74 5,642.59 4,544.09 16,559.42Actual Var Amt 0 41.27 45.56 86.83Restored Var Amt 0 0.6 0 0.6Actual No. Debts 1 5 2 8Restored No. Debts 1 1 0 2Actual Debt Amt 30.76 6,092.87 274.14 6,397.77Restored Debt Amt 291.95 3.44 0 295.39

Age Pension

Benefit Reivewed

Result Component

Carer Payment

Period Total


No of Reviews 2 0 1 3Fortnightly Amt 197.00 0.00 100.60 297.60Actual Var Amt 0 0 0 0Restored Var Amt 0 0 0 0Actual No. Debts 0 0 0 0Restored No. Debts 0 0 0 0Actual Debt Amt 0 0.00 0 0.00Restored Debt Amt 0 0 0 0No of Reviews 16 10 10 36Fortnightly Amt 7,589.88 4,010.15 4,923.61 16,523.64Actual Var Amt 3.34 0 0 3.34Restored Var Amt 469.5 19.07 0 488.57Actual No. Debts 2 3 2 7Restored No. Debts 0 1 0 1Actual Debt Amt 276.48 2,834.16 50.3 3,160.94Restored Debt Amt 0 0.52 0 0.52

Carer Allowance



No of Reviews 0 19 13 32Fortnightly Amt 0.00 2,810.23 1,245.20 4,055.43Actual Var Amt 0 167.98 62.9 230.88Restored Var Amt 0 0 0 0Actual No. Debts 0 1 0 1Restored No. Debts 0 1 1 2Actual Debt Amt 0 485.75 0 485.75Restored Debt Amt 0 101.55 84.8 186.35No of Reviews 0 31 0 31Fortnightly Amt 0.00 11,740.69 0.00 11,740.69Actual Var Amt 0 602.43 0 602.43Restored Var Amt 0 2434.07 0 2434.07Actual No. Debts 0 10 0 10Restored No. Debts 0 1 0 1Actual Debt Amt 0 2,061.21 0 2,061.21Restored Debt Amt 0 0.05 0 0.05

Special Benefit

Family Tax Benefit


2002/03 2003/04 2004/05 2005/06Sample Selected 1,493 1,500 1,495 1,489No. Cust with Debts 195 183 319 365No. Debts 239 232 425 461No. Cust w Prior Reviews 491 353 785 765No. Reviews 801 671 1,859 1,880Cust with Debts and Reviews 87 74 213 229Cust with Debts and No Reviews 108 134 106 136Cust with Reviews and No Debts 404 279 572 536Cust with no Debt or Review 894 1,013 604 588No. Cust wit Overlap 52 35 90 107No. Debts Overlap 58 42 116 118

Age Pension Random Sample Cases 02/03 - 05/06Overlap Between RSS Debt and Prior Review History

Age Pension

social security policy branch – random sample surveys

Documents

selected slide

ideal sample

representative sample

approval slide

remote area slide

sample population age

remote customers

centrelink officer