EERE Programs Briefing

March 3, 2011

Bruce Tonn

Weatherization Assistance Program: The Federal Perspective (Part 2)

NASCSP 2011 Mid-Winter Training Conference

# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov1Inspector General and General Accountability OfficeStatus of Retrospective Evaluation

Presenter: Bruce Tonn# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov2100% Response to State Program Information Survey

91% Response to All Agencies Program Information Survey

85% Response to Subset of Agencies Detailed Program Information Survey

Requests are being made for detailed weatherization data

Requests for billing histories are going out to utilities

Bulk Fuels and Indoor Air Quality Studies are underway Status of Retrospective Evaluation # | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov3Radon Study Approach

# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov4Preliminary Radon Results: 298 Homes Pre-Weatherization Basement/CrawlspaceLowest Main Living LevelRadon Stratumnmean (pC/l)s.e.% > 4 pC/lnmean (pC/l)s.e.% > 4 pC/l1 (low)130.50.20%220.30.10%2 (mid)602.70.520%991.50.28%3 (high, per EPA or LBL)704.10.537%1002.30.318%4 (high, per EPA and LBL)464.50.639%592.90.322%# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov5Preliminary Formaldehyde Results: Some consider .10 ppm as risk

# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov6Reprise State/Agency surveys for PY10

National Energy Savings Estimates for PY09-11

Focused evaluations of: Weatherization Innovation Pilot ProjectSustainable Energy Resources for Consumers Project

Special Studies: Energy Savings PersistenceDeferralsTerritoriesUnder-performersSocial Network impacts of weatherization WAP ARRA Period Evaluation Planning is Underway # | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.gov7State surveys and communications

Database migration

Sampled agency data collectionData Collection Lessons Learned# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.govIm going to share some key observations from the data collection process for the retrospective evaluation. Were using these lessons to adapt our process for the ARRA evaluation.

Focus on three key areas where well be making some changes to make things easier for states and their subgrantees:State surveys and communicationsDatabase migrationSurveys for sampled agencies

8Continuing the case management approach:Dedicated case manager for states and subgranteesNotify states before new data requests are sent to subgranteesFollow up regarding slow/unresponsive subgrantees

New approaches for fiscal data collection:Assistance in mapping available data to survey questionsEvaluation team can work directly with fiscal staff

WAP ARRA Period Evaluation: State Surveys and Communications# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.govThank states for their efforts in completing data requests and following up with subgrantees to encourage full participation.

Case management approach will be used again for ARRA evaluation states and subgrantees will be assigned a dedicated point of contact for evaluation matters.

For many respondents, compiling the requested fiscal data was the most challenging and time-consuming part of the state surveys.Many states found it challenging to map existing fiscal data to our survey questions. We are revising some questions for greater clarity / ease of interpretation We will assist states in mapping existing data to survey questions by reviewing fiscal data and reports If the program manager would like to cut out the middle man and have the evaluation team work directly with their fiscal staff we are happy to do so9Retrospective evaluation:19 state databases migrated for unit-level dataDemographicsUtility account infoBuilding characteristicsDiagnostics & measuresCosts189 sampled agencies had survey burden reduced

ARRA evaluation:More databasesMore comprehensive datasetsCodebooks are helpful!Database Migration# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.govBefore launching the agency data collection effort we contacted all states to determine whether there was a database that contained sufficiently detailed information to allow for database migration.

Goal was reducing survey burden for sampled agencies by extracting detailed information about individual weatherized units. We were able to reduce burden for 189 agencies nearly half of the 400 agencies that were sampled for extensive data collection.

Given the investments in database infrastructure under ARRA, we expect the next evaluation will involve an even more extensive migration effort.

Mapping the data correctly to our survey variables required a fair amount of communication with state offices. States did a great job of identifying a data contact to field questions from our analysts. We hope to use a similar process for the ARRA evaluation. The communications burden would be less for states that are able to share codebooks that define each data element. If you have developers working on databases, consider asking them to create codebooks as part of their effort.10Centralized data collection process: WI case studyState office used T&TA contractor to complete the Utility Info Survey and Housing Info Survey 9 sampled agencies955 units~400 hours of effort

Keys to success:T&TA fundsGame planOrientation to agency filesEvaluation team can provide training support

Sampled Agency Data Collection# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.govWe estimate the typical survey burden for a sampled agency is ~50 hours. One of the most time-intensive parts of the process is compiling the detailed unit-level information requested in the Utility Info Survey and Housing Info Survey.

WI Division of Energy Services wanted to keep agencies focused on production. Decided to use the states T&TA contractor (Wisconsin Energy Conservation Corporation) to complete the Utility Info Survey and Housing Info Survey for all sampled agencies in Wisconsin. While these surveys are time-intensive, it is largely an exercise of going through files. They can easily be completed by administrative staff that dont have an extensive background in weatherization.11Bruce TonnOak Ridge National Laboratorybet@ornl.gov(865) 574-4041

Claire CowanEnergy Center of Wisconsin ccowan@ecw.org(608) 238-8276 x117

Contact Info# | Weatherization Assistance Program: The Federal Perspective (Part 2)eere.energy.govWe welcome your feedback on any improvements that could be made to the data collection process for the ARRA evaluation. Ill be here through the rest of the conference; you can also call me or email me.12