An Assessment of EIA's Building Consumption Data
Background
The U.S. Energy Information Administration (EIA) routinely uses feedback from customers and outside experts to help improve its programs and products. As part of an assessment of its consumption surveys, EIA reached out to the National Academy of Sciences' Committee on National Statistics (CNSTAT) asking them to assess the Commercial Buildings Energy Consumption Survey (CBECS) and the Residential Energy Consumption Survey (RECS) and recommend improvements in data quality, geographic coverage, timeliness of data releases, and relevance of data for users.
On February 10, 2012, a panel from CNSTAT released their assessment of EIA's surveys on residential and commercial buildings end-use energy consumption, titled Effective Tracking of Building Energy Use: Improving the Commercial Buildings and Residential Energy Consumption Surveys. A summary of the major issues and recommendations identified in the report is provided below.
Next Steps
EIA and the Committee on National Statistics are now meeting with statistical and industry experts to discuss the recommendations and implementation ideas. EIA will also undertake methodological research later this year to test the concept of expanded use of administrative records and alternative modes of data collection.
Major Issues and Recommendations
The panel report included 33 recommendations across five broad areas where EIA could direct resources to improve the RECS and CBECS programs.
Explore methods to improve timeliness, increase frequency and measure change
RECS and CBECS currently rely on costly area probability sample frames, in-person interviews, and complex editing and estimation methods to achieve high response rates, accuracy on key survey items and good coverage of the sampled populations for Census Divisions, and, in the case of RECS, sixteen states.
- Evaluate use of a smaller and more frequent rotating sample design
- Revisit a subset of buildings and housing units over time
- Use multiple modes to collect data (in person, telephone, mail, and web)
- Release data faster by revising processing procedures and producing early estimates from a representative sample of the surveys
- Engage more energy suppliers to identify efficiencies in collecting consumption and expenditures data
- Conduct research on the advantages of alternative sampling frames
Increase access to more data for public, academic, and commercial research uses
Some RECS and CBECS data tabulations do not meet EIA standards for statistical precision, and in some cases, the sampled units are so unique that data groups or items must be truncated or suppressed. Other data items are omitted in the public micro-datasets or statistically perturbed or top-coded to meet strict legal requirements to protect the privacy and confidentiality of survey respondents. Federal statistical agencies, however, are moving to more open access via secure research data centers.
- Increase survey sample sizes to reduce data table suppressions
- Consider placing all microdata with small area geographic identifiers in one of the established, secure data research environments to foster innovation and insight in energy research
Work to balance respondent burden, data quality, and relevance
The current RECS interview averages about 50 minutes and includes physical measurements of the housing unit. The CBECS includes a respondent worksheet and a building interview, which averages about 35 minutes. Some survey questions are technical; respondents vary in their knowledge and ability to answer them.
- Explore use of 'short' and 'long' form CBECS and RECS questionnaires
- Where possible, use administrative records to replace or reduce the need for existing or new survey questions
- Use energy auditors and routinely debrief survey interviewers to understand the quality of respondents' answers
- Review and update question wording
Fill data gaps, prepare for new energy uses and data sources
On their current quadrennial cycle, RECS and CBECS are vulnerable to missing the adoption of fast emerging, new uses of energy in buildings (e.g., big screen televisions in conference rooms instead of projectors), quality improvements that reduce consumption for familiar ones (LED for Plasma TVs), or substitution of new for old uses (smartphones for computers). Furthermore, program gaps should be filled, and new data sources should be researched.
- Evaluate how well the surveys cover emerging energy uses
- Prepare to collect information on the capacity for electric vehicles
- Research uses of 'smart meter' data
- Fill the program gap on large multi-unit residential buildings by conducting a 'whole building' survey
Suggestions for 'state-of-the-art' improvements
The goals of the RECS and CBECS programs may also benefit from developing a variety of state of the art methods/tools.
- Develop a comprehensive survey of all energy-consuming premises
- Develop Interactive online tools to engage sample members in understanding their home/building's energy consumption
- Use digital photographs to capture building and equipment characteristics
- Develop survey modules on special topics related to energy issues
