Abstracts for the
Meeting of the American Statistical Association (ASA)
Committee on Energy Statistics
Energy Information Administration (EIA)
April 28 and 29, 2005
Open Meeting, Nicolas Hengartner, Chair
Greetings and Remarks, Guy Caruso, Administrator, Energy Information Administration (EIA)
Updates for the Committee since the fall 2004, Meeting, Nancy Kirkendall, Director, Statistics and Methods Group (SMG), EIA
Regionalizing the Short-Term Energy Outlook (STEO) Forecast: Margot Anderson, Director, EMEU Plenary session will discuss the key features of the Regional STEO model. Topics will include: (1) the rationale for regionalizing the model, (2) how EIA approached developing the regional model, (3) key components of the model; and (4) how the regional forecast will compare with the current STEO forecast.
STEO Performance Indicators: Diagnostics and Forecast Errors Margot Anderson, Director, EMEU Plenary session will present and discuss: (1) diagnostic tools for gauging forecast accuracy, (2) which variables make sense and which tools are practical, (3) how important are these in gauging model and forecast accuracy, and (4) how might EIA share the results with customers.
EIA completed the development of a thirteen-region electricity demand and supply model in the early 2005. Subsequently, EIA modelers used the generic least cost dispatching algorithm to simulate regional generation patterns and fuel choices in meeting a given set of regional electricity demand. Comparison of model results with 2002 and 2003 monthly generation data shows that cost captures only part of dispatching decisions facing electric utilities. And it became clear that modifications to the input assumptions are needed to improve the performance of the model.
This paper documents how EIA data are used to identify and estimate key model parameters that are important to the performance of the model. In addition, this paper will also demonstrate that the calibrated model will be as robust and can provide insightful information on winter heating fuel market, natural gas market, and summer electricity market.
The data used for model calibration include:
· EIA-906: monthly data on generation and fuel use at the generator or plant level by states.
· EIA-423 and FERC-423: monthly cost and quality of fuels for electric plants
· EIA-860: capacity of generators by technology and fuel at plant level
· Office of DOE Fossil Energy (FE): Annual Report of International Electric Import/Export data
Key model parameters EIA modelers have to estimate are:
· Designation of generators to one of the five dispatching groups
· Import constraints
· Numerical criteria for fuel switching
· Generation share of oil and gas in coal power plants
· Share of hydro power allocated to base load and the dispatching profile for non-base load power
Bureau of Census Frames Comparisons, Rick Hough and Vicki Haitot, Bureau of Census
The purpose of the frames evaluation being conducted by Census is to determine the coverage of the following five EIA frames: (1) EIA-3, Quarterly Coal Consumption and Quality Report, Manufacturing Plants, (2) EIA-5, Quarterly Coal Consumption and Quality Report, Coke Plants; (3) EIA-860, "Annual Electric Generator Report" (for combined heat and power plants in NAICS codes 31-33 only; (4) EIA-63a, "Annual Solar Thermal Collector Manufacturers Survey"; and (5) EIA-63b, "Annual Photovoltaic Module and Cell Manufacturers Survey.
Two representatives from the Census Bureau will present the following topics: (1) Methodology for matching and analysis of coverage (both by count and by volume) for all five EIA frames; (2) Results of matching and analysis for EIA-63a (solar thermal collectors), EIA-63b (photovoltaic cells and modules), and EIA-5 (coke plants); (3) Identify differences between EIA and Census frames which impact coverage analysis; (4) Identify characteristics of establishments missing on EIA frames; and (5) Identify next steps for assessing coverage of EIA-3(Manufacturers that consume coal) and EIA-860 (Combined Heat and Power Plants).
Hands-On Usability Testing of EIA's New
Website Design: Can You Find What
You're Looking For? Colleen Blessing,
Working since last fall, and based on input from many sources, the Web Redesign Group has created a new prototype design we would like the ASA members to "test drive." This session will be you sitting at a computer using the new site, commenting as you click through, giving us feedback on what you see, and finding answers to questions or finding what you usually use on the current site.
EIA conducted a similar usability test with ASA members before our redesign in 2000 and received useful feedback. There is no preparation necessary for this session, except if you want to record some reports or data tables you would like to be able to find on the new site. Looking at the new design for the first time in this session will allow us to capture your first impressions.
Follow-up on Frames Team Activities Howard Bradsher-Fredrick, SMG, EIA This paper will present a summary of the activities and conclusions of the various incarnations of the frames team from March 2004 through April 2005. The frames team was an inter-office EIA team composed of staff members from the EIA program offices, SMG and OIT. In general, all interested and affected offices had representation on each incarnation of the team
The impetus for the effort on improving frames emanated from the EIA Strategic Plan and specifically from the Strategic Information Technology Subgroup. These frames teams were charged with identifying efficiencies EIA can pursue regarding frames and the computer systems that support frames.
In particular, the first team was chartered with providing recommendations concerning activities in the frames area that will lead EIA toward saving resources, facilitating more commonality in data and software, improving processes, and resulting in frames and frame information that can easily be shared within EIA. The first frames team met from March 2004 to early May 2004 and developed a set of six recommendations.
The second incarnation of the team met from August 2004 to January 2005. The primary focus of its activities was to assess each unique EIA survey frame for “sufficiency.” In order for the team to ascertain sufficiency or insufficiency for individual frames, a number of tasks had to be accomplished, some of which involved extensive survey frame information gathering.
The third incarnation of the team began meeting in February 2005. It is charged with following up on the recommendations put forward by the first incarnation of the team.
EIA’s Proposed Strategy for Addressing Declining Response Rates in the Residential Energy Consumption Survey (RECS) Dwight French, EMEU, EIA Over the 25-year history of EIA’s Residential Energy Consumption Survey (RECS), response at various stages of this survey system has slowly but steadily decreased. This trend, which has been true for most government household surveys, is especially troublesome for RECS, which is actually a sequence of interconnected data collections: a survey of the household to get its energy-related characteristics, a request during the household survey to get the household respondent to sign a waiver form allowing the household’s energy suppliers to submit billing information, and then a survey of the suppliers to collect the energy consumption and expenditures information. All of these collections are necessary to get a complete set of survey data; therefore, non-response at each stage has a cumulative negative effect on data quality. This presentation will describe an aggressive, multi-faceted strategy that EIA has planned to address the comprehensive non-response issue, including additional and updated preliminary contacts, an incentive experiment, enhanced non-monetary incentives, the use of scanners to collect utility bill information at the household, a Spanish translation of the questionnaire, publicity about data confidentiality based on EIA’s new CIPSEA legislation, and a potential non-response sub-sampling process. The committee is asked to critique our approaches and suggest any others they might think would be useful.
Assessments: Presentations and a Panel Discussion Nancy Kirkendall, moderator
External Review of Survey Programs: A Progress Report, Brenda Cox, Battelle, under contract to EIA, and Nancy Kirkendall, EIA The Performance Assessment Rating Tool (PART) was instituted to boost the quality of Federal programs by encouraging rigorous performance assessment for government programs. These performance assessments identify program strengths and weaknesses, resulting in recommendations for improvement and recognition of desirable design elements already in place. For programs that involve a family of related surveys, the individual surveys that comprise the survey program need to be evaluated as well as the effectiveness of their integration to create the survey program. This paper presents the progress made in this investigation designed to develop templates for performing an external evaluation of a family of related surveys. The end products of this study will be a Program Evaluation Template and a Survey Evaluation Template, which can serve as guides for performing expert assessments of survey programs and their component surveys. At the Fall 2004 meeting of the ASA Committee on Energy Statistics, we described the project’s goals and presented the survey template that was to be concept tested later that year. This presentation updates that research by presenting the Program Evaluation Template and the insights gained from our proof of concept testing of both templates on the Petroleum Marketing Survey Program.
The overarching question that the external study team will address is:
Given continuing tight EIA budgets and other external drivers, is EIA doing the “right things” now, and what are the “right things” it should do in the next five years?
The question intentionally focuses on “is EIA doing the right things” (i.e., what products and services are provided to customers) and not on “is EIA doing things right” (i.e., how it conducts its internal business processes). At the ASA session, EIA will review its goals for this external study, outline project’s tentative timeline (scheduled to be completed in November 2005), and solicit ideas and suggestions from ASA Energy Committee members who may wish to share their experience with evaluative processes that resulted in positive change.
Discussion: Where Next? Howard Gruenspecht, EIA Deputy Administrator and Moderator
Progress on EIA’s 914: Response rate and kinds of challenges, John Wood, Reserves and Production Division (RPD), EIA At the fall 2004 meeting with the ASA Energy Committee, EIA introduced the new and approved Natural Gas Production Survey, Form EIA-914. The spring 2005 meeting will provide the first opportunity to discuss EIA’s preliminary assessment of the survey data for January, 2005, the first reporting month for this new survey. EIA will discuss the challenges faced during the early months of survey operation, as well as our plans for review, analysis, and comparisons of final natural gas production estimates based on the early sample of final production provided by the EIA-914 survey data with our previous methods of estimating final natural gas production. This review, comparison, and analysis is a critical step in the process of deciding when EIA should start publicly releasing the EIA based production estimates as our official EIA natural gas production estimates.
Estimate EIA’s 826
Since the Last Time: Differences, estimation groups, outliers and test results
Joseph Sedransk, Jim Knaub, Lindolfo Pedraza and
EIA Form-826 collects information, monthly, from regulated and unregulated companies that sell or deliver electric power to end users. It collects State-level sales volumes, sales revenues, and number of customers by end-use sector (residential, commercial, industrial, and total). The existing sample and methodology to estimate population totals were described at the fall 2004 meeting of the Energy Committee. This session will describe current efforts to form homogeneous subpopulations for estimation, and evaluations based on predictions using annual data (where truth is assumed known). We assess the importance of outlier detection, under alternative scenarios. We will be asking for advice on next steps for analysis, and on how to frame convincing recommendations for implementation.