Abstracts for the
Meeting of the American Statistical Association (ASA)
Committee on Energy Statistics
with the
Energy Information Administration (EIA)
April 28 and 29, 2005
Open Meeting, Nicolas Hengartner, Chair
Greetings and Remarks, Guy Caruso, Administrator, Energy Information
Administration (EIA)
Updates for the Committee since the fall 2004, Meeting, Nancy Kirkendall, Director, Statistics and Methods Group (SMG), EIA
Regionalizing the Short-Term
Energy Outlook (STEO) Forecast: Margot Anderson,
Director, EMEU Plenary session will discuss the key features of the Regional
STEO model. Topics will include: (1) the
rationale for regionalizing the model, (2) how EIA approached developing the
regional model, (3) key components of the model; and (4) how the regional
forecast will compare with the current STEO forecast.
STEO Performance Indicators:
Diagnostics and Forecast Errors Margot Anderson, Director, EMEU Plenary session will present and discuss: (1)
diagnostic tools for gauging forecast accuracy, (2) which variables make sense
and which tools are practical, (3) how important are these in gauging model and
forecast accuracy, and (4) how might EIA share the results with customers.
EIA completed the development of a thirteen-region electricity demand and supply model in the early 2005. Subsequently, EIA modelers used the generic least cost dispatching algorithm to simulate regional generation patterns and fuel choices in meeting a given set of regional electricity demand. Comparison of model results with 2002 and 2003 monthly generation data shows that cost captures only part of dispatching decisions facing electric utilities. And it became clear that modifications to the input assumptions are needed to improve the performance of the model.
This paper documents how EIA data
are used to identify and estimate key model parameters that are important to
the performance of the model. In
addition, this paper will also demonstrate that the calibrated model will be as
robust and can provide insightful information on winter heating fuel market, natural gas market, and summer electricity
market.
The data used for model calibration include:
·
EIA-906: monthly data on generation and fuel use
at the generator or plant level
by states.
·
EIA-826:
monthly electric utility sales and revenue report with state distributions
·
EIA-423
and FERC-423: monthly cost and quality of fuels for electric plants
· EIA-860: capacity of generators by technology and fuel at plant level
· Office of DOE Fossil Energy (FE): Annual Report of International Electric Import/Export data
Key model
parameters EIA modelers have to estimate are:
· Designation
of generators to one of the five dispatching groups
·
Import
constraints
·
Numerical
criteria for fuel switching
·
Generation
share of oil and gas in coal power plants
·
Share of
hydro power allocated to base load and the dispatching profile for non-base
load power
Bureau of Census Frames Comparisons, Rick Hough and Vicki Haitot, Bureau of Census
The
purpose of the frames evaluation being conducted by Census is to determine the
coverage of the following five EIA frames: (1) EIA-3, Quarterly Coal
Consumption and Quality Report, Manufacturing Plants, (2) EIA-5, Quarterly Coal
Consumption and Quality Report, Coke Plants; (3) EIA-860, "Annual Electric
Generator Report" (for combined heat and power plants in NAICS codes 31-33
only; (4) EIA-63a, "Annual Solar Thermal Collector Manufacturers
Survey"; and (5) EIA-63b, "Annual Photovoltaic Module and Cell
Manufacturers Survey.
Two
representatives from the Census Bureau will present the following topics: (1)
Methodology for matching and analysis of coverage (both by count and by volume)
for all five EIA frames; (2) Results of matching and analysis for EIA-63a
(solar thermal collectors), EIA-63b (photovoltaic cells and modules), and EIA-5
(coke plants); (3) Identify differences between EIA and Census frames which
impact coverage analysis; (4) Identify characteristics of establishments
missing on EIA frames; and (5) Identify next steps for assessing coverage of
EIA-3(Manufacturers that consume coal) and EIA-860 (Combined Heat and Power
Plants).
Hands-On Usability Testing of EIA's New
Website Design: Can You Find What
You're Looking For? Colleen Blessing,
Working since last fall, and based on input from many
sources, the Web Redesign Group has created a new prototype design we would
like the ASA members to "test drive."
This session will be you sitting at a computer using the new site, commenting
as you click through, giving us feedback on what you see, and finding answers
to questions or finding what you usually use on the current site.
EIA conducted a similar usability test with ASA members before our redesign in 2000 and received useful feedback. There is no preparation necessary for this session, except if you want to record some reports or data tables you would like to be able to find on the new site. Looking at the new design for the first time in this session will allow us to capture your first impressions.
Follow-up on Frames
Team Activities Howard Bradsher-Fredrick, SMG, EIA This paper will present
a summary of the activities and conclusions of the various incarnations of the
frames team from March 2004 through April 2005. The frames team was an
inter-office EIA team composed of staff members from the EIA program offices,
SMG and OIT. In general, all interested
and affected offices had representation on each incarnation of the team
The impetus for the effort on improving frames emanated from the EIA Strategic Plan and specifically from the Strategic Information Technology Subgroup. These frames teams were charged with identifying efficiencies EIA can pursue regarding frames and the computer systems that support frames.
In particular, the first team was chartered with providing recommendations concerning activities in the frames area that will lead EIA toward saving resources, facilitating more commonality in data and software, improving processes, and resulting in frames and frame information that can easily be shared within EIA. The first frames team met from March 2004 to early May 2004 and developed a set of six recommendations.
The second incarnation of the team met from August 2004 to January 2005. The primary focus of its activities was to assess each unique EIA survey frame for “sufficiency.” In order for the team to ascertain sufficiency or insufficiency for individual frames, a number of tasks had to be accomplished, some of which involved extensive survey frame information gathering.
The third incarnation of the team began meeting in February 2005. It is charged with following up on the recommendations put forward by the first incarnation of the team.
EIA’s Proposed Strategy for Addressing Declining Response Rates in the
Residential Energy Consumption Survey (RECS) Dwight French, EMEU,
EIA Over the 25-year history of EIA’s
Residential Energy Consumption Survey (RECS), response at various stages of
this survey system has slowly but steadily decreased. This trend, which has been true for most
government household surveys, is especially troublesome for RECS, which is
actually a sequence of interconnected data collections: a survey of the household to get its
energy-related characteristics, a request during the household survey to get
the household respondent to sign a waiver form allowing the household’s energy
suppliers to submit billing information, and then a survey of the suppliers to
collect the energy consumption and expenditures information. All of these collections are necessary to get
a complete set of survey data; therefore, non-response at each stage has a
cumulative negative effect on data quality.
This presentation will describe an aggressive, multi-faceted strategy
that EIA has planned to address the comprehensive non-response issue, including
additional and updated preliminary contacts, an incentive experiment, enhanced
non-monetary incentives, the use of scanners to collect utility bill
information at the household, a Spanish translation of the questionnaire,
publicity about data confidentiality based on EIA’s new CIPSEA legislation, and
a potential non-response sub-sampling process.
The committee is asked to critique our approaches and suggest any others
they might think would be useful.
Assessments: Presentations and a Panel Discussion Nancy Kirkendall, moderator
External Review of Survey Programs: A Progress Report, Brenda Cox, Battelle, under contract to EIA, and Nancy Kirkendall, EIA The Performance Assessment Rating Tool (PART) was instituted to boost the quality of Federal programs by encouraging rigorous performance assessment for government programs. These performance assessments identify program strengths and weaknesses, resulting in recommendations for improvement and recognition of desirable design elements already in place. For programs that involve a family of related surveys, the individual surveys that comprise the survey program need to be evaluated as well as the effectiveness of their integration to create the survey program. This paper presents the progress made in this investigation designed to develop templates for performing an external evaluation of a family of related surveys. The end products of this study will be a Program Evaluation Template and a Survey Evaluation Template, which can serve as guides for performing expert assessments of survey programs and their component surveys. At the Fall 2004 meeting of the ASA Committee on Energy Statistics, we described the project’s goals and presented the survey template that was to be concept tested later that year. This presentation updates that research by presenting the Program Evaluation Template and the insights gained from our proof of concept testing of both templates on the Petroleum Marketing Survey Program.
The overarching question that the external study team will address is:
Given continuing tight EIA budgets and other external drivers, is EIA
doing the “right things” now, and what are the “right things” it should do in
the next five years?
The question intentionally focuses on “is EIA doing the
right things” (i.e., what products and services are provided to customers) and
not on “is EIA doing things right” (i.e., how it conducts its internal business
processes). At the ASA session, EIA will review its goals
for this external study, outline project’s tentative timeline (scheduled to be
completed in November 2005), and solicit ideas and suggestions from ASA Energy
Committee members who may wish to share their experience with evaluative
processes that resulted in positive change.
Discussion: Where Next? Howard Gruenspecht, EIA Deputy Administrator and Moderator
Progress on EIA’s
914: Response rate and kinds of challenges, John Wood, Reserves and
Production Division (RPD), EIA At the
fall 2004 meeting with the ASA Energy Committee, EIA introduced the new and
approved Natural Gas Production Survey, Form EIA-914. The spring 2005
meeting will provide the first opportunity to discuss EIA’s preliminary
assessment of the survey data for January, 2005, the first reporting month for
this new survey. EIA will discuss the challenges faced during the early months
of survey operation, as well as our plans for review, analysis, and comparisons
of final natural gas production estimates based on the early sample of final
production provided by the EIA-914 survey data with our previous methods of
estimating final natural gas production. This review, comparison, and analysis
is a critical step in the process of
deciding when EIA should start publicly releasing the EIA based
production estimates as our official EIA natural gas production estimates.
Estimate EIA’s 826
Since the Last Time: Differences, estimation groups, outliers and test results
Joseph Sedransk, Jim Knaub, Lindolfo Pedraza and
EIA Form-826 collects information, monthly, from regulated
and unregulated companies that sell or deliver electric power to end users. It
collects State-level sales volumes, sales revenues, and number of customers by
end-use sector (residential, commercial, industrial, and total). The existing
sample and methodology to estimate population totals were described at the fall
2004 meeting of the Energy Committee. This session will describe current
efforts to form homogeneous subpopulations for estimation, and evaluations
based on predictions using annual data (where truth is assumed known). We
assess the importance of outlier detection, under alternative scenarios.
We will be asking for advice on next steps for analysis, and on how to
frame convincing recommendations for implementation.
.