ABSTRACTS

for the Fall Meeting of the

American Statistical Association (ASA)

Committee on Energy Statistics

October 16 and 17, 2003

with the

Energy Information Administration

1000 Independence Ave., SW.

Washington, D.C.  20585

 

Thursday, October 09, 2003

 

I.  Strategic Plan –

 

Title 1: EIA's Strategic Plan and Performance Goals for 2003-2008  (Plenary Session):

 

Note: Session emphasis is to be on Action plan for Goal 1, the first of the three EIA Goals:

 

Goal 1: EIA's information program is relevant, reliable and consistent with changing industry structures, and EIA's information products are high quality and timely. 

Goal 2: EIA's resource base is sufficient to accomplish its mission

Goal 3: EIA employees rate EIA high in the areas of leadership management, and meaningful work; and they rate themselves high in motivation and productivity

 

Documentation for Goal 1 will include the strategic plan (nearly done) and the action plan. 

 

Quantifying Analytical Quality Through Assessments by Independent Expert Reviewers   Bill Weinig, SMG, EIA  Paper developed by Howard Bradsher-Fredrick, Tom Broene, Stan Freedman, Herb Miller and Bill Weinig.

 

EIA has been conducting surveys and employing other methodologies to assess the quality, timeliness and other attributes of EIA products for a number of years.  However, a systematic quantitative assessment of these attributes related to EIA analytical products has not been conducted.  In this paper, we are proposing a limited survey of subject matter experts who review EIA draft products through the Independent Expert Review program and other review media.

 

Questions to the Committee:

 

  1. Are EIA’s proposed survey questions balanced and comprehensive given the goal?
  2. Are the Independent Expert Reviewers likely to be the appropriate survey audience?
  3. Is the pretest group of two or three reviewers, described earlier, sufficient?
  4. Are there other questions we should be asking?

 

II.  Natural Gas –

 

Overview:  EIA’s Data Initiatives in  Natural Gas  Presentation by Elizabeth Campbell, Office of Oil and Gas, EIA (Plenary Session). Overview topics will include DOE Secretary’s Data initiative and relationships to recent and planned work including natural gas production estimates, price estimation, and proposed new LNG survey. (Plenary Session)

 

Criteria to Select and Implement Estimation Procedures:  Comparison of Texas Production Methodologies  Presentation by Kara Norman, SMG, EIA  (Break Out Session)

 

The Statistics and Methods Group within EIA performed an assessment of two methods for estimating natural gas production in the state of Texas, a Parametric Model versus a Multinomial Model.  The Evaluation of Methods utilizes the following criteria: transparency, timeliness, accuracy and reproducibility.  The goal of SMG is to apply these evaluation measures to future comparisons of any two estimation methods.

 

Questions for the ASA Committee:

 

  1. Are the outlined criteria adequate and appropriate?
  2. Was the Evaluation of Methods applied correctly and sufficiently?
  3. Would (or how should) the criteria change based on the comparisons of different methodologies?
  4. Now that a new method, the Multinomial Model, has been selected, what improvements could be made to the chosen estimation procedure?

 

Estimating Industrial Natural Gas Prices  Presentations  Presentations by Ruey-Pyng Lu, SMG, EIA and Michael Taylor, University of Michigan graduate student and SMG Summer Intern, EIA. (Break Out Session)

 

The Energy Information Administration (EIA) collects and publishes data on prices and volumes of natural gas delivered to customers in four sectors (residential, commercial, industrial, and electric power). EIA publishes the average price for a state computed as revenue averaged by volume. Natural gas industry restructuring has resulted in industrial customers consuming less than 20 percent of the gas purchased from companies that physically deliver it to the facilities.  EIA’s respondents cannot report a revenue stream associated with the 80 percent for which they are transporting “for the account of others”.  To estimate the price of the natural gas consumed by the industrial sector, EIA has been exploring the use of another survey, Form EIA-423 [Monthly Cost and Quality of Fuels for Electric Plants] which surveys all nonutility electricity generating facilities with a nameplate capacity of 50 megawatts or greater.  At the April 2003 meeting we showed the committee some plots of the relationship between price and volume from that survey. Since there was not a discernible relationship the committee suggested that we develop a model using the cost and volume from the Form EIA- 423 , and data reported by deliverers of natural gas on the EIA-857.

 

Meanwhile we have been receiving and reviewing additional EIA-423 data, which we believe represent off-system prices (the deliveries for the account of others) for industrial facilities that generate electricity and meet the threshold for the EIA-423 survey.  Since the EIA857 represents the on-system prices, the question is how best to estimate the off-system prices for the industrial facilities that either don’t generate electricity or generate electricity, but do not qualify for Form EIA-423. 

 

Our approach is to model the difference between natural gas prices paid by EIA-423 respondents in 2002 (the first year of the survey’s existence) and a day-weighted average of monthly Henry Hub (a large natural gas distribution hub in Southern Louisiana) prices.  Six models were to be made, corresponding to the following areas: the Northeast Census Region, the Midwest Census Region, the South Census region excluding Louisiana, Texas and Oklahoma (which were found to be significantly different from the remainder of the region), Louisiana, Texas and Oklahoma as a block, the West Census Region excluding California, and California by itself following the same rationale used above.  The state-level purchased price reported by EIA would be a weighted average of prices reported on surveys EIA-857, EIA-423 and the predicted price difference from the model added to the day-weighted monthly average of spot gas prices. An enhanced model is adopted to use a day-weighted average of monthly regional Henry Hub (Chicago, New York, California, Oregon and Louisiana) prices. There are some promising results in this modeling approach.

 

The subsequent task to improve the estimation of industrial natural gas price is to use the count of natural gas use facilities and the natural gas volume consumed by them in the industrial sector of EIA-906 [Power Plant Report] and Manufacturing Energy Consumption Survey (MECS) 2002 data (available in Fall 2004) to validate the natural gas industrial price estimated by the difference of regional Henry Hub prices and EIA-423 respondents.  We will be asking the committee for guidance in determining the tabulations we should request from MECS to validate our estimate of the industrial price and also for guidance on presenting our estimates.

 

Natural Gas Production Estimation in the Gulf of Mexico   Presentation by John Wood, Director, Reserves and Production Division, OOG, EIA, in Dallas, Texas. (Break out session).

 

Completion level production data collected and reported by the Minerals Management Service is used to estimate current Gulf of Mexico gas production.  This reported data is essentially complete in mid 2001 and progressively less complete closer to current months.  The basic concept assumes that the incomplete data in recent months is a sample distribution of what will ultimately be reported when complete.  Both complete and incomplete (sample) distributions are very skewed.  Estimates are made for the missing data in each class in the sample distribution based on an expected number of producing completions, which depends on an average or standard complete distribution.  Various methods and procedures for filling in the classes of the sample distributions will be discussed.

Evaluation of EIA-910 Survey of Natural Gas Marketers:  Commercial and Residential Prices   Presentation by Tom Broene, SMG, EIA.  (Break out session)

 

EIA has been conducting the 910 survey in 5 states since August 2001.  The survey supplements the 857 survey of utilities.  An evaluation by SMG shows that:

  1. The volumes of natural gas sold by marketers do not match what the utilities claim to be transporting for resale, but are judged to be acceptable.
  2. Compared to the utilities, the marketer prices are little difference for residential, lower for commercial, but higher for both in GA.
  3. Minimal growth in the number of customers purchasing natural gas from marketers.
  4. Minimal impact on national prices.

 

Questions for the committee are:

  1. Does the form make sense?  This is the usual set of questions that EIA uses to obtain a weighted average price.

2.  EIA currently publishes only the average (integrated) price, which is the weighted average of the price from the marketer and the price from the utilities.  Is this reasonable, or should EIA publish both prices?  Should EIA include a variance or range for the marketer’s prices?

Friday, October 17, 2003

 

EIA and the Confidential Information Protection and Statistical Efficiency Act (CIPSEA) Presentation by Jay Casselberry, SMG, EIA (Plenary session)

 

The E-Government Act of 2002 (Public Law 107-347) was signed into law on December 17, 2002.  The law included the Confidential Information Protection and Statistical Efficiency Act (CIPSEA).  Information collected under CIPSEA has two important components:  (1) its must be treated as confidential and (2) it is limited to use for  exclusively statistical purposes unless a respondent provides informed consent for nonstatistical uses.

 

With enactment of CIPSEA, EIA began a process of determining what information will be collected under CIPSEA in 2004.  The process included identifying the reasons why CIPSEA should (and should not be) used for specific surveys in 2004.  After considering the 58 EIA surveys that collect confidential information, it was agreed that 10 surveys would begin collecting information under CIPSEA.

 

Questions posed to the Committee will include:

1.   EIA has developed proposed informed consent agreements that survey respondents would be asked to sign so that EIA could share CIPSEA information  with approved government organizations for nonstatistical uses (e.g., preparing for and/or responding to emergency situations).  EIA is requesting the Committee’s advice on the proposed informed consent agreements as well as any experience the members have with informed consent processes. 

2.            CIPSEA requires that when a statistical agency collects information that may be used for nonstatistical purposes the agency must notify the public of the possible  nonstatistical uses.  Legislation requires EIA to share non-CIPSEA information for official purposes.  At the time of collection, EIA can not be sure of all the possible nonstatistical official uses that may be made of non-CIPSEA information.  Does the Committee have any suggestions on how EIA can effectively communicate the possibility of nonstatistical uses while not unduly raising concerns by EIA’s survey respondents?

3.   EIA will not use CIPSEA for some surveys because it currently has sharing agreements with other Federal organizations and was not sure at this time if the uses would be classified as for “statistical purposes.”  Some examples of EIA sharing information with other government agencies will be described.  The Committee is asked for their views on whether the uses described in the examples are consistent with “statistical purpose’ as described in CIPSEA.

 

Electricity Transmission Data Needs   Presentation by Douglas R. Hale, EIA Senior Scientist on loan to the Lawrence Berkeley Laboratory.  (Plenary session)

 

For much of the year EIA analysts have been working on a report on transmission data. This work has been in part motivated by the Department of Energy’s concerns with reliability and modernization of the high voltage grid and by the Federal Energy Regulatory Commission’s (FERC) reliance on the grid to promote competition. The major motivation was the Energy Information Administration’s (EIA) interest in providing policy relevant information.

 

The first objective of this report is to itemize transmission data currently collected by the Federal government.  An equally important objective is to examine whether the data informs the discussion of basic transmission policy issues. The study is restricted to information that is produced by the Federal and State governments, their agents and regulated entities, such as Independent System Operators (ISOs). North American Electric Reliability Council (NERC) data that are routinely supplied to DOE, EIA and FERC are also included. Privately produced data and proprietary analysis tools are not considered.

 

Data alone cannot answer many policy questions. For example in February 2003, Congress directed the Secretary of Energy to analyze

 

…the possible impact on regional electricity prices of FERC's proposed rule for Standard Market Design (SMD)…This independent analysis must compare wholesale and retail electricity prices and the impact on the safety and reliability of generation and transmission facilities in the major regions of the country both under existing conditions and under the proposed new rule.

 

Answering these kinds of questions requires a comparison of an existing with a hypothetical situation. That is almost always done by creating a mathematical model of the electrical-economic system and seeing how it functions under different policy rules. [1]

 

A third objective of this report is to assess the extent available models are able to help answer policy questions. The availability and quality of data severely limit what models can reliably predict. At the same time developments in economic research and in electricity markets are indicating the importance for behavior of new kinds of information. For example, congestion and transmission rights were of little public interest a decade ago. Now investors, customers and policy makers want to know future congestion costs and the value of transmission rights.

 

The final objective of this report is to suggest specific improvements in data collection and analysis tools that would support better answers to policy questions.

 

This talk is an update on the project’s status and a preliminary discussion of general issues of data collection and dissemination raised by the complexity of transmission data. In particular, the detailed electrical data required to answer questions about reliability and the extent of individual markets cannot be generally available because of National Security concerns. But, some of this data must be available by some means to inform public debate.

 

Market data-prices, production, imports, congestion-are increasingly available on web sites operated by ISOs. The amount of market data, some at 5-minute intervals, is overwhelming. The ISOs do not report the same data the same way, so it is difficult to compare across ISOs.   While it makes little sense for the Federal Government to compile and archive this data it may be worthwhile for FERC to promote standardization.

 

There are some limited areas where additions to EIA’s on going data collections may provide a better picture of transmission relevant data. Some preliminary suggestions are discussed. There are also areas where changes to FERC’s Form 1 might allow a clearer picture of the economics of transmission organizations.

 

Data Edits for the EIA-920, Previously the EIA-906  Presentation by Bob Rutchik.  (Break out session)

 

EIA has designed and tested a new survey to go to combine heat and power facilities.  The major data elements collected are:

  1. Total fuel consumed at the facility,
  2. Fuel used to generate electricity by prime mover type, and
  3. Stocks of coal and petroleum products at the facility.

 

This survey is new, both in layout and in some of the data elements.  Most importantly we are no longer collecting information on useful thermal output from these facilities.  Instead, we are asking for fuel used to generate electricity at the prime mover level.

 

EIA is working on designing new edits for this survey.  We are interested in edits that the respondent can use in the internet data collection, while filling in the form, and summary edits that are used by survey staff to verify the data submissions. 

 

We would like the committee’s help in the appropriateness of proposed edits for the new survey and in suggesting edits for it.  

 

Proposed Guidance on Relative Standard Errors (RSEs)   Presentation by Shawna Waugh, SMG, EIA (Break Out Session)

 

The purpose of this session is to:

  1. define relative standard errors (RSEs) and their importance;
  2. identify current uses of RSEs by end-user and supplier surveys;
  3. recommend best practices; and
  4. determine whether to adopt the proposed guidance on Relative Standard Errors.

 

Relative Standard Errors are typically a tool for evaluating sampling errors (but not bias) of estimates. Some managers use RSEs to edit data; other managers use RSEs to select a sample.  The most prevalent practice is to publish RSE Tables in which each cell in the RSE table corresponds to estimates in a published table, thereby providing users with information on the reliability of estimates.

 

RSEs serve to improve the design and implementation of surveys and assess reliability of published estimates.

  1. RSEs may be used to select sample design or to select a sample.
  2. RSEs may be used to identify errors prior to publication.
  3. RSEs should be used to restrict publication of unreliable estimates.
  4. RSEs should be to inform data users about the reliability of estimates.
  5. RSEs may be used to compare data.

 

Over time on a single survey and among current surveys the criteria select for restricting publication of unreliable estimates differs.  Some managers would advocate restricting any estimate with an RSE greater than 10; other managers support publishing estimates with an RSE less than 50; and a few managers would support publishing all estimates regardless of the RSE. When selecting criteria to restrict publication of estimates, it is important to balance accessibility to estimates with importance of publishing reliable estimates.

 

Questions

  1. Should guidance on how to use RSEs be provided?
  2. If so, what are best practices for improving data quality?
  3. If so, what are best practices for inform customers how to interpret and use RSEs to evaluate reliability of estimates?
  4. What experience have members had with RSE?

[1] Occasionally the alternative policy is followed somewhere outside of the area considering a change. The UK’s early success with restructuring its electricity industry was used by supporters of California’s restructuring as evidence that competitive markets would increase generation investment and lower electricity prices. Another way to evaluate a proposal is to create artificial markets in a controlled setting and see how real people act when the rules are changed. A good analysis method, either mathematical modeling, historical analogy or experimental, would correctly anticipate real world responses to alternate policies.