U.S. Energy Information Administration - EIA - Independent Statistics and Analysis
Analysis & Projections
Annual Energy Outlook Retrospective Review
Release Date: March 25, 2015 | Next Release Date: March 2016 | Report Number: DOE/EIA-0640(2014)
Evaluation of 2014 and Prior Reference Case Projections
The U.S. Energy Information Administration (EIA) produces projections of energy production, consumption and prices each year in the Annual Energy Outlook (AEO). Each year, EIA also produces an AEO Retrospective Review document, which presents a comparison between realized energy outcomes and the Reference case projections included in previous editions of the AEO. The purpose of the Retrospective is to show the relationship between past AEO projections and actual energy indicators, enable trend analysis, and inform discussions of potential improvements to the AEO.
The projections in the AEO are not statements of what will happen but of what might happen, given assumptions and methodologies. The AEO Reference case projection assumes trends that are consistent with historical and current market behavior, technological and demographic changes, and current laws and regulations. The potential impacts of pending or proposed legislation, regulations, and standards are not reflected in the Reference case projections.1
The discussion and analysis in each year's AEO primarily focuses on a Reference case, lower and higher economic growth cases, and lower and higher oil price cases. In addition to these standard cases, numerous alternative cases have been included in recent AEOs. Readers are encouraged to review the full range of cases, which address many of the uncertainties inherent in long-term projections.
The AEO Retrospective comparison, by contrast, summarizes the differences between actual energy indicators and past AEO Reference case projections (alternative AEO cases are not included) by calculating the average absolute differences and the average absolute percent differences for several of the major variables for AEO1994 through AEO2014.2 The average absolute percent difference is the simple mean of the absolute values of the percentage difference between the Reference case projection and the actual value. The historical data are primarily from the Monthly Energy Review (MER).3 This edition includes tables for prices in constant dollars in addition to nominal-dollar tables. Projections for consumption concepts depend on real prices, so constant-dollar price comparisons give a clearer picture of the linkage between consumption projection differences and price projection differences.
The annual projection process begins with the development of assumptions for two key drivers—the world oil price and the macroeconomic growth baseline—that are determined exogenously to the National Energy Modeling System (NEMS) used to prepare the AEO. While the integrated nature of NEMS results in some dynamic feedback that slightly modifies the initial assumptions about the world oil price and the macroeconomic growth baseline, these feedbacks tend to be relatively small, so the initial assumptions for these drivers largely determine the overall projection environment. To the extent that the initial assumptions used to drive the NEMS projections deviate from the realized environment, the NEMS projection results for energy markets will also generally deviate from realized energy outcomes.
Table 1 provides a summary of 25 comparisons based on 21 primary concepts (the other four concepts are energy price projections presented in both real-dollar and nominal-dollar valuations). The detailed results that underlie Table 1 are presented in Tables 2 through 22. These tables show the Reference case projections, historical values and percent differences between projected and actual for all years available. They also provide the average absolute difference across all AEOs for each year of history. The tabulated results can change from one year's evaluation to the next due to data revisions for prior years that can occur in the Monthly Energy Review (MER), and, occasionally, in gross domestic product (GDP).
The first column in Table 1 is the simple average of the absolute value of percent differences for each concept across all AEOs since the 1994 edition and across all available historical years. In general, energy consumption quantities tend to be more stable and thus projected with greater accuracy than the relatively volatile concepts of net imports and energy prices. This is because energy consumption trends reflect underlying sources of inertia in the energy-consuming capital stock, substantial lead times for capital purchase decisions, locked-in contract periods, and consumer perceptions of and responses to price changes. The average absolute percent differences for energy consumption, production, and carbon dioxide emissions range roughly between 5% and 11%. The absolute percent differences for energy prices range between 10% and 38%. Net imports tend to exhibit even larger average absolute differences, ranging between 25% and 55%, primarily influenced by pre-2008 AEOs which projected much higher net imports.
The second column of Table 1 provides a summary of the percentage of cases in which a particular concept is overestimated relative to its actual value out of the total number of annual projections made for which actual values are available across all AEOs since the 1994 edition. A zero would represent a concept that was always underestimated and 100% would represent a concept always overestimated. In an "unbiased" projection, one would expect the percentage of overestimates to be close to 50% when the sample size is large enough.
|Variable||Average Absolute Percent Differences||Percent of Projections Over- Estimated|
|Gross Domestic Product|
|Real Gross Domestic Product (Average Cumulative Growth)* (Table 2)||0.9||45.8|
|Imported Refiner Acquisition Cost of Crude Oil (Constant $) (Table 3a)||37.7||17.3|
|Imported Refiner Acquisition Cost of Crude Oil (Nominal $) (Table 3b)||36.6||18.7|
|Total Petroleum Consumption (Table 4)||7.9||70.7|
|Crude Oil Production (Table 5)||8.1||51.1|
|Petroleum Net Imports (Table 6)||24.7||73.8|
|Natural Gas Price, Electric Power Sector (Constant $)** (Table 7a)||29.5||28.9|
|Natural Gas Price, Electric Power Sector (Nominal $)** (Table 7b)||28.7||31.1|
|Total Natural Gas Consumption (Table 8)||8.0||67.1|
|Natural Gas Production (Table 9)||7.4||60.4|
|Natural Gas Net Imports (Table 10)||54.5||74.7|
|Coal Prices to Electric Generating Plants (Constant $)*** (Table 11a)||20.1||36.0|
|Coal Prices to Electric Generating Plants (Nominal $)*** (Table 11b)||21.1||39.1|
|Total Coal Consumption (Table 12)||10.8||66.7|
|Coal Production (Table 13)||7.8||69.3|
|Average Electricity Prices (Constant $) (Table 14a)||10.4||32.9|
|Average Electricity Prices (Nominal $) (Table 14b)||12.2||38.7|
|Total Electricity Sales (Table 15)||4.9||53.8|
|Total Energy, Carbon and Intensity|
|Total Energy Consumption (Table 16)||6.8||80.9|
|Delivered Residential Energy Consumption (Table 17)||5.4||71.6|
|Delivered Commercial Energy Consumption (Table 18)||6.6||50.7|
|Delivered Industrial Energy Consumption (Table 19)||11.3||86.2|
|Delivered Transportation Energy Consumption (Table 20)||7.9||80.9|
|Total Energy Related Carbon Dioxide Emissions (Table 21)||8.5||68.0|
|Energy Intensity (Energy Consumption / Real $ GDP) (Table 22)||10.0||93.3|
|AEO – Annual Energy Outlook.
Source: These statistics summarize the calculations in Tables 2 through 22. The data in Tables 2 through 22 are based on the 1994 through 2014 AEO Reference case projections. Historical data are taken from theÂ Monthly Energy Review September 25, 2014, DOE/EIA-0035(2013/08) (Washington, DC, September 2014), except for GDP data which are from the Bureau of Economic Analysis, U.S. Dept. of Commerce, September 2014, http://www.bea.gov/national/xls/gdplev.xls.
* The basis for GDP comparison is the projection differences in the cumulative average growth rate of real GDP from the first year shown for each AEO. The summary information for projection differences given for GDP growth rates is absolute percentage point differences; for all other AEO concepts, the comparison basis is absolute percent differences.
** As of 2013, the wellhead price of natural gas was no longer reported by EIA. With this edition of the Retrospective, the natural gas price to the electric power sector replaces the wellhead price.
*** Beginning in AEO2003, EIA electric generating projections incorporated combined heat and power (CHP) electricity generation in electricity generating plants. Prior to AEO2003, coal price projections reflected data collected, estimated, and reported to electric utilities and excluded CHP power generation.
The underlying reasons for deviations between the AEO Reference case projections and actual outcomes are numerous and varied. Changes in economic, market, and technological trends and in policies affecting energy choices are often important drivers of these deviations. EIA monitors these developments and uses the best available data to continuously update NEMS and the AEO projections. EIA makes extensive use of scenarios in the AEO to explore the implications of major breaks from historical trends. Even though the precise timing and magnitude of a change in trend cannot be predicted, the likelihood that there will be some changes in trend is virtually certain, and the impact of such changes is identified using the AEO scenarios. Thus, the full range of potential outcomes cannot be evaluated with the AEO Reference case alone, but only in conjunction with the relevant AEO scenarios and other analysis as applicable to a specific situation.
Some notable factors contributing to deviations between actual outcomes and AEO projections are:
- Macroeconomic factors play a very important role in determining energy usage. The financial crisis and resulting economic weakness starting in 2007 was not anticipated in AEO modeling as NEMS does not project future business cycles. AEO2001 through AEO2009 over-projected macroeconomic growth (Table 2). Earlier AEOs systematically underestimated cumulative growth, even through 2007-2009, due in part to the robust 3.9% average growth in real GDP from 1993 through 2000. Post-2000 real GDP growth has decreased to an average of 1.8%, and AEOs produced during this period have tended to overestimate, even before the impacts of the latest recession. In general, the AEO tends to lag behind macroeconomic trends as a result of an analytic framework which returns economic factors to long-run trends, rather than attempting to predict business cycles or other economic phenomena.
- The imported refiner acquisition cost of crude oil in 2013 was just under $100 per barrel, down nearly 5 dollars from its peak in 2011 (Tables 3a, 3b). Many of the AEOs prior to AEO2006 projected nominal prices less than $35 per barrel in 2013. Underestimation of crude oil prices since 2000, and higher vehicle efficiency standards that were not included in the Reference Case assumptions, contributed to overestimation of petroleum consumption for these AEOs. This coupled with the effects of the 2007-2009 recession has generally resulted in overestimates of petroleum consumption for the years since 2008 (Table 4).
- Petroleum net imports peaked in 2005, falling successively in each year since. Before AEO2006, with few exceptions, AEOs had projected rising net imports post-2005. From AEO2006 on, projections for net imports have included years of decline (Table 6). This effect was caused primarily by the economic downturn, with a lesser contribution from increasing vehicle fuel efficiency. More recently, rising domestic oil production has also played an important role. Going forward, the phased increase in light-duty vehicle fuel economy standards through 2025 will continue to reduce projected petroleum consumption (with gasoline use falling enough to compensate for some continuing freight-driven growth in diesel fuel use).
- Natural gas production has increased each year after 2005 after a long period of relatively flat production, and achieved another record high level in 2013 (Table 9). Natural gas net imports peaked in 2007, falling successively in each year since, with 2013 at the lowest level shown (Table 10). AEOs since AEO2006 have tended to underestimate production and consequently overestimate net imports.
- Prior to 2009, the fuel with the largest absolute percent difference between the projections and historical consumption has generally been natural gas. This can be partially traced to the performance of natural gas price projections. In earlier projections, the natural gas price was influenced by the changes in the world oil price, but this relationship has decoupled since 2000. Through AEO2005, the Reference case has generally overestimated natural gas consumption (Table 8), in part due to the tendency for significant underestimates of the natural gas price (Tables 7a, 7b). In addition, industrial natural gas usage is highly sensitive to business cycles, leading to greater volatility in overall natural gas consumption.
- Coal consumption is dominated by electric generating use and usage generally grew modestly until 2009 (Table 12). Coal prices to electric generating plants have been increasing smoothly since 2000 until they flattened in 2012, then dropped slightly in 2013 (Tables 11a, 11b). In 2009, the natural gas price to the electric power sector dropped nearly 50% primarily due to increases in natural gas production (Tables 7a, 7b). As a result, 2009 coal consumption dropped by over 10% and has not achieved the 2009 level since. These combined outcomes have led to coal consumption being overestimated by 10 to 31% since 2009 (Table 12).
- Electricity prices were almost always underestimated in the Reference case projections from AEO1998 on (Tables 14a, 14b). In deregulated markets, natural gas tends to determine marginal electricity prices, and with part of the United States remaining regulated, this presents a complex pricing environment for a national projection; but overall, the underestimation of natural gas prices pre-2009 and of coal prices generally (noted above) also results in underestimated electricity prices.
- In 2013, total energy consumption experienced a 3% increase but was still lower than it was in 2000. Nearly all AEOs projected rising consumption over the historical period, with AEO2008, AEO2009 and AEO2012 being exceptions by exhibiting single-year declines early in their projection horizons. This is the result of earlier AEOs not anticipating the strong effects of the recent recession as well as underestimation of reductions in the energy intensity of the economy, as discussed below. Projection differences tend to be relatively small; however, some pattern of overestimation is evident (Tables 16 to 20).
- Projected energy intensity has been the concept most often overestimated (Table 22). Energy intensity for the retrospective comparisons is defined as the ratio of two other projected concepts: total energy consumption and real GDP. As a ratio, the projection differences are dependent upon relative differences in energy consumption and real GDP. The overestimates of energy intensity are the result of two "reinforcing" features of past projections. During the period through AEO2000, GDP tended to be underestimated, which imparts a tendency to overestimate energy intensity.4 Post-1998, energy consumption has tended to be overestimated, once again imparting a tendency to overestimate intensity.
1Each full Annual Energy Outlook includes a variety of other cases with different assumptions – for AEO2014, 29 alternative cases exploring different assumptions and different policies were presented in addition to the Reference case.
2The National Energy Modeling System (NEMS) has been used to prepare the Annual Energy Outlook since AEO1994, and this publication covers only these AEOs. The AEO2009 results are from "An Updated Annual Energy Outlook 2009 Reference Case Reflecting Provisions of the American Recovery and Reinvestment Act and Recent Changes in the Economic Outlook" (passed after the official publication of the AEO2009). This version is only available on the EIA website at: http://www.eia.gov/oiaf/servicerpt/stimulus/index.html. Finally, projections from AEOs for 1994 and 1995 ended in 2010, so beyond 2010, entries for these two publications are now blank in Tables 2 through 22.
4The recent comprehensive revisions to GDP included with the update from 2005 chained dollars to the BEA current use of 2009 chained dollars increase nominal GDP in 2005 by about 3.7%. This was the chained dollar year for AEOs 2011 through 2014. Since the prior AEOs could not have anticipated this level adjustment, Table 22 includes an upward adjustment of 3.7% in GDP across all of the projection years. Energy intensity is still the most often overestimated concept after this adjustment. AEO 2015 will be based on the 2009 chained dollar concept and will not be adjusted in future Retrospectives.