Information Quality Guidelines
"Independent Statistics & Analysis."
- Background of the U.S. Energy Information Administration (EIA)
- Office of Management and Budget Information Quality Guidelines
- Department of Energy Information Quality Guidelines
- EIA Information Quality Guidelines
- EIA Information Subject to Information Quality Guidelines
- Questions about EIA Information and Requests for Correction of Disseminated Information Not in Compliance with Information Quality Guidelines
1. Background of EIA
EIA, created by Congress in 1977, is a statistical agency of the U.S. Department of Energy (DOE) and is one of the ten principal statistical agencies in the Federal government. EIA provides policy-independent data, forecasts, and analyses to promote sound policymaking, efficient markets, and public understanding regarding energy and its interaction with the economy and the environment. EIA information products are used by public and private analysts and policymakers to monitor the current status and trends of energy supply and consumption.
EIA's activities in the creation, collection, maintenance, and dissemination of information include:
- Developing concepts and methods
- Planning and designing surveys and other means of collecting data
- Collecting data
- Processing and editing data
- Analyzing data
- Producing estimates and projections
- Reviewing information products
- Disseminating information in published reports, electronic files, and other media requested by users.
(For a general discussion of major Federal statistical agencies' activities, including their quality programs, see the Federal Register (67 FR 38467) notice jointly issued by the agencies on June 4, 2002.)
EIA's information dissemination activities adhere to separate information quality guidelines issued by the Office of Management and Budget (OMB), DOE, and EIA. EIA's intention is that those guidelines shall apply to information prepared by EIA for public dissemination and disseminated on or after October 1, 2002, regardless of when the information was first disseminated. Discussions of the applicable quality guidelines follow.
2. Office of Management and Budget (OMB) Information Quality Guidelines
In response to section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554), the U.S. Office of Management and Budget (OMB) issued final information quality guidelines on February 22, 2002 (67 FR 8452) and directed Federal agencies to issue guidelines for ensuring and maximizing the quality of information they disseminate.
3. Department of Energy Information Quality Guidelines
The Department of Energy (DOE) information quality guidelines apply to all DOE components. DOE is also providing a mechanism for a person to request correction of DOE-disseminated information that the person believes does not comply with applicable information quality guidelines.
4. Energy Information Administration Information Quality Guidelines
EIA has performance standards to ensure the quality (i.e., objectivity, utility, and integrity) of information it disseminates to the public. Quality is ensured and maximized at levels appropriate to the nature and timeliness of the disseminated information. EIA also strives for transparency about information and methods in order to improve understanding and to facilitate reproducibility of the information. Additional information about EIA's quality program is available at our information quality guidelines Web site.
EIA's commitment to quality and professional standards of practice includes:
- Use of modern statistical theory and practice in all technical work
- Development of strong staff expertise in the disciplines relevant to its mission
- Implementation of ongoing quality assurance programs to improve data validity and reliability and to improve the processes of compiling, editing, and analyzing data
- Development of a strong and continuing relationship with appropriate professional organizations in the fields of statistics and relevant subject-matter areas.
To carry out its mission, EIA assumes responsibility for:
- Determining sources of data
- Establishing measurement methods
- Selecting methods of data collection and processing that provide useful information while minimizing respondent burden
- Employing appropriate analytical and forecasting methods
- Ensuring the public availability of data and supporting documentation.
In conjunction with its information collection and dissemination activities, EIA is fully in compliance with the Paperwork Reduction Act of 1995 (PRA), Public Law 104-13. EIA shall use the PRA clearance process to help improve the quality of information disseminated to the public. EIA demonstrates in its PRA submissions to OMB the "practical utility" of a proposed information collection that EIA plans to disseminate. Additionally, for a proposed information collection that EIA plans to publicly disseminate, EIA shall evaluate the collection in light of applicable information quality guidelines and state in its PRA clearance submission to OMB that the information will be collected, maintained, and used in a manner consistent with the OMB, DOE, and EIA information quality guidelines.
With available resources, EIA continually works to improve its systems in order to provide high quality information needed by public and private policymakers and decisionmakers. EIA standards underlie its quality guidelines. The standards document the professional basis upon which EIA expects to be judged by our stakeholders and the level of quality and effort expected in all our activities, including those of our contractors.
For many years, EIA has utilized standards, policies, and other operational guidance to ensure the quality of its activities. EIA's current Standards Manual is available.
Following is a general discussion of the EIA's commitment to performance standards for quality.
4a. Objectivity
Objectivity involves ensuring the substance of the information is accurate, reliable, and unbiased, and the information is presented in an accurate, clear, complete, and unbiased manner. Objectivity is achieved for EIA's information products by having qualified people use reliable data sources and sound analytical and forecasting techniques to prepare information products that undergo careful reviews prior to dissemination.
EIA shall use reliable data sources
Much of the information disseminated by EIA is based on surveys sponsored by EIA. EIA-sponsored surveys are conducted using methodologies that are consistent with generally accepted professional standards for all aspects of surveys, including frame development; statistical design; questionnaire design and testing; data collection; and control of sampling and nonsampling errors through nonresponse analysis, imputation of missing data, and development of weights, adjustments, and variance estimates, as appropriate. EIA surveys and information systems are documented and explanatory materials are made available for EIA information products.
EIA's surveys are also approved by OMB under the Paperwork Reduction Act of 1995. (An inventory of EIA surveys is available in the EIA Survey Forms.)
EIA information products may also use data produced or maintained by other U.S. government agencies, other information organizations, and foreign organizations. EIA's use of this information facilitates public access. External information is assessed prior to EIA's dissemination of the information. However, the performance standards applied to information generated by EIA cannot be applied to information from external sources. In disseminating non-EIA information, EIA focuses on using the best available information and ensuring the transparency of that information.
Data users are encouraged to consider the initial sources of information presented in EIA's information products and to determine the suitability of such information for their purposes.
EIA shall use sound analytic and forecasting techniques
EIA analytical reports are prepared using a variety of analytical techniques, including simple tabulations with descriptive summary statistics, multivariate statistical methods, and econometric models. Analytical techniques are reviewed for their appropriateness to the data and the analysis being conducted and are clearly identified in reports.
EIA uses models to conduct analyses and produce forecasts. Most models used by EIA have been developed by EIA staff or by contractors under direction from EIA. When EIA uses a proprietary model from another organization, EIA undertakes an analysis of the model to ensure its appropriateness before including the model results as part of an information product. Models are documented, tested, and evaluated prior to their use in conjunction with EIA information products.
EIA models are based on best judgments of current and future behavioral relationships and methods of projection. The models are periodically updated to reflect input from internal and external reviews and research findings on behavioral relationships. All updates are documented.
EIA models have detailed documentation describing the goals and objectives of the model, the data sources being used, and the methodologies and assumptions employed. Model documentation is available on EIA's Web site at Recent Energy Model Documentation or from the model manager.
EIA shall emphasize quality in the preparation of information products
EIA information products are based on surveys, models, and external information sources. Appropriate quality control procedures are used in all steps of preparing information products. Documentation of EIA information products is designed to improve understanding of the information so that users may assess the suitability of the information for their needs.
EIA shall conduct quality reviews of information prior to dissemination
Information products are reviewed by technically qualified staff prior to dissemination to ensure their quality. Products that are considered to be more technically complex may also be reviewed by independent expert reviewers from outside EIA to provide additional perspective and expertise. The level of review an information product is subjected to prior to dissemination is determined by the characteristics of the product and EIA-established review procedures.
EIA shall provide information on methodologies and data quality issues
EIA informs users of the concepts and methodologies used in collecting and processing the data, the quality of the data disseminated, and other features that may affect the use or interpretation. By providing information on methodologies and concepts to information users, EIA enables users to make judgments and verify that the data they are using are similar in conceptual framework and definitions to the data they need to complete their work.
The information also allows users to better understand possible sources of error which might restrict their uses of the data. In the area of statistical information, objectivity requires acknowledging that errors in statistical estimates are unavoidable. These errors generally fall under the categories of sampling and nonsampling errors. Sampling errors result when estimates are based on a sample and not a complete enumeration of the population of interest. EIA provides information regarding what is known about the magnitude of these errors, such as variances or coefficients of variation to quantify the magnitude of sampling errors. Though quantifying nonsampling errors is more difficult, EIA does provide information to assist users in understanding those possible error sources.
EIA shall correct errors and issue revisions of previously disseminated information, as appropriate
If a substantive error is detected after a product is disseminated, EIA will make correction and issue an errata notice or other notification as appropriate.
EIA's information may be revised after initial dissemination to reflect more complete information or other changes in the underlying data. EIA information products may include information that is preliminary and is expected to be revised, and revisions made to information previously disseminated. Preliminary and revised data are noted.
4b. Utility
Utility refers to the usefulness of the information to its intended users. Utility for EIA's information products is achieved by continuously monitoring information needs and developing new information sources or by revising existing information collection methods, models, and information products when appropriate.
EIA's information products are widely available and broadly accessible. EIA's efforts to ensure the utility of information products shall include:
- Establishing and adhering to schedules designed to ensure quality information is released in a timely manner
- Providing information products in ways accessible to a broad range of information users
- Providing explanatory materials to assist users in understanding and interpreting the information
- Analyzing customers' information requirements by use of specific tools, such as customer surveys
- Assessing the products themselves to help ensure timeliness and relevance
- Holding discussions with policymakers and analysts in Congress, the Executive Branch, and State and local governments
- Consulting with data providers and data users (e.g., solicitation of comments through the Federal Register, feedback from Web customers)
- Conducting cognitive interviews and focus groups to evaluate information collection and dissemination methods and instruments
- Sponsoring user conferences (e.g., the annual EIA Energy Conference)
- Attending energy industry conferences and other meetings
- Conducting other outreach activities to adapt EIA's information programs as appropriate
- Sponsoring the evaluation of EIA programs and information products by the American Statistical Association Committee on Energy Statistics.
4c. Integrity
Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. EIA ensures the integrity of its information products by adhering to applicable policies and programs.
EIA shall ensure the integrity of its disseminated information in the following ways:
- EIA has an established Operations Security Program.
- EIA employs rigorous information technology controls that have been identified as desirable security practices.
- EIA utilizes procedures to protect survey information collected under pledges of confidentiality.
- EIA meets its statutory requirements regarding information integrity, including the Privacy Act of 1974; Freedom of Information Act; Paperwork Reduction Act of 1995; and other relevant laws.
4d. Influential Information — Transparency and Reproducibility
Influential scientific, financial, or statistical information shall include a high degree of transparency to facilitate its reproducibility.
- Influential when used in DOE means information: (1) that is subject to embargo until the date of its dissemination by the Department or DOE Element disseminating the information because of potential market effects; (2) that is the basis for a DOE action that may result in an annual effect on the economy of $100 million or more; or (3) that is designated by a DOE Element as "influential"
- Transparency means clear and concise information on such topics as information sources, survey and analytical methods, accuracy, and reliability.
- Reproducibility means capability of being substantially reproduced, subject to an acceptable degree of imprecision. With respect to analytical results, "capable of being substantially reproduced" means that independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error.
EIA has several information products that are considered influential because they are subject to embargo procedures given their impact on markets. For influential information, EIA shall have a high degree of transparency of data and methods ensuring the capability to reproduce the information.
EIA also disseminates important energy information that does not fall within DOE's influential information category, but the information does have many public and private uses. EIA shall strive for transparency about such information and methods in order to improve users' understanding and to facilitate reproducibility.
To help ensure transparency and reproducibility, EIA requires documentation of systems and models and appropriate explanatory materials (e.g., information sources, survey and analytical methods, accuracy, and reliability) to accompany disseminated information. Standards also require freezing or archiving the input and output data used in disseminated information, as well as archiving the version of any model used. Estimates and projections in EIA's information products are directly reproducible by agency officials and contractors. Many underlying data sets contain proprietary or sensitive information and are treated as confidential; hence they could not be available to, or be reproduced by, the public. Analytical results may not be easily reproducible by third parties due to the complexity and detail of the methods and data. EIA places great emphasis on its review process to ensure the quality of information disseminated. EIA will ensure reproducibility for all influential information using commonly accepted scientific, financial, and statistical standards. EIA will apply rigorous robustness checks to all analytical results before information is disseminated.
EIA also achieves transparency through wide dissemination of its information by use of the EIA Web site primarily, but also by printing some products. Releases of information products are announced on the EIA Web site and most electronic versions can be accessed and downloaded directly from the site.
5. EIA Information Subject to Information Quality Guidelines
EIA's intention is that all applicable information quality guidelines shall apply to information prepared by EIA for public dissemination and disseminated on or after October 1, 2002, regardless of when the information was first disseminated. An information product disseminated by EIA before October 1, 2002, that is no longer updated or maintained but is still available from EIA (e.g., archived information in EIA's files or in electronic files that EIA continues to distribute on its Web site) is not subject to the guidelines or to the request for correction process.
EIA information not subject to information quality guidelines includes:
- Press releases, including but not limited to fact sheets, press conferences or similar communications in any medium that announce, support the announcement or give public notice of information EIA has disseminated elsewhere
- Any inadvertent or unauthorized disclosure of information intended only for inter-agency and intra-agency communications
- Correspondence with individuals or persons
- Testimony and other submissions to Congress containing information EIA has disseminated elsewhere
- Responses to requests for EIA records under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act, or similar laws
- Information in public filings (such as public comments received by EIA in rulemaking proceedings), except where the EIA distributes information submitted to it by a third party in a manner that suggests that EIA endorses or adopts the information, or indicates in its distribution that it is using or proposing to use the information to formulate or support a regulation, guidance, or other EIA decision or position
- Information contained in subpoenas or documents filed in connection with adjudicative proceedings
- Procedural, operational, policy, and internal manuals and memoranda prepared for the management and operation of EIA not primarily intended for public dissemination
- Archival records (including information made available to the public on EIA's Web site to document historical EIA actions)
- Communications intended to be limited to government employees or DOE contractors or grantees
- Information disseminated by EIA employees, contractors, or grantees that is not put forth as an EIA information product (e.g., materials presented by an individual at a professional meeting)
- Information disseminated by others that is accessible through hyperlinks on EIA's Web site.
Questions about EIA Information and Requests for Correction of Disseminated Information Not in Compliance With Quality Guidelines
Questions — EIA is fully committed to working with both information providers and users. If you have questions about EIA, its surveys, its models, or its information products, please contact EIA's Office of Communications (OC) or an EIA staff person who is an expert on a specific energy topic.
Office of Communications — Provides statistical and analytical energy data, information, and referral assistance to the government and private sectors, academia, and the public.
Office of Communications (OC)
1000 Independence Avenue, SW
EI-40, Room 2G-090, Forrestal Building
Washington, DC 20585
E-Mail: infoctr@eia.doe.gov
EIA Energy Experts — EIA staff members who are experts on specific energy topics (e.g., coal, electric power, natural gas, petroleum, nuclear, renewable energy, or energy consumption).
NOTE: Respondents to EIA surveys who report using EIA forms should not use the procedure below to correct or revise information submitted on an EIA survey form. For correcting information submitted on an EIA form, please refer to the appropriate form and contact that form's manager using the information at EIA Survey Forms.
Requests for Correction
DOE has established administrative mechanisms allowing persons to seek and obtain, where appropriate, timely correction of information maintained and disseminated by EIA that does not comply with the OMB, DOE, or EIA information quality guidelines. For more information about the correction process or to request correction, visit the DOE Information Correction Request Web Site.
An appeal with respect to EIA information will be forwarded to EIA's Statistics and Methods Group (SMG) for review to determine what action is appropriate. SMG is independent from the EIA program office that disseminated the information and that handled the initial request for correction.