About EIA

EIA Standards

 

Statistical Standards

EIA has adopted the Office of Management and Budget (OMB) Standards and Guidelines for Statistical Surveys. These standards will be supplemented with EIA specific standards which are currently under development.

 

COOP, Records Management, and IT

Model Documentation

Communication

EIA will develop, as needed, standards related to external communication and electronic dissemination on topics including plain language, information accessibility, media relations, and other communication-related issues.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Energy Information Administration Standard 2002-2

Title: Information Technology (IT) Systems

Superseded Version: Standards 88-02-01, 88-03-02, 88-03-03, and 88-03-04

Purpose: To ensure that the EIA IT systems are developed, maintained, and secured using best IT practices and that they adhere to established policies, standards, and procedures.

Applicability: All IT systems at EIA.

Background: EIA has standards for developing and documenting computer systems. These standards ensure that systematic procedures are used when new IT systems are developed and existing systems are revised; that these systems meet business and functional requirements; and that the systems can be operated, modified, and backed up. With the advent of the Internet, the EIA IT standards have been expanded to address data access and security issues arising in the Web environment.

Required Actions: All EIA and contractor staff involved in developing and maintaining IT systems are required to follow applicable Federal Information Processing (FIPS), Department of Energy, and EIA-specific IT system standards, policies, guidelines, and procedures listed below.

Related Information:

1. Federal Information Processing Standards (FIPS)
2. Department of Energy IT Standards
3. 2002-30, Business Process Documentation
4. 2002-31, EIA Continuity of Operations
5. 2002-32, EIA Records Management
6. EIA Information Systems Security Policy
7. EIA Password Generation, Protection, and Use Procedures
8. EIA Operations Security Program Manual
9. EIA Software Systems Life Cycle Standard
10. EIA Web-Based Data Collection Systems Standard
11. EIA Security Architectural Standard For Web-Based Data Collection Systems
12. EIA Desktop PC Standard

Approval Date: September 26, 2002

 

Energy Information Administration Standard 2002-3

Title: Information System Documentation

Superseded Version: Standards 88-03-02 and 88-03-03

Purpose: To ensure that EIA information systems are documented adequately to allow personnel unfamiliar with them to become knowledgeable about them and to operate them, if necessary.

Applicability: All EIA information systems.

Required Actions:

1. If an EIA information system is a modeling system, it must follow the requirements of EIA Model Documentation Standard 2002-26.

2. All other EIA information systems (e.g., survey systems, secondary data systems, and other systems supporting other key EIA business practices) must have documentation of all operations (both automated and manual) necessary to operate, maintain, and update the systems.

  • All new information systems should have up-to-date Users and Developer's Reference Manuals (see Enterprise Architecture Guidance).
  • All information systems created prior to the approval date of this standard should have up-to-date Operations and Programmer's Maintenance Manuals as required by the predecessor
    EIA Standards/Statistics Manual.
  • When documenting an information system that is not a modeling system, the following topics should be covered, if applicable: 1) an overview of integrated manual and automated operations, workflow, interfaces, and personnel requirements; 2) frames development and updating; 3) sampling design and methodology; and 4) a detailed description of each step in collection/processing/dissemination, including distribution of survey materials, establishment of access to on-line reporting options, data collection methodologies, respondent contact handling, non-response procedures, data entry procedures, data editing procedures, error correction/adjustment procedures, estimation methodology, quality comparison procedures, withholding procedures, revision procedures, dissemination, and back-up and archiving procedures. This information may be incorporated into the existing documentation or written as a separate document.

Related Information:

1. 2002-30, Business Process Documentation
2. 2002-31, EIA Continuity of Operations
3. 2002-32, EIA Records Management

Approval Date: September 26, 2002

 

Energy Information Administration Standard 2002-26

Title: Model Documentation

Superseded Version: 96-01-03

Purpose: To ensure that the procedures, equations, and assumptions which define EIA models are publicly available.

Applicability: All models developed and maintained by EIA or its contractors. (In a large model or system, separable components or modules may be considered as individual models for documentation purposes.)

Required Actions:
1. Model documentation should correspond to a specific archived version of the model.

2. The documentation should include the required components listed in Checklists A and B and the optional components as the program office deems appropriate. For items in Checklist B, options are offered for presenting the materials either separately or along with the materials in Checklist A.

3. Documentation should include discussions of the theoretical basis for the model, empirical support for the model, critical assumptions (as judged by the modeler), mathematical specifications, and other information that the program office believes is relevant to potential users.

4. The program office determines whether documentation will be available in one or multiple volumes, the format (hard copy or electronic), and the organization of the materials.

5. In cases where minor documentation revisions are necessary, the program office may produce a new version of the documentation or a supplemental document that describes only the revisions that have taken place in the model.

6. All documentation should undergo a quality assurance review by the program office prior to dissemination. In addition, new or significantly revised model documentation may undergo reviews by other EIA offices and/or independent expert reviewers from outside EIA when appropriate.

7. When a product using model outputs is sent to the Administrator for approval, the responsible Office Director should specify what documentation is available. If up-to-date documentation is not available, the office should provide a schedule for completing the documentation within 90 days of the product's release or should request an exception.

8. The most current model documentation should be available on EIA's Web site (see Recent Energy Model Documentation main page) or from the model manager.

Checklist A. Explanatory Model Documentation Components

Elements are required unless "optional" is specified. Materials need not be presented in the order discussed here.

a) A reference to the appropriate archive package.

b) Model overview: A concise description of the model, its purposes and uses, how it generates forecasts, critical assumptions, and a discussion of any significant departures from accepted theory or practice.

c) Process flow diagram (optional): A flowchart showing the sequencing of the data inputs, calculations (processes), and outputs of the model.

d) Mathematical specifications: The equations of the model, or, for a linear programming model, the objective technology matrix and constraint vector. The relevant equations include those used to transform input data and parameters into model data and parameters, as well as equations that characterize the solutions of algorithms. Refer to the Guidelines for Mathematical Specifications in Model Documentation.

e) Variable and parameter definitions: All variables and parameters used in the documentation and their units of measurement. If this is prepared with the mathematical specifications, then it is not necessary to present these separately. (It is not necessary to include definitions of computer-code variables that have no exact counterpart in the model documentation, such as temporary computer-code variables, or variables used only in debug statements.)

f) Inputs, outputs, and data: All should be defined, and their units and sources specified. It should be made clear which data are used as direct model input and which data are used for estimation of parameter values. If data were transformed or manipulated prior to use, an explanation should be provided. (The material in this section need not be presented as a unit. The information on data used for estimation could well be presented with the discussion of "model estimation procedures." The discussion of data transformations could well be included with the "Mathematical Specifications" section, see item b above.) See Checklist B, item a, for guidelines for presenting values of data used as direct input into the model run. The program office may elect to include the data sources for this direct-input portion of data as descriptive comments within the input file included in the archive package, rather than having it in the explanatory portion of the documentation.

g) Model estimation procedures: The methods and data sources used to estimate parameters and other quantities in the model should be identified. Enough information about the estimation technique should be given to allow an expert to exactly reproduce the estimation results. This includes a precise citation of data sources, the data series used, and an exact description of which portions of the data series are used for each estimation. (This material need not all be presented in the same section of the documentation; e.g., the data sources could be presented with the other items listed in item f above.)

h) Existence and uniqueness of solutions (optional): For iterative or optimization problems, in cases where the existence or uniqueness of a solution has been demonstrated by analytical means, a description may be presented. In other cases, computer runs should be conducted to test whether the solution methodology is dynamically stable. Also, tests from different initial value conditions should be conducted to provide evidence of the uniqueness of solutions.

i) Sensitivity analysis (optional): Tests should be performed to determine whether changes in key model inputs cause key model outputs to respond in a sensible fashion. If available, include the most recent sensitivity analysis in the documentation or reference an information product with the analysis.

Checklist B. Supplementary Model Implementation Documentation Components

For each of these items, options are provided for presenting the material on a computer file, or alternatively, with the other documentation materials.

a) Direct input data:

When data are used as a direct input to the model, the values should be given and clearly labeled, possibly on an accompanying computer file or files (e.g., ASCII, spreadsheet, or database files may be used). Another option is to reference the archive package with the direct input data. When data are presented in a computer file, identify each data series including labels identifying each column to ensure the meaning of each row is clear. (If the input files included in the model archive package are labeled in this manner, then this automatically satisfies the requirement.) In cases where the archived input files are labeled to satisfy this requirement, the program office furthermore has the option of including descriptive comments to identify sources of the data. When this is done, it is not mandatory to include this information in the explanatory portion of the documentation (Checklist A, item f).

b) Parameter estimates from regressions:

In cases where parameters are estimated by a statistical regression or econometric package, the parameter estimates, R2, t-statistics, and other key information which the program office chooses to report, can be presented in one of two ways:

  1. A discussion may be included in the documentation, or
  2. An output listing from a statistical or econometric package may be presented. Enough information should be included to allow a reader to understand which equation in the documentation is being referred to, and how the parameter names in the package output correspond to names used elsewhere in the documentation.

Explanatory lines within the package outputs may be added for explanation, where necessary. In such cases, a distinctive mark, such as ">>" (or whatever a writer prefers to use) should be placed at the beginning of the comment, so that a reader can distinguish the added comments from the output of the package.

c) Correspondence between variable names appearing in the documentation and those in the computer code:

When possible, the variable names in the documentation should match the corresponding variable names used in a model's computer source code. In cases where the name in the code differs from the name in the documentation, provide a cross-reference list. (Computer-code variables which have no exact counterpart in the documentation, such as temporary variables, would not be cross-referenced.) Cross-referencing can be accomplished in one or a combination of two ways:

  1. A cross-reference table may be presented (included either in an electronic file or in the documentation), or
  2. When there is a declaration in the computer code of a variable which is the equivalent of a documentation variable, a comment in the code would indicate the name of the equivalent variable in the documentation, in the distinctive form {vn} where vn represents the variable name in the documentation. The distinctive form makes it easy to locate the name electronically. (If another distinctive form is used, then the documentation should state the form.) Comments could alternatively be placed next to variable names (e.g., a FORTRAN COMMON block) or another location in the code that the program office chooses.

If the program office elects not to update the variable cross-reference table further, then for all subsequent variable declarations where there is an equivalent variable in the documentation, the documentation name should be identified in a code comment. If the program office elects to use the computer-comment approach throughout the entire code for all of the documentation variables, then publication of the cross-reference table is optional.

If the variable cross-reference table is no longer being updated, the documentation should state this and should explain the format for identifying documentation variable names in code comments.

Guidelines for Mathematical Specifications in Model Documentation

The mathematical specification of the model should be an unambiguous formal statement of the modeler's concept of the methodology and structure to represent real world phenomena. Given the description of the model's methodology and structure along with knowledge of the models inputs and transformations, in principle, one should be able to form an understanding of the specific interrelationships represented in the model, the underlying assumptions, and the essential model outputs. In practice, an interested person may investigate the documentation and computer code as well as discuss the model with the manager to more fully understand the intricacies of the model.

This requirement obviously does not mean that the documentation needs to present one equation in the documentation for each equation in the computer code. Often a single mathematical equation in the documentation is the mathematical equivalent of several computer code statements. The more compact representation would be used in the documentation. This means that certain computer "temporary variables" that are used to temporarily store values (in code) need not be discussed at all in the documentation. In some cases, words rather than an explicitly written equation can be used to specify the model, but only if it is clear how the stated idea would be expressed mathematically.

In cases where an optimization problem is stated, or an iterative process is used to find a solution of some mathematical problem, it is important to state precisely what optimization problem is being solved, or to state precisely, in mathematical terms, the solution obtained when the iterative process is complete. Where iterative processes are used, the basic solution methodology should be described. If there are key parameters, such as a damping parameter, that are involved in the iterative process, then these should be described and the relevant equations containing those parameters given. References to published literature should be given, where applicable, but it is not necessary to describe the iterative process used in the code in complete detail. The mathematical specification in the documentation need not deal rigorously with the possible problems of non-convergence or multiple solutions. However, if an iterative procedure in the code has a default condition which allows the computation of the program to continue even if convergence is not attained, then this condition should be stated.

In some circumstances, one equation in the model documentation can appear several times in the computer code when parallel relationships are used in several separate sectors. In such cases, the documentation should clearly describe the several sectors to which this general equation applies, and relevant subscripts or arguments for representing the various sectors should be clearly defined. (It is not necessary to repeat the equation in the documentation for each instance in which it appears in the code, as long as the documentation makes it clear how the specific instances of the equation fit in to the model structure.)

If a linear program problem is of such a size and complexity that it is impractical to include the complete specification, provide a description of the structure of the problem in as much detail as practical. In addition, provide information needed by an expert to access electronically any desired coefficient or constant in the problem, to determine the dimensions of the objective function vector and constraint matrix, identify which constraints involve strict or non-strict inequality, and to obtain all information necessary to generate the results of the optimization problem.

Portions of the mathematical specifications can be included in an appendix or appendices if the program office chooses to do so. The text and the appendices jointly should provide a complete specification of the model. (If a complete mathematical specification is provided in a report, it is not necessary to repeat these same equations in an appendix, unless the program office chooses to do so.)

In the variable cross-references (Checklist B, item c), it is possible that one documentation variable name might correspond to several computer variable names. In some circumstances, the computer code might use two different variables which have the same definition but are in different units. If the preference is to use one variable name to represent both variables, include a special notation or explanation in the variable cross-references in cases where the correspondence is among variables with the same definition but measured in different units.

--------------------------------------------------------------------------------

1Graphic Standard: Web Graphics that Translate Well in Black and White, Herbert Miller, Update and Results of Cognitive Testing of EIA Graphs, paper presented at semi-annual meeting of the American Statistical Association's Advisory Committee on Energy Statistics (Washington, DC, April 19, 2001).

2An example of where users had difficulty occurred when the two y-axes represented the same variable but at different orders of magnitude; e.g., both y-axes were in billions of barrels but of different orders of magnitude; e.g., both y-axes were in billions of barrels but one had a scale of 5, 10, 15 and the other had a scale of .5, 1.0, 1.5. Users did not notice that changes in one variable were 10 times the magnitude of changes in the other variable. Also, percent change is not a good alternative to dual y-axis graphs because users were confused as to whether the graph presented percent change from a base year or from the previous year.

Related Information:
1. 2002-27, Model Archival
2. 2002-28, Proprietary Models
3. Standard 2002-26 Guidelines for Mathematical Specifications in Model Documentation
4. Review, Clearance, and Dissemination of EIA Products

 

Energy Information Administration Standard 2002-27

Title: Model Archival

Superseded Version: 91-01-04

Purpose: To ensure that EIA model calculations are reproducible.

Applicability: All models used by EIA.

Required Actions:
1. A model archive package should be prepared by the program office when model outputs are used in an EIA product that is publicly disseminated.

2. A model archive package should contain:
a. All source code and program control files needed to compile, link, and execute the reference or base case scenario of the model. If alternate source code versions were used to run the model for other scenarios cited in the same product, provide these versions or create a scenario-specific archive.
b. Input files used by the model for the reference or base case cited in the model, along with other file versions that would be needed to run the model for other scenarios cited in the same product.
c. Primary output (as opposed to debugging or trace files) from the reference or base case scenario used in the product. It is not necessary to provide all output files for all scenarios cited in the product, as long as those outputs can be regenerated from runs of the archived model and the primary outputs from the model can be verified against disseminated results.
d. Instructions for compiling and running the model and comparing the results to disseminated results. A description of changes needed to run the alternative scenarios published in the report or to create scenario-specific archives should be included.
e. The source of the proprietary data and software should be included in the archive instructions. However, an archive package should exclude proprietary data and software used in the model.

3. The program office should create and verify an archive package within sixty days of disseminating an EIA information product utilizing model outputs.

4. The archive package must be retained until no longer needed for current business. Consult with the EIA Office of Resource and Technology Management (ORTM) regarding records retention requirements.

5. (Optional) Develop a policy for public dissemination of the archive that addresses such topics as:
a. The model's transportability.
b. Additional software required (such as proprietary embedded models) that is not provided by EIA with the model.
c. EIA's expectations for how a public user will identify the model if the model has been modified by a user from outside EIA.
d. Limits on EIA support.
e. Any other issues pertinent to outside use.

Related Information:
1. 2002-26, Model Documentation
2. 2002-28, Proprietary Models

Approval Date: September 26, 2002

 

Energy Information Administration Standard 2002-28

Title: Proprietary Models

Superseded Version: 91-01-05

Purpose: To permit the use of proprietary models in EIA modeling systems and in conjunction with EIA products.

Applicability: To all models available to EIA through license, purchase, or subscription.

Required Actions:
1. Every agreement for the acquisition or use of a model should provide for:
a. Publicly available documentation of the model's design, theoretical basis, empirical implementation, and objective capabilities.
b. A means for EIA to replicate model calculations for a period of three years after each application in an EIA product.

2. For an active EIA model:
a. The model documentation should be available to the public.
b. The model version and archive of all model inputs and outputs associated with a disseminated information product must be identified so the results can be replicated.
c. All changes EIA makes to the model should be documented and archived to EIA standards.
d. A proprietary model should not be embedded in an EIA modeling system unless the model is commercially available.

Related Information:
1. 2002-26, Model Documentation
2. 2002-27, Model Archival

 

Energy Information Administration Standard 2002-30

Title: Business Process Documentation

Superseded Version: None

Purpose: To ensure that EIA's essential business processes are adequately documented so they can be operated efficiently and effectively by anyone with the necessary skills, information input and technical resources.

Overview: Documentation requirements for specific portions of EIA software systems, surveys and models are delineated in several existing EIA Standards. This Standard covers the documentation requirements for the remaining portions of these processes and for EIA's other business processes not covered by existing standards (e.g., administrative, financial, customer service). The documentation will consist primarily of manuals that provide instructions and reference materials to individuals learning how to perform the function. These manuals would be used not only by staff newly assigned to the function, but also by veteran employees who must perform the function when the usual performers are unavailable. This might occur when the usual performer is absent from work, retires, or is unavailable during post-disaster recovery operations.

Applicability: All business processes that are essential to EIA's Continuity of Operations Plan. There are no waivers to this Standard.

Required Actions:
This Standard requires the following:

  • Creation and maintenance of Process Performance Manuals (may also be called User Manuals, User Guides, Operations Manuals, Process Manuals, Instruction Manuals, etc.) for all business processes designated as essential.

  • A Process Performance Manual (or its equivalent) will contain the following information:
    1. A narrative description of the process, its purpose, operating environment, components, information inputs (and sources) and information outputs (and destinations);
    2. An overview chart showing the connectivities of the process components among themselves and with outside entities;
    3. A set of instructions suitable for an untrained but skilled individual that explains how each process component is performed;
    4. An optional troubleshooting guide and set of frequently-asked-questions that provide help to those encountering difficulty in performing the process; and
    5. Names of individuals to contact who can provide assistance in the performance of the process.

Related Information:
1. EIA Data Systems Standards, 2002-2, 2002-3
2. EIA Models Standards, 2002-26, 2002-27, and 2002-28
3. EIA Software System Life Cycle Standard

Approval Date: September 26, 2002

 

Energy Information Administration Standard 2002-31

Title: EIA Continuity of Operations

Superseded Version: None

Purpose: To establish the minimum requirements for planning for continuing operations of the Energy Information Administration in case of emergency or catastrophe.

Overview:
The EIA Continuity of Operations Plan designates essential functions, provides for the emergency delegation of authority to carry out tasks normally performed by the Administrator, assigns management responsibility for operations under various contingencies, outlines alert and notification procedures and a communications plan, and categorizes vital records necessary to maintain operations or reconstitute agency programs. Lower level organizational planning must be consistent with the requirements of the EIA COOP Plan.

Applicability: All Offices in EIA.

Applicable Laws and Regulations:

Required Actions:
Each Office will create an implementation plan. These plans shall include, at a minimum, the following basic elements:

  • Standing delegations of authority to act for management positions in the organization in the absence of managers.
  • Designation of an assembly point for staff in the case of a required building evacuation, and the maintenance of a staff roster to enable the rapid identification of any missing staff.
  • A method for the alert and notification of staff of the existence, nature, and required response actions appropriate for the emergency situation.
  • An inventory of vital records, updated on a regular basis, to insure their availability as a part of the EIA vital records program.
  • A description of the processes the Office will implement to accomplish its mission objectives in various emergency scenarios.

The Office of Planning, Budget, Procurement, and Evaluation will create a disaster recovery plan to reconstitute EIA's IT infrastructure in the event of an emergency or disaster, in support of EIA's essential functions.

The EIA Office of Resource and Technology Management (ORTM) will create a plan to provide necessary administrative support (personnel, procurement, and finance) for the level of EIA operations conducted under emergency conditions.

These plans will be tested regularly and updated as necessary to insure their operability should an emergency situation arise.

Related Information: See applicable laws and regulations above.

Approval Date: September 26, 2002

 

Energy Information Administration Standard 2002-32

Title: EIA Records Management

Superseded Version: None

Purpose: To ensure that procedures and systems are implemented and maintained to assure efficient and effective records management.

Overview:
As the basic administrative tool by which the Government does its work, records are a component of each agency's information resources. Like other resources, records must be managed properly for the agency to function effectively and comply with Federal regulations. This document establishes required standards for activities with respect to records creation, maintenance and use, and disposition, which are essential to sound management of EIA business lines.

Applicable Laws, Regulations, Policies, And Procedures:

Applicability: This standard applies organization-wide to both program and administrative records. There are no waivers to this Standard.

Required Actions:
This standard requires the following actions:
1. EIA will establish a planned coordinated set of policies, procedures, and activities to manage the organization's records.

2. EIA will designate a Records Management Officer, John Paul Deley to provide overall policy guidance in records management on an organization-wide basis.

3. Each EIA office will designate an individual or individuals to act as Records Management Contacts and forward the routing symbol and telephone number of the designees to the EIA Office of Communications.

4. Each EIA office will create and maintain a records disposition program designed to provide for (1) the removal from valuable office space all records and nonrecord materials which are no longer essential for current operations and (2) the protection of records that should be kept for long periods.

5. EIA will develop a policies and procedures manual governing the maintenance and disposition of all EIA records, including electronic media.

6. The EIA Records Liaison Officer will coordinate the retirement and retrieval of all EIA records to or from Federal Archives and Records Centers.

7. When planning and implementing all technology applications, EIA offices will include records management requirements in order to maximize the organization's ability to document its activities.

8. EIA offices will develop a standardized filing system or structure, which provides a records management mechanism that facilitates ease of records use, access, and disposition.

Related Information:
EIA Software System Life Cycle Standard.

Approval Date: September 26, 2002