Bulk download facility

The bulk download facility provides the entire contents of each major API data set in a single ZIP file. A small JSON formatted manifest file lists the bulk files and the update date of each file. The manifest is generally updated daily and can be downloaded from http://api.eia.gov/bulk/manifest.txt. The manifest contains information about the bulk files, including all required common core attributes:

Each bulk file is downloaded from accessURL as a .zip file, and contains a single .txt file of the same name. The extracted text file contains all the dataset's series (ordered by most recent update data first) followed by all of the dataset's categories (ordered by hierarchy from top to bottom). Each line of the extracted text file is a complete object in JSON.

Latest bulk download files:

• State Energy Data System (SEDS)
• Electricity
• Natural Gas
• Petroleum
• Coal
• Short-Term Energy Outlook
• Crude Oil Imports
• International Energy Data
• Total Energy
• U.S. Electric System Operating Data
• CO2 Emissions
• International Energy Outlook

Sample series line (carriage returns and spacing added here for readability):

{
  "series_id":"NG.RL2R04SOK_1.A",
  "name":"Oklahoma Natural Gas Plant Liquids, Reserves Revision Decreases, Annual",
  "units":"Million Barrels",
  "f":"A",
  "unitsshort":"MMbbl",
  "description":"Oklahoma Natural Gas Plant Liquids, Reserves Revision Decreases",
  "copyright":"None","source":"EIA, U.S. Energy Information Administration",
  "iso3166":"USA-OK",
  "start":"1979",
  "end":"2008",
  "last_updated":"13-AUG-13 11.49.51 AM",
  "data":[
     ["2008","136"],
     ["2007","73"],
     ... 
     ["1980","69"],
     ["1979","54"]
  ]
}

Sample category line (carriage returns and spacing added here for readability):

{
  "category_id":"456175",
  "parent_category_id":"456174",
  "name":"U.S. Dry Natural Gas Production",
  "notes":"",
  "childseries":[
    "NG.N9070US2.M",
    "NG.N9070US2.A"
  ]
}

The manifest allows a consumer of EIA data to:

1. Quickly check the manifest for updates
2. Only download and process bulk data files with a more recent last_updated field in the manifest than your previous download
3. Associate common core attributes, such as description and keywords, with the data
4. Process the series that are newer than your last download
5. Optionally ingest and replicate the data set's complete category structure