0389
 1    AMERICAN STATISTICAL ASSOCIATION
 2     COMMITTEE ON ENERGY STATISTICS
 3                WITH THE
 4   ENERGY INFORMATION ADMINISTRATION
 5   
 6   
 7   
 8   
 9   
10   
11   
12   
13   
14   
15   
16         Friday, April 7, 2006
17   
18            Washington, D.C.
19   
20   
21   
22   
0390
 1                     C O N T E N T S
 2   Open Meeting, Nicholas Hengartner,          391
     ASA Chair
 3   
     Preliminary Research Results on Respondent  394
 4   Cut-off Dates for EIA Electricity Data
     Collection, Howard Bradsher-Frederick and
 5   Alethea Jennings, SMG, EIA
 6   ASA Discussant on An Empirical Evaluation   454
     of the Relationship Between Crude Oil and
 7   Natural Gas Prices, Cutler Cleveland
 8   ASA Discussant on Preliminary Research      462
     Results on Respondent Cut-off Dates,
 9   Walter Hill
10   EIA-914 Data Expansion Challenges to        470
     Include Crude Oil Production, John Wood,
11   OOG, EIA
12   ASA Discussant, Mark Burton                 501
13   Invitation for Public Comments              532
14   Committee Suggestions for topics at the     536
     fall 2006 meeting
15   
     Adjournment of the ASA spring 2006 meeting  543
16   with EIA, Nicholas Hengartner, ASA
     Committee Chair
17   
18   
19                      *  *  *  *  *
20   
21   
22   
0391
 1                  P R O C E E D I N G S
 2                                            (8:33 a.m.)
 3             THE CHAIR:  Good morning, ladies
 4   and gentlemen, welcome back to the second
 5   half of the ASA meeting on energy statistics.
 6   Again I remind you this is an ASA meeting,
 7   not an EIA meeting.  Nancy Kirkendall remains
 8   the official federal, what is it -- I'm
 9   looking for the name.
10             MS. KIRKENDALL:  Ambassador.
11             THE CHAIR:  Ambassador -- no, it's
12   officer, and she of course has a --
13             MS. KIRKENDALL:  Designated federal
14   official, but I think that's different now,
15   because it used to be that the committee was
16   an advisory committee under FACA -- don't ask
17   me what that stands for -- and it had all
18   sorts of regulations, and now you're not
19   under FACA anymore and so we can do what we
20   want.
21             THE CHAIR:  Okay.
22             MS. KIRKENDALL:  And the big reason
0392
 1   for that is that you don't give us unanimous
 2   consensus advice, instead it's a
 3   brainstorming thing, only ------.  So the
 4   advisory committee is frequently -- a true
 5   advisory committee under that law would have
 6   to deliberate and give consensus advice.
 7   This is much better for all of us, I think.
 8             MS. FORSYTH:  Plus we can do what
 9   we want.
10             MS. KIRKENDALL:  Plus we can do
11   what we want.
12             THE CHAIR:  And we don't have to
13   agree.
14             MS. FORSYTH:  And you don't have to
15   agree and Bill doesn't have to do this
16   paperwork drill every however many years.
17             THE CHAIR:  Okay, just in terms of
18   mechanics for the new members, the way it
19   works is that you send in the receipts for
20   dinner and so forth -- hotel, travel,
21   whatever you have -- to the ASA office;
22   Kathleen Wert, who is at the entrance desk,
0393
 1   has all the paperwork and has all the
 2   information for you on how this goes and how
 3   to do that, and so if you want to hook up
 4   with her and ask her questions, that would be
 5   a good thing.  So that's on the technical
 6   side.
 7             On the more interesting side for
 8   today let me -- we're starting out with
 9   break-out sessions.  So I will invite Mark
10   Burton, Cutler Cleveland, Jae Edmonds, Neha
11   Khanna, Nagaraj Neerchal, Tom Rutherford, and
12   Susan to go down to the fifth floor to 5E-069
13   to discuss the paper on empirical evaluation
14   of relationship between crude oil and natural
15   gas prices.  The remaining committee members
16   should remain up here.
17             MR. CLEVELAND:  You'd think we'd
18   get it right by now, huh?
19             SPEAKER:  Well, actually, they've
20   got it right all the other times in the past.
21                  (Recess)
22             THE CHAIR:  Okay, good morning,
0394
 1   everybody to the break-out session on
 2   preliminary research results on respondent
 3   cut-off dates for EIA, Electricity Data
 4   Collection, and we have the pleasure of
 5   having a presentation of Howard
 6   Bradsher-Fredrick and Alethea Jennings this
 7   morning from the Statistics and methods
 8   group.  Welcome Howard.
 9             MR. BRADSHER-FREDRICK:  Okay, we're
10   ready to go?
11             THE CHAIR:  The floor is yours.
12             MR. BRADSHER-FREDRICK:  Okay.  Can
13   everybody hear me okay?
14             THE CHAIR:  Everybody else --
15             MR. BRADSHER-FREDRICK:  Most
16   importantly the transcriber.
17             THE CHAIR:  Court Reporter.
18             MR. BRADSHER-FREDRICK:  Okay.  Let
19   me know if I get too far away.  Okay, I am
20   Howard Bradsher-Fredrick, and today Alethea
21   Jennings and I are going to talk about our
22   preliminary research work on respondent
0395
 1   cut-off dates for EIA electricity data
 2   collections.  And I like to emphasize
 3   preliminary because unlike a lot of my
 4   presentations, this really is preliminary.
 5   There's also a disclaimer here essentially
 6   saying that this is a work in progress.
 7             Okay, here's the outline of the
 8   presentation that we're going to give.
 9   First, I'll provide a short introduction.
10   The analysis focuses on two annual
11   electricity surveys, the EIA-860, the annual
12   electric generator report, and the EIA-861,
13   the annual electric power industry report.
14   Then I'll discuss the overall goal and the
15   short term objectives.  And next I'll discuss
16   the data submission profile by month of
17   submission.  First, we'll talk about the
18   overall and we also have data stratified by
19   fuel type.
20             Then Alethea will take over and
21   she'll discuss her analysis on the EIA-861,
22   which involves a data submission profile
0396
 1   stratified by end use sector.  Then she'll
 2   conclude the discussion of our work and pose
 3   questions to the Committee.
 4             Okay, the EIA-860 is an annual
 5   census of all existing and planned electric
 6   generating facilities with a nameplate
 7   capacity greater than one megawatt.  The
 8   survey collects information at the generator
 9   level on plant design and capabilities, but
10   not on actual operations.  It should also be
11   mentioned that the 860 frame serves as the
12   frame for several other monthly electricity
13   surveys and one annual survey.  Thus it also
14   serves as a mechanism for updating the master
15   frame.
16             Significant resources are spent on
17   non-response follow-up on this survey.  The
18   due date is April 1st, but follow-up
19   continued in 2005 until September 9th.
20             We thought that the first item to
21   be investigated was the issue of when are the
22   forms actually submitted and how does this
0397
 1   submission profile appear to in terms of each
 2   fuel type.  While this work is in its early
 3   stages, we also began thinking about what
 4   related investigations should be undertaken.
 5             Okay, the overall goal:  Thus our
 6   initial overall goal is to quantitatively
 7   determine the impact on the data submission
 8   profile of various alternative policies,
 9   cutting off non-response follow-up earlier
10   than is presently done.
11             Okay, the blue came out nicely
12   here.
13             As these are our short terms
14   objectives, the first two objectives in blue,
15   these are the objectives we've done some work
16   and we're now presenting.  These are to
17   determine the effect on coverage overall and
18   stratified by fuel, if EIA decided to cut off
19   the survey response stream on the EIA-860 on
20   a variety of different dates.  And two, to
21   determine the effect on coverage of large
22   producers and consumers on the EIA-861 by
0398
 1   using a variety of cut-off dates.
 2             And the third objective is to
 3   determine whether or not earlier respondents
 4   submitting data require less follow-up than
 5   later respondents.  This issue of the quality
 6   of data of earlier versus later respondents
 7   was one I began to try to investigate, but
 8   ran into some difficulties, so I don't have
 9   any results yet.  Something that could be
10   followed up on if we chose to do so.
11             Okay, here's the first table.  This
12   table shows the submission profile of all
13   fuels.  This is operating status only.  These
14   are the 2004 data as submitted in 2005.  The
15   due date on the form is specified as April 1,
16   2005.  Let's have a little bit of a look at
17   this table.  These are essentially the
18   generator count that came in before May 1st,
19   and it's somewhat over 8,000, and this
20   represents 57 percent in terms of the
21   generators.  In terms of the volume as
22   measured by nameplate capacity we have 64.7
0399
 1   percent by that time, and the average
 2   capacity of those generators was 77.7
 3   megawatts.
 4             Now we get into May, and you can
 5   see we're getting over 3,000 during that
 6   month, which represents 21 percent.  In terms
 7   of the cumulative total we have 78.1 percent.
 8   In terms of the volume, we're up to 85.6
 9   percent at the point of May 31st.
10             In June, we're starting to see
11   again some drop-off here.  We have 1,700 in
12   terms of the count, which in cumulative
13   represents 90 percent.  In terms of the
14   volume we're up to 95.1 percent.  We started
15   seeing further drop-off as we go along until
16   we get down to September 9th, and we have at
17   that point 98.3 percent of the cumulative
18   generators responding so to speak, and we
19   have 99.8 percent of the capacity, and we
20   have non-respondents here.
21             So then we started stratifying by
22   fuel to see how our profile looked.  The
0400
 1   first one here is the non-hydro renewables.
 2   This was thought to be an area where EIA
 3   would want to be sure to have sufficient
 4   coverage due to the usual considerable
 5   interest in non-hydro renewables, so here I
 6   can give you a chance to have a look at this
 7   table.
 8             By May 1st, we have 52.5 percent of
 9   the generators reporting, and we have 60.2
10   percent of the capacity, and during May, we
11   get another 246 so we're getting 18.1
12   percent, so we have 70 percent at this point,
13   73 percent of the volume, and by the time we
14   get to June 30th, we're up to 90.3 percent of
15   the generators and 91.0 percent of the
16   capacity.  Now, we start seeing a fairly
17   steep drop-off at this point.  We'll go on to
18   the next slide.
19             The natural gas is the input fuel,
20   and here I can give you another chance to
21   look a bit at the table we see before me.
22   First, 63 percent of the count and 65.4
0401
 1   percent of the capacity, and we start seeing
 2   a drop-off here, but it may not be quite as
 3   significant as we were seeing in -- well, I
 4   was going to say maybe it isn't that
 5   dissimilar than some of the other ones.  You
 6   can use your own judgment on that.
 7             So by June 30th here, we have 93
 8   percent of the generators reporting and 96
 9   percent of the capacity covered, and by the
10   end we have 99.0 percent of the count and
11   99.7 percent of the volume.
12             And the last one for the 860 is
13   coal, and you'll notice here -- you can look
14   at the numbers -- that it seems as though we
15   tend to get the coal-fired plants in fairly
16   early in comparison to the others.  We have
17   69.7 percent in by May 1st and 74 percent of
18   the volume, and by June 30th here we have
19   96.8 percent of the count and 97.8 percent of
20   the volume.
21             Okay, I'll turn it over to Alethea
22   to talk about the 861.
0402
 1             MS. JENNINGS:  Okay.  Howard has
 2   talked to you a little bit about the 860.
 3   Now, we're going to talk a little bit about
 4   the 861.  The 861 survey is the annual
 5   electric power industry report, and that
 6   survey collects information from electric
 7   power industry participants that are involved
 8   in the generation, the transmission, and the
 9   distribution of electricity.  The 861 also
10   serves as the frame for a monthly survey,
11   which is a cut-off sample from the 861 and
12   that is the 826.  So this is also a very
13   important frame as well as the 860.
14             The data that are referenced here
15   are for the year 2004, the same as the data
16   for the 860 that you just saw.  The data
17   submissions, however, were received by EIA
18   throughout the entire year of 2005, from the
19   month of January through the month of
20   December.
21             Now, the tables that Howard showed
22   you display the 860 data submission profiles
0403
 1   stratified by month of submission and by fuel
 2   type.  We're going to look at the 861 data
 3   submission profiles stratified by month of
 4   submission and by end use sector.  For the
 5   remaining tables that you're going to see,
 6   you'll notice that the first four columns of
 7   data are going to be exactly the same, as
 8   they relate to respondent counts and
 9   percentages, both cumulative and by
10   submission date.
11             Now, the total of the second column
12   indicates that the survey currently has about
13   3,300 active units.  So you'll notice as we
14   look at these tables that the reporting
15   pattern is actually quite similar to that of
16   the 860.
17             This particular table shows total
18   electricity sales data, and that's for all of
19   the sectors combined.  Column 7 shows us the
20   cumulative percentage of total sales volume.
21   Actually now, that's right here, 89 percent.
22   So you'll notice that by May 31st you have 89
0404
 1   percent of your total sales volume, and then
 2   by our fictitious date of June 31st, we have
 3   97 percent of the total sales volume.
 4             The remaining tables that you're
 5   going to see are going to show you the data
 6   for the individual sectors.
 7             THE CHAIR:  Before you go forward,
 8   you've blacked out -- or blued out a few
 9   boxes.  Non-responses?
10             MS. JENNINGS:  Exactly, and that is
11   because the non-respondent totals just
12   pertain to the count of respondents that we
13   have that we did not receive data for, so we
14   wouldn't have sales and percents of sales and
15   cumulative percentages available for them.
16   So that's just a respondent count number and
17   the percentage of the total respondents at
18   the bottom.
19             DR. FEDER:  Do you have historical
20   data for how big they are?
21             MS. JENNINGS:  We don't show it on
22   this table; we don't have that available on
0405
 1   this table, so we didn't exactly pull that,
 2   but that is something that we could
 3   definitely look at.
 4             Okay, let's look at the residential
 5   sector.  Again, by the end of May
 6   approximately 88 percent of the total sales
 7   volume had already been reported and again by
 8   the end of June, approximately 97 percent of
 9   the total sales volume had been reported.
10             Let's move on to the commercial
11   sector.  Same pattern here.  By the end of
12   May you have 89 percent, by the end of June
13   you have 98 percent with very little coming
14   in after the end of June.
15             And then finally, for the
16   industrial sector, approximately 89 percent
17   of the total sales volume was reported by May
18   31st, and again 97 percent by the end of
19   June.  So within each sector about 97 percent
20   of the total volume had been reported prior
21   to July 1st.
22             Now, the data submission files for
0406
 1   the 860 showed us a similar pattern with an
 2   overall 95 percent of the cumulative volume
 3   of generation being reported prior to July
 4   1st.
 5             I want to mention that in thinking
 6   about other related areas of investigation,
 7   we do recognize that it's possible to analyze
 8   the movement of utilities from being ranked
 9   as one of the largest providers of
10   electricity and then becoming a lower-ranked
11   provider.  We can certainly investigate this
12   if we determine that it's a worthwhile
13   effort.  So with that we have a couple of
14   questions for the committee.  We want to ask
15   you if you think it would make sense for us
16   to utilize a cut-off date, for example June
17   30th, to limit the number of non-response
18   follow-up, reminding you also that this would
19   enable us to have a more timely data
20   dissemination and also we want to remind you
21   that the bias will potentially be a problem
22   and it could affect other surveys, because,
0407
 1   as we mentioned earlier, these surveys are
 2   the basis for frames of other surveys as
 3   well.
 4             And finally, we want to know what
 5   other background analysis would you recommend
 6   in order to further illuminate the pertinent
 7   issues that we've raised here?
 8             THE CHAIR:  Moshe?
 9             DR. FEDER:  I'm going to speak as a
10   service statistician; I don't have the
11   expertise in energy statistics as such.  My
12   only concern is if one wanted to do analysis
13   of small plants, which are maybe represented
14   more highly -- there could be some reasons
15   why they are late.  Maybe it's because there
16   accounting system is different.  That would
17   pose some problem.  But timeliness is also
18   very important.
19             So what I'd like to say is that you
20   need to -- I would ask the stakeholders, what
21   do you think it is that you want?  You want
22   us to release maybe some preliminary by June
0408
 1   30th or -- I saw you actually had quite a bit
 2   of response in July and August, I don't know,
 3   maybe the end of August.
 4             Or another approach that could be
 5   taken is take this cut-off, release estimates
 6   and then do a special study of -- if they
 7   wanted to respond later and release it as a
 8   special report, but I don't know what really
 9   is the research interest here, and to me any
10   statistical question must be driven by a
11   research question, because that's why we do
12   it, and those data users will tell you,
13   that's why I wish Neha or Mark were here,
14   because they probably will tell you what they
15   need.
16             But if they are looking at the
17   overall sales and so on, then it doesn't
18   really matter, so --
19             MS. JENNINGS:  One of our greatest
20   concerns, and I think Howard mentioned this
21   earlier, was the amount of resources that get
22   tied up in follow-up of non-response, and
0409
 1   that's one of the reasons that we're digging
 2   into this, because there's a lot of time and
 3   effort by analysts following up on
 4   non-response issues.  And we don't want to
 5   disregard those respondents that did not send
 6   their submissions in on a timely basis.  This
 7   is mandatory.  So that's another reason why
 8   we don't want to just say we ignore it, but
 9   then how much of an effort should we really
10   be putting into the follow up non-response?
11   That's one of the basic questions that we
12   have.
13             DR. FEDER:  All I wanted to say is
14   that it could be the result they are late is
15   because of the accounting system, and that
16   will be costly just to wait, so is it because
17   they are -- like some people file the income
18   tax on April 15th or how they like, others
19   who wait because the numbers they're looking
20   at.  So I think it will pay to ask those late
21   responders, while I don't know if you can, is
22   there a reason or would you like to get more
0410
 1   time next year, let us know what is the
 2   reason you are not reporting on time.
 3             MS. JENNINGS:  Okay.  So
 4   investigate the reasons for the non-response.
 5             MS. FORSYTH:  With a debriefing.
 6             DR. FEDER:  Debriefing, yeah.  We
 7   do that in all surveys.  We have a debriefing
 8   but we usually debrief the interviewers, you
 9   don't have that, so I don't know what --
10             MS. FORSYTH:  But you could debrief
11   with a follow-up.  You're calling them,
12   right?
13             MS. JENNINGS:  They're being
14   called.
15             MS. FORSYTH:  And so as part of
16   their non-response, when they finally respond
17   you could have a telephone conversation with
18   them about, okay, it took a while this time,
19   what was up with that?
20                  (Laughter)
21             MS. JENNINGS:  Or at least dedicate
22   some resources to that for a period of time,
0411
 1   that's what you're saying.
 2             MS. FORSYTH:  Right, because if it
 3   is just that accounting, then like, what she
 4   said, we don't need to be spending a lot
 5   of --
 6             MS. JENNINGS:  Right.
 7             DR. FEDER:  Domestic electricity
 8   sales, you have to do a meter reading to have
 9   that.  Every utility company has different
10   procedures.  For instance mine now doesn't
11   even have to come to the house; they drive by
12   and it picks up the signal and they have it
13   almost on real time.  Other companies might
14   need to get access, so maybe they say we are
15   not going to give you this number that early,
16   because we don't have it.
17             MS. JENNINGS:  Okay.
18             THE CHAIR:  Okay, the way I see
19   this, there are actually two questions here.
20   Or I am going to make it into two questions.
21             MS. JENNINGS:  That's fine.
22             THE CHAIR:  The first question is
0412
 1   what about the frames, because you are using
 2   the response for this survey as input to
 3   other surveys, and lo and behold, if I'm
 4   going to make frames and someone else is
 5   going to use them, I want them to be as
 6   complete as possible.  So, on the other hand,
 7   if it trickles in slowly I'm still going to
 8   get the frame ultimately, and it's not so bad
 9   for reporting the energy numbers.
10             So that comes to -- my second part
11   is somehow if we are going to think of a
12   cut-off, like, suppose we take even a June
13   1st cut-off, just making up numbers.  June
14   1st cut-off will have actual non-response
15   that if I dig hard enough will come in.
16             There's nothing that prohibits me
17   to sample from these non-respondents and
18   aggressively get those numbers and then
19   actually be able to adjust the non-response
20   total based on that sample, right?  And in
21   fact the big problem with having those
22   non-responses is you don't really know what
0413
 1   the guys who haven't answered, what their
 2   numbers would be, but if you go with samples
 3   then I could have just put half of my energy
 4   that I'm putting in now because I want
 5   everybody inputted on that smaller sample
 6   with the cutoff, so it's like a certainty
 7   sample plus a real random sample, then you
 8   may be able to reduce the amount of energy
 9   you expend, and at the same time get timely
10   numbers.  But that will not address the frame
11   issue, you see?
12             MS. JENNINGS:  You're right.
13             THE CHAIR:  And so that's where I
14   see two really distinct issues, and I don't
15   know what to advise at this stage, except to
16   say, well, you go aggressively on some of
17   them to be able to get the energy numbers
18   out, and hopefully you just hope that Moshe
19   is right and it's a reporting or accounting
20   problem, and that the other ones will
21   eventually trickle in as well, but for the
22   purpose of your reporting energy statistics,
0414
 1   if you get an early release that is adequate
 2   for the users' perspective, at least
 3   that's --
 4             MS. JENNINGS:  At least for that
 5   sample.
 6             THE CHAIR:  For that sample,
 7   exactly.
 8             MS. JENNINGS:  Okay.
 9             THE CHAIR:  Now, you're the experts
10   on this, so I'm just opening my big mouth,
11   and I realize I don't know much about this.
12             DR. FEDER:  Who's the expert?
13             MS. JENNINGS:  Right.  Must be
14   them.
15                  (Laughter)
16             THE CHAIR:  Over there.
17             DR. FEDER:  Now, I think it always
18   has to go back to what users need.
19             MS. JENNINGS:  Good point.
20             THE CHAIR:  But here one of the
21   users is the EIA.
22             DR. FEDER:  Then those users need
0415
 1   to say.
 2             MS. FORSYTH:  And with regard to
 3   the --- point you've made, I mean, that
 4   wasn't quite what I was going to make, but an
 5   additional point is that you're looking at
 6   weighted --
 7             MS. JENNINGS:  Total volume.
 8             MS. FORSYTH:  Volume, right.  And
 9   the weighted volume is probably what you need
10   for your purposes, but for frame purposes
11   there may be other weights that are important
12   and so that you've got 96 percent of your
13   sales volume doesn't necessarily mean that
14   the frame is 96 percent for other users'
15   purposes.
16             MS. JENNINGS:  Right.  And again,
17   as Moshe pointed out while I was talking
18   earlier that we do want to take into
19   consideration the sizes of those entities,
20   where the volume actually lies.  So I
21   definitely follow you.
22             MS. FORSYTH:  But it's sales
0416
 1   volume, as opposed to -- there may be some
 2   other volume that's relevant, too.
 3             MS. JENNINGS:  Uh-huh.
 4             THE CHAIR:  Walter, and then you
 5   over there.
 6             MR. HILL:  I have the paper ---
 7   that was a -- I have the paper; it was a
 8   ----- to find problem, and I thought it was
 9   well handled.  In the paper there were
10   comments about the size of the plants, which,
11   maybe you alluded to briefly, and maybe
12   that's -- there may be again two problems.
13   One of them is whether or not the response
14   rate is good for the large plants and another
15   one may be whether or not it's good for the
16   small plants.  You could certainly do it,
17   although it's not appears -- you might
18   actually have the data.  In the tables it
19   looks like the large plants are actually
20   responding at a very high rate and the small
21   plants are not.  So what you might be
22   missing -- because you've got average
0417
 1   capacity that was ----- to the table.  But it
 2   looks like you're getting the large ----.  So
 3   there might be a substantive bias or
 4   difference in the types of --------
 5             MS. JENNINGS:  And we can look at
 6   that more closely.
 7             MR. HILL:  You might even be
 8   --------- 100 percent, 99 percent of the
 9   large plants in that first wave, first or
10   second wave, and you're missing the smaller
11   ones as you go on, so there's a question
12   about whether or not the large plants --
13   what's more important to you, those large
14   plants or the small plants?
15             MS. JENNINGS:  What's more
16   important.
17             MR. BRADSHER-FREDRICK:  Well, I
18   think it caused some consternation with some
19   people with the non-hydro renewable that it
20   looked like there were some of -- I mean,
21   unfortunately, you got into August.  The
22   number went up a bit from the average to 16.8
0418
 1   megawatts is the average capacity.  Of course
 2   those two in September were big ones, but
 3   other than that I think that what you're
 4   saying sort of bears out that it was just --
 5   some people pointed out those two as a bit of
 6   an anomaly that we needed to be aware of.
 7             I didn't mean to interrupt, but
 8   just to add some information that you may not
 9   have noticed.
10             THE CHAIR:  Yes, sir?
11             MR. SITZER:  Scott Sitzer, EIA.  It
12   kind of seems hard for me to think that there
13   are accounting reasons that these companies
14   would not have their year-end data by a
15   reasonable time during the first half of the
16   year.  I'd be surprised if they even know
17   what their 2005 sales were by the middle of
18   2006, for example.
19             On the 861, I believe that the due
20   date that we have on the survey is February
21   the 15th, but that's not when we begin to do
22   intense non-response.  We don't open the IDC
0419
 1   data collection system until early January.
 2   So we're really giving them at least 60, and
 3   probably more like 90 days, before we start
 4   making intensive non-response efforts.
 5             We do have a cut-off.  It may not
 6   be June 30th, but definitely there is a time
 7   where we cut off the process because we have
 8   to publish our electric power annual in the
 9   fall.  In order to do that we've got to cut
10   this process off.  So whether it's June 30th
11   or July 30th, there is a cut off date at some
12   point.
13             I think the issue to me is the
14   mandatory nature of the survey and if you're
15   going to say these people don't have to do it
16   because they missed the cut-off date, it's
17   probably not a big deal for them because
18   they're small and they don't effect the
19   numbers that much.  But if word gets out, and
20   I'm afraid it does, then the issue becomes,
21   well, who else is now going to be a
22   non-responder.  So I think we have to
0420
 1   continue to make those efforts, even past the
 2   cut-off date if necessary, but we do have
 3   limited resources so there's a trade-off
 4   there that you have to take in to account.
 5             THE CHAIR:  But you see, if that's
 6   an issue, then you do like the IRS, you see.
 7   We're going to come aggressively against you
 8   and get those numbers, but you can't afford
 9   to get everybody right, so you're going to
10   take a sample.  That's what the IRS is doing.
11   And it kind of tries to enforce the mandatory
12   aspects of taxation.  At least that's what we
13   hope.  So it's not saying that, for example,
14   looking at the sample, having the cut-off and
15   then just saying we're going to select a
16   subset and follow through this year on these
17   guys, that that would say the other ones go
18   off scot free.
19             I hear what you're saying and I
20   think it's a very valid point.  I don't think
21   we are advocating this.  Yes, Barbara?
22             MS. FORSYTH:  I have a question.
0421
 1   Is it the same people, the same clients every
 2   year, or the same entities every year that
 3   are late?
 4             MS. JENNINGS:  Actually, we had
 5   that question.  That's part of what we can
 6   investigate.  We did not investigate on
 7   either survey to that degree.  That will take
 8   a little more work on our part, but that
 9   question occurred to us as well:  Were they
10   repeat non-responders.
11             MS. FORSYTH:  Probably different
12   strategies if they are or if they aren't; I'm
13   not real sure if ------
14             MS. JENNINGS:  Someone actually
15   from the ---- office might be able to answer
16   it.
17             MR. SITZER:  We have the project
18   manager for the survey right here, Tom
19   Leckey.
20             MR. LECKEY:  Hi, I'm Tom Leckey,
21   EIA; I work on the 861.  I think there would
22   be a high correlation in year over year
0422
 1   timings.
 2             MS. FORSYTH:  ----- costs.
 3             DR. FEDER:  Definitely you're right
 4   about not letting them off the hook because
 5   there's a peril here.
 6             MR. LECKEY:  And I guess I would
 7   just add one other thing.  You were
 8   mentioning about the accounting, and Scott,
 9   you followed up on accounting systems as a
10   possible contributor to a slow response.  I
11   think the biggest factor in the response is
12   that the larger the entity the more likely
13   they are to have resources committed to
14   regulatory reporting.  So the smaller they
15   are, the more likely it is that they have an
16   ad hoc regulatory reporting system, which is
17   going to slow down the response rate for
18   people.
19             MS. FORSYTH:  And that's really
20   important for your frame purposes if what
21   you're missing are small ones, because they
22   have different reporting systems.
0423
 1             DR. FEDER:  Then again, if you get
 2   a response after the cut-off date, you can
 3   still use it imputation for the next year, so
 4   that's useful.
 5             THE CHAIR:  Yes?
 6             MS. HOWELL:  Shannon Howell, EIA.
 7   Speaking of imputation with relating to other
 8   surveys and the impacts this could have, if
 9   it's the smaller facilities that are having
10   more trouble reporting, that could impact on
11   the 826, which is the monthly that uses the
12   861 as a frame, where a lot of those smaller
13   facilities' monthly values are imputed for
14   based off the 861.  So if you don't have 861
15   data, or you only have two-year-old data,
16   that's very problematic.
17             Now, if the data is continued to be
18   collected after the cut-off date, provided
19   it's not really, really late into the next
20   year, it can still be used, but not
21   collecting it or potential bias in having the
22   smaller ones be not collected could cause
0424
 1   problems with other surveys.
 2             MR. HILL:  You can easily come up
 3   with a story, as our economist members do,
 4   that at the larger plants there's somebody
 5   sitting there whose job is every month to
 6   push the button that sends you the number,
 7   whereas at the small ones that doesn't
 8   happen.  Your question in some sense is what
 9   are you gaining or losing by not having those
10   extra respondents in there.  You'd like to do
11   something like a cost-benefit analysis of
12   what are we losing by not having that last 1
13   percent, 5 percent, 10 percent in that
14   sample.  If that can be done; if you could
15   somehow establish costs of those specific
16   values.
17             MS. JENNINGS:  Of course, what's
18   obvious to us is what we've alluded to
19   earlier and what Shannon reiterated.  Those
20   are the things that are obvious.  We know we
21   have that loss there, that impact, and that's
22   a loss to us.  But you're suggesting a
0425
 1   cost-benefit analysis, just to find out
 2   specifically what's missing?
 3             MR. HILL:  If you can do that; if
 4   you can somehow assign costs.  What do we
 5   lose by not having that extra five percent?
 6             MS. JENNINGS:  Okay.
 7             MS. FORSYTH:  Remembering that part
 8   of what you use is the imputation.
 9             MS. JENNINGS:  Okay.
10             MS. FORSYTH:  It's very complex,
11   it's multi-dimensional.
12             THE CHAIR:  Now, this problem is
13   also reminiscent of what we have seen in this
14   committee over the last few years from John
15   Wood on natural gas production in the sense
16   that we get -- it's the same thing, same
17   thing absolutely.  And the question this begs
18   is can we actually use what we've learned in
19   those contexts for natural gas production.
20             In this case, here, again, not for
21   completing the frames, understanding that the
22   frames will not be complete if we have a
0426
 1   cut-off point, but at least being able to
 2   provide early reporting of the totals, which
 3   is one aspect that you want to solve.  And
 4   again, it's not letting anybody else off the
 5   hook, it's just for us to be able to publish
 6   the good numbers early; that's the name of
 7   the game.
 8             MS. FORSYTH:  What are the dynamics
 9   of the follow-up process?  I guess what I'm
10   wondering is whether you think it's the
11   repeated contacts that eventually bring these
12   people in, or just giving them time?
13             MS. JENNINGS:  Actually, Tom is
14   probably a better person to answer that,
15   because Tom's --
16             MR. LECKEY:  Right, repeated
17   contacts is the element that brings in a
18   responsive entry.
19             MS. FORSYTH:  So it really is the
20   effort?
21             DR. FEDER:  Contact by phone or --
22             MR. LECKEY:  That's usually the
0427
 1   most effective way.
 2             MR. SITZER:  It is the most
 3   effective but there is a process of
 4   contacting non-respondents initially with a
 5   postcard or something.  And then maybe you
 6   don't get anything; in another month, it's
 7   another postcard.  Then if you don't have
 8   anything from them, you may need to escalate
 9   it, so a higher-level EIA person sends a
10   higher-level person at the company a letter.
11             MS. FORSYTH:  And so that would be
12   like the June?
13             MR. SITZER:  Something like that,
14   yeah.
15             MS. FORSYTH:  So actually you're
16   getting a pretty good response from your
17   postcards and letters, because you get up to
18   90 percent, right?
19             MR. SITZER:  I think those are
20   effective, yeah.  I couldn't tell you,
21   though, how many people need that at any
22   given point in time.  I'm not sure.
0428
 1             DR. FEDER:  But then why not
 2   approach the repeat offenders that year after
 3   year are late, earlier in the process.  Just
 4   target those with a phone call because you
 5   know they're not going to do anything with
 6   the paper reminder.
 7             MR. SITZER:  I think you've got
 8   people, though, who are recalcitrant and
 9   really don't care how many times we call.
10   But I think they tend to be non-significant
11   in terms of the numbers they're giving us,
12   but nevertheless there is that
13   word-gets-around concern, I think.
14             DR. FEDER:  Yeah, but companies
15   develop certain habits and sometimes the next
16   person on the job could be a better
17   responder, but he or she is told by the
18   previous POC, "Just don't worry about this,
19   they always send this, and we always recycle
20   their card."
21             THE CHAIR:  Barbara, what about
22   biases?  We see coverage going up to pretty
0429
 1   high; is that something I should be concerned
 2   about?
 3             MS. FORSYTH:  Well, Moshe made the
 4   point that if it's the small ones that are
 5   coming in late --
 6             DR. FEDER:  It could be influential
 7   in regression analysis for instance.  If you
 8   do any kind of multivariate analysis where
 9   the interest is in the relationship between
10   production and cost or whatever, I think that
11   might have an effect.
12             MS. FORSYTH:  And a ripple.  The
13   thing I guess I'm focusing on is the ripple
14   effect across the other surveys, if the small
15   institutions missing -------- data from them.
16             DR. FEDER:  I think we agree that
17   if you get a response, even if you get it
18   late, and you don't use it for this year's
19   estimate, it could be used for imputation and
20   other purposes for the next year and so on;
21   it's better than no data.  So if a small
22   company responded last year in July, I could
0430
 1   still impute that value for the next year,
 2   maybe with a factor for this year.  It's
 3   better than reporting zero.  So it pays to do
 4   it even if it's not going to come in time.
 5             MS. FORSYTH:  And so maybe another
 6   way of asking a slightly, like a morph of
 7   your question is, are there ways to reduce
 8   the costs of the respondent follow-up.  It
 9   sounds like it's the telephone calls that are
10   the major cost, and is there a way of
11   reducing that, either perhaps by getting less
12   skilled non-response on unit, or, I mean, not
13   as highly paid.
14             MS. McDANIEL:  I'm Karen McDaniel
15   at EIA.  We do a combination of various types
16   of non-response follow up.  We begin with
17   e-mails; we do do postcard reminders.  At the
18   very end we do, as you said, send out a
19   letter to the higher up in the company,
20   whether it's the mayor or whoever it is,
21   depending on if it's a municipality or
22   whatever, but do our share of e-mails.
0431
 1             We have had, I don't know if --
 2   working on 861 right now -- but I have worked
 3   on other surveys, but we have had a temporary
 4   agency come in to do non-response at certain
 5   times.  So we've tried a number of things.
 6             MS. FORSYTH:  Yeah.  I'm just
 7   thinking that e-mails can be automated so,
 8   for us at least the labor costs are the --
 9             MS. McDANIEL:  Right, and you can
10   hit a bulk number at a time we use a ------
11   program to reach a greater number but that
12   does cause some problems because different
13   people have different systems, and they have
14   firewalls.
15             A lot of these companies have
16   turnover, meaning different employees.  The
17   people who have quarterly surveys have ----
18   told us we have found that every quarter
19   there could be a different respondent.  So
20   e-mail addresses change.  There are issues
21   with e-mail response as well, but time-wise
22   and money-wise it's probably greater.
0432
 1   Greater thing to do.
 2             MS. FORSYTH:  Assuming I'd worry
 3   about bias.
 4             THE CHAIR:  You do.
 5             MS. FORSYTH:  But I'm a purist.
 6   I'd worry about bias, I think.
 7             THE CHAIR:  Now, what is the
 8   impact?  Suppose that I do update my frame as
 9   the respondents trickle in.  I don't have a
10   frame that's cast in stone for a given any
11   time.  Would that make a difference?  Because
12   suppose June 1st, on some other EIA agents or
13   other survey, they're going to use the
14   current frame, and it goes down to August;
15   the frame has changed because some other
16   people have trickled in.  It has been
17   updated.  How would that impact uniformity
18   across the surveys because the frame has
19   changed?  Is there a way to adjust for that?
20   Should that be a concern to me?  Or is that
21   the vanquished question?
22             MS. FORSYTH:  Well, I think of it
0433
 1   operationally.  It's a nightmare when the
 2   frame changes, when you have schedule late,
 3   you have to draw a sample by it, and I'm not
 4   sure how the monthly survey samples are drawn
 5   but operationally, it could have a ---
 6   impact.
 7             DR. FEDER:  What we do -- perhaps
 8   you can address this, but we produce the
 9   sample data set or database on a temporary
10   basis and keep updating it and all users of
11   the data keep updating the programs.  Once
12   it's set up, it usually doesn't take that
13   long to run the program again and recognize
14   that those data sets are temporary.  But we
15   have people that do the frame, others do the
16   weighting and so on, so they're all working
17   on it.  If you don't have this many people to
18   do it then it could be quite an onerous task,
19   I guess.
20             MR. SITZER:  Basically, the frame
21   doesn't change because somebody didn't
22   respond.  The frame is held constant in the
0434
 1   sense that we need a good reason to take
 2   someone off the frame.  They may have not
 3   responded because the company was sold, it
 4   went bankrupt, but we usually need some kind
 5   of evidence of that before we would actually
 6   take them out of the frame.  So the frame can
 7   contain non-respondents and other surveys may
 8   still use it.
 9             MS. FORSYTH:  I don't understand
10   that.
11             THE CHAIR:  Okay.  I didn't
12   understand that part either.  So the frame is
13   essentially set?
14             MR. SITZER:  It's set, although we
15   have to augment it or reduce it depending on
16   the information we get, but it's not just
17   they didn't respond and therefore they're out
18   of the frame.  We need to know that they
19   really don't exist anymore in some sense
20   before they're going to be dropped from the
21   frame, and that's why we continue the
22   non-response process.
0435
 1             DR. FEDER:  Scott, does that apply
 2   to the surveys that this survey feeds into,
 3   such as the one that Shannon was talking
 4   about?
 5             MR. SITZER:  They're using that
 6   frame.  The 826 is using the 861 frame, but
 7   it's a subset of it.
 8             THE CHAIR:  I see.  So now it
 9   becomes an interesting question.  How are the
10   non-responses across surveys that use the
11   same frame, right?  Is that what you're going
12   to say, Barbara?
13             MS. FORSYTH:  Well, you're ahead of
14   me, but yeah.
15             THE CHAIR:  I suspect that if they
16   didn't answer the 860 or 861, they will not
17   answer the other surveys as well?
18             MS. FORSYTH:  Probably because
19   they're not in business anymore.
20             THE CHAIR:  Well, then they
21   shouldn't be in the frame in the first place.
22             MS. FORSYTH:  Exactly, but --
0436
 1             DR. FEDER:  They could go out of
 2   business after the year, in February of the
 3   following year.
 4             THE CHAIR:  Or they may not respond
 5   period, I mean they --
 6             MS. FORSYTH:  Exactly.
 7             THE CHAIR:  Yes?
 8             MS. McDANIEL:  Karen McDaniel, EIA.
 9   But you have to remember also too, not every
10   survey, even if they feed from one another,
11   have the same contact person.  They could
12   have a different contact person for two
13   surveys that are similar, or one that's a
14   sample of the other, they don't always
15   necessarily have the same exact contact
16   person.  And as he said, just because they
17   don't respond doesn't mean they're off the
18   frame.  We have to find out who is
19   responsible for reporting for that particular
20   survey, even though that person may be a
21   different person this year or this quarter or
22   this month, but somebody at that company has
0437
 1   to report.
 2             MS. JENNINGS:  In addition to that,
 3   let me just remind you that the frequency of
 4   the collection of the data differs from
 5   survey to survey.  The 860 and 861 are annual
 6   and the other surveys are monthly.  There are
 7   other surveys that even quarterly in other
 8   areas of EIA.  So the frequency makes a
 9   difference as well.
10             You will probably here from the
11   monthly people a lot more, possibly more
12   timely, I can't speak to that with statistics
13   right now, but the annual surveys report once
14   a year, so that does make a difference.
15             MS. HOWELL:  I don't exactly work
16   on the 826 directly, but I work with the data
17   in terms of imputation on a monthly basis,
18   and I think that the issue for non-response
19   there is not as dramatic, but the 826 is a
20   cut-off sample based on the 861.  So the
21   smaller entities aren't reporting there, and
22   they are imputed for.  So in sense of having
0438
 1   the frame knowing if they exist or not
 2   determines, of course, whether or not you
 3   could keep them to impute for them, but if
 4   they exist and don't report data, it makes it
 5   very difficult to impute for them, if not
 6   impossible.
 7             THE CHAIR:  Everybody is looking at
 8   me.  That's not a good sign.
 9                  (Laughter)
10             MR. HILL:  I'm going to ask you a
11   naïve question.  Is this terribly much more
12   difficult than the monthly, the quarterlies?
13   Is that a possible reason for non-response?
14   It's much more time consuming for the
15   respondents, than say the monthly?
16             MS. JENNINGS:  Oh, your question
17   is, is it more time consuming for a
18   non-response follow up for an annual survey,
19   versus --
20             MR. HILL:  Are you ---- getting
21   better responses on the monthly survey
22   because --
0439
 1             MS. JENNINGS:  Are we getting
 2   better --
 3             MR. HILL:  Because again, it was
 4   more difficult to --
 5             MS. JENNINGS:  I'm just going to
 6   ask.  Tom, do you know the answer to that?
 7             MR. LECKEY:  I'm sorry, what's the
 8   question?
 9             MR. HILL:  Is the annual survey
10   much more difficult for the respondents?
11   Does it take them several hours to respond to
12   it, as opposed to 10 minutes.
13             MR. LECKEY:  Yes, the annual survey
14   is much longer.  I think it has 150 data
15   items on it.  Not everybody has to fill out
16   all 150, but it's got many, many more, the
17   826 would have, I don't know, 20 or 25.  So
18   it's certainly longer.
19             MS. McDANIEL:  I think another
20   issue is that annual, where they don't report
21   but once a year, there's more opportunity for
22   change of contact in that company.  If
0440
 1   they're reporting monthly, you, I think, are
 2   better able to keep up with who should be the
 3   contact or who is going to be the contact
 4   because it's occurring every 30 days.
 5             THE CHAIR:  If I got it right, the
 6   monthly surveys are actually certainty sample
 7   plus a random sample from the larger
 8   companies, and we exclude the small ones, is
 9   that correct?
10             MS. HOWELL:  It should be a strict
11   cut-off sample measuring the largest
12   facilities, I believe as a census, with
13   occasional non-response, usually caught a
14   month or so later.  Hurricane Katrina for
15   instance gave us a little bit of trouble.
16   But it should be a cut-off sample with the
17   smaller facilities completely excluded.
18             THE CHAIR:  And so the 860 is the
19   only opportunity you have to see the smaller
20   ones.
21             MS. HOWELL:  The 861.
22             MR. LECKEY:  It's definitely 862.
0441
 1             THE CHAIR:  The 862?  So there is
 2   no way to piggyback on the month, is it --
 3             MS. FORSYTH:  Right, to use it for
 4   the follow-up.
 5             THE CHAIR:  To use it for the
 6   follow up, so I'm just thinking, borrowing
 7   strength across questionnaires or surveys,
 8   but this one here has a very special purpose.
 9   Of course you told us that in the beginning
10   and we just didn't realize.
11             I mean, the fact is you've
12   identified indeed some of the problems with
13   the bias and you are already cognizant of the
14   importance of getting as complete a sample as
15   possible or getting all the data possible,
16   and I'm a little bit worried about saying or
17   giving a blessings of yes, you can use a
18   cut-off sample or have a cut-off, because of
19   the importance of getting the non-response
20   data or getting the data as accurately as
21   possibly from the 860 and 861, at the same
22   time recognizing that it costs a lot of money
0442
 1   to get diminishing returns, and that is where
 2   we always are, right?
 3             It's always this cost-benefit thing
 4   that we're talking about, and here it's
 5   clearly again the case that in terms of
 6   numbers.  I mean, you say I get over 97
 7   percent of total sales, why should I spend a
 8   million to get it to 99 percent, right?
 9   We're all asking that question.  I don't know
10   what to advise you, except that I think that
11   maybe going aggressively against a handful, a
12   sample of them as I initially suggested might
13   help, although will not resolve the problem.
14   And definitely, I would not close the books
15   on latecomers, because of the importance of
16   those latecomers, even if they come in late,
17   right, you may not use them to report the
18   sales numbers but you may need them later on
19   for other purposes, and so if they report
20   late, don't exclude them.  But you already
21   knew that, so I feel a little bit at a
22   disadvantage here.  Any words of wisdom
0443
 1   beyond that, except that Derek here would
 2   tell us something.
 3             MR. BINGHAM:  I will?  No.
 4             THE CHAIR:  Moshe, do you have
 5   anything you want to add?  Maybe I should
 6   turn the tables.  Are there other questions
 7   or clarifications that Alethea may ask us.
 8             MS. JENNINGS:  No, Howard may ask
 9   us.
10             THE CHAIR:  Oh, Howard, yes.
11             MR. BRADSHER-FREDRICK:  Well, I
12   guess with regard to background analyses, I
13   mean, I guess I've heard people saying that
14   you could look into repeat offenders.  I'm
15   just trying to think of some technical things
16   that could be done.  In the end, Scott is
17   going to have to make a decision about what
18   actually would happen with regard to
19   surveying the people or having cut-off dates
20   and so forth.  Is there anything else that we
21   could do?
22                  (No response)
0444
 1             MR. BRADSHER-FREDRICK:  I guess the
 2   answer is no.  But people are still thinking.
 3             THE CHAIR:  Suppose you find repeat
 4   offenders?  What would you do?
 5             MR. BRADSHER-FREDRICK:  Well, this
 6   at least would tell one that maybe one should
 7   spend one's resources on looking at those
 8   repeat offenders, perhaps, rather than having
 9   cut-off dates.  That would be one approach.
10             DR. FEDER:  Call them earlier in
11   the process.
12             MR. BRADSHER-FREDRICK:  Yeah.
13             MR. BINGHAM:  Can you estimate, so
14   if they are repeat offenders you have the
15   previous years?
16             MR. BRADSHER-FREDRICK:  I'm not
17   sure.  These are some of the things we'd have
18   to look at.  How much data do we really have?
19   This has been a problem with looking at some
20   of these issues.
21             MR. BINGHAM:  It's already been
22   asked, but if you have information on a
0445
 1   particular -- as to whether they're a repeat
 2   offender or not, can you, if they haven't
 3   responded by the cut-off date, can you
 4   estimate what their responses might be?
 5             MR. BRADSHER-FREDRICK:  Well, they
 6   do imputation -----
 7             MR. BINGHAM:  So based on the
 8   previous years, right, okay, so that's what
 9   you're --
10             MR. LECKEY:  Yes.
11             MR. BINGHAM:  Okay, so you've been
12   talking about that, okay.
13             THE CHAIR:  Now, the question was
14   more since it is a mandatory survey, do we
15   have any enforcement tools to diminish the
16   repeat offenders?  I think again of the IRS.
17   Do we have -- are there any tools available
18   to you, besides the carrot, to get data in,
19   especially if you identify repeat offenders,
20   and that would be the use of it.
21             MR. SITZER:  The form makes it very
22   clear that this is a mandatory form, and if
0446
 1   they don't file it they're subject to
 2   monetary penalties.  I don't know that we've
 3   ever done that.  And I don't know what the
 4   process would be, but I think a long time ago
 5   we decided that that might be a path that we
 6   didn't want to go down.
 7             THE CHAIR:  It's an honest question
 8   that's -- Moshe?
 9             DR. FEDER:  Yeah, that's on the
10   earlier point of imputation.  This is an
11   example where you want to have the data from
12   those companies because if you want to be
13   able to do a good job imputing values for
14   small companies you need data from some small
15   companies.  So here is a situation where even
16   internally, within the EIA, having that
17   information is useful for future imputation
18   models.  If you use regression, imputation,
19   or even past year data, anything like that.
20   So this is why I was concerned that just to
21   decide to cut off early might cause us to
22   lose information that's needed for specific
0447
 1   uses such as imputation.
 2             THE CHAIR:  Yes.  We're at the end
 3   of our wits.
 4                  (Laughter)
 5             MS. FORSYTH:  We can only empathize
 6   with how you feel.
 7             SPEAKER:  Just get more data ------
 8             THE CHAIR:  What do you say more
 9   data, Moshe?
10             DR. FEDER:  Like, check on repeat
11   offenders, and --
12             THE CHAIR:  But then what?
13             DR. FEDER:  You can learn a lot by
14   looking at who are these repeat offenders and
15   target them.  Go after the repeat offenders
16   earlier in the process, check out why they
17   are late.  Somebody mentioned that it could
18   be in small companies the POC changes every
19   year.  So it might make sense to call them up
20   in December and ask them who's going to be my
21   contact person when I send the questionnaire
22   soon.
0448
 1             MS. FORSYTH:  Or work with the
 2   facility to identify ways of finding points
 3   of contacts proactively for the repeats, just
 4   a small group.
 5             THE CHAIR:  And have proxies for
 6   points of contact because you have points of
 7   contact for other surveys.  Can you leverage
 8   that?  I know in one company one person may
 9   not know the other one, but still is there a
10   way that -- this may be a way to get the
11   information to the right person.
12             DR. FEDER:  So here's another
13   example where you look at the repeat
14   offenders and see is the POC the same for the
15   other surveys or not, so you can learn about
16   that and find out what's the best way to
17   approach them.
18             MR. LECKEY:  Right.  I kind of
19   think what Moshe's getting at here is an
20   alternative.  It's an entirely new
21   alternative where the legislation calls for
22   penalties.  So you can pursue penalties or
0449
 1   you can pursue information through the threat
 2   of penalties.  But the other way to do it
 3   that never tried is to incentivize people or
 4   to engage with them before the fact.  Now,
 5   that's resource-intensive too, and I don't
 6   know where those resources would come from,
 7   but it is new and it might yield a different
 8   result from the tried and --
 9             MS. FORSYTH:  And it might be
10   resource-intensive the first time, but if it
11   works maybe in the longer term it will
12   require less.
13             THE CHAIR:  I am willing to call it
14   quits five or ten minutes early.  I thank you
15   very much for your time.
16             MS. JENNINGS:  Thank you.
17             THE CHAIR:  Again, if you have any
18   questions, feel free -----
19             MS. JENNINGS:  We appreciate the
20   effort though, and the comments.
21                  (Recess)
22             THE CHAIR:  Okay ladies and
0450
 1   gentlemen.  Okay, let's get back to work
 2   please.
 3             Before we continue, I'd like to
 4   take this opportunity to do some shameless
 5   free advertisement for Shawna here, who is
 6   actually organizing a session on energy
 7   statistics.  Actually it's at the JSM, in
 8   Seattle.  So, you have the program.  There's
 9   a list of a number of talks.  I think this
10   was the result of some of the discussions we
11   had at the committee last time, and I really
12   am pleased that Shawna tucked the ball and
13   rolled with it and actually made this happen.
14             It's wonderful to see that what
15   we're discussing here in the committee gets
16   larger exposure to the general statistical
17   community, because I think a lot of the
18   problems we're having here are good problems
19   that hopefully will stimulate other
20   interactions in the future.  So thank you
21   very much for doing it, again.  If I read
22   here --
0451
 1             MR. MOSHE:  Who's going to be the
 2   chair of the session?
 3             THE CHAIR:  I already have the
 4   chairs, of course.  Listen, I'm getting all
 5   the practice here.  I'm going to be polished
 6   by the time I get there, I promise you that.
 7             And the other nice thing is it's an
 8   afternoon session, so you can go and party as
 9   much as you want the night before.
10                  (Laughter)
11             THE CHAIR:  Oh my God, I know.  So
12   Shawna really hit the nail on with this one.
13   So I think that's all I want to say.  Again,
14   please show up, support EIA by your presence
15   at the JSM, if you go there, and thank you
16   for everybody who agreed to talk at this --
17   actually none of us is talking.
18                  (Laughter)
19             THE CHAIR:  Howard is, sorry,
20   sorry.
21             MR. BRADSHER-FREDRICK:  I am.
22             THE CHAIR:  You are.  Okay.
0452
 1             MS. KIRKENDALL:  And Will Gifford
 2   is also.
 3             THE CHAIR:  Yes.  No, that's fine.
 4   Excellent.  Now, on a more important note I'd
 5   like to ask -- it was Shawna who --
 6             MS. KIRKENDALL:  It's Vicki and
 7   Rick.
 8             THE CHAIR:  I would like to ask
 9   Vicki and Rick to come down here.  Those are
10   our friends from the Census Bureau who did a
11   wonderful job -- yes, and Shawna, yes -- who
12   worked with Shawna on this dual system
13   estimation and validation of our frames.  So
14   I think they did a fantastic job, and EIA
15   would like to recognize their efforts.
16             MS. KIRKENDALL:  Yeah.  So this is
17   for Rick.  Let me show you the right one --
18   and to Vicki Haitot.  They are Rick Hough and
19   Vicki Haitot --
20                  (Laughter)
21             MS. KIRKENDALL:  -- for their
22   efforts in the frames evaluation.  This was
0453
 1   one of the first projects we've had with the
 2   Census Bureau to evaluate our frames.  They
 3   looked at five different frames that match
 4   into their surveys, and gave us back results
 5   that helped us to understand the quality of
 6   our frames.  And we thought it was a
 7   wonderful collaboration.  They came over the
 8   last two years, and talked to you several
 9   times.  It's been a great partnership.  And
10   this is just a little certificate to thank
11   them for their work.
12             MR. HOUGH:  Thank you.
13                  (Applause)
14             MS. KIRKENDALL:  Yeah, and Shawna
15   has been the person that they've interfaced
16   with over the whole time.  So she's been this
17   part of the effort, too.
18             THE CHAIR:  Yes.
19             MR. HOUGH:  I hope we can get one.
20             MS. KIRKENDALL:  I hope so too.
21             THE CHAIR:  Yes, yes.
22                  (Laughter)
0454
 1             MR. HOUGH:  This is a marketing
 2   wonder anyway.
 3             THE CHAIR:  Yes.
 4                  (Laughter)
 5             THE CHAIR:  So next on the agenda
 6   is going back to a summary of the two
 7   breakout sessions.  I understand that the
 8   breakout session downstairs in the
 9   fifth-floor room was very lively, and I think
10   Cutler will give us a summary of all the
11   fireworks.
12             MR. CLEVELAND:  Thanks Nick.  We
13   had a presentation from José and Fred about
14   an interesting and longstanding issue in
15   economics and energy markets, and that is the
16   relationship between oil and gas prices.  And
17   José and Fred developed a model using a
18   co-integration framework to test the
19   relationship between Henry Hub prices and WTI
20   crude oil prices to see if in fact there was
21   some kind of long-run relationship, as well
22   as a short-run relationship.  And their
0455
 1   research was motivated by the fact that in
 2   the recent years there have been some
 3   significant perturbations to this long-run
 4   relationship which you look at when you do
 5   the biocular excitement test as my
 6   econometrics professor used to call it.
 7             When you look at the prices there
 8   were periods, particularly during cold snaps
 9   and other shortages of gas, where the prices
10   deviated.  So this raised a question of, is
11   there a long-run relationship, and, if so,
12   has it changed, disappeared, and so on and so
13   forth.  So it was a lively discussion.  I
14   think some of us -- I guess I'll speak for
15   myself -- who have followed the statistical
16   work of certain parts of NEMS and the EIA are
17   quite happy to see this type of modeling
18   framework being used to look at time series
19   data.
20             Some of the work that has
21   underlined certain parts of NEMS in the past
22   has not been really up to snuff in terms of
0456
 1   staying up to speed with what the state of
 2   the art is in time series analysis, and this
 3   to me was a very positive thing to see, and I
 4   would really encourage everyone at EIA
 5   involved in time series analysis to embrace
 6   this type of approach to analyzing long-run
 7   statistical data.
 8             I think that one of the best things
 9   about the discussion was the interesting
10   questions that it really raised, which a good
11   model should, about what, really, the
12   structural relationship is between natural
13   gas and oil prices.
14             In the past there were reasons why
15   you would expect oil and gas prices to be
16   coupled at the wellhead and reasons why you
17   would expect them perhaps to deviate and one
18   of the, in, I think not surprising, their
19   results showed that in fact there was a long
20   run, what's called cointegration relationship
21   between oil and gas prices, that they do move
22   together, that oil prices drive gas prices
0457
 1   with a lag, but not vice versa, and the
 2   second major empirical finding was that there
 3   is an important time trend apparent in the
 4   data, where gas prices appear to be growing
 5   faster than oil prices.  And so those two
 6   results raise a number of questions,
 7   particularly about how this relationship
 8   might change in the past given the changing
 9   dynamics particularly in the gas market.
10             For example the model was estimated
11   with oil that in the past whose price has
12   been determined in the world market,
13   conventional oil.  What happens when
14   unconventional oil, oil sands, tar sands,
15   start to come online, since they are not a
16   co-product with natural gas, and what
17   implication would that have for this
18   structural relationship?
19             We also see an increasing
20   globalization of the natural gas market,
21   largely through some increasing pipelines
22   that connect markets that had previously been
0458
 1   sectioned off, but increasingly with their
 2   improvements in the technology of LNG, LNG is
 3   now becoming a major player.  How might that
 4   affect this relationship since, in the U.S.,
 5   we pretty much had a closed market for
 6   natural gas, but now you have natural gas
 7   coming in from other parts of the world, and
 8   how might that change this long-run
 9   relationship and might some of these factors
10   be important in producing this time-trend
11   that we see of gas prices tending to grow
12   faster than oil?
13             There was an issue about what the
14   existence of this time-trend meant for the
15   reliability or convincing nature of the
16   results, and I think from my econometrics
17   experience the existence of a time-trend
18   always means you have a homework assignment,
19   which is to figure out what's really going on
20   into producing that time-trend, because time
21   itself obviously is not doing it.  And I
22   think José and Fred are aware of that.
0459
 1             There was also a discussion about
 2   if it should be used in NEMS, and I think
 3   we're only beginning to explore that issue.
 4   Is this really useful for a long-run
 5   forecasting tool -- I think there were some
 6   serious questions raised about that -- but on
 7   the other hand it also has a short-run
 8   component in it which may be useful.  So I
 9   think that's something that the EIA is going
10   to have to look at more carefully.
11             We also had a distinction of ----
12   again looking forward to what might happen in
13   the future.  The question was, well, should
14   we really be focusing on the demand side
15   instead of using wellhead prices, which is
16   what this is using, looking at end user
17   prices, and looking at regions where oil and
18   gas actually compete at the point of end use,
19   in particular boiler applications, and so on
20   and so forth, which is really a kind of a
21   short-run phenomenon.  On the longer-run
22   issue the point was made that really what's
0460
 1   competing now is decisions made to drill for
 2   non-associated gas or oil, and so the
 3   associated gas issue, which might have
 4   produced some of this couple in the past,
 5   might become less of an issue in the future,
 6   and so that was something that needed to be
 7   looked at.
 8             To some rate, I think the general
 9   consensus was that there were some powerful
10   forces here at work in the past, which
11   produces very close relationship, and there's
12   a need to understand what produced that
13   structure, the mechanisms that produce that
14   structure, and then again this time-trend
15   variable, which represents this gas price
16   moving away from oil more recently, what is
17   driving that, and how might that affect the
18   structure of this model in the future?
19             So if there were other folks in the
20   session that want to amend my remarks or add
21   to them, please do so.
22             I do just want to say for guys, I'm
0461
 1   going to reiterate one remark that I made at
 2   the beginning, was that as someone who has
 3   been a long-time user of these models and a
 4   long-time critic of some parts of the
 5   statistical procedures used in NEMS, that
 6   this type of cointegration framework is
 7   really good to see.  And it really represents
 8   an example of how EIA should be embracing
 9   current state-of-the-art tools and time
10   series econometrics, and I would encourage
11   you to do that to the full extent possible
12   throughout all your work.  But it's great to
13   see.
14             THE CHAIR:  Thank you very much.
15   Anybody else wants to add?
16                  (No response)
17             THE CHAIR:  Well, it seems although
18   the session was lively everybody's in roaring
19   agreement.
20             MR. CLEVELAND:  Yeah.
21                  (Laughter)
22             THE CHAIR:  Next I'd like to invite
0462
 1   Walter Hill to give us a summary of our
 2   discussions we had on preliminary research
 3   results on respondent cutoff dates.  Walter?
 4             MR. HILL:  In the presentation the
 5   word "preliminary" was mentioned a couple
 6   times, and also that it's the head of a
 7   working paper.  It is on electricity
 8   production and consumption.  I thought that
 9   the problem was clearly stated, and the
10   attack was more or less straightforward.  The
11   issue is that series of surveys are done to
12   collect information on electricity, say,
13   consumption, with surveys that were
14   non-response.  This is a non-response issue.
15   The question was whether or not there should
16   be an early cutoff on the date for which
17   responses are taken, and I think we can
18   quickly say we thought, no, we don't know.
19   That was a short answer.  You're getting
20   numbers, as, say, roughly -- something little
21   feedback here -- roughly 50 percent or so in
22   the first month, maybe roughly two thirds or
0463
 1   ---------- the second month.  Some feedback
 2   there.  Want me to talk louder?  And then
 3   there's a -- you tail off -- and then there's
 4   a reduction -- no, there's still some
 5   feedback.  ------ Thank you.
 6             Monthly data were collected.
 7   You're getting response rates around 50
 8   percent, say half in the first month, 80
 9   percent for the second, up to 90 percent in
10   the third, and there's a reduction in the
11   response rate as you continue on through
12   July, August, and September.  So the question
13   is whether or not it's worth the cost to
14   continue to try to follow up, to get
15   responses on those later months.
16             We thought it might be worthwhile
17   to ask stakeholders -- and there are a number
18   of stakeholders -- whether or not that
19   additional information was useful.  And the
20   stakeholders even include EIA among other
21   people who might be using the data.
22             There are questions about whether
0464
 1   or not the frame -- there are no question
 2   about the frame -- whether or not the frame
 3   changed at all during the period of the
 4   analysis.  The short answer is no, it did not
 5   change.
 6             Question about whether or not the
 7   non-respondents change from year to year, and
 8   since there were a number of -- I don't know,
 9   recidivists -- we're seeing in some
10   literature that, no, they looked like there
11   were a number of producers who were not
12   responding from year to year on this annual
13   survey.  There's also a fear -- oh, so the
14   question of whether or not you go after those
15   specific non-responders, there was a, oh,
16   concern that since these are mandated, a
17   concern that if you went after the
18   non-responders that looks bad for the -- in
19   fact, I'm reading between the lines here --
20   bad for the agency, that the agency's going
21   after these users, on the one hand.  On the
22   other hand, since it's mandatory you expect
0465
 1   the producers to respond to the survey
 2   without any sticks or carrots.
 3             We noted that the non-responders
 4   tended to be the smaller producers, and so
 5   you're getting most of the responses -- most
 6   of the larger units are coming in.  But still
 7   the smaller units -- again, a number of
 8   issues came up, whether or not you really
 9   wanted those smaller units, and I think our
10   short answer is yes, you did want those.
11   Some people may be doing regression analysis
12   regarding the size of the units, you might
13   want the complete data just in any case.  As
14   a rule, having a complete data set is better
15   than not having the data.
16             People tended to think that it is
17   worthwhile imputing the data -- and that's
18   concurrently done -- from the previous years.
19   We thought even if the data came in late,
20   it's worth getting the data.  Maybe you
21   release some of the initial in preliminary
22   form, and then release the final version at
0466
 1   some later time.
 2             We had some discussion on how the
 3   units were contacted for responses.  We know
 4   that there is e-mail.  Those were followed up
 5   by one or two postcards, and those were then
 6   followed up by letters to more important
 7   people within the organization.  Most of the
 8   costs were telephone costs.  It was noted
 9   that perhaps there was a way of reducing the
10   telephone costs, which just means money paid
11   to the workers, and whether or not it would
12   be possible for my less skilled workers to do
13   that part of the work.
14             We know that you could update the
15   database when numbers trickled in.  The
16   question about whether or not it's worth
17   doing a cost-benefit analysis to see whether
18   or not it's worth getting that additional
19   one, two, five, or ten percent in those last
20   three months.
21             The IRS was mentioned a couple of
22   times, maybe because it's in April, as to
0467
 1   whether or not the IRS has ways of trying to
 2   get people who don't respond.  In general we
 3   thought that might be too heavy-handed, and
 4   it doesn't look good for other government
 5   agencies that they've adopted a very
 6   heavy-handed approach to getting responses.
 7             And we had some general comments
 8   about what happens with this sort of an issue
 9   of trying to get a response from the entire
10   frame.  Just note that there are increasing
11   costs, there are diminishing returns for the
12   extra cost, you may or may not need the
13   latecomers.  There are enforcement mechanisms
14   in place that have not been implemented.
15   People here maybe want to add or give
16   corrections to that summary.
17             THE CHAIR:  Thank you very much,
18   Walter.  Any other comments from this
19   session?  Again, I wanted to thank both the
20   presenter, Roger Frederick (?), and Alethea
21   Jennings, because this is very early on, it's
22   preliminary work, and I know it's always hard
0468
 1   to present preliminary works, because one
 2   hasn't thought through all the details, but
 3   in this case it seems that most of the bases
 4   were already covered, and the committee just
 5   reassured ourselves that, yes, we agreed with
 6   most of the comments they made, and the
 7   questions, I hope we were able to answer, to
 8   some extent, namely, is it worthwhile to look
 9   at the cutoff, and the question is always,
10   well, what's the purpose, why are we
11   collecting this data.
12             There seem to be two issues.  One
13   was confirmation of the data frame, and the
14   other one was the publication of energy
15   statistics.  And for the frame purpose it
16   seems that it's worthwhile continuing to get
17   the data in as it comes, because it's
18   confirmation that the frame, which is used in
19   other surveys, is good, and on the other side
20   it is possible using methods maybe not even
21   that different from what John Wood will
22   present, or has presented -- I got your
0469
 1   attention there --
 2                  (Laughter)
 3             THE CHAIR:  Essentially, using
 4   possibly even similar methods that are used
 5   in natural gas to try to predict what the
 6   final number might be after cutoff.  So these
 7   are some of the ideas that were floated
 8   around.  So it was a very good presentation.
 9   I wanted to thank again the EIA for allowing
10   us to comment early work, because I know how
11   difficult that is.  I mean, one is always put
12   on the spot, and one isn't quite sure if it's
13   good or not, and having -- has always the
14   fear we haven't thought through it
15   completely.  So let me reassure you this
16   committee appreciates your courage in doing
17   so.
18                  (Laughter)
19             THE CHAIR:  I want to move on to
20   our next item on the agenda, and I'd like to
21   welcome John Wood again, who is a known
22   quantity to this committee.  He has presented
0470
 1   here several times and today he's going to
 2   talk to us about the 914 data expansion
 3   challenges to include crude oil productions.
 4   Welcome, John.
 5             MR. WOOD:  Thank you.  Let's see.
 6   If I press "up" for example, something
 7   happens, and if I were to press this one it
 8   will go backwards.  EIA wants to go forward.
 9                  (Laughter)
10             MR. WOOD:  The purpose of the paper
11   is to solicit the committee's input on the
12   proposal to modify the form EIA-914 to also
13   collect crude oil production data, how to
14   select the states that will be estimated
15   individually in that process, and the
16   proposed modification of the EIA-914 natural
17   gas production estimation methodology.  Now,
18   again a number of members of this committee
19   have been involved in both setting up and
20   assessing the 914 project to start with, and
21   some time series estimation of production of
22   natural gas, and the overall approaches one
0471
 1   might want to take to time series assessment,
 2   and so we have in fact enjoyed and benefited
 3   from the prior discussions, and expect that
 4   to continue.
 5             In case you wandered in, or don't
 6   want to wait until I get there, the 914 was a
 7   tremendous success, brilliantly executed,
 8   cost effective, and is actually producing
 9   estimates which have very low sampling and
10   modeling errors, and the errors that are
11   associated with it in the beginning were
12   production reporting problems which we helped
13   straighten out, and often we improved the
14   data actually going to the states as part of
15   that process, and helped certain operators
16   understand what they were actually doing.
17             And the other problem -- again,
18   it's inherent in time series -- and that is,
19   we start somewhere.  And so we calibrate a
20   model with -- in the case of what we are
21   running right now, we calibrated it against
22   2003 data, when 2003 data wasn't final yet.
0472
 1   And so we did this at the operator level.
 2   And to the extent that the calibration set
 3   was, let's say, incomplete, then that would
 4   reflect forward into the model predictions.
 5   And we've got a suggested way to handle that.
 6             Going on to oil production,
 7   domestic crude oil production accounts for 34
 8   percent of the U.S. total supply, and
 9   increased attention will be devoted to the
10   domestic supplies in the light of the
11   continuing high crude oil prices and renewed
12   interest in reducing imports.  And then there
13   are a large number of things that are, I
14   think, going to impact both domestic and
15   North American production that are not
16   obvious in the historical statistics.  And
17   that is a rapid increase in the North
18   American production from non-conventional oil
19   resources such as Canadian tar sands and oil
20   shales, and especially production associated
21   with oil shales.
22             Now, just mechanically here, EIA is
0473
 1   at high risk -- and this is the calm way of
 2   saying it -- of not being able to produce
 3   timely and accurate monthly crude oil
 4   production estimates at the state and
 5   national level.  And first the Monthly Oil
 6   Production Update -- MOPUP -- system that is
 7   currently used has been in need of
 8   replacement for several years, although
 9   budget constraints have not made this
10   possible.  By the way, the preceding system
11   that actually Nancy Kirkendall helped put
12   together was the Monthly Energy Supply
13   System, MESS.
14                  (Laughter)
15             MR. WOOD:  And MOPUP was the
16   follow-on.  MOPUP uses monthly production
17   data collected on form EIA-182.  It's
18   designed to get prices, but it's an early
19   indicator of oil production.  The form
20   EIA-182 survey is going to be discontinued,
21   therefore both a new crude oil production
22   estimation system and a new survey instrument
0474
 1   that captures monthly crude oil production
 2   data are needed.  And again our approach is,
 3   we're going to take the 914 system that has
 4   been very successful in producing natural gas
 5   production estimates, and modify it to
 6   collect crude oil production data and produce
 7   monthly crude oil production estimates.
 8             Now, as we go through this
 9   modification process, again we need a new
10   survey data collection instrument.  EIA
11   currently connects two types of natural gas
12   production data on the form.  One of them is
13   gross withdrawals.  All of the gas that comes
14   out of the well, and lease production; that
15   portion of the production that actually
16   leaves the lease that the gas production
17   takes place on.
18             The form is laid out mechanically.
19   They have a part one that collects survey
20   respondent information and part two collects
21   the production data, and we collected it for
22   the six top producing areas, and we collected
0475
 1   it for the other states, excluding Alaska,
 2   and the logic for that was that primarily we
 3   are interested in the lower 48 gas system.
 4             What we would do in the
 5   modification process is add a second report
 6   to the form EIA-914 family called Monthly
 7   Crude Oil and Lease Condensate Production
 8   Report.  We'd have a part one with the same
 9   respondent information, and a part two that
10   mirrors the form that would collect two types
11   of production data, crude oil and lease
12   condensate.  And in order to have individual
13   coverage for the six largest crude oil
14   producing states, Alaska, California, and
15   Colorado would have to be added to the list
16   of states that are currently used for the gas
17   production data.
18             The selection of the six producing
19   areas was somewhat arbitrary.  They did
20   generally have relatively large fractions of
21   the total U.S. gas production share.  We
22   would recommend roughly to include the top
0476
 1   ten natural gas producing states in the
 2   modified form, and if we did that we would
 3   have to add Kansas.
 4             To have the top ten oil producing
 5   states we would have to add North Dakota and
 6   Montana.  And this also gives us the
 7   opportunity to have direct regional -- at the
 8   state level -- coverage of what we expect to
 9   be very important production plays in Montana
10   and North Dakota.
11             Both of them have relatively small
12   oil and gas production now, but in Montana
13   there is a field, the Elm Coulee field,
14   discovered in the year 2000, that produces
15   from the middle member of the Bakken shale.
16   This is production associated with oil shales
17   of which the United States and North America
18   has more than its fair share.  It is clearly
19   the largest onshore oil field discovery in
20   the last fifty years, and the Williston Basin
21   is basically covered with that formation, and
22   this could be an enormous supply in the
0477
 1   future.  And we feel like we ought to start
 2   covering it.
 3             We also think that we should cover
 4   the offshore Pacific region, especially the
 5   Pacific federal offshore.  It is a federal
 6   domain, and it has significant oil
 7   production, and potentially, more
 8   importantly, it has very large resources.
 9   And so if drilling programs are started up
10   again in the Pacific federal offshore we are
11   confident there'll be significant increases
12   in oil production.
13             MR. CLEVELAND:  Sorry to interrupt.
14   Looking at those forms, what the EIA does --
15   when you go to the annual reserve report for
16   example, we have detailed state data on many
17   more states than you're reporting here, so
18   where does that data come from?
19             MR. WOOD:  That comes from
20   individual annual reports, and approximately
21   92, 93 percent of it comes from detailed
22   field-level numbers, but there's also a very
0478
 1   large demand for monthly production,
 2   especially in areas where there is rapid
 3   change or in circumstances when there's a
 4   rapid change in production.  In something
 5   that is physically changing, the oil
 6   production of Montana basically increased 50
 7   percent from 2004 to 2005.  That dominant
 8   field increased 100 percent, and --
 9             MR. CLEVELAND:  So you would
10   continue with the two forms.
11             MR. WOOD:  They are not exactly the
12   same data.  The EIA-914 collects gross
13   production, and the reserves report collects
14   wet after lease separation, where most of the
15   non-marketable gases like CO2 have been
16   removed, and the gas that is re-injected has
17   been removed, and that is particularly
18   important in those areas which have high
19   concentrations of non-marketable gases, which
20   in fact is true in most of the Rocky
21   Mountains, which is one of the major plays,
22   and in Alaska.
0479
 1             And I'll touch on Alaska in a
 2   little more detail, but, going back to the
 3   potential for the federal offshore Pacific,
 4   again, and this is why it's nice to know the
 5   physical things you are studying.  There is a
 6   major formation that covers most of the Santa
 7   Maria Basin.  It is the Monterey Formation.
 8   We had a three or four year argument with the
 9   Mineral Management Service over, with 1990's
10   technology, whether one percent of that would
11   be recovered or two percent.  EIA boldly said
12   two percent.  MMS stuck with their one.
13   After three years we decided to disagree.
14   With horizontal drilling and fracking of
15   horizontal wells the potential is to go up to
16   10 percent or 20 percent, or something, so
17   these resources that were almost
18   unrecoverable 10 years ago are potentially
19   very economic and very large.
20             This is the form that we are
21   sending out now, and again it has two
22   categories of production, and then you report
0480
 1   in the areas that -- individually six of
 2   them, and then we asked them to report the
 3   rest of their production without including
 4   their Alaskan production.  And there's not
 5   there many companies that actually have
 6   Alaskan production, and they are dominated by
 7   just a few major companies.
 8             If we change the report to add,
 9   first, the new part, this would be a part
10   three, we'd have gross withdrawals of crude
11   oil, black oil, and then gross withdrawals of
12   lease condensate.  And that is the liquid
13   that is associated with natural gas
14   production.  And we often publish liquids.  A
15   data series which we publish is crude oil,
16   including lease condensate.  And so the
17   mechanics of it, though, for our data
18   processing systems, they're set up to get two
19   types of gas production, they would be set up
20   to get two types of oil production, or two
21   types of liquid production.
22             If we add 10 states -- and we may
0481
 1   well want to go to a column format just for
 2   ease of looking at it -- there are some
 3   discussions in a prior meeting, and it's good
 4   to go over it again.  When you have an oil
 5   well -- or a well, period -- the full well
 6   stream that comes out usually includes some
 7   oil, some lease condensate, some water.
 8   We've been measuring in the 914 the gross
 9   withdrawals of gas that is run through
10   facilities.  Some of it goes back in the
11   ground and re-pressuring and re-injected,
12   some of it is vented and flared -- a rather
13   small quantity -- there's fuel used on lease
14   mostly for power, and then there's
15   non-hydrocarbons removed -- some fields have
16   very high concentrations of CO2 or nitrogen
17   or some other non-marketable gas.
18             Now, if we actually go after the
19   production for the top ten gas states and the
20   top ten oil states and the Pacific offshore,
21   do we have to have a really large increase in
22   the number of operators we go to?  And the
0482
 1   answer is no.  When the EIA-914 survey was
 2   approved by OMB, EIA stated that we could
 3   meet our error targets of 1 to 5 percent with
 4   an operator sample in the 250 to 350 range,
 5   and we can stay in that operator survey range
 6   and meet the targets for the top 10 natural
 7   gas producing states and the top 10 oil
 8   producing states.  And we'll go over a little
 9   bit of the results that make us confident we
10   can do that.
11             And, again, one of the things that
12   makes that possible is that we showed in
13   figure one almost all the wells produce oil
14   and gas, and therefore most of the large
15   natural gas producers are also large crude
16   oil producers.  So we're already surveying
17   them, and one of the questions in the
18   interviews that were run during the
19   development process that operators ask, "Why
20   aren't you collecting oil production data
21   too?"  And it was a good question and now we
22   plan to.
0483
 1             Now, experience with the survey and
 2   some improved modeling will allow the error
 3   targets to be met, and again with the
 4   operator samples that actually have a smaller
 5   percentage of the total production.  For the
 6   U.S. we targeted that the operators that were
 7   sampled would have around 90 percent, and for
 8   individual areas I think there were none that
 9   had less than 82 or 83 percent of coverage.
10   In some states you have to add a lot of
11   operators to get up to 85 percent of the
12   production, and that's basically not
13   necessary.
14             MR. MOSHE:  Is it an annual sample,
15   or it should be one?
16             MR. WOOD:  The frame that is used
17   for this is collected on a monthly basis at
18   an operator level, and so you can actually
19   see the production percentages of the sample
20   change by month, and let's say the
21   calibration year.  And you can see the
22   percentage of production of a state, for
0484
 1   example that one operator has, over any
 2   monthly period you choose to look.
 3             Again, the same formal methodology
 4   will be used to estimate oil production as we
 5   used to estimate gas production.  A sample
 6   will be selected that covers roughly 85
 7   percent of total production, and we are not
 8   going to be worried if that drops to 75
 9   percent, and we certainly don't worry when
10   it's 98 percent, as it is in the federal
11   offshore of Gulf of Mexico, an area
12   dominated, almost all, by large companies
13   because of the technology and the expense
14   involved in going offshore.
15             Speaking of real challenges that
16   actually worry me, budget and OMB come to
17   mind.  I think there is a strong consensus in
18   the EIA that oil production data should be
19   collected, and that the 914 modification is a
20   very cost effective way to do this.  And we
21   really ought to start now.  The FY-2006
22   budget is tight, tough decisions have been
0485
 1   made, tough decisions remain to be made, and
 2   the EIA-914 modification project has not yet
 3   been funded.  And I hope that changes
 4   shortly.
 5             And then we haven't gone through
 6   the OMB clearance process, but we of course
 7   expect that to go smoothly and expeditiously.
 8   And in fact in some ways I believe that,
 9   because we've even set this up not to be a
10   new form, but to be a mere modification of an
11   existing survey which has no change in the
12   authorized respondent burden.  And those
13   would be the main concerns of OMB, and I
14   think we've met them going in, and it's
15   needed.
16             Once again our targets in the
17   original 914 gas program were to have
18   estimate 60 days from the end of the
19   production month, and that the sampling error
20   would be within 1 percent for the lower 48
21   states, and within 1 to 5 percent for -- now
22   we would change it to 16 areas; originally,
0486
 1   it was 6.
 2             I think there's some very
 3   interesting information in here that not
 4   everyone would be aware of, including many in
 5   EIA.  Again, the ranking on these states
 6   which we are proposing to collect from,
 7   natural gas rank is in red, and this is with
 8   preliminary 2005 production, and various
 9   states have moved around from 2003 to 2004 to
10   2005.  But Texas is the number one gas
11   producing state with about 25 percent now.
12   Alaska is number two in gross gas production
13   with about 15 percent of the gross gas
14   production in United States.  All of that gas
15   is used on the North Slope that is counted as
16   crude reserves.  And that gas that is counted
17   as marketed production is also only that gas
18   that is used on the North Slope.  Most of the
19   gas is re-injected.
20             The pipeline economics are quite
21   feasible to come down, especially if we were
22   to come down through Canada.  With $5 natural
0487
 1   gas prices, there's a tremendous amount of
 2   interest in doing it.  And, as you can see,
 3   and going back to some other discussions, if
 4   we started marketing in the United States all
 5   of the -- or 90 percent of the gross gas
 6   production in the lower 48 states, you could
 7   increase the production by 10 percent or 5
 8   percent, or 2 percent, depending on how big
 9   the pipeline was, and that would have a
10   significant market impact.
11             As you go down this list you notice
12   that most of the high-ranked natural gas
13   producers are also relatively high-ranked oil
14   producers.  Oil rank is in blue.  Number two
15   is Texas, number three is Alaska, and number
16   one is usually the federal offshore of Gulf
17   of Mexico.  Wyoming is seventh, probably
18   increasing, et cetera.  California, which is
19   not included in the current survey, is the
20   fourth-largest producer, and we would go on
21   down.
22             And there's one logic here for
0488
 1   breaking it off at around the 10th largest
 2   one.  It is about at 10 or 11 that these
 3   states are running at about 1 percent of the
 4   national total and that's another criterion
 5   that is sensible, that we'd like to know
 6   individually on those states with our own
 7   survey.
 8             I want to show three slides that
 9   show how we are doing with the 914 estimates.
10   This black line is our best current estimate
11   based on state production data.  It is a very
12   reliable estimate in 2004, and probably
13   pretty reliable in the first half of 2005.
14   And then may or may not trail off depending
15   on what state we are talking about.  And as
16   we all know there's a significant lag in
17   getting complete state report of production
18   from the state.
19             These four percent error bands were
20   drawn, and that was the average monthly error
21   in the published number for 1997 through
22   2002.  And so the number that EIA published
0489
 1   for New Mexico, the average absolute error
 2   was around four percent in that six-year
 3   period, on a monthly level.  And so as long
 4   as you're inside that level one could say
 5   you're doing pretty well, at least as well as
 6   you used to with the 914 survey.
 7             This blue line is what we actually
 8   published in the Natural Gas Monthly as our
 9   first estimate for New Mexico.  And you will
10   notice in '04 that we weren't doing terribly
11   well, and in June and July here we were low
12   by almost 20 percent, then came back up on
13   track, and basically we're staying inside
14   that 4 percent error band.
15             The green line is the estimates
16   made with the 914 data.  They are in
17   relatively close agreement with the estimates
18   made from the state data right now, and the
19   state data -- the black line estimates are
20   probably only good to within one or two
21   percent, so basically these are tracking
22   together rather well.  After mid-year the 914
0490
 1   data is probably more reliable.
 2             This is the Texas number.  My
 3   office actually makes these estimates, but we
 4   use the Crystal method, which was basically
 5   developed through the efforts of the ASA
 6   committee, and it is a reasonable time series
 7   approach, but like any time series approach,
 8   its reliability depends upon the assumptions
 9   that underlie it still holding.  In January
10   of '05 the state of Texas changed its
11   production reporting form, changed its
12   production collection system.  If we had used
13   the Crystal method in December, there would
14   have been a 12 percent error because it
15   disrupted the reporting back in '04 also.
16   And we went with a couple of empirical
17   estimates which we settled in on the one we
18   were going to use, and we're basically
19   staying inside.
20             Again for Texas our historical
21   record was plus or minus two percent.  And so
22   we were basically staying in there, and it
0491
 1   was a period of enormous change in the Texas
 2   data.  Among other things they corrected a
 3   process that had been over-reporting six
 4   percent of their production.  C02 that should
 5   not have been counted, was counted.  And they
 6   made the change starting in January '05, and
 7   it took just about to the end of '05 before
 8   everyone or mostly everyone was actually
 9   reporting that.  And due to the contact we
10   have with operators through the 914 program,
11   we were well aware of that problem and took
12   steps to start correcting it.  And, in fact,
13   the Texas 914 system was, therefore, adopted
14   as the official estimating series some months
15   ago.
16             MR. BINGHAM:  Why were there two
17   percent error bands instead of four percent
18   that you had for New Mexico?
19             MR. WOOD:  Because when you
20   compared the first published estimate made
21   about three months after the end of the
22   production month to final data, the average
0492
 1   absolute error was less than two percent in
 2   that state.  In Wyoming during that same
 3   six-year period the absolute average error
 4   was 18 percent.  And so this two percent is
 5   rather good, especially since it was based on
 6   very incomplete preliminary state data.
 7             Right here is the kind of thing
 8   that would have been -- if we got it right,
 9   it was just because expert guessing sometimes
10   is a reasonably reliable way to estimate
11   production.  There was a hurricane, it did
12   impact Texas, not like it impacted Louisiana
13   and the federal offshore Gulf, but instead of
14   the average half a percent change per month
15   it dropped six times that; it dropped three
16   percent.  And basically this was a very good
17   estimate of it, and then that production
18   returned.
19             This is even more important.  The
20   MMS official reporting of data has a year to
21   two year lag before you're up at 99 percent.
22   Of course everyone wants to know what is
0493
 1   happening.
 2             Given that lag, I think our track
 3   record from 1997 to 2002, including
 4   hurricanes, was an absolute average error of
 5   four percent for the federal offshore.  And
 6   that's what these red bands are.  And you see
 7   in '04 we generally stayed in there.  We were
 8   getting off a little bit.  And in fact we
 9   used the 914 data, and were comparing it, and
10   decided we'd better recalibrate the other
11   method of estimating the Gulf.  And when we
12   did so, that error -- again, we did not use
13   the 914 during this period of time, but we
14   did recalibrate the model we were using that
15   was based on preliminary well-level data.
16   When the hurricane came there was a 50
17   percent drop in production, and I assume that
18   we are certainly within a couple of percent
19   of getting that.  And again, there would have
20   been no way to reliably know that information
21   without the 914 survey.
22             I want to switch to how we actually
0494
 1   estimate the total production.  And what I
 2   also want to go over is in some sense within
 3   getting to the error bounds we want, this is
 4   a modeling process.  And if you lay it out
 5   that way it's easier to understand where
 6   we're going.  The total production is just a
 7   sum of the production from the surveyed
 8   operators plus an estimate of the production
 9   for the operators not in the sample.  In
10   Texas, for example, about 15 percent of the
11   production is from operators that are not
12   sampled, and that's around 5,000 operators,
13   whereas the number of operators that are
14   actually sampled is more like 140.
15             And I want to focus the attention
16   on how we estimate this number, the
17   production for the non-sampled operators.
18   And the simplest model, and the one that our
19   own SMG group was certainly most familiar
20   with and thought would be a good starting
21   point, was to simply calculate what the ratio
22   is in the calibration year of the non-sampled
0495
 1   to the sampled.  And then over time when
 2   we're trying to estimate one and two years
 3   out, just multiply that ratio times the
 4   production from sampled operators.  Now, that
 5   actually would get you usually within a few
 6   percent.
 7             There are some inherent problems
 8   with it.  One is that this implies for the
 9   one percent target that this ratio would be
10   constant over time and it is not.  And
11   secondly, something we tested and looked at,
12   are there groups of operators that are overly
13   influential, that are very large compared to
14   everyone else?  And Nancy had written a paper
15   on overly influential operators and we tested
16   it very thoroughly.  And as we'll see there's
17   a lot of logic to being very careful about
18   using the data from the overly influential
19   operators.
20             Out of roughly 140 operators, the
21   top 3 have the first quartile of production.
22   The top 3 operators have 25 percent of the
0496
 1   production.  And they do not actually behave
 2   the same way as all of the 5,000 operators
 3   that are not sampled, as you might expect.
 4             This is the actual behavior of the
 5   production of the non-sampled operators and
 6   you notice they grew from around 1.9 BCF a
 7   day up to over 2.4 billion a day.  This was
 8   an increase of around 17 percent.  If we use
 9   that ratio model then you get the black line
10   and -- which would -- again, you're assuming
11   that the ratio between non-sampled operator
12   production and sampled operator production is
13   constant, and it's not.  And so it just shows
14   it's flat.  So you underestimated this amount
15   of production.
16             The worst indicator was when you
17   broke it off and modeled it on just the first
18   quartile, the top three operators, and they
19   were going the other way.  The next worse was
20   then use the first two quartiles.  Again,
21   they were going the other way.  When we used
22   the lower 40 percent, they at least were
0497
 1   behaving in a somewhat similar way until the
 2   constant ratio was a better indicator.  It
 3   was the best of the subsets of the sampled
 4   data and straight ratio model, but it's not
 5   good enough.
 6             What we did do in the first model
 7   was to use a constant slope so the ratio
 8   would have been whatever it was in the
 9   calibration year plus a constant times time,
10   and it increased about a half a percent a
11   month.  But sometimes it did not increase,
12   and sometimes it increased 2 percent a month.
13   And whole years, the percent change would be
14   one year 5 percent and another year 20
15   percent, 2 years out.
16             And so we also added a drilling
17   parameter.  And basically when prices are
18   high and demand is high the activity to bring
19   on more gas supply increases, drilling goes
20   up a lot.  And that was a very good
21   correlator with the relative change of the
22   non-sampled operators compared to the sampled
0498
 1   operators.  And then we multiplied that
 2   variable ratio, variable slope, times the
 3   lower 40 percent.
 4             Now, I obsess over things like the
 5   following:  This is the ratio over a series
 6   of months.  You notice it is going up at a
 7   relatively strong, positive slope.  And so
 8   that ratio you had multiplied times your
 9   sample is increasing, and then it turned
10   around and it dropped.  And then it started
11   increasing rapidly again, and then it started
12   dropping again.
13             If that's what the data looks like,
14   you know you cannot model it with great
15   precision if you use a constant slope.  So we
16   look for obvious correlators, and the level
17   of drilling and the change in the level of
18   drilling seemed to work reasonably well.  So
19   this is the resulting model run.  This
20   particular curve was generated with basically
21   a two-parameter model for testing six
22   calibration years and 72 points that were
0499
 1   three years out.  And very small errors, and
 2   it does have a great deal of flexibility.
 3   The errors using this model are very small.
 4             If you want to sit around and play
 5   you can add another parameter and just
 6   actually make this come down and go up again.
 7   But you have to pay a lot of attention to
 8   make sure that it doesn't get out of control
 9   each time you add a new parameter.
10             We did for those 72 months -- and
11   again, those 72 months are basically three
12   years out from the calibration year, and this
13   time period starts at the beginning of
14   January of a calibration year and goes
15   forward.  So this is the start of the third
16   year.  And so we're looking at 24 months out
17   to 36 months out.  This is the ratio method,
18   and this was the best ratio method.  And it
19   had an absolute average error of 7.4 percent
20   and an average error of -7.4 percent.  It is
21   a very biased model.  The largest error was
22   -14.1 percent, and if you used just the total
0500
 1   sample, that error grew to 16 percent.
 2             Here we went to a constant slope,
 3   and that did two things:  One, the absolute
 4   error is 2.5 percent and the average error is
 5   0.4.  And you notice there's a fairly
 6   reasonable distribution of errors around the
 7   zero error line, and the largest error
 8   dropped from 14 percent to 6.7 percent.
 9             With the variable slope model the
10   average error was -0.1 percent, the average
11   absolute error was 1.4 percent, and the
12   largest error or the smallest error was -4.1
13   percent.
14             Now, that -4.1 percent is an error
15   in estimating the non-sampled production.  So
16   the error in the total estimate is much
17   smaller than that.  So for a test of six
18   calibration years, 72 monthly estimates in
19   the third year out, in Texas the largest
20   error in production for non-sampled operators
21   was 4 percent in one month.  Most of them
22   were quite a bit smaller than that.  And the
0501
 1   non-sampled operators have 15 percent of the
 2   total production.  So the largest error in
 3   the total production was 0.15 times 4
 4   percent, was 0.6 percent, the largest error
 5   in the two-parameter model for 72 months.
 6             And our conclusion is that only
 7   small errors result from the sampling and
 8   estimating modeling.  Significant errors can
 9   occur because of operator reporting errors
10   and initial model calibration, because
11   calibration year data at the operator level
12   may be incomplete.  Thank you.
13                  (Applause)
14             THE CHAIR:  Okay.  Thank you very
15   much, John.  I'd like to invite Dr. Burton to
16   comment on your talk.  Mark?  Press the
17   button.  Yes.
18             DR. BURTON:  I really have more
19   questions than I have comments.  Thank you.
20   Could be my own ---.
21             Given what I would guess is a
22   uniform desire to see monthly data and given
0502
 1   the circumstances you described, is there any
 2   alternative?  Is there any alternative to
 3   modifying the 914 that would compete in terms
 4   of a possible course for EIA?
 5             MR. WOOD:  Nein.
 6             MS. KIRKENDALL:  I don't think so.
 7   We've had this estimating procedure based on
 8   slow and not very accurate data for a long
 9   time, and for a while it didn't seem to
10   matter.  But if there's interest in high
11   quality data on production, we're probably
12   better off collecting it ourselves.
13             DR. BURTON:  Right.  It sounds like
14   the existing system is fragile at best and
15   scheduled to be discontinued.  So then the
16   question ultimately is, is the monthly data
17   something that's essential to both the EIA
18   programs and to your users, and after sitting
19   downstairs an hour or so ago, I have a
20   profound and newfound desire to know more
21   about petroleum.  I have a feeling that may
22   be common among a lot of people, so that -- I
0503
 1   mean in a sense, if that is data that we want
 2   and I believe it is, and if this is the best
 3   alternative, then I guess we really should
 4   say, well, gee, we're really lucky that we
 5   have this alternative.
 6             The rest of the questions I have
 7   really have to do more with just kind of some
 8   of the specifics.  When you did the gas, you
 9   said '93 was the calibration year?
10             MR. WOOD:  Well, the year that we
11   calibrated from was 2003.
12             DR. BURTON:  2003, I'm sorry.
13             MR. WOOD:  We tested over '97 to
14   2003 to build the model.
15             MR. CLEVELAND:  How often do you
16   anticipate having to recalibrate for the
17   existing gas and for oil if you do it?
18             MR. WOOD:  We plan to recalibrate
19   in a fundamental way once a year.  Now, we
20   have re-sampled from the 2004 data.  And we
21   have not put that model into production yet.
22   And in combination with some of our
0504
 1   developing methodology for estimating state
 2   level production using time series such as
 3   the Crystal method, we have the opportunity
 4   to start during a period and make sure that
 5   we had a correct calibration volume when we
 6   started.
 7             So almost monthly, starting about
 8   now for several states, we would start
 9   recalibrating the 2004 model on -- for
10   example in Texas and in Wyoming we would have
11   very good estimates now for the first month.
12   Well, certainly all through 2005 on the
13   volumes -- and we're talking about one or two
14   percent here.  So it would be almost a
15   continuous recalibration for small changes in
16   magnitude and a major calibration once a year
17   as you get a complete set of operator-level
18   data.
19             DR. BURTON:  I notice that the
20   overall number of sampled firms wouldn't go
21   up substantially above what you're doing now.
22   Would it largely be the same set of firms
0505
 1   that are included in the gas sample?  If you
 2   introduce oil, are you going to be
 3   introducing it to a set of respondents that
 4   are largely already familiar with the gas?
 5             MR. WOOD:  Yes.
 6             DR. BURTON:  I know OMB and others
 7   are concerned with how burdensome it is to
 8   the firm; I don't really care.  I just want
 9   to --------
10             MR. WOOD:  To be more precise,
11   every single one of the gas operators
12   currently picked would be included in the
13   combined survey.  And they make up more than
14   three quarters.
15             DR. BURTON:  But you would be
16   adding states presumably, or at least that's
17   the goal is to add states --
18             MR. WOOD:  Right.
19             DR. BURTON:  -- so that it's
20   possible then in moving to the larger firms
21   that as you put a new state into the process
22   or into the mix that you'd pick up firms that
0506
 1   don't currently participate.
 2             MR. WOOD:  That is true.  But in
 3   Colorado, which is not individually estimated
 4   now, we went in and checked, as we did
 5   actually in every state, and roughly 75
 6   percent of the production for Colorado comes
 7   from people already in the sample.
 8             DR. BURTON:  Already part of ----
 9             MR. WOOD:  They simply are not
10   asked to report individually for Colorado.
11             DR. BURTON:  The only other
12   question I have, and it looks like a really
13   fun area to explore, is the overall sort of
14   issue of the ratio of the non-sampled firms,
15   their production.  And you said something
16   that motivated the question.  You said that
17   the changes in that ratio were highly
18   correlated to drilling activity which was
19   probably highly correlated to changes in
20   price.  And what it sort of made me wonder is
21   whether or not the smaller producers, at
22   least in the states that you're looking at,
0507
 1   represent a lot of the sort of slack capacity
 2   that can be drawn into production when
 3   there's a price signal to motivate that.  Am
 4   I taking it too far?
 5             MR. WOOD:  A little.  The --
 6             DR. BURTON:  That's charitable.
 7             MR. WOOD:  And again, we had some
 8   discussion of this at the breakout session
 9   this morning.  We went from mid '86 to late,
10   like, 1999, the gas bubble and maybe a 25
11   percent overcapacity tended to go away.  And
12   so in most areas of the country there's very
13   little spare capacity.  So there isn't much
14   of that in what you're talking about.  I
15   think a better explanation is in good times
16   the small operators -- I mean, the vast
17   majority of the companies, by the way --
18   again, 5,000 or 5,200 in Texas -- have more
19   access to capital because they fund it out of
20   their income streams and therefore they can
21   afford to drill a lot more and their attitude
22   and inclination is to do so until they start
0508
 1   going broke again.
 2             Again, as you saw those curves, you
 3   can map it.  The drilling peaked, and when
 4   the curves were going up the steepest in the
 5   non-sampled operator production, and when the
 6   price cratered and the drilling cratered,
 7   they just turned around and had a negative
 8   slope.  They were doing worse than steady
 9   state during those periods of time.
10             And the big operators, the reverse
11   of that, they have great more access to large
12   amounts of capital to continue their program
13   even if the price isn't that great.
14             DR. BURTON:  They're not going to
15   be nearly as cyclical; they're not going to
16   move with price nearly to the same degree.
17             Nick, I've done the best I can to
18   get you back on schedule.  That's all I've --
19                  (Laughter)
20             THE CHAIR:  Thank you very much.  I
21   was going to have first the comments from
22   Derek, but I see that hands are going up, so
0509
 1   what I'm thinking is first let's -- if you
 2   can write down your question we'll come back
 3   to it if it's not disruptive.
 4             DR. EDMONDS:  Well, actually my
 5   question might be helpful in that -- I was
 6   just going to ask John, is there anything in
 7   particular you would like the committee's
 8   opinion that you think we might be helpful or
 9   is the basic -- is it to provide an
10   assessment as to whether the 914 is really
11   something we need to be doing and potentially
12   to concur with you and to add our weight to
13   that argument?  If you could just say
14   something to that.
15             MR. WOOD:  Yes, I think your formal
16   advice on what time this afternoon the
17   project should be funded would be helpful.
18                  (Laughter)
19             MR. WOOD:  As I said, there is a
20   consensus, and that consensus extends to the
21   administrator and the deputy administrator.
22   As far as this is a fine idea, funding it is
0510
 1   slightly different.  We are well along on
 2   negotiating how that might happen, that to
 3   avoid what I said we're at high risk of, and
 4   the reason I said that is because we are at
 5   high risk, is to start work on that as a
 6   recommendation to avoid that as your opinion
 7   of a high priority mission for EIA, I think
 8   would be very helpful because it's at the
 9   outside --
10             MR. CLEVELAND:  To that end, if you
11   look at the academic literature -- which is
12   what I'm most familiar with -- look at some
13   of the most important work done investigating
14   the relationship, for example, between supply
15   and price over the past 20 years, a lot of it
16   has used monthly oil production data.  So
17   from an academic researcher's perspective
18   it's provided a lot of, I think, really
19   crucial insights into what the dynamics of
20   that relationship are.  When you need monthly
21   data to answer some of the -- there are just
22   some important questions that annual data
0511
 1   cannot answer.
 2             MS. KIRKENDALL:  Well, there is
 3   monthly data available.  It's just available
 4   with about a two-year lag.
 5             MR. CLEVELAND:  Well, okay, fine.
 6   In the oil business, that's an eternity.
 7                  (Laughter)
 8             MS. KIRKENDALL:  Yeah, because --
 9             MR. WOOD:  The cycle is already
10   over.
11                  (Laughter)
12             MR. CLEVELAND:  It's just --
13             MS. KIRKENDALL:  Yeah, so you can
14   investigate the past quite well.
15                  (Laughter)
16             SPEAKER:  If you're into economic
17   history.
18             MR. WOOD:  For Oklahoma and
19   Louisiana, I think you might make that four
20   years.
21                  (Laughter)
22             THE CHAIR:  Would you mind having
0512
 1   Derek --
 2             DR. FEDER:  It's a follow-up of --
 3             THE CHAIR:  If it's a follow-up,
 4   we --
 5             DR. FEDER:  I just have a technical
 6   comment.  The Crystal method is a time series
 7   method that ---- ratio and the estimation of
 8   the ratio that's a time series.  In looking
 9   at your graph, it looks to me that what you
10   have here is a -- some of it could very well
11   be modeled by a basic structure or model with
12   a random slope.
13             Nancy actually gave a talk a few
14   years ago on ---- models in one of the JSMs
15   which I heard.  The model that you had
16   especially was the dt term.  I thought maybe
17   even a hurricane term could be useful.  I
18   just thought maybe a better model could
19   include the estimate somewhat, although the
20   estimate of the total is not too bad.  To
21   study the properties of the ratio as a time
22   series might be useful.
0513
 1             MR. WOOD:  Actually, I agree.  In
 2   fact, I hope you noticed we are doing that.
 3             DR. FEDER:  Yeah.
 4             MR. WOOD:  And to formally present
 5   it in various analytical guises would be
 6   useful.  Also, when you say a hurricane term,
 7   what did you actually have in mind?
 8             DR. FEDER:  It's like the dt term
 9   you had that went back to demand.
10             MR. WOOD:  That was the basic
11   drilling level.
12             DR. FEDER:  Yeah.  I think the
13   presence of an intervention term could -- we
14   know hurricanes trigger some changes in the
15   series.
16             MS. KIRKENDALL:  Like a dummy
17   variable.
18             MR. WOOD:  Okay.
19             DR. FEDER:  It's a stand --
20             MR. WOOD:  And so then you'd want
21   to know that the actual occurrence of a major
22   hurricane which took out 50 percent of the
0514
 1   production --
 2             DR. FEDER:  ------ just the ratio.
 3             MR. WOOD:  Actually, for a
 4   substantial period of time did that change
 5   the relationship between small operators
 6   and --
 7             DR. FEDER:  Between the sampled and
 8   non-sampled, yeah.
 9             MR. WOOD:  Right.  Now, we have not
10   published anything on that, but we have
11   certainly looked at it.
12             DR. FEDER:  Okay.
13             MR. WOOD:  Is that something you'd
14   like to see it investigated and formally
15   reported?
16             DR. FEDER:  It might be useful; it
17   might improve the accuracy of the prediction
18   for the ratio.
19             MR. CLEVELAND:  He's suggesting put
20   it as a formal term in your model.
21             DR. FEDER:  Yes.
22             MR. CLEVELAND:  So for most months
0515
 1   it would have no impact?
 2             MR. WOOD:  I can't say.  I'm just
 3   trying to think of a kinder way to say this.
 4                  (Laughter)
 5             MR. CLEVELAND:  Don't be kind.
 6             MR. WOOD:  I believe it would be
 7   only of academic interest.
 8             DR. FEDER:  Oh, really?
 9             MS. KIRKENDALL:  I'm not sure that
10   it's not significant.
11             MR. WOOD:  In the way we've looked
12   at it is that the large companies behave
13   differently than the small companies in the
14   presence of a hurricane.
15             MR. CLEVELAND:  Yeah.
16             MR. WOOD:  And basically not enough
17   to move anything to one percent.  And, in
18   contrast, the behavior of the sampled
19   companies in the hurricane was a very good
20   predictor of how the non-sampled operators --
21   so in fact we were quite curious about that
22   ourselves, but in the Gulf of Mexico where
0516
 1   the data is the cleanest and the best, we
 2   have a 98 percent production weighted sample.
 3   And so effectively you were looking at
 4   everybody when you looked at the sample.
 5             DR. FEDER:  Okay.  Right.
 6             MR. WOOD:  And so it's a small
 7   effect even where you have smaller samples,
 8   but that it's not a big effect, even --
 9             DR. FEDER:  Okay.  Well, my main
10   comment was just on the model that you had
11   for the ratio to maybe modify --
12             MR. WOOD:  And there are other
13   things.
14             DR. FEDER:  Yeah.
15             MR. WOOD:  Yes, and in fact, again,
16   that ratio was up there to work on all 6
17   years and all 36 months for each of the base
18   calibration years, and we used two
19   parameters.  When I just played with one year
20   at a time, I can make it follow those curves,
21   and then you can physically interpret what is
22   going on and who causes it.
0517
 1             DR. FEDER:  Okay.
 2             THE CHAIR:  Derek?
 3             MR. BINGHAM:  I'll jump in
 4   actually, because my question were related to
 5   this.  Just out of curiosity, when you showed
 6   that plot where you went out like 48 months,
 7   and so you had I guess the base year, the
 8   calibration year, you created a ratio and you
 9   said, okay, let's say it's constant, and you
10   moved forward.
11             MR. WOOD:  Correct.
12             MR. BINGHAM:  And then after a
13   certain period of time it doesn't look good.
14   But if you were to update every year, of
15   course successfully -- you said you would
16   recalibrate every year -- does it actually
17   look that bad?  Because the one-year window
18   that you had with all of the methods didn't
19   do so bad, right?
20             On your figure three it shows four
21   years later it's terrible, because those
22   ratios are changing.  But on a one-year
0518
 1   window they all look kind of clumped up, and
 2   that could be because of a visual thing.
 3   What happens with the plot, after four years
 4   they're really far apart, so they look
 5   relatively close together with the one-year,
 6   but --
 7             MS. KIRKENDALL:  The problem is
 8   that you use a ratio based on a calibration
 9   year two years later, because the calibration
10   year is two years in the past from the time
11   you're starting to collect data.
12             MR. BINGHAM:  So the answer to, I
13   think, one of the previous questions was that
14   you recalibrated every year, and I thought
15   that that meant -- because for the one ------
16             MR. WOOD:  You changed the
17   calibration year.  You moved the calibration
18   year forward one year --
19             MR. BINGHAM:  Right.
20             MR. WOOD:  -- and then you hold
21   that, but it's during a two-year period.  So
22   when we calibrated against '03, the year we
0519
 1   were actually predicting for was '05.
 2             MR. BINGHAM:  Okay.
 3             MR. WOOD:  On the other hand, one
 4   of the improvements to the modeling is to do
 5   that continuous interim calibration.  And
 6   again in Wyoming we know within tenth of a
 7   percent within six months, going to the
 8   detailed state data, which you can verify at
 9   the well level, and then if you're running a
10   little low or high you can simply multiply
11   and change your calibration slightly and be
12   dead on.  And so the opportunity to do
13   recalibration during the process is a very
14   good idea.  And it's almost the last place
15   where we can add a significant improvement to
16   the model.
17             Now, in answer to your direct
18   question, the constant slope variable we used
19   in there again covered all the monthly
20   changes for two years.  And on average, it
21   was about half a percent a month.  So over a
22   two-year period you're down a couple of
0520
 1   percent.
 2             Generally -- and we haven't
 3   formally studied this, but it looks like the
 4   first year out, if you calibrated off of
 5   2003, the 2004 constant slope -- it might
 6   have been 0.4 percent a month, and in the
 7   2005 year it might have been 0.6.  And so, in
 8   fact, when we go back in and add a parameter,
 9   do one year out with one constant slope and
10   then two years out with another constant
11   slope, it knocks another percent or so off of
12   the largest error, etcetera.  And in fact we
13   will probably do that:  Instead of using one
14   constant parameter for two years we will use
15   two parameters for one year at a time in our
16   next recalibration cycle.
17             MR. BINGHAM:  Okay.
18             MR. WOOD:  And by the way, an
19   immediate answer that you're off by a percent
20   or two by December one year out.  And so it's
21   almost a linear increase in error.
22             MR. BINGHAM:  Okay, I'm going to
0521
 1   guess that that's what motivated the
 2   regression model.  So let me ask you this.
 3   The RT -- I didn't quite understand how it's
 4   used -- is it still multiplied by -- so what
 5   you do is you estimate a ratio at a given
 6   time.  Do you multiply that by the total for
 7   the sampled at that point, or is it --
 8             MR. WOOD:  You can, and the one we
 9   use -- and can we throw it back up there?
10   You notice it's multiplied --
11             MR. BINGHAM:  Twenty-two.
12             MR. WOOD:  -- by a parameter that's
13   labeled S sub L.
14             MR. BINGHAM:  Yeah.
15             MR. WOOD:  And that was the lower
16   40 percent.  The smaller operators had made
17   up 40 percent ---- which again was the large
18   majority of the operators, but they were much
19   smaller than the majors.
20             MR. BINGHAM:  Okay.  And this is
21   giving you your ratio?
22             MR. WOOD:  Right.
0522
 1             MR. BINGHAM:  Which is then --
 2             MR. WOOD:  This just says -- and I
 3   think Nancy described it once -- basically
 4   when you're in the calibration year you have
 5   a certain coverage, the ratio, let's say 85
 6   percent of the sample.  And what all the data
 7   shows is that you lose that coverage over
 8   time, and so that the non-sampled operators,
 9   the smaller, the operators that make up the
10   smallest 15 percent of the production grow in
11   relative production over time, so that if you
12   do not change the ratio to grow, you're
13   multiplying it times a smaller percentage all
14   the time.  And if it's constant, then you get
15   a biased low answer like you saw for all the
16   constant models.  If you increase the ratio
17   directly in proportion to the loss of
18   coverage from a sample, then you get an
19   accurate estimate.
20             MR. BINGHAM:  Right.  So let me ask
21   you this.  One of the reasons the constant
22   slope approach doesn't work is because it's
0523
 1   not as highly correlated with the bottom 40
 2   percent or at least -- maybe I should say
 3   this differently.  What you found is that
 4   there seems to be more correlation among that
 5   un-sampled total with the bottom 40 percent
 6   than with the whole --- everybody together.
 7             MR. WOOD:  Absolutely.
 8             MR. BINGHAM:  So why not build your
 9   ratio estimator or regression-based ratio
10   estimator simply by using SLT as your
11   multiplier?  Because what you're doing, it
12   seems if I understand you correctly, is
13   you're basing this RT, then you're
14   multiplying it by STT, the S -- the total at
15   time T.  And so maybe the thing that's really
16   correlated, the thing you want to base here
17   your entire -- and hat -- I'm throwing out
18   all this stuff from the minutiae, but maybe
19   you want to base your estimate of the total
20   entirely on SLT there, and building a ratio
21   estimator for it.  So in your estimate of the
22   total production, you've got S of T plus and
0524
 1   hat of T, right?  And the -- so --
 2             MR. WOOD:  I was thinking that's
 3   what I did.  If you look at this and you zero
 4   out A and you zero out B --
 5             MR. BINGHAM:  Right.
 6             MR. WOOD:  -- then you've got a
 7   constant ratio, and you are multiplying it
 8   times --
 9             MR. BINGHAM:  But you've got R of T
10   on the left-hand side.
11             THE CHAIR:  Derek, Derek, it's --
12             MR. BINGHAM:  Oh, that's the N of
13   T?
14             THE CHAIR:  Yeah.
15             MR. BINGHAM:  Okay.  All right.  I
16   just want to make sure.
17             THE CHAIR:  Misnomer, misnomer.
18             MR. BINGHAM:  Okay.  Because the
19   left-hand side is the wrong thing.  Is that
20   what you're telling me?
21             MS. KIRKENDALL:  Yeah.
22             THE CHAIR:  This "R" should be an
0525
 1   "N."
 2             MR. BINGHAM:  Because you said it's
 3   the variable ratio, not the --
 4             MS. KIRKENDALL:  Yeah, because
 5   you've multiplied your ratio.  If you didn't
 6   adjust that --- that would be a ratio.
 7             THE CHAIR:  It's a small thing.
 8             MR. BINGHAM:  Okay.
 9             MS. KIRKENDALL:  Yeah, that would
10   make the difference.
11             MR. WOOD:  I feel like -- the
12   left-hand side is missing the volume of the
13   sample and the R sub T represents those items
14   in the programs.  From here to here.
15             MR. BINGHAM:  That's your ratio?
16             MR. WOOD:  What I described.
17             MR. BINGHAM:  All right, okay.
18   That's actually what I wanted to know.
19             MR. WOOD:  I really wondered what
20   you were talking about, and we were talking
21   about me not writing the right formula on the
22   board.
0526
 1             MR. BINGHAM:  Yeah I didn't -- so
 2   all right.  So I had no problems.  My other
 3   question was more like what Moshe was saying,
 4   essentially.  There are a lot of different
 5   time series models that one can build, so I
 6   guess that what you seem to be doing is quite
 7   reasonable now that I understand what this
 8   is.  And so I guess I'll finish up there, I
 9   guess, because I think that one could look at
10   a lot of different time series models for
11   that ratio.
12             MR. WOOD:  And in fact we are
13   playing with a fairly large set of them all
14   the time.  In fact, the original
15   formulations, we could use one operator or
16   all of the operators in the models and then
17   we also run a large number of other time
18   series analysis, and we're always comparing.
19             MR. BINGHAM:  Actually a last
20   question.  Are the "A"s and "B"s in this
21   model dynamic or are they fixed at a
22   calibration year as well?
0527
 1             MR. WOOD:  At this time, we fix
 2   them and leave them in the next cycle which
 3   we haven't presented to the review committee
 4   yet.  They would be slowly adjusted over
 5   time.
 6             MR. NEERCHAL:  It would be like one
 7   of Moshe's ------
 8             MR. WOOD:  Yes.
 9             THE CHAIR:  There's the option of
10   more comments from the committee.  Yes?
11             MR. CLEVELAND:  How was EIA
12   collecting data on gas and oil from
13   unconventional sources?  Do you collect it?
14   Is it reported?
15             MR. WOOD:  As a specific item?
16             MR. CLEVELAND:  Yeah.
17             MR. WOOD:  There's one that we
18   collected specifically, and it's probably one
19   of the most important ones to date, and that
20   was collected on the EIA-23 reserves form.
21   And we collect coal bed methane, which has
22   grown from essentially nothing to more than
0528
 1   10 percent of the U.S. production.
 2             Now, when you say "collect," we
 3   don't collect it with our own survey tool.
 4             MR. CLEVELAND:  Right.
 5             MR. WOOD:  We do identify the
 6   fields and reservoirs that produce this and
 7   do some analysis on it.
 8             MR. CLEVELAND:  I'm just thinking
 9   over the last couple days how many times
10   we've said how important these different
11   forms of gas and oil come online, how that
12   will affect X, Y, and Z, and so analytically
13   it would be important to have that
14   information as it unfolds in the future.  I'm
15   sure you guys are thinking about that, but it
16   would be pretty important to have.
17             MR. WOOD:  I heartily agree, and on
18   your second budget recommendation, we are
19   actually going to publish a short article in
20   roughly a month and put it on "what's new" or
21   something on the EIA web that talks about the
22   impact of production from oil shales on U.S.
0529
 1   production and reserve.  And that is done by
 2   supplementing the data reported to us
 3   directly with other sources of data and
 4   direct data from the state, direct data from
 5   the operators, and, following all of that, we
 6   would probably very shortly thereafter do the
 7   same type of thing for the shale production,
 8   the unconventional gas production from the
 9   shale, which actually is substantially more
10   important right now as a percentage of
11   production.
12             MR. NEERCHAL:  John, you said you
13   are pitching this to OMB as a modification to
14   914.  And then you mentioned about respondent
15   burden, and I'm wondering how much more
16   expensive this modification is going to be?
17   Why would OMB say no?
18             MS. KIRKENDALL:  My guess is they
19   won't say "No."  Because the sample is still
20   within what we told them we'd have in the
21   beginning.  It does double the burden because
22   you're asking them more than twice as many
0530
 1   questions.  But the number of companies
 2   doesn't change from what we already have
 3   approval for.
 4             MR. WOOD:  In fact, I would be
 5   cautious in saying it doubles the burden.  I
 6   would say it would be more like increases the
 7   burden by five percent or something.
 8             MS. KIRKENDALL:  You get it from
 9   twice as many data points.
10             MR. WOOD:  They collect all --
11   well, if you --
12             MS. KIRKENDALL:  The burden's
13   measured by how long it takes them to fill
14   out the form.
15             MR. WOOD:  Right.  Yet it's how
16   that this is calculated.  Number of data
17   elements per respondent is OMB's --
18             MS. KIRKENDALL:  It's supposed to
19   be length of time for each respondent to fill
20   out the form.  That's what the burden hours
21   are to them.
22             MR. WOOD:  Still, if you double the
0531
 1   number of states individually reported, and
 2   just did gas, it would be a very small
 3   increase.  Now, we're also -------
 4             MS. KIRKENDALL:  You're not the one
 5   to say that; the respondents are the ones to
 6   say that.
 7             MR. WOOD:  Yeah, but -- well, as I
 8   would repeat, in the site visits we made,
 9   many of them ask in the first place why
10   aren't you collecting the oil.  Most of the
11   time --
12             MS. KIRKENDALL:  --------- -- for
13   anything you need to do --
14             MR. NEERCHAL:  ------- down if you
15   include the ID variables in the denominator.
16             MS. KIRKENDALL:  Well, that doesn't
17   change.
18             MR. NEERCHAL:  Yeah I --
19             MS. KIRKENDALL:  What OMB does,
20   what we have to do, is put out a Federal
21   Register notice saying that we're planning
22   this change.  And if we get lots of
0532
 1   complaints then they aren't going to be
 2   liking it very much.  But if nobody
 3   complaints, or few people complain and we can
 4   argue that it's -- then the industry hasn't
 5   said they don't like it and so it's much
 6   easier.
 7             So if industry thinks we should be
 8   collecting it, particularly the companies
 9   that report to you, there really would not be
10   a problem.  If they don't want to report it,
11   then there's a problem.  But we've usually
12   gotten our clearances through.  There have
13   only been a few that haven't.
14             MR. NEERCHAL:  Okay --- last ---
15   issue up at the list.
16             THE CHAIR:  Good.  Thank you very
17   much.
18             At this time I'd like to invite the
19   general public questioning, silent so far, to
20   voice any questions, comments, that might
21   have occurred during this entire proceeding.
22                  (No response)
0533
 1             THE CHAIR:  There's no general
 2   public, is there?
 3             MS. KIRKENDALL:  Yes, there is.  We
 4   have one general public.
 5                  (Laughter)
 6             MR. POLUN:  I have two brief
 7   comments.  I guess the first if I may is I'd
 8   just like to thank --
 9             SPEAKER:  Would you identify
10   yourself, sir, please?
11             MR. POLUN:  Sure.  Yeah.  My name
12   is Jason Polun.  I work in the private sector
13   for an investment management firm that from
14   time to time may or may not invest in
15   publicly traded energy companies.
16             So the first -- really I just want
17   to thank you.  Everyone today I've met has
18   been so gracious and so hospitable, I'm
19   really glad I made the trip and I really
20   enjoyed myself.  So thanks a lot.
21             The second is, I really think you
22   can't overestimate the value that you bring
0534
 1   to the private sector, basically the
 2   financial markets.  Any way that you increase
 3   transparency -- we're talking about a
 4   commodities here, right?  Oil and gas.  And
 5   the way that they're traded is basically on
 6   demand and supply and the fact that you
 7   collect the data and help us understand
 8   supply helps us make the best decisions
 9   possible which ultimately helps our clients
10   and shareholders.  So, Dr. Burton, when you
11   said, "Should we collect the data?" my heart
12   almost stopped, right?
13                  (Laughter)
14             MR. POLUN:  So I think you do a
15   wonderful job and anything that you do to
16   increase that transparency is very helpful.
17             And two examples of that today,
18   first a very small example, the fact that you
19   keep this meeting open to the public is
20   great, that the process is transparent.  In
21   that regard it's like you had me at hello.
22   It's a great thing.
0535
 1             The second piece is EIA-914.  I
 2   think that has helped with the timeliness of
 3   the information and I think it's beginning to
 4   trickle into the financial markets, and it's
 5   creating a buzz, that, look, the EIA is doing
 6   this, it's out there.  And I saw a research
 7   note today actually that someone put out, an
 8   energy cell site firm who said, "Hey, did you
 9   guys see the EIA is doing this?"  So the
10   message that I want to send to you is, A,
11   thank you very much and B, please keep doing
12   what you're doing.  It's very helpful from
13   our end.  So thank you.
14             THE CHAIR:  Thank you.
15             MR. CLEVELAND:  Can we ask your
16   brother, John?
17                  (Laughter)
18             THE CHAIR:  I was going to ask ----
19             MR. WOOD:  Younger.
20                  (Laughter)
21             THE CHAIR:  -- in writing.  I think
22   the EIA would appreciate those kind of
0536
 1   feedbacks from time to time, especially at
 2   budget time.
 3             MR. POLUN:  Yeah, absolutely.
 4   Sure, I'll talk to you after 2:00.
 5             THE CHAIR:  Thank you very much.
 6   Any more comments?
 7             At this time I'd like to ask the
 8   committee to voice their interest about
 9   possible topics for the next meeting which
10   occurs in six months, mainly suggestions,
11   topics you'd like to hear that you haven't
12   heard this time around or on topics that
13   you'd like to follow up.  I assume that the
14   modelers will ask for more modeling --
15                  (Laughter)
16             THE CHAIR:  -- since this seems to
17   have been very successful this time around.
18             MR. RUTHERFORD:  I don't mind which
19   model they have, but I just request that the
20   contractors building the model should be here
21   to talk to us.  Any model they want to bring
22   is fine but I'd like to talk to the people
0537
 1   actually doing the model.
 2             MR. CLEVELAND:  I'd like to see the
 3   world oil -- the SAGE thing in more detail.
 4   It would be a good thing to come back to.
 5             MS. KHANNA:  And the oil production
 6   model that's lying behind SAGE.
 7             SPEAKER:  Yeah.  Lay out there.
 8             DR. BURTON:  We spent a lot of time
 9   over the last couple of days talking about
10   oil and gas, and I would be remiss as a
11   former West Virginian to remind you there's
12   another fuel.
13             THE CHAIR:  Coal.
14                  (Laughter)
15             MR. WOOD:  Thank goodness it can be
16   gasified and liquefied.
17                  (Laughter)
18             THE CHAIR:  Yes.
19             DR. EDMONDS:  Along the same lines,
20   there's a very interesting area that's
21   opening up in the form of the LNG trade that
22   we're sort of sitting on the edge of, and
0538
 1   then a little bit more on this notion of the
 2   unconventional fuels and how they really
 3   potentially affect the way we collect and
 4   understand what's going on in the energy
 5   business.
 6             MR. CLEVELAND:  Another possible
 7   topic would be the modeling of how
 8   achievement of wind energy and the
 9   integration of wind energy into the grid, and
10   the interesting problems that poses when you
11   start adding which is our fastest growing
12   source of electricity in the world other than
13   gas.  How you integrate an intermittent
14   source into the current grid and regulatory
15   structures is a really critical issue.
16             THE CHAIR:  So you're looking more
17   in how this is done physically or how we
18   report data, how we collect data, how we
19   model this energy?
20             DR. EDMONDS:  This is more of a
21   modeling issue, I think.
22             MR. CLEVELAND:  It's a modeling
0539
 1   issue.
 2             MS. KIRKENDALL:  ------
 3             DR. EDMONDS:  It's fundamentally
 4   different than modeling a base load polarized
 5   coal plant coming online.  You've got a
 6   higher variance term.
 7             MR. CLEVELAND:  And it gets
 8   penalized in the current way in which we do
 9   it because of its intermittence, and that's
10   simply in part due to the fact that there are
11   real costs associated with its intermittence,
12   but also the way in which we tend to treat it
13   in our modeling and analytical frameworks,
14   that intermittent is bad.  Anyway, it's an
15   interesting topic, I think.
16             THE CHAIR:  Neha?
17             MS. KHANNA:  Yeah.  If we do go
18   with wind energy I think we've got a lot of
19   other issues apart from just modeling.  One
20   other issue would be siting.  New York with
21   its renewable energy portfolio is pushing
22   wind energy in a big way, but they're having
0540
 1   huge problems trying to site the wind towers.
 2   Classic NIMBY is, it ruins my view, it ruins
 3   my property values, I don't want it.  We had
 4   a huge public outcry right out in Ithaca, a
 5   sleepy town which nobody knows about except
 6   people that live there, but it was a huge
 7   issue.
 8             John, I was very happy you
 9   mentioned coal gasification because I think
10   there's a lot of potential there.  What's the
11   potential for it?  And do we have information
12   on that?  Is the EIA looking at it seriously
13   in terms of -- or actually more than coal
14   gasification, even liquefaction of coal as a
15   transportation fuel?
16             THE CHAIR:  The one item I'd like
17   to add to the agenda next time is looking at
18   the recommendation of the review committee,
19   right?
20             MS. KIRKENDALL:  Yes.
21             THE CHAIR:  That review committee
22   will give us a lot of food for thoughts.  It
0541
 1   may be that that report that they will
 2   generate may suggest themes for us to look
 3   at, and I just want to make sure that this
 4   will be prominently on the agenda since I
 5   think we should take that seriously.
 6             MS. KIRKENDALL:  Yeah, that's a
 7   good idea.
 8             MR. CLEVELAND:  I'll add another
 9   one.  That is nuclear power.  In a carbon
10   constraint world I want -- last year from my
11   energy class I teach, I was getting
12   information on levelized costs of power
13   generation.  I asked I don't know who it was
14   at the EIA to give me that data and they
15   said, "Well, we don't -- I can't give it to
16   you because we don't calculate it because no
17   one's planning to build nuclear power
18   plants."
19             THE CHAIR:  Not true anymore.
20             MR. CLEVELAND:  No, but anyways, it
21   raises the issue of how EIA is looking at the
22   issue of nuclear power, what information do
0542
 1   we have -- I mean the nuclear industry makes
 2   these claims, but I think our -- many of them
 3   are true about how that their power plants
 4   are more reliable and how the newer plants
 5   would cost less and so on.  And so it would
 6   be interesting to know how EIA is looking at
 7   a possible renaissance in nuclear power,
 8   particularly since our President is
 9   encouraging it.
10             THE CHAIR:  I think that looks like
11   an exciting meeting ahead.  Alternative
12   energies, nuclear --
13             MS. KIRKENDALL:  Wind.
14             THE CHAIR:  Wind, yes.
15             DR. BURTON:  Coal.
16             THE CHAIR:  And coal.  We won't
17   forget Mark.  I thank you, everyone who has
18   participated either as a presenter or as a
19   commenter or as a discussant in these
20   successful meetings.  I'd like to adjourn
21   them for now.  I want to remind the committee
22   members, those who have not had a receipt for
0543
 1   the dinner last night, I have photocopies
 2   here.  You're welcome to take them.  And I
 3   wish you all a safe trip home.  Those of us
 4   who want to stay and have lunch, there's
 5   lunch downstairs on the first floor where we
 6   had lunch yesterday.  And this ends these
 7   proceedings.  Thank you very much, everyone.
 8                  (Applause)
 9                  (Whereupon, at 11:53 a.m., the
10                  PROCEEDINGS were adjourned.)
11                      *  *  *  *  *
12   
13   
14   
15   
16   
17   
18   
19   
20   
21   
22