AMERICAN STATISTICAL ASSOCIATION (ASA) MEETING OF THE AMERICAN STATISTICAL ASSOCIATION COMMITTEE ON ENERGY STATISTICS WITH THE ENERGY INFORMATION ADMINISTRATION (EIA) DAY TWO Washington, D.C. Friday, October 6, 2006 2 1 P R O C E E D I N G S 2 (8:41 a.m.) 3 MR. HENGARTNER: Good morning, 4 ladies and gentlemen. Welcome back to 5 another exciting session of the ASA Committee 6 to the EIA. And I guarantee you that this is 7 going to be the last time I'm going to chair 8 this session. So -- 9 MR. BURTON: Well, you fired me 10 yesterday. 11 MR. HENGARTNER: I know. You, out. 12 Go. 13 This morning, we actually have some 14 really cool stuff on the program. Harry 15 Vidas is another contractor, and what EIA 16 wants us to do is exactly look and interact a 17 little bit with the contractors, because they 18 actually provide some valuable input to the 19 EIA. And it's a part that we usually don't 20 see much. And so this is an opportunity for 21 us to see another side of EIA. And as such, 22 I think it's a great opportunity. And so I BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 3 1 would like to welcome Harry Vidas to the 2 podium. 3 SPEAKER: Might be a good chance to 4 just -- 5 MR. HENGARTNER: Oh, yes. Before 6 we start, I would like everybody who had not 7 signed in, or rather, said their name aloud 8 in the microphone, to do so at this time just 9 for the record. 10 MR. STAUB: John Staub, EIA. 11 MS. MAYNE: Lauren Mayne, EIA. 12 MR. CLINE: Matt Cline, EIA 13 contractor. 14 MR. LADDY: George Laddy, 15 contractor. 16 MR. VIDAS: And Harry Vidas, 17 contractor as well. Good morning, everybody. 18 My name is Harry Vidas. And as the 19 introduction said, I'm a contractor that 20 occasionally does do work for EIA. And what 21 I want to do today is talk about a project 22 that we did earlier this year to look into BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 4 1 the possibility of improving the way that the 2 SAGE model represents non-OPEC oil 3 production. 4 The objectives as were given to us 5 was first to provide a detailed description 6 of conventional oil resource base, with the 7 ability to look at alternative resource base 8 assumptions. Number two was to develop a 9 bottom-up estimate of oil discovery, 10 development, and production cost, and then 11 having the ability to alter that based on 12 assumptions about future upstream technology 13 advances. 14 Number three is to create an 15 algorithm that could go into SAGE to allow a 16 price response to non-OPEC production 17 forecast for sensitivity analysis, and for 18 the regular AEO production forecast. And we 19 were asked do it in a way that it should be 20 understandable, traceable, verifiable, and 21 unburdensome to update. 22 So the scope of work that we had BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 5 1 included the following steps that we decided 2 to try to do. Number one, we had already 3 developed a way of looking at oil resource 4 and natural gas resource base costs to add 5 into the future using what we call the World 6 Assessment Unit model which I'll describe in 7 a minute. So that gave -- we already had an 8 ability to look at the long run, what we call 9 the resource cost of oil and gas production. 10 And then the second thing was to 11 try to -- was to develop a standalone model 12 in Excel that would look at the annual 13 forecasts of oil production as a function of 14 the variables we talked about: Price, 15 technology, and so on. 16 And then what we wanted to do was 17 to be able to extract from that -- it's 18 called Proof of Concept Model, the parameters 19 that could then go into SAGE, and to give EIA 20 a report describing the data methodology and 21 so on for how to do this, and also what the 22 data were. The idea was that it will be BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 6 1 something that EIA would have to work with 2 and evaluate and test. And then they would 3 have the decision to make as to whether or 4 not that algorithm or some variation of that 5 was worthwhile to put into SAGE. So our work 6 did not include actually working with the 7 SAGE model, but to do this Proof of Concept 8 outside the model in Excel. 9 We did the work, I guess, starting 10 earlier this year. We finished up the model 11 itself back in February; did a preliminary 12 version of it, did some testing on it. And 13 then produced the final version in March. 14 And then we delivered a report in April, and 15 that was finalized in May. And those of you 16 who have not seen it yet, it's available on 17 the website. And also I think there is some 18 printouts out in the front of the report. 19 And the first part of this is what 20 we call the WAU model. And what this is is a 21 model that EIA developed back in 2004 22 primarily to look at long-term natural gas BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 7 1 supplies around the world and the ability to 2 supply the U.S. with LNG. But at the same 3 time, it also does the resource cost 4 calculations for oil, for conventional oil 5 production. And what this is is what we call 6 a kind of bottoms-up calculation of resource 7 cost looking at what we call a discovery 8 process model that has within it an 9 assumption about how many main fields there 10 are, the field-size distribution of those 11 fields, and then development cost for those 12 fields. 13 And we used algorithms that we 14 developed in the past in looking at U.S., 15 Canadian, and other oil and gas resources 16 around the world. So we developed these 17 costing functions that said here is how much 18 it costs to drill a well on the onshore at a 19 certain drilling depth, and here is what it 20 costs for offshore by different water depths 21 and different drilling depths and so on. 22 And therefore, these algorithms BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 8 1 were then put into this model with a detailed 2 resource base assumption that came from the 3 USGS Year 2000 world resource assessments, 4 with some modifications that I'll talk about. 5 So therefore, the -- we did -- this 6 provided EIA then with the bottoms-up 7 calculations that they were looking for that 8 allowed them to vary resource base 9 assumptions like field size distributions, 10 and to do a very detailed bottom-up cost 11 calculation based on distributions of field 12 sizes, locations, onshore/offshore water 13 depth, drilling depth. The finding -- that 14 is to say, the drilling success rates as well 15 as the financial parameters in terms of 16 internal rate of return criteria, royalty 17 rates and so on. 18 And this was done at what's called 19 the assessment unit level, which is what the 20 U.S. Geological Survey used to do their world 21 resource assessments, and there are about 520 22 of these assessments units around the world. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 9 1 And they represent individual basins, and in 2 the some cases, certain formations within 3 those basins. And then those WAU curves are 4 then aggregated up into country curves. So 5 the 520 WAU units are then added up into 6 about 92, I think, countries that we have in 7 the model. 8 Now, that is the first piece of the 9 puzzle, which is to say, at any given price 10 in any given technology assumption, how much 11 oil and gas is economic at a certain price. 12 And the second part of the puzzle is, well, 13 how do I then use that to forecast future 14 production? 15 And what we came up with was a kind 16 of a modification of the Hubbard curve 17 methodology. I sometimes call it Hubbard 18 Light in that it's not really using all of 19 the assumptions and all of the underlying 20 assumptions and beliefs that go into a 21 Hubbard curve. All it's really doing is 22 trying to use some kind of function to BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 10 1 forecast long-run production such that the 2 area under the curve is equal to the entire 3 resource base. 4 Because what we are trying to do in 5 the first step with the WAU model is answer 6 the question, at a given price, how much oil 7 is economic? And then we are using this part 8 to say, well how does that get produced over 9 time? And even though the SAGE model doesn't 10 have a 100-year time frame, we wanted to 11 include in this model a very long time frame, 12 because we wanted to play out what that 13 resource base assumption means for a long-run 14 production for purposes of looking at 15 consistency, logic. And also because people 16 want to know what the long-run potential is 17 for conventional oil production. 18 So the basic idea behind this is, 19 as I said, the Hubbard curve, and within that 20 Hubbard curve, there's a lot of different 21 ways of mathematically -- I don't want to go 22 through all of the details here. But the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 11 1 basic, simple calculation is the production 2 rate, annual production rate, divided by the 3 cumulative production, is a linear function, 4 with an intercept and a slope. 5 And by starting with that 6 assumption, you can develop all sorts of 7 corollary functions that go along with that. 8 They can then be manipulated in that model. 9 And without going into the mathematics, what 10 conceptually we are trying to do is we are 11 saying, well, we know the historical 12 production rates. And we have recent 13 last-year production rates. We have from 14 reserve estimates and from the WAU model an 15 estimate of how much would be economic to 16 develop at a given price. 17 And what we were trying to do in 18 essence is trying to draw a curve with that 19 resource base that starts at the historical 20 production rate that kind of looks like a 21 smooth Hubbard curve from now forward. So 22 therefore, the Hubbard curve aspect of this, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 12 1 or the logistic curve, which also what it's 2 called, is using -- is trying to develop the 3 shape of the future production, whereas the 4 WAU model is being used to look at the 5 economics of how much gas and oil is -- or in 6 this case, how much oil is available. 7 So we are not using the logistic 8 curve or the Hubbard curve to determine the 9 resource base, which is one of the ways it 10 has been used in the past. We are using it 11 to do the scheduling of the production, not 12 the resource base itself. And that's kind of 13 a modification. 14 Now, as part of doing that then, we 15 then have to work backwards and say, well, we 16 don't know if this is the last year's 17 production rate. And we have from the other 18 source, the resource base, the area under the 19 curve that we're trying to then use the model 20 to do that, to draw that function. And 21 that's essentially what we're doing here. 22 Now, another criticism of the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 13 1 Hubbard curve or the Hubbard methodology has 2 to do with the shape of the curve. And what 3 we wanted to do here was to allow the model 4 to look at, either endogenously or through 5 other assumptions, changing the shape of the 6 curve. And this is through -- you'll see in 7 the report the functional relationship. 8 Basically the idea is the blue curve is your 9 symmetric Hubbard curve that you've all seen 10 in different reports. And what we said was, 11 well, you might have a situation where that 12 production gets accelerated, or in fact 13 decelerated. 14 So what we have in the model then 15 is the ability to change the shape of that 16 curve based on, in this case, price 17 function -- although you could do it, as I 18 said, as just as another assumption. So that 19 allows us then to look at what I think 20 conceptually makes sense, which is at high 21 oil price situations, you may decrease 22 drilling levels which may have an BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 14 1 acceleration affect on production that may 2 not match necessarily the symmetric Hubbard 3 curve. 4 Now, one of the criteria that EIA 5 gave us was they wanted to be able to update 6 this frequently. Well, not frequently, but 7 once in a while, and not have it be very 8 burdensome. So the model relies on public 9 data, which in many cases is already produced 10 at EIA. And then what this does is it allows 11 you to take those data, put them into the 12 spreadsheet, and the spreadsheet will 13 automatically recalculate the parameters that 14 go into the model. 15 Cumulative production comes from 16 the USGS estimates that were put into the 17 resource assessment for the year 2000, the 18 year 2000 world assessment. And that 19 originally comes from Petro Consultants, and 20 that gives us the cumes through a certain 21 date. And then the production since that 22 time comes from the EIA website which has BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 15 1 various sources for that oil production 2 information. 3 Now, the reserves in the model also 4 come from EIA. And those are reprints of the 5 Oil & Gas Journal reserve estimates, with 6 some adjustments for the non-conventional 7 gas, probably non-conventional one. 8 We also -- EIA wanted to look at 9 well counts as well. So we put into the 10 model historical well counts and also do a 11 calculation of what future well counts would 12 be consistent with the forecast. And those 13 come from the Oil & Gas Journal and the World 14 Oil Magazine that are public information. 15 Unfortunately, some of that data 16 is -- you have a long time frame for that 17 information. So some of it is only 10 years 18 worth of data rather than the longer time 19 that we would want to have. And then of 20 course, the key component is the resource 21 cost curves which come from the WAU model I 22 described earlier. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 16 1 Now, the conceptual idea behind 2 this was we wanted to use this information in 3 SAGE. So you can see on the left-hand side 4 here that we are using as inputs, of course, 5 the original USGS resource assessment, our 6 costing algorithms and assumptions about 7 financial parameters, technology parameters 8 and so on. And those go into the WAU model, 9 and those produce the cost curves. 10 And then on the right-hand side, we 11 have the oil production model, which is the 12 spreadsheet model I just described. And that 13 includes a lot of historical data from the 14 EIA website, Oil & Gas Journal, World Oil 15 Magazine and so on, which gives us the 16 production history of the reserves and so on. 17 And those calculations are going to the WAU 18 model with an assumption about the future oil 19 price. 20 And you can either run that model 21 with different oil prices and then get you a 22 production curve that could be put into SAGE, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 17 1 or you can take the algorithms out of the 2 model and put them into SAGE. And that's 3 what EIA is looking at now is, do they like 4 how that works and how they want to put it 5 into SAGE and whether or not they want to do 6 that at all. 7 So that was the long-run goal. And 8 the report discusses how you could do that 9 and what equations you would take out of it 10 and so on. Now, the supply curves 11 themselves, we represent by a three-point 12 curve. And the only reason for doing that is 13 just for simplicity purposes. You can have 14 any kind of recommendation you want in a 15 long-run curve. We did it this way so that 16 you can have a compact representation that 17 makes it easy to change the resource base; we 18 don't have to recalculate the supply curve. 19 And I don't need to go into all the 20 different parts of it, but essentially, it's 21 just a -- you take the WAU output and then 22 fit to a three-point curve. And then the key BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 18 1 parameter in the model is the representation 2 of production at any point in time as a 3 function of prior production, the prior year 4 cumulative production, and then the resource 5 base, which is what is recalculated in the 6 model. 7 And then by substituting the supply 8 curve function for the representation of the 9 variable resource base, you could then put 10 together a -- kind of a long equation which 11 allows you to forecast production as a 12 function of price for any given period of 13 time. And the idea we have is that this 14 would be done within SAGE as a 15 pre-calculator. Within each five-year SAGE 16 period, you then run the model at different 17 price steps, and then that curve would 18 represent the potential production for the 19 next period in the SAGE model. And that 20 would be one way of doing it that would be 21 very straightforward and easy to implement. 22 As I mentioned to you, one of the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 19 1 criticisms of the Hubbard curve is the 2 inflexibility of the shape. And what we did 3 is we put into the model the ability to do 4 this shape adjustment, which we put in for 5 the purpose of testing as a function of price 6 that allows the model to accelerate 7 production if prices go up. And this is the 8 representation that we have in the model for 9 that. 10 And then what we have in the report 11 is the regional production curves. And what 12 these are showing you is just at some 13 hypothesized pricing levels of $25, $45 and 14 $65, the price responsiveness of different 15 regions, and what they would look like at 16 different price levels. 17 And again, this is representing 18 conventional production, not non-conventional 19 and not tertiary recovery. Now, any one of 20 these, as you probably know, was very 21 controversial in terms of what assumptions go 22 into it and what the implications are. And BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 20 1 what we tried to do in the report -- and also 2 in discussions with EIA is try to do -- kind 3 of alert to them what kind of issues there 4 are in terms of doing these long-run 5 forecasts, which I'm sure most of you are 6 familiar with. 7 One is this question of 8 technological advances. How do costs change, 9 how do recovery factors change in the future, 10 and if you're doing a long-run forecast, how 11 do you implement that? And there are some 12 ways within the structure that we've just 13 showed you how to do that. But the 14 question's how do you pick those parameters? 15 Like for example, how do drilling 16 costs change over time? Obviously, they are 17 a function of oil and gas prices -- we can 18 look historically and determine that. But it 19 is also a function of long-run technology, 20 and how do you determine what that number 21 should look like. 22 Other issues. Just sort of BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 21 1 modeling questions, like we have the 2 five-year period in SAGE, what does that 3 represent in terms of this model. Is it the 4 last year, the average year, and how do you 5 coordinate this model with everything else in 6 SAGE? 7 The field size -- the resource base 8 itself is quite controversial. One of the 9 problems is that if you look at, for example, 10 Canada and the production that's forecast in 11 Canada, it looks kind of anomalous. And the 12 reason you have a very rapid drop-off in 13 production in Canada is because of the 14 resource base that went into it based on the 15 USGS assessment. 16 If you take, instead using the USGS 17 resource estimates, the one we came up with 18 for the NPC back in 2003 when we were -- we 19 did the big U.S.-Canadian gas study, we also 20 as part of that did an assessment of oil. 21 And that has a much different assumption and 22 would lead to a much different forecast. So BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 22 1 one of the questions is how do you take the 2 USGS information and use it, and to what 3 degree do you look at these sort of anomalous 4 production tracks versus what you would 5 expect or what other people expect to go back 6 and adjust it? 7 And I don't know how much leeway 8 EIA has to make those adjustments, but 9 they're obviously -- things can look kind of 10 funny in the -- and you can kind of see them, 11 and you need to address them. The other 12 issue is the growth issue, which is to say 13 fields don't -- aren't static, that over 14 time, more reserves are added to those 15 fields. And one of the questions is when we 16 look at the world as a whole, how do we try 17 to assess how much further growth we're going 18 to get in the existing fields, because that's 19 a very large portion of the resource base. 20 If you look at the original 21 assessment done by the USGS, that showed 22 612 billion barrels of growth in existing BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 23 1 fields expected out into the future. Now, 2 for the purpose of the WAU, we made some 3 adjustments to that based on what we thought 4 was a reasonable number to have for the 5 growth. And we did that as a function of 6 proven reserves and ultimate recovery. And 7 we set up certain constraints, and so we 8 wouldn't expect the future growth to be much 9 more than a certain fraction of the known 10 ultimate recovery. 11 On the other hand, we wanted to 12 make sure that it was at least a minimum 13 amount of proven reserves. And the problem 14 why this comes up is twofold. One is the 15 USGS does not know the age of all the fields. 16 And the basic assumption used by USGS was 17 that there is what's called a growth curve 18 which will tell you how much reserves are 19 added to a field based on its age. So a 20 field that, let's say, is 10 years old and 21 has a reserve estimate of 100 million 22 barrels, is expected to be growing by BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 24 1 45 percent between its 10th year and let's 2 say its 80th year, which is the end of its 3 lifecycle. 4 So what the USGS does is, they say, 5 well, based on the U.S. patterns for growth 6 that they have seen, they're going to apply 7 that same growth curve to all of these 8 foreign fields, and we are going to assume 9 that curves are the same. And that's what 10 they did, except for some cases, they didn't 11 know the age of the fields, so they didn't 12 actually calculate a growth for that 13 individual field. So when you go through the 14 data, you don't have a growth variable for 15 certain areas. So they have to -- I think 16 they had to extrapolate that, and we had to 17 do the same kind of thing. 18 And the way we did it was by 19 looking at how much reserves there were, how 20 much ultimate recovery, and assume there to 21 be a minimum amount. 22 Well, the problem you have of BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 25 1 course is that it changes the forecast. In 2 other words, if I just put into the model the 3 USGS assessment of 612 billion barrels of 4 growth, that delays -- or that allows 5 production to go much higher out into the 6 future, as opposed to using our constraint 7 growth number of 390. And we can talk about 8 this more, but obviously, the resource-based 9 assumption both for the growth and the new 10 fields is very critical. 11 And then the other issue is 12 definitions. You know, the U.S. has certain 13 definitions, the EIA has a whole program 14 devoted to estimating reserves in the U.S., 15 and as you all well know, a very elaborate 16 statistical methodology that they used to 17 estimate that. But there isn't one for the 18 world as a whole. And therefore, what does a 19 reserve mean, how do different countries 20 estimate those reserves? And, you know, what 21 should be done to try to be consistent across 22 different countries? And also, production is BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 26 1 not measured the same way in terms of NGLs 2 and so on. 3 And then the other issue is that if 4 you look at the forecast, you see that the 5 Western Europe has kind of an odd shape to 6 the forecast, it kind of levels off, then it 7 comes back up again. And the reason for that 8 is there's a lot of Arctic oil assumed in 9 Greenland in the USGS assessment, a large 10 resource base there. And we assume that 11 starting in some future year, that would 12 be -- start to be exploited. 13 Well, obviously it affects the 14 forecast, depending upon what you think is 15 going to happen there. So there are some 16 countries and some areas that really have not 17 had any exploitation with large resource base 18 and then when you put that into the forecast 19 it could affect it. So what do you do about 20 those kinds of errors? 21 So I think the assessment -- my 22 conclusion of this is that this methodology BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 27 1 does make sense. It allows EIA to try to do 2 the things we talked about, which is the 3 current bottoms-up resource-based estimate, 4 kind of a cost-based assessment of the cost, 5 with the economic -- it has a pretty 6 reasonable flexible way of doing an annual 7 forecast. But it doesn't really resolve all 8 these underlying issues, which are primarily 9 data-driven, definitionally-driven 10 assessments. 11 And one of the things I talked to 12 the EIA about was in the long run, how do you 13 try to resolve them? And of course obviously 14 that's a very difficult thing to do because 15 of the difficulty of the data. So those are 16 my prepared compared remarks. I'm willing to 17 take questions or comments. 18 MR. HENGARTNER: Can you go to the 19 microphone please and identify yourself. 20 MR. BROWN: Hi. I'm Bill Brown, 21 I'm with OIF, the AEO half of the office. 22 You mentioned in your presentation that you BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 28 1 have adjustments to accelerate the production 2 of the Hubbard curve. 3 MR. VIDAS: Yes. 4 MR. BROWN: I was curious if that 5 affects -- by pushing it forward, would it 6 have any effect on the technical recovery of 7 the different price levels as you go forward 8 in the forecast. 9 MR. VIDAS: The way it's 10 represented in this model is that the area 11 under the curve is a function of price. But 12 we also allow the price to change the shape. 13 But in a mathematical sense, they are two 14 independent factors, although they are 15 related to each other, so that when you have 16 higher oil production -- higher prices, you 17 get both a bigger area under the curve, as 18 you would expect, plus you get acceleration, 19 although you could represent it -- you know, 20 that's what I was saying earlier, that you 21 could represent -- the shape is a kind of a 22 different issue. It could be sort of BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 29 1 regulatory-related in terms of whether or not 2 the country is open to foreign investment. 3 It is necessary to -- 4 MR. BROWN: You may be missing my 5 question. One of the things we observed in 6 this country was that when lots of 7 development went in early on, the act of 8 doing -- sort of accelerating the development 9 actually reduced the recoverable amount at 10 any given price. 11 MR. VIDAS: Are you talking the 12 production of too much gas along with oil? 13 MR. BROWN: Exactly. 14 MR. VIDAS: No, we've not looked 15 into that, no. And for, you know -- of 16 course, there was a lot of ignorance back in 17 the '20s and '30s in terms of recovery 18 factors. And I don't know that that's 19 happening now; maybe it is. 20 MR. CLEVELAND: So the issue of 21 price is one that I -- so you have prices 22 skewing the curve at the left, which is BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 30 1 understandable. But you also have -- of 2 course, prices will expand the resource base 3 as well. And so it wasn't clear to me from 4 looking at your document how -- what's the 5 mechanism by which a higher price environment 6 increases ultimate recovery? 7 MR. VIDAS: That's the purpose of 8 this other model, which is the cost curve 9 model. So what we are literally doing is 10 taking that other model and playing out the 11 discovery process and development process and 12 saying that at a particular price, at $5 per 13 barrel, $10 per barrel, $15, here's how much 14 more oil and gas, or in this case oil, we 15 would expect to be able to get out at that 16 price level. 17 MR. CLEVELAND: So price is an 18 independent variable in your discovery model? 19 MR. VIDAS: Yes, that's right. 20 MR. CLEVELAND: So there is some 21 implied price elasticity there between -- 22 MR. VIDAS: Right. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 31 1 MR. CLEVELAND: How discoveries 2 respond to prices. 3 MR. VIDAS: Right. But we've got a 4 long-run resource base; so in other words, we 5 have a discovery process model that tells us 6 for any given level of drilling how many 7 fields we are going to find. 8 MR. CLEVELAND: Right. 9 MR. VIDAS: And then there is a 10 development calculation that says when I do 11 find it, is it economic to develop. And you 12 could essentially create a long-run curve by 13 cranking through that, and for any given 14 drilling level, say here's what the minimum 15 price is for this drilling level, and if I 16 had that minimum price, here's what I want to 17 develop. 18 And then you keep going to the next 19 increment. And eventually what you're doing 20 is you're adding more increments of drilling 21 at higher price levels and you're developing 22 smaller and smaller fields. So you can BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 32 1 develop a curve which is a time-independent 2 long-run resource cost curve. And that's 3 what you have to put into the model that 4 tells us how to change the area under the 5 curve as a function of price. So it's 6 essentially an exogenous calculation using it 7 in every process model to give you the 8 resource base size as a function of price. 9 And then this goes in as -- 10 MR. CLEVELAND: So the parameter 11 associated with price in that model, how is 12 that calculated? 13 MR. VIDAS: It's literally taking 14 the different steps. I don't know if the WAU 15 model has tabular form. It says at this 16 level of drilling, that's a $15 resource base 17 increment. You can calculate in the next 18 increment; that's $11, so on and so on and so 19 on, until I drill 20,000 wells, exploratory 20 wells and so many development wells. 21 And at each increment, I'm adding 22 more resources at higher price. That table BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 33 1 then for simplicity purposes -- let's just 2 feed it to a three-point function. And that 3 three-point function then goes into the WAU 4 model. Now, what I was saying was the shape 5 of this -- 6 COURT REPORTER: Would you go back 7 to the mic? 8 MR. VIDAS: I'm sorry, that -- 9 MR. HENGARTNER: The microphone. 10 COURT REPORTER: Would you repeat 11 that please, sir? 12 MR. VIDAS: What I said was that 13 the information on how much of the resource 14 base is economic at any given price is 15 calculated in the WAU model. And it comes 16 out as a table that says that for each 17 increment, here is what the marginal resource 18 cost is and here's what the incremental and 19 cumulative reserves added of oil would be at 20 that price. And then that table of 21 information, which gives us price-quantity 22 pairs, goes into the WAU. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 34 1 And we could have put it in any way 2 we wanted to, but we just chose to do as a 3 three-point function because it allows us to 4 condense the -- 5 MR. CLEVELAND: No, I see what 6 you're doing there and I think that's 7 reasonable. The question is I have is that 8 there is an implied price elasticity in all 9 that. 10 MR. VIDAS: Yes, that's right. 11 MR. CLEVELAND: And I'm sort of 12 wondering where exactly -- like to see -- it 13 would be interesting to know what exactly 14 that is. 15 MR. VIDAS: Yeah, that would be a 16 function of -- in terms of how we actually 17 did the calculation, the cost of the wells, 18 the cost of developing each of the fields, 19 and then our discovery process algorithm, 20 which was based on kind of looking at U.S. 21 History, because we don't have the full 22 history of all these different areas. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 35 1 MR. CLEVELAND: Right. 2 MR. VIDAS: So it's primarily a 3 function of those two things, the discovery 4 process itself, the field size distribution 5 and then the economies of scale and the 6 economies of each of the areas. So for 7 example, if you take an offshore deepwater 8 area where you need to spend hundreds of 9 millions of dollars just for the production 10 platform, or for a sub-sea production unit, 11 you're talking about needing 25 to 50 million 12 barrels to be economic. 13 So that has a much different 14 economic cost curve for the field development 15 than you do onshore, where you are drilling a 16 3000-foot well, where field size is down to 17 50,000 barrels and 25,000 barrels is economic 18 to develop. So you have your hundreds to one 19 ratios in terms of the cost depending on 20 where it is. So what we've done is we've 21 taken the WAU units, which are about 520 22 units, and broken them down, because each one BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 36 1 of those can have a distribution of drilling 2 depths, and in the case of offshore, water 3 depths. 4 And we've taken those and broken it 5 down into sub-areas and say that a certain 6 portion of the resource base is at this 7 drilling depth, or this water depth, and we 8 give that part of the resource base its own 9 economics. And then that's what goes into 10 these curves. 11 Now, what you typically see, if you 12 look either at the United States, 13 Canada -- or when we look at the other world 14 areas, it's a large part of the resource base 15 is relatively inexpensive. And there are two 16 reasons for this. One is there are huge 17 economies of scale in field development. So 18 if you look at the productivity of wells, 19 they tend to be related to the size of the 20 fields. So large fields tend to have very 21 thick reservoirs, multiple pay zones and you 22 get a lot of oil or gas out of each BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 37 1 individual well. 2 And the second reason why they tend 3 to be economic is because the discovery 4 process tends to favor finding larger fields 5 earlier, for the simple reason that it's 6 bigger. And even if you did random, you're 7 more likely to find them. So that's why you 8 tend to see in any resource curve, if you're 9 starting from zero exploration, there's going 10 be a very long bottom part of it. And that 11 is true for every area that we ever looked 12 at. 13 And then you find smaller and 14 smaller fields, and that's where you start to 15 get to the very steep part of the curve, and 16 eventually get to the point where you only 17 have very small fields, where incremental 18 drilling has very tiny amounts of reserves 19 but has a very sharp cost function. And that 20 curve is universal, although the exact 21 parameters of course are really based on the 22 economics I had talked about earlier. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 38 1 MR. CLEVELAND: Can I keep going? 2 MR. HENGARTNER: Yes, you still 3 have the floor. 4 MR. CLEVELAND: The reserve 5 growth -- I'm glad to see -- the USGS has 6 been hammered on this reserve growth thing. 7 And I think I'm glad to see your -- use the 8 reserve growth numbers with caution, because 9 I think that that's appropriate. But I think 10 what's an important -- if you guys are going 11 to use those numbers, the most important 12 explicit -- implicit assumption is, that as I 13 think you've surmised, is that those reserve 14 growth curves are based on U.S. oil and gas 15 fields. 16 So when you apply those reserve 17 growth factors to the rest of the world, you 18 are assuming that the same level of 19 technology and know-how is going to be 20 applied to every other field in the world as 21 was done in the U.S.; right? 22 MR. VIDAS: That's one of the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 39 1 assumptions, yes. 2 MR. CLEVELAND: And that's a 3 gargantuan assumption. And so some regions 4 that would probably be true, but when you 5 start getting into Asia and some of 6 developing nations, there are many reasons to 7 believe that that would never happen. So I 8 think that's important to -- as I think 9 you've -- you've done to some extent, 10 acknowledged the important assumptions in 11 those reserve growth numbers. 12 MR. VIDAS: Yeah, the criticisms 13 that are -- not criticisms, but cautions that 14 are raised about this methodology is as you 15 said, one is the technology one. Although I 16 would guess that if we are looking out 20 or 17 30 years, given how the oil and gas industry 18 has become a service-oriented industry where 19 there are independent companies that are 20 doing a lot of development, the world as a 21 whole is sharing that technology quite a bit. 22 And that's why I would think that BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 40 1 would -- although that is true historically 2 and certainly true in terms of the timing, I 3 think ultimately, everything that the U.S. 4 learns in terms of technology is being 5 applied. In fact, if you look at offshore 6 Angola or Nigeria and so on, it's exactly the 7 same technology that's being been used in 8 deep water Gulf of Mexico. 9 If you look at horizontal well 10 drilling in Saudi Arabia, that came from the 11 U.S. as well. So these things do eventually 12 get there. So in terms of the long run, it 13 may not be an issue. 14 The more problematic part of it is 15 this very simple question: What is a 16 reserve? And when we talk about reserves in 17 the U.S., we're not really talking about the 18 same definitions in other countries. And as 19 you know, when OPEC went to a reserve-based 20 allocation of -- the allocations of 21 production, that created incentive for the 22 countries to up their reserve estimates, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 41 1 which is what they did. 2 So are those as valid as otherwise? 3 Now, there was some work that U.S. Gas has 4 done, taking the Petro Consultants database 5 that I talked about earlier and looking at 6 different vintages of that production, 7 probably the reserve estimates. And they did 8 see that if you look across large fields -- I 9 think they looked at 170 or 180 giant fields, 10 that there was in general large amounts of 11 reserve growth that took place between, I 12 think it was 1981 and 1996. So you know, 13 their claim is they did look at some actual 14 data. But they haven't tried to go back and 15 I don't think tried to fit it or try to see 16 whether or not that reserve growth pattern is 17 different than the U.S. 18 And I wouldn't expect it to be for 19 the reasons that -- the reasons are 20 different, and as you're saying that the rate 21 at which technology is applied is different 22 historically. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 42 1 And then you have this political 2 factor about how the reserve estimates 3 themselves tend to be for bragging rights, 4 and in the case of OPEC allocations, have an 5 effect as well. So that's why it's very 6 difficult to do an apples-to-apples 7 comparison, and that's why you always -- I'm 8 afraid cautious has to be the word in every 9 instance, there's really no way around it as 10 far as I know. 11 MR. CLEVELAND: The representation 12 of technology and technical change -- so the 13 way in which it shows up in your model is in 14 the drilling -- the cost model; correct? 15 MR. VIDAS: That's correct. And 16 that would be the -- unless you try to put it 17 into the accelerator, too, but that's a kind 18 of a -- that would be another way you could 19 put it in -- in terms of -- that's what I was 20 saying earlier, that if we said that the 21 Hubbard curve was the true representation of 22 reality in the U.S., that would be done under BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 43 1 historical technology. But we now have ways 2 of increasing production rates, in effect. 3 MR. CLEVELAND: That has always 4 been true. Technology is just -- here's 5 always been huge innovations in the industry. 6 MR. VIDAS: Right, that's what I'm 7 saying. 8 MR. CLEVELAND: Based on the 9 innovation -- I guess the question is, is the 10 pace of innovation now so much greater 11 they're producing some kind of structural 12 break in the data series? And I -- 13 MR. VIDAS: Well, what I'm thinking 14 is that now when you find a field, you're 15 using today's technology, whereas in the past 16 you may have found it in 1920, and used 1920s 17 technology first, and you're into 1930s. So 18 therefore, the historical pattern has 19 embedded in the certain technology pattern, 20 whereas today we have a different pattern. 21 That's all I'm saying. 22 MR. CLEVELAND: So in the case of BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 44 1 your model, the way in which technology 2 increases ultimate recovery is by lowering 3 drilling costs -- 4 MR. VIDAS: That's correct. 5 MR. CLEVELAND: And hence 6 increasing -- so I guess, one question there 7 is that you've aggregated exploration 8 development production into one activity. 9 And it's really -- production certainly is a 10 different universe than exploration 11 technology. 12 MR. VIDAS: Well, it actually is 13 two different steps in the WAU. But for the 14 purposes of representing it here, we wanted 15 to have a price-quantity pair. In fact, 16 there are two dimensions whenever you 17 explore, the exploration takes place given a 18 certain price and technology level. And then 19 you have inventory of discoveries, which are 20 field sizes that have been discovered. And 21 you can pick as to what you're going to 22 develop based on technology, too. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 45 1 So you could -- depending on when 2 you did it and what the prices were, you'd 3 have a different inventory of discoveries 4 that you've developed. And for the purposes 5 of simplicity in the model, we simply say 6 we're going to develop a curve by taking the 7 minimum cost that's possible for every 8 incremental drilling, which includes 9 developing every field up to that marginal 10 cost. And that way, we can throw in the 11 fields for each increment as we go up the 12 curve. 13 And as you point out, it is kind of 14 a modeling construct that isn't necessarily 15 realistic, because you could, for example, 16 when the prices go up, you can develop a lot 17 of inventory of discoveries you have already 18 have -- have developed. And that would 19 happen differently than the smooth curve that 20 we have in the model. 21 SPEAKER: Uh-huh. 22 MR. VIDAS: So that is correct, but BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 46 1 it's just an artifact of how we -- I mean, 2 you could actually put the discovery process 3 model in SAGE and do it on a real -- on a 4 time-interval by time-interval basis. But 5 then it becomes an unwieldy kind of -- it's 6 not an easy thing to update, and that's why 7 we try to do it this way. 8 MR. HENGARTNER: One thing that 9 occurs to me is, the WAU model is very 10 interesting. However, in terms of 11 exploration and the impact in oil discovery, 12 the price of oil needs to be above a certain 13 level for quite some time, because there is a 14 lag, right? We had a spike of oil going to 15 what -- 70s maybe? 16 MR. VIDAS: Yes. 17 MR. HENGARTNER: Did we explore 18 very much more? Maybe, maybe not. The price 19 is back down, all right, and so the 20 volatility in price now will have an impact 21 on how we perceive and want to do exploration 22 in the future. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 47 1 MR. VIDAS: Right. And the way 2 we've done that in the spreadsheet model is 3 to use an averaging price, so that way you 4 don't take just this year's price, you take 5 an average of the last five years and weight 6 it 30:20, whatever the ratios were to get 7 around that. And of course, that's something 8 you can also vary in the model, too. But you 9 wouldn't expect big changes like, you say, 10 for -- in one year, based on just one year's 11 worth of price. And that isn't how we are 12 the model. 13 MR. CLEVELAND: You did see it back 14 in the first few price shocks, but they 15 learned their lesson. 16 MR. HENGARTNER: Yes. Exactly. 17 MR. CLEVELAND: I guess that will 18 be the problem is it becomes -- it's price 19 expectations, not price. Well, that's why 20 you go to moving averages to -- 21 MR. VIDAS: Right. And then in the 22 SAGE model, of course, we're trying to set BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 48 1 this algorithm up to do the next five years 2 of -- so we're going to have to use that as 3 part of developing that curve for the next 4 five-year period. 5 MR. CLEVELAND: So in the curves 6 that you showed with the different prices, is 7 technological -- that one percent increase in 8 technology -- 9 MR. VIDAS: That's correct. 10 MR. CLEVELAND: That's embodied in 11 that. 12 MR. VIDAS: Yeah, we didn't change 13 the technology between that. Now -- 14 MR. CLEVELAND: So those curves 15 that you showed has the assumption of a one 16 percent -- 17 MR. VIDAS: Yes. 18 MR. CLEVELAND: Per year decline in 19 the cost of finding and developing oil. 20 MR. VIDAS: That's right. So it's 21 already built in and it doesn't change based 22 on price. So the only thing we are varying BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 49 1 is the price. 2 MR. HENGARTNER: Harry, sorry, I'd 3 like to get back to you, but I'd like to have 4 Tom Rutherford add his planned discussion to 5 mix up a little bit our comments here. And 6 then it's going to be your turn. I won't 7 forget you. Thank you. 8 MR. RUTHERFORD: I think that I 9 should first apologize for being a couple of 10 minutes late; in the rain, I was a little bit 11 slow. There is always a challenge -- in 12 modeling, it's always a cheap complaint to 13 say, well, you've not done enough, or you've 14 not looked at this. So now let us focus on 15 what we have as a model here, and think about 16 ways that -- what would -- so what's useful 17 about this is that we try to be a coherent 18 framework for talking about resources and 19 reserves, and the means of connecting to 20 available geologic information, quantifying 21 how changes have -- and estimates of those 22 things and what they mean for the economics. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 50 1 So all these things are good, 2 that's definitely what we want to be moving 3 forward. The sort of the downside I see 4 about the framework is that it's a little bit 5 patchwork. So we're taking data from 6 different places. I personally -- I'm the 7 type that I like to have everything sort of 8 laid out coherently in one framework. So I 9 don't have to know that I get stuff from this 10 model and then run it in that one. It's a 11 little bit easier if everything is in one 12 certain setting. 13 But that's less of a concern to me 14 than -- I'm a little bit concerned about the 15 treatment of the international component. I 16 recognize the world is a very diverse place. 17 And if we start worrying about what the 18 nature of oil exploration is and all -- there 19 are lots of different countries, lots of 20 different contexts -- but I think that for 21 oil, probably there is not -- if you were to 22 pick the top ten other locations and focus on BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 51 1 those, you might -- it is one of these things 2 where it's a small number of countries that 3 really have a major impact, and sort of 4 trying to pay attention to adapting the model 5 to those contexts to provide -- I'm thinking 6 about trying to how better improve the 7 representation of oil extraction or recovery 8 in other countries. This could be quite 9 important. 10 And if we think about developing 11 country contexts, or indeed, even Russia or 12 Kazakhstan or these places, it is not just 13 oil price, it is also sort of foreign direct 14 investment, sort of investment climate, cost 15 of production. 16 There are a number of other factors 17 that also I think would affect what your 18 forecast would be -- in a first order sort of 19 affect, would be the degree to which those 20 things -- so I think that a model that's a 21 little bit more adapted to the specifics of 22 the major suppliers could be a productive way BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 52 1 to improve the framework. And then of 2 course, the final thing I just raised has to 3 do with the role of expectations. Your talk 4 was much more coherent than -- I've looked at 5 this model before, and I couldn't quite make 6 sense. 7 But your discussion is pretty clear 8 that you have a clear idea of what the model 9 is doing. And so I -- it will be very -- I 10 would really like to see the model developed 11 in a framework where you could have it all in 12 one setting, the equations are all kind of 13 clear -- stuff comes in, stuff goes out, and 14 you can sort of make sense of it. And then 15 it provides a better framework for thinking 16 about extensions either to incorporate other 17 countries, but also when thinking about 18 dynamics, about what the role of price 19 expectations are, it's something that 20 again -- but that's a bigger issue than just 21 this model. 22 But this role of sort of recursive BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 53 1 expectations of prices and the distinction 2 between what the price is the model is 3 producing or what the expectations are of the 4 agents of the model, or something -- I'm a 5 little bit uncomfortable. 6 But just a recap, I think the 7 attempt to try to reconcile resource and 8 reserve estimates and to be able to draw on 9 more recent estimates of the actual resource 10 base to try to provide a framework for 11 understanding future developments of 12 world markets, that's truly important. And I 13 think there is enough writing on this aspect 14 of the energy system that some more attention 15 to international -- specific international 16 things are definitely important. 17 MR. CLEVELAND: Follow-up before we 18 move on. Just a general comment following up 19 on that. Ten years ago, you never would have 20 seen EIA doing this, the Hubbard -- the whole 21 long-run forecasting. You never -- they 22 never would have -- they were doing their BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 54 1 NEMS thing. And the whole peak oil debate 2 has gotten so much attention that I think 3 EIA, and the USGS to a lesser extent, has 4 been forced to respond. 5 And I'm happy to see that in fact 6 you have, because it is important, because 7 there are a lot of kooks out there mis-using 8 doing some really rudimentary silly curve 9 fitting that has confused the debate more 10 than it has helped. And I think that 11 applying -- taking essentially the Hubbard 12 curve, which a lot of these folks use -- or 13 some form of it, and trying to introduce some 14 structure to it, some transparency to it, and 15 some economic rationality to it, albeit in a 16 very simple way here, I think is a positive 17 thing to do. And so I think it's good that 18 you're doing this. 19 MR. BROWN: It's just simply -- 20 MR. HENGARTNER: Could you go to 21 the mic, please? 22 MR. BROWN: Oh, sure. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 55 1 MR. HENGARTNER: Okay. 2 MR. BROWN: I'm simply curious if 3 you knew of the next sort of USGS or 4 equivalent type of agencies' plans for oil 5 resource assessment. 6 MR. VIDAS: Well, the one thing 7 that's going on now that maybe Howard could 8 talk about is the NPC study that DOE's asked 9 the National Petroleum Council to look at 10 this issue of world oil and gas resources. 11 And my understanding is that what they are 12 doing is doing a survey of what's been done, 13 how different people are modeling oil and gas 14 production out into the future, what 15 assumptions they are using, and they are 16 trying to not necessarily reinvent the wheel, 17 but to try to provide some expert commentary 18 on what they like about that -- what their 19 deficiencies are, and what recommendations 20 they would give to the government in terms of 21 how to look at these long-run issues. 22 And that's -- and I'm sure that BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 56 1 everybody including different government 2 agencies were part of that as they were just 3 part of the gas study that we finished up 4 three years ago. 5 MR. HELKIE: I just had one 6 question. I just wondered if you had 7 investigated that assumption that technology 8 is kind of independent -- technology -- the 9 trend of technology is independent of the 10 price. 11 MR. VIDAS: As I was alluding to 12 earlier, one of the controversies in doing 13 these long-run forecasts is how do you put in 14 technology? And as I was saying earlier, as 15 the NPC had done a study, both in 2003, that 16 I was a large part of, and also in 1999, 17 looking at this technology trends. 18 And they did it primarily from 19 expert opinion because they couldn't find any 20 good way of looking at it mathematically. 21 Though there have been attempts that have 22 been done to try to look at, for example, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 57 1 drilling costs as a function of activity 2 levels, prices, and technology trends, or 3 recovery factors, and so on. And some of the 4 numbers just come out to be I think crazy in 5 terms of what they come up with, given what I 6 know about technology advances. 7 So the answer is, I have not seen, 8 nor have I ever myself done, a rigorous 9 statistical analysis that I believe that 10 would give us a look at whether or not high 11 prices induce technology or retard it, 12 because people are able to make money with 13 the existing technology and don't bother to 14 try to figure out how to develop that field 15 at $25 -- if the price is $50, they know how 16 to develop it at 50 bucks and don't have to 17 bother -- or work at it. 18 One of the interesting things that 19 came out of the NPC -- and this -- was the 20 idea that if you look at what actually is 21 happening to individual, for example, 22 material science, information processing BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 58 1 science, is that a lot of the things that are 2 happening now are all originating outside the 3 oil and gas sector, and are being brought in 4 and adapted to oil and gas technology. For 5 example -- composites to do an offshore 6 platform as opposed to steel, that wasn't 7 done for that purpose, but it was adapted. 8 If you look, all this computer 9 technology has been a big source of 10 improvement that was brought in from other 11 sectors. If you look at things like 12 measurement while drilling and those kinds of 13 technology, those sensors and so on weren't 14 necessarily developed for oil and gas, but 15 they were adapted. 16 So all the different 17 drivers -- really is not sectorially 18 specific. It's actually an issue of the 19 industry -- of the economy as a whole, and 20 how it advances technology. And oil and gas 21 is one of the consumers of the technology, 22 and that's -- I think the way the 2003 study BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 59 1 was written, it was talking much more in 2 terms of technology as being an economy-wide 3 phenomenon, not necessarily specific to oil 4 and gas, although obviously there are 5 individual aspects of it as well. So it's 6 still an issue, and I have not resolved it, 7 because I don't model it for the long run. 8 MR. HELKIE: It would seem to me 9 though that the relevant technology for the 10 application is not what's laying out in the 11 field, but what's being applied. And it 12 would seem to me that what technology is 13 applied would be a function of the price, 14 which is I think what you're saying, right? 15 MR. VIDAS: It is, but if you look 16 at something like -- look at how the big 17 gas -- most of what we have got is just oil 18 and natural gas is -- domestic and Canadian 19 gas. And if you look at some of the hot 20 places now, like say, the Barnett shale or 21 the Fayetteville shale, and the Woodford 22 shales, and what's being done, and how they BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 60 1 are advancing, they are using horizontal 2 drilling technology that has been around for 3 15 years. And they are using fracing 4 techniques that have been around for 30 5 years. But they are adapting them and 6 learning how to apply them to an individual 7 occasion. And what's happening is, they are 8 being driven by the higher prices, but they 9 are also -- 10 MR. HELKIE: Exactly, yeah. 11 MR. VIDAS: But if you actually 12 look at the costs here, in some cases they're 13 coming down. So in other words, if I looked 14 at what it was costing to do -- a deep 15 Bossier field development in the Vernon field 16 was $10 million per well, now it's down to 17 like $6. So it's kind of funny how -- and 18 people know this is happening. So they are 19 actually sometimes doing things that are 20 uneconomic, with the expectation that they'll 21 learn enough to make them economic. 22 So it is really complicated when BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 61 1 you look at individual locations and what 2 people are doing. And as you say, it's 3 sometimes -- it's the doing that drives down 4 the price -- it's the actual effort to do it 5 that causes the price -- the cost to go down. 6 MR. HELKIE: No, I think it's 7 interesting, and this is not a criticism of 8 your work. But I just think it's something 9 that's really worthwhile investigating. I 10 mean, we've had this economy-wide 11 productivity boom, or had it in the '90s. 12 Nobody has got a clue why it happened. 13 MR. VIDAS: Right. 14 MR. HELKIE: I mean, they are just 15 utterly clueless, right? And -- 16 MR. VIDAS: Let me suggest that -- 17 MR. HELKIE: And I think it is just 18 something that's really worth investigating. 19 MR. VIDAS: I think it may be 20 something as simple as how many scientists 21 are there. 22 MR. HELKIE: I'm sorry? BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 62 1 MR. VIDAS: If you look at our 2 population, in terms of what percent of our 3 population is scientists, our population is 4 growing. The percent of the number of 5 scientists out there and engineers working on 6 this is growing. The number of papers that 7 are being published is growing. So we have 8 a -- it may be a very simple input-output 9 function. 10 MR. HELKIE: That's an interesting 11 point. Intuitively, it makes sense. But the 12 scientific population outside of the United 13 States has been growing a lot more from 1985 14 to 2005 than it did in the U.S. The 15 productivity increase has all been in the 16 U.S. -- 17 MR. VIDAS: Because of our -- I 18 think, I finally struck -- 19 MR. HELKIE: And Canada, and not 20 elsewhere, right? 21 MR. VIDAS: I think it is a -- 22 MR. HELKIE: And it's just BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 63 1 really -- this is just one example of where 2 it is really interesting, but this is the 3 example we really care about, and -- 4 MR. VIDAS: Yeah, I think most 5 economists, and I guess in a roomfull of 6 economists, I can say this. 7 MR. HELKIE: Yeah, but -- 8 MR. VIDAS: We think it's the 9 market structure. I mean, the fact that you 10 have the incentives. 11 MR. HELKIE: But I think in the 12 long-run application, it's really an 13 important assumption to say that the 14 technology advance is independent of the 15 price. 16 MR. VIDAS: So you disagree with 17 that, you're saying. 18 MR. HELKIE: I disagree with it, 19 yeah. 20 MR. VIDAS: Yeah. 21 MR. HELKIE: But that doesn't mean 22 I'm right. I disagree -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 64 1 (Laughter) 2 MR. VIDAS: So having spent hours 3 and hours with petroleum engineers just 4 discussing this sort of issue and going 5 around and around, the more you talk about 6 specifics, the more hard it is to do 7 generalities. I understand why you're saying 8 that. 9 MR. CLEVELAND: Can you put up your 10 curve for world oil production? 11 MR. VIDAS: I think it's the 12 non-OPEC -- 13 MR. CLEVELAND: Non-OPEC? Okay. 14 MR. VIDAS: But the report has the 15 whole lot -- 16 MR. CLEVELAND: I'm curious as to 17 what the area under that curve is, and what 18 that is -- well, what that is what ultimate 19 recovery is compared to what the USGS raw 20 numbers. I mean, how much does your economic 21 truncation of the things -- do you have any 22 idea? Just curious. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 65 1 MR. VIDAS: The whole resource 2 base, I believe, which is the ultimate 3 recovery of the weekly USGS is, 4 3300-something billion barrels. And ours is 5 3100. 6 MR. CLEVELAND: That's it? Okay. 7 MR. VIDAS: So we've knocked it 8 down 200 -- just for the -- 9 MR. CLEVELAND: In other words, 10 that will be the infinite price -- amount 11 here. 12 MR. VIDAS: Right. 13 MR. CLEVELAND: I'm not sure 14 exactly what this is. I guess if you could 15 document that, but it's easily -- it easily 16 comes out of the model. 17 MR. HENGARTNER: Jae. 18 MR. EDMONDS: When I first started 19 doing long-term global energy modeling, 20 working with colleague Ralph Ruddy (?) who 21 was very enamored by King-Hubbard. And he 22 would -- have he would use these King-Hubbard BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 66 1 curves, and he was always trying to grapple 2 with how you reconcile them with historical 3 data. And he would add a parameter and then 4 that wouldn't quite do it. And maybe add 5 another parameter. These things got very 6 elaborate. And so we eventually just set 7 that aside. 8 And what we did is we just went to 9 a straight graded resource. Essentially, on 10 the previous figure, you have effectively a 11 graded-resource curve defined by three 12 points. What you might want to think about 13 though is that the potential, that's based on 14 a notion of what we would call today 15 conventional oil. 16 MR. VIDAS: Right. 17 MR. EDMONDS: And as I think you 18 have already pointed out in a couple of 19 points here in the talk, that definition kind 20 of is squishy. I mean, it is sort of always 21 moving or -- sort of moving out on you. So 22 if you get to these with -- at an infinite BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 67 1 price, you would be surprised, or you 2 probably wouldn't be surprised -- but you're 3 not going to have a cleanly truncated 4 resource because you get into these shales. 5 And once you start getting into the shales, 6 you really have a resource that nobody has 7 ever gone out looking for. It's like that's 8 a bad outcome when you're out there 9 prospecting. And so you need a way to 10 actually have that curve -- rather than just 11 shut itself down as the price goes to 12 infinity, you actually need a way for you to 13 open that up. 14 And the way we did it was we just 15 added a few extra points. And so you 16 actually weren't constrained by 17 King-Hubbard's thinking about the problem. 18 You actually had a graded resource. And I 19 think we used five different grades of the 20 resource. It makes reconciliation with the 21 real world just so much easier. So you may 22 want to think about that. And then you take BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 68 1 your reserves out of the resource phase as 2 you go out and explore further. And that 3 works out pretty well. And so I just put 4 that forward as a thought. You might want to 5 think about it. 6 MR. VIDAS: Yeah, when we discussed 7 the results with EIA, we talked about that. 8 And I think EIA is well aware of the 9 phenomenon of non-conventional resources and 10 how they might fit in. And there might be 11 two things you could think about -- or they 12 could think about doing. One would 13 be -- implicit in this is that there is 14 something called a conventional oilfield. 15 It has a certain amount of 16 production that comes out of a field of a 17 certain size. And what that represents in 18 any hearing census or recovery factor that 19 says that there is a certain amount of oil in 20 place, and that with conventional production 21 techniques and certain well-space of a 22 certain amount and maybe a water flood you BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 69 1 can get a certain percent of that oil out. 2 But you could, in theory, if you had a much 3 higher price, do surfactant floods, you could 4 do, you know start recovering with nitrogen 5 or CO2 -- 6 MR. EDMONDS: Yeah. 7 MR. VIDAS: You could do a 8 thermo-enhanced oil recovery and so on. So 9 there is kind of a technological price for 10 phenomena, which gives you extraordinary 11 recovery factors in conventional fields, and 12 that's kind of in theory part of the growth. 13 Although if you look historically, some of 14 it's there, but some of it isn't there. So 15 one recommendation that we talked about when 16 we had our discussion was to a bit more 17 explicitly deal with this growth in the 18 conventional fields as a function of -- as 19 you were saying, price -- 20 MR. GRUENSPECHT: This 21 conventional/unconventional thing is 22 very -- sometimes it is talked about as the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 70 1 resource, but there is -- I guess you include 2 secondary recovery in your sort of what most 3 people call secondary recovery in a 4 conventional field that is part of your 5 conventional thing. But the question is, is 6 tertiary area on the lot. 7 SPEAKER: Right, got it. 8 MR. GRUENSPECHT: In conventional 9 fields, some people would view that as -- I 10 think that's not in here now; right? 11 MR. VIDAS: Well, that's what the 12 commission does. 13 MR. GRUENSPECHT: How long that -- 14 MR. VIDAS: When you ask somebody 15 is it in there or not they would say we're 16 not really sure because we're trying to look 17 at historical data. Obviously, there is 18 tertiary recovery that has happened in the 19 U.S. 20 MR. GRUENSPECHT: Right, in the 21 (inaudible) and in other places. 22 SPEAKER: International fields. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 71 1 MR. CLEVELAND: But oil -- shale 2 oil is fundamentally a different geologic -- 3 MR. VIDAS: Right. And that's why 4 we're trying to -- 5 MR. CLEVELAND: So it's a different 6 technology. 7 MR. VIDAS: Right. And that's why 8 I was trying draw the distinction between 9 looking at the population that presumably is 10 in here, which is to say conventional fields, 11 and apply unconventional or new ideas to 12 those conventional fields. 13 MR. CLEVELAND: Right. 14 MR. GRUENSPECHT: Right -- fields, 15 right. 16 MR. VIDAS: Which will be 17 represented as increase in recovery factor. 18 MR. GRUENSPECHT: Right, people 19 kind of look at this thing as conventional 20 and unconventional, like two box models. But 21 I think it's sort of like a third box, which 22 is -- they sort of -- I don't know where you BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 72 1 draw the line, but it is a continuum of 2 technology. But I think in these kind of 3 things, again, you raise this issue of 4 unconventional technology in conventional 5 resources. 6 MR. VIDAS: Right. And the way the 7 USGS actually will do it is they'll do it the 8 way Howard has said, which is that there is a 9 certain type of reservoir which we're going 10 to call conventional. And there is other 11 types of shales and tar sands and so on which 12 are a different geologic setting. And we're 13 going to give those a different name and 14 apply different technologies. Now, there is 15 no resource assessment, world resource 16 assessment that the USGS has done for 17 non-conventional resources. 18 There are placeholders. If you 19 look at their databases, there are 20 placeholders for them, but they haven't 21 assessed them. So if you are really trying 22 to do this, and in the long run it has to be BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 73 1 able to do this just to calculate -- not 2 calculate, but at least have a representation 3 of those conventional and non-conventional, 4 and then have within the conventional this 5 idea of technology and technology advances 6 which increase recovery factor, and it's 7 actually the same thing for the 8 non-conventionals which have the same issues 9 in terms of recovery factors and prices and 10 technologies. 11 MR. HENGARTNER: Now, all this 12 discussion suggests actually that in terms of 13 the Hubbard curve, we made be interested in 14 not one curve but a mixture of curves. 15 MR. VIDAS: For different 16 resources. That's right. 17 MR. HENGARTNER: Right. 18 MR. VIDAS: Right. And this of 19 course was just the first step, which is 20 looking at conventional resources using 21 historical recovery factors which are what 22 essentially is in the USGS resource BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 74 1 assessment. Like you're saying that -- you 2 would logically want to do this and then add 3 to it the non-conventional techniques in 4 conventional fields, and then have another 5 resource for the non-conventional, which of 6 course is a -- actually is in SAGE now, as I 7 understand it. And so it is in there in a 8 structural sense already, but it could be 9 investigated as you were saying. 10 MR. HENGARTNER: Yes. 11 MR. RUTHERFORD: If everyone wants 12 an argument from Y2050 -- projecting the 2050 13 makes sense, let's say this diagram -- 14 MR. HENGARTNER: Well, making sense 15 is a big word. 16 MR. RUTHERFORD: I'm not saying we 17 have statistical precision, but the debate 18 goes -- conceptually, 2050 is not that far 19 out. 20 MR. CLEVELAND: Yeah, it's 21 interesting to me and this is -- I'm doing 22 some work at the Robert Kaufman, whose work BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 75 1 you probably know. And we have a 2 paper -- it's in here somewhere, I don't know 3 where; where we basically, it's much cruder 4 than this, where we just take different 5 estimates, take the range of estimates for 6 what ultimate recovery is, and just fit a 7 Hubbard curve and it's remarkable how little 8 difference it makes about when the peak is, 9 you know -- 10 MR. VIDAS: Twenty years, you're 11 thinking. 12 MR. CLEVELAND: Maximum. 13 MR. VIDAS: Yeah. 14 MR. CLEVELAND: Maximum. 15 MR. VIDAS: Peak of what? 16 MR. CLEVELAND: World oil 17 production. 18 MR. VIDAS: Conventional -- 19 MR. CLEVELAND: Conventional oil 20 production. 21 MR. VIDAS: Conventional oil. 22 MR. CLEVELAND: Yeah. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 76 1 MR. VIDAS: Conventional -- using 2 conventional recovery factors or -- 3 MR. CLEVELAND: For all the 4 assumptions that are pretty much implied in 5 this as well. 6 MR. VIDAS: Okay. So -- okay. 7 MR. CLEVELAND: You know, all the 8 assumptions -- so you have people who think 9 there is X zillion barrels and people will 10 think production is peak. And those imply a 11 huge range of assumptions about technology 12 and price -- whatever they are, we just took 13 that full range and fit the curves to it and 14 its remarkable how little difference it 15 makes; the die is in some fundamental way 16 cast. 17 MR. VIDAS: Of course, that's five 18 administrations, right? Twenty years is five 19 presidents. 20 MR. CLEVELAND: Envision how 21 little -- sensitivity of price here is very 22 little. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 77 1 MR. VIDAS: Yeah, and as I was 2 saying before the talk, it has to do with the 3 economies of scale in oil production, and 4 then big fields trying to be economic with 5 $10 or $15, and they're going to be found 6 first and developed first. And that's why 7 you get that long bottom part of the curve. 8 MR. CLEVELAND: Right. 9 MR. VIDAS: The cost curve, I mean. 10 Well, thank you for your attention. I -- 11 MR. HENGARTNER: Thank you very 12 much. 13 MR. VIDAS: Thank you. 14 (Applause) 15 MR. HENGARTNER: I think we will 16 break now and then reconvene in the break out 17 session. The people up here are Johnny 18 Blair, Moshe Feder, Barbara Forsyth, myself. 19 No, I'm not here. 20 (Laughter) 21 MR. HENGARTNER: Walter Hill will 22 be up here. And we're going to go BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 78 1 downstairs, Mark Burton, myself, Cutler 2 Cleveland, Jae Edmonds and Thom Rutherford 3 and Susan Sereika. 4 (Recess) 5 SPEAKER: Good morning. 6 MR. HILL: Yeah. Good morning. 7 I've been deputized to run this session, the 8 chair of the break-out session. I assume we 9 go until about 11:00, with the usual format. 10 So your presentator discussing and comments 11 from horseshoe crab -- people in the 12 horseshoe crab here in the audience. You're 13 on. Good luck. 14 MR. LU: Okay. Good morning 15 everybody. My name is Ruey-Ping Lu. I'm 16 from Statistical and Method Group. I'm going 17 to present the study results of the weekly 18 monitoring of stocks of oil from monthly 19 data. Okay. Well, typically we have to put 20 up the disclaimers, for this is ongoing 21 project, so just present to you and solicit 22 your advice, so we can do better job to serve BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 79 1 the public. 2 And let me give you a little bit 3 the story about the stock of crude oils and 4 the petroleum products. And EIA published 5 monthly petroleum -- supply monthly, okay. 6 That's monthly data. 7 And the major petroleum products 8 are the crude oil, motor gasoline, distillate 9 fuel oil, residual fuel oil, jet fuel, 10 unfinished oil, propane and other oils. 11 These are all from surveys from the 12 refineries, the petroleum supplies, so we 13 have no problems. These other oils in the 14 monthly basis, we have no problem either. 15 But in the weekly basis, we do have a story. 16 The problem comes -- thing is we 17 tried to estimate other oils, the weekly 18 other oil stock. And this one is published 19 in the weekly petroleum status report. Okay, 20 this is the latest one. They published it 21 last week. And the one I showed before they 22 published end of last month, which you will BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 80 1 see it's September. And included was the 2 data for July 2006. See there is a two-month 3 lag; I'm going to talk about that. 4 So here, we want to determine the 5 best way to estimate weekly stocks of other 6 oil based on weekly data from the major 7 petroleum products, plus monthly data for 8 other products. And also, we try to evaluate 9 alternative methods by comparing couple of 10 different estimates. 11 In these projects, we use the data 12 from 1993 to 2005, the monthly stocks, also 13 the weekly stocks. You can see weekly stocks 14 at many other -- major petroleum products. 15 In -- compared to the monthly and the weekly, 16 they are two months' lag, that's exactly here 17 you can see. Okay? It published last 18 month's, but in October, we don't have August 19 data. We only have July data, so we still 20 have to use the July data -- or up to the 21 July data to help us to estimate the October 22 weekly numbers. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 81 1 See the story here? Because we 2 don't have a survey to collect this, the 3 components of the other oil stocks. See, the 4 components of other oil stocks are aviation 5 gasoline, kerosene, and all these small 6 pieces. They add up as not as small pieces. 7 Okay? Altogether, and we have a monthly 8 survey to collect all this, but we don't have 9 a weekly survey to collect all this. And our 10 customer wants weekly number because they can 11 what, check on the financial market. We've 12 those -- some of those components play some 13 roles in the financial market. 14 So you can see the components of 15 all these were about 11 or 12 different 16 pieces. And when we displayed the numbers of 17 the weekly number, okay, check to the 18 months -- end of the months, and then compare 19 to the monthly number, the blue one is the 20 PSM, is the Petroleum Supply Monthly. Okay, 21 so two months behind by the blue one. And 22 this is the weekly. I interpret the weekly BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 82 1 to the end of the months, plot it. Here 2 that's above 20 percent difference. Now, it 3 looks like it's not much, but if you look at 4 the scale, okay, that's above 20 percent 5 discrepancy. We're now going to say which 6 is, but generally we'll use this as a 7 benchmark. So we would say there is a 8 discrepancy. 9 Somehow, there's a story of that is 10 natural gas price shoot up. Okay, and the 11 monthly number goes down, and we didn't catch 12 that. But later, yes, we did. Okay. So our 13 front office and the customer noticed this, 14 so we're going to study is there any way we 15 can improve the estimate of the weekly 16 number. As it caused not this much 17 discrepancy, because there'll be something 18 like that. 19 Next. Okay, because of the problem 20 in the 2003, like January to April, in the 21 first week of April 2004, we start collecting 22 the weekly propane. Before, the propane BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 83 1 number was only collected during the winter 2 month on a weekly basis, otherwise on a 3 monthly basis. And propane is a big 4 component in the other oil stock, so we try 5 to solve part of the problems is -- get a 6 weekly number out of propane. So we can have 7 big chunk which is more precise than trying 8 to estimate the others. Okay. 9 So my work is trying to look for 10 different methodology or alternative methods 11 to model the weekly stock of the other oils. 12 So what I did is using the multiple 13 regression modeling and unobserved components 14 models. I don't know you -- I mean, it was 15 this one -- this was published by Harvey in 16 the late '80s in British, so that time, it 17 did not have the computer program, until two 18 years ago, assess the -- publish a procedure, 19 which can run -- called UCM, unobserved 20 components model. 21 Then we will compare the forecast 22 or the estimate of the Petroleum Supply BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 84 1 Monthly number, okay? I treat the multiple 2 regression model using the monthly data from 3 1993 to 2001, and the dependent variable will 4 be the major petroleum product, but the 5 stocks. So you can see those -- model 6 gasoline, jet fuel, distilled. And plus, the 7 months lag and so on. Okay. So then we can 8 predict using the 2002 up to the 2005 monthly 9 number. 10 But in doing this, I didn't gather 11 the weekly number. Once I get a monthly 12 projection or the monthly estimate, then I 13 used that to interpret in the weeklies. 14 Okay, which means I still can compare the 15 estimate, but in order to get the weekly 16 number, I have to do some other 17 way -- intervals or make the monthly and get 18 the weekly number. 19 The unobserved components model is 20 you have a time series. You list this time 21 series as several different components, like 22 you have a trend, okay? And you have BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 85 1 seasonal and you have a cycle, and you have, 2 auto regressive terms, plus multiple cycles 3 and other things, which will be sort of the 4 marriage of ARIMA model and the multiple 5 regressions. Okay, the only thing they have 6 to assume is, oh, these different components 7 are independent. 8 Okay? So you can see here, we can 9 express the numbers in terms of the 10 trans-season cycle, auto regressive and the 11 lag variables plus regression. That's a 12 multiple regression piece. Okay. So let's 13 look at the result of that. And in the 14 multiple regression, I run two different -- I 15 run a bunch of models and these two performs, 16 well, better feet (?) and better prediction. 17 So you can see it's the range of the mean 18 absolute present error and also the range. 19 Unobserved components model is pretty good 20 compared to the regression. 21 And we have another division to 22 the -- we call the STEO, it's a short-term, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 86 1 energy outlook. They did three months ahead 2 or a month -- a year ahead prediction. So 3 they have another different multiple 4 regression model running. Okay. So that's 5 the number they provided me. So I compare 6 that, it's with the same intervals. And this 7 is we published. Weekly petroleum supply 8 report. Okay. And this 24 is the circle we 9 have to start off this project. Okay, now, 10 if we check on the -- what, from year to 11 another year, to 2002, 2003, 2004, 2005. 12 Okay, the comparison, you can see 13 unobserved components model is pretty good 14 compared to currently used. Okay? So if I 15 give you the picture of this -- this, the 16 problem we had, okay, 2000, early -- that's 17 the year, beginning of the -- what, 2003 18 January to the April 2003, okay? And look in 19 the blue ones, okay? It's -- we see above up 20 and down 10 percent, I hope -- get to the 21 what? 5 percent. That'll be what, ideal, so 22 we can have better estimate. The things may BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 87 1 be used sometimes, when you're free to 2 unobserved components model. Some 3 coefficient may not be, or some variables may 4 not be significant. So I have to add more 5 data into it. Okay. So that turns 6 out -- the model is there whether when you 7 feed in the numbers or the datas, it may 8 change a little bit before what you've 9 forecast. 10 Okay? That's the result that we 11 have, then I would like to have your advice 12 is, are we ready to use this unobserved 13 components model or called structural model 14 to estimate other oil stocks? Currently, our 15 petroleum supply division is running 19-20 16 different models, then they pick up one, 17 which is plus the expert opinions, say this 18 is the one, we saw them marketed in your 19 concern, they put up as a weekly other oil 20 stocks. 21 Then it's good, you can see which 22 is comparable to what I did on the modeling, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 88 1 but the only thing is when they pick that, 2 they have an expert opinion, it may not be 3 reproducible. So which is published, okay, 4 but they may not be reproduced again and 5 again. If I have the modeling simply as 6 adopted then we can rang it to a (inaudible). 7 And a year later, we checked the model, fit 8 it, how good is, then we try to adjust that. 9 And another thing is in the Short 10 Term Energy Outlook Division or -- that is a 11 EMEU office, they'll have another number 12 published, whether that's the monthly number. 13 So we're trying to reconcile, which is EIA to 14 publish almost the same number, not to do it 15 one number and the other one is little bit 16 different. They are discrepant each other, 17 okay. 18 And that is my presentation. So I 19 would like to have your advice how we go on 20 this. Yes. 21 MR. HENGARTNER: Thank you very 22 much. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 89 1 MR. FEDER: Actually, I have lots 2 of questions, so maybe others should ask 3 first, and I'll ask my questions. 4 MR. HENGARTNER: Go ahead. Oh, you 5 want to -- 6 MR. FEDER: No, I'm going last 7 because I have many questions. 8 MR. HILL: I have one 9 technical -- when you put up your model, 10 there was a -- it's a regression model. You 11 typically worry about the correlation of the 12 error (?) terms. Back two screens? 13 MR. LU: I think before that, yeah. 14 This one? 15 MR. HILL: Yes. Let me look at it 16 again. Entered into the IID. 17 MR. LU: Uh-huh. 18 MR. HILL: Usually it's hard often. 19 Firstly you taught us how to make that 20 assumptions. 21 MR. LU: Yes. 22 MR. FEDER: I don't know if you -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 90 1 MR. LU: Generally, they assume 2 each component is a stochastic. Okay. You 3 can test that, if not then you assume it is 4 deterministic. 5 MR. FEDER: You didn't give a model 6 for MuT? What is the model for MuT? 7 SPEAKER: They're trying to -- 8 MR. FEDER: Yeah, yeah, there's a 9 model for it. Are you assuming you have a 10 stochastic? What is the model for that? How 11 do we always say it's a model for the trend? 12 MR. LU: Oh, model for MuT is just 13 the -- it's a number. 14 MR. FEDER: It's not because you 15 don't have the stationary theories. Usually 16 we have been -- most of his models, MuT is 17 given as MuT minus one, plus stochastic 18 slope, which is around the work. And I 19 think -- but I don't know what T is in your 20 case, but usually there is a model from 21 MuT -- 22 MR. LU: MuT is the trend and the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 91 1 gamma T is the season. 2 MR. FEDER: I know, I know. Yeah, 3 but the question is what model used, what is 4 the stochastic nature of MuT? 5 MR. LU: Oh, MuT. 6 MR. FEDER: I'm pretty sure what 7 you've from MuT -- is -- MuT equals MuT minus 8 one plus stochastic slope, and usually for 9 the slope, you assume a random block model. 10 MR. LU: Yes, I did check it, 11 whether it's random work or not. Yes. 12 MR. FEDER: From not -- but not 13 from MuT, for the slope, which is MuT minus 14 MuT minus one. 15 MR. LU: Oh, yes. We check -- is 16 the Y T minus Y T -- 17 MR. FEDER: No, no, not Y T, MuT. 18 MR. LU: I know, but when you look 19 at the Y T and the Y T minus 1 -- 20 MR. FEDER: Right. 21 MR. LU: Then you can have all 22 those. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 92 1 MR. FEDER: But for that, you 2 also -- 3 MR. HILL: Okay, maybe the 4 question's whether MuT is a constant -- we 5 suspect it's not -- if you go by the stat -- 6 MR. LU: Let's go by the T -- 7 SPEAKER: Because MuT is -- 8 SPEAKER: Go by the T. 9 SPEAKER: It's not a constant, not 10 a constant. 11 MR. FEDER: Not a constant, not a 12 constant. And not just -- just independent, 13 because -- the trend should change slowly 14 over time. 15 SPEAKER: To change -- 16 MR. FEDER: And to make this -- you 17 make this -- the difference being a random 18 log. But okay, I have a few questions here. 19 In your modeling what is your -- yeah, Y T. 20 Do you model also the monthly basis in this 21 model or you fit this model only to the 22 weekly -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 93 1 MR. LU: Okay, I have the monthly 2 number. Okay. For the -- for everything. 3 SPEAKER: Right. 4 MR. LU: Then I convert to the 5 weekly number for the major product. 6 MR. FEDER: So this model is fitted 7 to both the monthly data and to the total 8 petroleum stocks and petroleum, I mean, the 9 weekly data that you have to both, right? 10 MR. LU: Yes, I convert everything 11 to the weekly, then free the weekly numbers, 12 do the forecast for 14 weeks, because I need 13 these three months ahead. Like here we say, 14 we publish this in the end of September. 15 MR. FEDER: Right. 16 MR. LU: And I need to estimate or 17 forecast the October weekly number. And I 18 can only use up to the July number. 19 MR. FEDER: Yeah. 20 MR. LU: Okay? So I need to focus 21 August, September, and October. 22 MR. FEDER: Yeah -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 94 1 MR. LU: But I will not use 2 October -- August and September, I only use 3 October number. 4 MR. FEDER: No -- I understand 5 that, and you said that you use the data 6 from -- beginning for 1993 to -- 7 MR. LU: To 2001, yes. 8 MR. FEDER: So in -- when you 9 fitted the model and estimated all those 10 parameters, did you use all the data that you 11 had, the converted monthly data and the 12 weekly data, all of that? 13 MR. LU: Yes. 14 MR. FEDER: Okay. 15 MR. LU: So maybe the weekly base 16 can go along. 17 MR. FEDER: Okay, I -- 18 MR. LU: And now I have, for 19 example, forecast the October, I do use the 20 weekly published major petroleum product in 21 the model. 22 MR. FEDER: In the model? BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 95 1 MR. LU: Yes. If they are in 2 the -- because they are here. 3 MR. FEDER: No, the seasonality, is 4 it quarterly here or monthly? What did you 5 say? 6 MR. LU: It's yearly. 7 MR. FEDER: No, the seasonality. 8 MR. LU: I know. 9 MR. FEDER: I know that cycle -- 10 what is -- is it a -- the seasonality terms, 11 how many seasonal terms did you have there? 12 MR. LU: Just one. Yeah, so I used 13 it a -- come out with 53 -- 52. It's 53, 14 yes. 15 MR. FEDER: A week? 16 MR. LU: Yes, yes. 17 MR. FEDER: Yes, also you have 52 18 dummies there. 19 SPEAKER: That's -- yeah, that 20 might be over 15. 21 MR. LU: Yes, I have a -- 22 MR. FEDER: You've been able to BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 96 1 estimate it, okay. 2 MR. LU: Well, you find out 3 seasonality is not significant. Okay, so 4 when I run the actual model, some components 5 are significant, some are not. Okay, when I 6 run these -- propane is the one, will be my 7 regressive. 8 MR. FEDER: It may be that your 9 seasonal terms turn out to be non-significant 10 because you had weekly seasonality which may 11 be too much. If you tried -- actually, I 12 wrote the -- 13 MR. LU: The cycle? 14 MR. FEDER: I wrote the paper 15 on -- I used executive same models, actually 16 Nancy has a copy. 17 MR. LU: You mean the UCM, this 18 one, structure model. 19 MR. FEDER: Yeah, but it used that. 20 It was -- yeah. Dr. Ferry (?) commented that 21 trigonometric representation of 22 seasonality -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 97 1 MR. LU: Yes. 2 MR. FEDER: Is better than the 3 dummy variable in representation. 4 MR. LU: Okay, yes. 5 MR. FEDER: And I think if you use 6 that, and if you had just quarterly or maybe 7 monthly seasonality instead of weekly, you 8 would get more significant terms of the 9 seasonal. I -- this is my guess. I -- 10 MR. LU: So you mean I should 11 use -- 12 MR. FEDER: Because you have so 13 many parameters, then getting significance 14 can be a problem. 15 MR. LU: So instead of 53, I should 16 use 13 weekly? 17 MR. FEDER: I would -- 11. 18 MR. LU: Yeah. 19 MR. FEDER: 12 minus 1, yeah. 20 MR. LU: Oh, okay. 21 MR. FEDER: Monthly -- I think so, 22 maybe even quarterly. I'm not familiar with BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 98 1 the subject matter to say it with certainty, 2 but I would at least suggest to try. 3 MR. LU: Okay. But if you'll say 4 11 -- 5 MR. FEDER: The number -- oh, well, 6 the number of seasonal terms is number of 7 periods minus one. That's the -- 8 MR. LU: Yeah, but three months 9 will be 13 weeks, not 12 weeks, right? Any 10 places, I didn't know what happened. 11 MR. FEDER: I understand as to 12 what -- 13 MR. LU: Yeah, quarterly, right? 14 MR. FEDER: Twelve months, yeah, 15 that's what I'm -- 16 MR. LU: Oh, you go by a month. 17 Okay, 12 okay. 18 MR. FEDER: Just -- 19 MR. LU: I thought it was a weekly. 20 MR. FEDER: No, because your model 21 has so many parameters. 22 MR. LU: Yes. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 99 1 MR. FEDER: That's usually a 2 problem unless you have a very long series, 3 and you have only 10 years of data, so -- 4 MR. LU: Yes, I have 400 something, 5 yes. And you did -- I used the 400 -- 440 6 and 0.2 feet. (?) 7 MR. FEDER: It's helped to saying 8 thus -- actually running it. You may have 9 enough data, I'm not sure. 10 MR. LU: Yeah, but the full model 11 won't be good. So I had to kick out 12 some -- and make it determinalistic. So in 13 this stochastic sense, there are only few of 14 them. Yeah, most of the audit, it didn't 15 come out significant. 16 MR. FEDER: Now, you showed us the 17 time series plots beginning with 2002. Have 18 you looked at the entire series from 1993? 19 Because we didn't see, I mean, that -- 20 MR. LU: Oh, no. 21 MR. FEDER: You might learn 22 something from it. I mean, just looking at BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 100 1 the -- 2 MR. LU: You mean, I should have -- 3 MR. FEDER: Yeah -- 4 MR. LU: Forecast back? 5 MR. FEDER: Yeah, yeah, yeah -- 6 MS. FORSYTH: To evaluate it, it's 7 like -- 8 MR. FEDER: Yeah, yeah. Just to 9 evaluate the fitness of the model. 10 MR. LU: Okay. You mean, the 11 monthly or the weekly base? Because here 12 it -- we publish weekly numbers knowing we 13 would never adjust, or just to let it go. 14 But in monthly, we may, or we have with us 15 what, annual numbers. 16 MR. FEDER: No, I would say for 17 evaluation purposes, to see how good -- 18 MR. LU: The model. 19 MR. FEDER: Unobserved components. 20 MR. LU: Okay. 21 MR. FEDER: Yeah, yeah, I think you 22 are right, you have to do it monthly or BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 101 1 quarterly, just to get a visual impression of 2 how good the models are, because there could 3 be some things that happen where there was a 4 break in the series for some economic 5 reasons, so the model might be affected. The 6 fitness of the model might be affected by 7 some spikes in those series and things like 8 that. 9 MR. LU: Yes. Since you have 10 experience over the UCM, I once -- this in 11 some month, the feats are good under the AIC, 12 Akaike Information Criteria is good. But in 13 some months they just -- boom, blow out. So 14 what happened? 15 MR. FEDER: Yeah. I don't know. I 16 was fortunate to work with series that are 17 more stable. 18 SPEAKER: Oh. 19 MR. FEDER: You mentioned that SAS 20 came out with their procedure two years ago. 21 I wrote my own procedures. I didn't -- I 22 wrote them actually in Gauss, so I -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 102 1 MR. LU: Oh, you used the Gauss, 2 oh. 3 MR. FEDER: Okay, there is actually 4 also a package that -- 5 MR. LU: Stem. 6 MR. FEDER: Gilbert from the Bank 7 of Canada wrote for -- 8 MR. LU: Yes. 9 MR. FEDER: That's been around for 10 a long time. 11 MR. LU: But do they have a UCM? 12 MR. FEDER: Yeah. 13 MR. LU: Oh, then -- 14 MR. FEDER: It's called -- 15 MR. LU: If they have (inaudible) 16 then -- 17 MR. FEDER: It's called DSM, 18 Dynamic Structural Models. 19 MR. LU: Okay. 20 MR. FEDER: You wrote it a long 21 time ago, it's available for free. 22 MR. LU: For free, yes. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 103 1 MR. FEDER: And (inaudible) as 2 opposed to -- his name is Gilbert, first 3 name, Paul -- Paul Gilbert. 4 MR. LU: Okay. 5 MR. FEDER: From Bank of Canada. 6 MR. LU: What happened is when I 7 run it, so many (inaudible) one or two 8 months. The feets are terrible and the AIC 9 numbers are way off. Then go on and stable 10 again. So I'm wondering something missing. 11 MR. FEDER: Right. 12 MR. LU: That will being 13 your -- the case, which I have (inaudible) 14 forecast. 15 MR. FEDER: That's a good question. 16 This is where subject matter people need 17 to -- 18 MR. LU: Well, he is the -- Mike is 19 the manager of the -- this weekly and monthly 20 production -- publication. 21 MR. CONNER: The question 22 is -- tell me again what the question is. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 104 1 MR. FEDER: Well, he saw some 2 misfit in some when you go back historically 3 and you look at the expected -- I mean, what 4 you do is for every point of time T, you can 5 use the data up to the time T to predict the 6 next observation, and then you look at the 7 difference. And if the prediction -- that's 8 called -- the difference is called a 9 prediction error. And sometimes that is 10 small, sometimes it's more than you expect -- 11 SPEAKER: Yeah. 12 MR. FEDER: Which is the model 13 doesn't give a very good prediction of the 14 next value. And that could be because 15 something happened and the model doesn't 16 know, but subject matter -- subject matter 17 people might now. So this is where we 18 statisticians have to say, I don't know what 19 happened here, do you guys know? But I would 20 check because I personally don't use SAS 21 much, and I don't know what those (inaudible) 22 procedures do. I prefer -- I think it's BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 105 1 unnecessary whether to say, but I mean you 2 have to look and see what the model is and 3 what the model for the trend is, and what 4 kind of seasonality they use and compare it 5 to other packages as well. 6 Actually I want to say that I like 7 the unobserved components model and I think 8 that's the way to go, but maybe even that can 9 be improved. And I think Barbara also 10 mentioned that maybe there is some other 11 symptomatic variable that you can use in your 12 regression. 13 MR. LU: Oh, okay. 14 MR. FEDER: That might improve it; 15 that can explain the spike -- I can give an 16 example how we actually did a paper on the 17 effect of seat belt legislation in Britain on 18 fatalities, highway fatalities. And they had 19 an intervention term which accounted for the 20 introduction of the legislation. In your 21 case, maybe some kind of change in the market 22 could also be entered into the model to BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 106 1 explain a big discrepancy between the 2 observed and the expected or things like 3 that. So this is where -- subject where the 4 people can bring in other variables and other 5 knowledge into the model. 6 MR. CONNER: Yeah, and actually one 7 of the ones that -- that's always been sort 8 of my head theory, if you will, is that 9 there's actually -- we talk about the fact 10 that we have inventory data in all these 11 major products. But we don't have any survey 12 data on the other oils. 13 But it's always been -- been my 14 feeling, just having looked at the data in 15 sort of an informal way, that there are in 16 fact relationships between the inventories 17 for which we do have survey data and the 18 inventories of the other oils, specifically 19 because in the refining business, nobody 20 shows up at the refinery for work and says, 21 lets make other oils today. They say, we're 22 going to make gasoline today. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 107 1 And the other oils are large 2 by-products of the other processes in the 3 refinery, and therefore, to the extent that 4 you know the crude runs and the production of 5 these various other things from the survey 6 data, there ought to be things that you can 7 infer about where the other oils inventory 8 should be going based on that, as well as 9 other factors, you know? You obviously got a 10 lot of -- 11 MR. LU: The propane is the one 12 highly correlated with the other oil stocks. 13 MR. FEDER: But since you asked 14 us -- you asked the committee about 15 recommendation, can I just say -- so maybe 16 it's not the committee, but my recommendation 17 would be yes, to go with that. Consider a 18 few things in the model, such as including 19 more variables, including -- look if you can 20 use -- now you also converted monthly data 21 into weekly -- 22 MR. LU: Into weekly, yes. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 108 1 MR. FEDER: Which you can -- you 2 don't have to do it if you write the 3 observation equation in the (inaudible) 4 models in the right way, so maybe you don't 5 need to do the conversion. But that's too 6 technical; we haven't seen enough details to 7 comment on that. I would also try to fix 8 seasonality which is less, shall I say 9 greedy, not have too many seasonal terms, 10 just because when models become very big, 11 their performance become less predictable. 12 MR. LU: Okay. But the -- in fact, 13 the seasonality is not there. Or can I use 14 any cycle? Even though there's no 15 seasonality or just cycle, cyclic effect. 16 MR. FEDER: The cycle is -- has a 17 larger -- that's the economic cycle, right? 18 MR. LU: No, you can have three or 19 four, the cycle. 20 MR. FEDER: Yeah, I don't know. 21 MR. LU: The quarter is in numbers. 22 Like a -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 109 1 MR. FEDER: But then it becomes a 2 seasonality, right? Cycle in this context 3 usually is a multi-year cycle, I'm not an 4 economist, but I think that's what 5 they -- Walter, do you know if the term cycle 6 here usually used for longer period? 7 Business cycle, I know that business cycle 8 usually applies to a longer period. 9 MR. LU: Oh, okay. 10 MR. FEDER: But I don't know about 11 SAS -- you calling cycle here. If this is 12 less then it's confounded with seasonality, 13 then you're not going to get anything out of 14 it. 15 MR. LU: You may. Yeah, you may 16 confound it. 17 MR. FEDER: Yeah, I think you're 18 talking here about a business cycle, which is 19 a longer period than the seasonality. 20 MR. LU: Yeah, generally you pick 21 up one or the other. We either use in cyclic 22 effect or -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 110 1 MR. FEDER: And again, one more 2 thing is plot the series beginning from -- 3 MR. LU: The data I have. 4 MR. FEDER: And look at all this 5 and the subject matter, people should look at 6 all the big prediction errors to see if 7 there's an explanation for that. As a way 8 of, as Barbara said, to see if how well the 9 model fits. 10 MR. LU: Okay. Yeah, because we 11 saw that the biggest problem is in 2003. So 12 we'll concentrate on there and go on 13 (inaudible) whatever happened to before, 14 because before it's done so we has a -- hey, 15 here's the problem, we don't want a similar 16 thing happen in the future, in 2003 -- 2004, 17 2005 or even this year, you know. What 18 happened is end of 2005, we have a little bit 19 of sense this committee coming out again. 20 So we generally have to rely on the 21 experts and say, do you sense any discrepancy 22 of your weekly and monthly numbers at the end BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 111 1 of the 2005 or at the beginning of 2006. Or 2 a missing market will tell you and we don't 3 know. Because so long it was three months 4 lag. You know, even we say two months data 5 available, it's a three months lag. 6 MR. FEDER: Yeah. 7 MR. LU: A missing happening in 8 these three months is a big intervention; 9 right? 10 MR. FEDER: Yeah. 11 MR. LU: It may give the weekly 12 number off by a lot. You know, like I 13 believe in 2003, there was an embargo from 14 Venezuela, is that right? 15 MR. CONNER: Yeah, there were some 16 issues with crude production in Venezuela 17 around that time. 18 MR. FEDER: That has to go in to 19 the model as a fixed -- as opposed to 20 non-random effect, and that can take care of 21 it. I can give you an example 22 what -- actually, it's in this paper which I BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 112 1 sent Nancy a couple of years ago when we did 2 the estimation of -- for households in Canada 3 from -- and we had data coming from three 4 different sources. 5 Like, okay, so you had, one was the 6 census, which in Canada is every five years, 7 not every 10 years like here. The other 8 source of data was the big source of data 9 that was the labor force survey, which 10 is -- which had given us monthly data, and 11 then we also had the demographic series of 12 the number of people. Number of people is 13 linked to the number of households because 14 everybody has to live somewhere and so we 15 used that, too. 16 And what we did, we wrote, like 17 yours, unobserved component model, we call it 18 the -- 19 MR. LU: Structure. 20 MR. FEDER: Structure model, right. 21 MR. LU: Yeah, dynamic structure. 22 MR. FEDER: And we wrote with the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 113 1 observation equation written in such a way 2 that all the data we had was represented, the 3 selenium function of the unobserved 4 components. 5 MR. LU: Okay. 6 MR. FEDER: And I can tell you that 7 I did this in -- I did it in 1999 and in 2001 8 there was a census and I visited in 9 Canada -- I believe this -- yeah, I was then 10 in Britain and the demographer that worked on 11 my data told me that the predictions were 12 within one percent of the actual numbers. 13 MR. LU: Wow. 14 MR. FEDER: So, but, we don't have 15 volatility like that with households -- 16 MR. LU: Human beings are easier to 17 observe. 18 MR. FEDER: Huge improvement of 19 what was known before. And so -- but I know 20 your data are much more volatile, and you can 21 see it in the plots. And we had much better 22 data. This -- I also serve in Canada, the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 114 1 labor force survey is 70,000 people every 2 month. 3 MR. LU: Yeah, pretty good. 4 MR. FEDER: So what I'm saying is 5 by writing the observation equation in your 6 states based model correctly, you can capture 7 all the data that's in your -- 8 MR. LU: Different components. 9 MR. FEDER: In the three sources of 10 data that you have here. 11 MR. LU: Okay. 12 MR. FEDER: But yeah, it's not an 13 easy -- I mean, there's a lot of work that 14 needs to be done here. But I think -- and 15 you can account for the lag, the three months 16 lag and for the difference in periods, 17 everything can just go in to the model. My 18 only concern is I'm not sure the SAS 19 procedure has the flexibility in it, you may 20 need to either write your own procedures or 21 not then make some compromises, that's all. 22 I think Gilbert's package may have some BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 115 1 capabilities, too, that package I mentioned 2 to you before as -- 3 MR. LU: Okay. Yeah, in R -- 4 MR. FEDER: Yeah, yeah. 5 MR. LU: Here, SAS is easier to 6 input, then you can write and then they tell 7 you the components. You can specify the 8 components in different terms, then you check 9 the significance of each one. 10 MR. FEDER: I don't use SAS enough 11 to say I don't know. 12 MR. LU: Okay. 13 MR. FEDER: But maybe, I mean, SAS 14 is a good product, so maybe they have all 15 these capabilities, I just don't know. 16 MR. LU: But it is an interesting 17 piece because the whole theory was published 18 in Britain, to start with. 19 MR. FEDER: Yes. 20 MR. LU: So British people should 21 have a better (inaudible) programs to run 22 this. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 116 1 MR. FEDER: The Bureau of Labor 2 Statistics is using all its monthly estimates 3 now based on states based model. 4 MR. LU: Okay. 5 MR. FEDER: I can -- 6 MS. BLUMBERG: There were some 7 papers at the joint staff meetings on that I 8 went to. 9 MR. FEDER: Yes. 10 MS. BLUMBERG: I can't -- 11 MR. FEDER: You can talk to Stuart 12 Scott, William Bell, and Richard Tiller. All 13 these people are using these models and have 14 written papers on it -- and here in town. 15 MR. LU: Okay, can I send you the 16 e-mail then you'd -- 17 MR. FEDER: Absolutely. 18 MR. LU: Yeah, you tell me, how who 19 I can contact and get the census. 20 MR. FEDER: I can actually -- yeah, 21 yeah. But, all the -- I mean, I know the 22 Bureau of Labor Statistics in fact, they're BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 117 1 also having an effort now to use benchmarking 2 procedures built on states based modeling, 3 which is the same as the unobserved 4 components model to benchmark the (inaudible) 5 data to the national estimate. So, -- 6 MR. LU: Yeah, there's a -- 7 MR. FEDER: They're doing a lot of 8 work on that. Richard Tiller, T-i-l-l-e-r, 9 is involved in all these things, so you can 10 contact him for example. Stuart Scott and 11 William Bell used to work on it as well. 12 MR. LU: This one have a several 13 difference. Some people call dynamic 14 structure, now you'll see that we'll call it 15 structural models. 16 MR. FEDER: Technically speaking, 17 that the structural model, the stripes, what 18 we're assuming about the process, and then 19 you can represent it at the states based 20 model. So technically, the two aspects of 21 slightly different things, but you're right, 22 people use different technology and -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 118 1 MR. LU: Yes. 2 MR. FEDER: Yeah, but actually, 3 since how we started that, we've used it in 4 Canada, I wrote a paper with two 5 collaborators on Australian data and then, as 6 I said, actually one of them wrote this 7 papers with filler here in Washington, I'm 8 using this method, so it's used here too. 9 MR. LU: Okay. 10 MR. FEDER: In fact, I'm told that 11 all the BLS estimates of unemployment are 12 based on states-based modeling. 13 MR. LU: Do they use that to adjust 14 or do they use that to benchmark? 15 MR. FEDER: No, no. Right now to 16 adjust but there's a ongoing project to do 17 benchmarking. 18 MR. LU: Okay. 19 MR. FEDER: To improve the 20 precision of those estimates. 21 MR. LU: Yeah, because I know they 22 have a weekly number and they also have a BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 119 1 monthly number. You know, the only thing 2 weekly number they won't coincide is the said 3 date. Well, you probably see the monthly and 4 we may not be probably seeing the weekly 5 number. Here we probably see weekly on every 6 Friday, right Mike? 7 MR. CONNER: That's correct. 8 MR. LU: And then in the monthly -- 9 MR. CONNER: I think they're 10 collected as of Friday, we publish on 11 Wednesday. 12 MR. LU: Yeah, yeah, collected on 13 Friday and then published on Wednesday. 14 MR. FEDER: If you Google the names 15 of Tiller and Pfefferman, you'll probably hit 16 the website at Southampton when they have a 17 working paper on that which you can download 18 as well. And you're -- 19 MS. BLUMBERG: Scott -- Stuart 20 Scott and Bell both gave papers at -- just go 21 in because we all got that -- we all got a CD 22 with all the abstracts on it, so you can just BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 120 1 pull that off. 2 MR. FEDER: Do you mean this 3 year's? 4 MS. BLUMBERG: This year, yes. 5 MR. FEDER: From the abstract web 6 page you can download a lot these papers, 7 including Pfefferman's papers that he's done 8 in the 1990s. 9 MS. BLUMBERG: And I think I have 10 one of them, I remember some test, I went to 11 the session, but -- I'll make a copy and send 12 it over to you. 13 MR. LU: Okay, so 2006, JSM. 14 MR. FEDER: Ruey-Pyng, you're 15 actually right that historically it started 16 in -- with Harvey and Europeans were doing it 17 more, but I would say the last 10 years it 18 become more and more common in this year, and 19 now the BLS, that's what they use, so 20 it's -- and now as I learned from you it's 21 also part of SAS. And Gilbert in Canada had 22 written that packages so and so -- it's BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 121 1 definitely becoming more widely adopted. 2 MR. LU: Okay. 3 MR. FEDER: But I think if -- I 4 cannot speak for the committee but for 5 myself, I endorse this approach, I think it's 6 great, adjusting that -- you need to keep 7 working and making sure that those spikes 8 will be smaller. 9 MR. LU: Okay. But you don't think 10 I need to convert it to the weekly time 11 series to work on (inaudible) on the monthly. 12 MR. FEDER: I can give you example 13 from what we did when we needed to use the 14 demographic series of number of people. 15 Because the other -- the census numbers and 16 the labor force numbers were directly the 17 number of households, so that we did not 18 conversion, but how do you use the number of 19 people? We just wrote an observation 20 equation, which says the total number of 21 people is the sum of the following, number of 22 households of size one plus two times the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 122 1 number of households of size two plus three 2 times the number of households of size 3 three" -- and so on. 4 MR. LU: Uh-huh. 5 MR. FEDER: Unfortunately, we 6 didn't have the number of households of size 7 10 or whatever so we made some kind of 8 assumption about households of size four and 9 bigger. But essentially we wrote, in the 10 observation equation, we wrote the fact what 11 we knew. And the same thing here. You have 12 a monthly series, you say that the monthly 13 number is equal to the following -- 14 MR. LU: Components. 15 MR. FEDER: Components. And you 16 just sum it up. Again, I'm not sure exactly 17 how the input into SAS is, so I -- but I know 18 in theory that's what you have to do. 19 MR. LU: Uh-huh, yes, uh-huh. 20 Okay, yeah. 21 MR. FEDER: Yeah. But conversion 22 might also work. I'm not saying it's a BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 123 1 better provision (inaudible). All I'm saying 2 is use as much data as you have, plot the 3 series starting from 1993 to today, look at 4 all the differences. If you see a different 5 stock -- I'm not sure -- 6 MR. CONNER: Mike. 7 MR. FEDER: Talk to Mike and say, 8 hey Mike, look at this, why do I have a big 9 difference in 1994? And he'll say, oh, 10 that's because in that year Venezuela decided 11 to shut down something. And they said, okay, 12 I'll edit as a fixed effects in my model, or 13 whatever and you proceed until you have a 14 good faith. 15 MR. LU: Okay. 16 MR. FEDER: So, I'd say good 17 approach and just -- 18 MR. HILL: Do you think household 19 data are much more stable for both? I don't 20 know if it's -- probably it's there might be 21 spikes in here that you wouldn't have in 22 households. The price of housing doesn't BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 124 1 jump suddenly in Toronto overnight. 2 MR. FEDER: Exactly. 3 MR. HILL: Whereas the oil price 4 can jump. 5 MR. FEDER: Exactly, exactly. 6 MR. HILL: And maybe or maybe that 7 may remain, that may remain. These are 8 stockpiles, so I assume it's less sensitive 9 to short term pricing changes. But there may 10 be affects like that in this data series that 11 you wouldn't find in housing. 12 MR. FEDER: Yeah. 13 MR. LU: STEO is another office in 14 EIA, published the prediction numbers 15 monthly. And their number is the pink one 16 here, you can see that, which may not 17 consider the time series effect, although 18 regress in the moving everyday pieces. It's 19 purely just the -- 20 MR. FEDER: You just said something 21 on what -- 22 MR. HILL: That's the beginning of BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 125 1 the Gulf War, this Iraq war number two, March 2 '03. 3 MR. LU: No, that's in '02. 4 January, 2002. Or was it -- 5 MR. HILL: I think that's the war, 6 yes. 7 MR. FEDER: Maybe that is to -- I 8 just want to say but the spikes in the 9 housing in fact there is something that 10 happens every year when students go to 11 college day that the household composition 12 changes. So there are some things that -- 13 MR. LU: Okay. 14 MR. FEDER: But, we didn't have a 15 big effect so we -- and the models somehow 16 adjusted for it. But -- 17 MR. HILL: Seasonality. 18 MR. FEDER: You may be right, I 19 mean, could be some things that happened that 20 Mike might be able to come up with some 21 auxiliary data that you can include in your 22 model. I mean it's nice that in your model BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 126 1 you had some fixed effects as well, so you 2 might be able to find the right variables to 3 put in the model to explain the discrepancy. 4 MR. LU: Okay. 5 MR. HILL: That is the start of the 6 war. The Iraqi phase of the -- 7 MR. LU: January -- 8 MR. HILL: March, it happened 9 during the spring, I -- we were at spring 10 vacation, academic spring vacation. 11 MR. LU: Spring, 2003? 12 MR. HILL: Yes. 13 MR. LU: Oh, yeah. Before that, 14 it's a -- it started with January, 2003, 15 right? That's the -- 16 MR. CONNER: I think the actual 17 invasion was when the U.S. actually invaded 18 Iraq was in March. 19 MR. LU: March, yeah. 20 MR. HILL: Yes. 21 MR. CONNER: I think it was March. 22 I mean there's probably, you know -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 127 1 MR. HILL: It had some effect on 2 the oil market and stockpiles, yes, around 3 the time of the invasion. 4 MR. LU: Yeah, I needed to 5 (inaudible) to that to the PCM and the weekly 6 number to see how close they are. That's the 7 start of they didn't catch up with -- we call 8 it sudden impact or some intervention 9 factors. 10 MR. FEDER: Can I just ask you 11 since I'm the -- discussed it, I just want to 12 make sure I will not say things that are 13 wrong; the source of the monthly basis series 14 is stocks in other oils, which has a 15 two-month lag, right? 16 MR. LU: Yes. 17 MR. FEDER: Against that it was 18 publication delays. 19 MR. LU: Yes. 20 MR. FEDER: And the other sources 21 of total petroleum stocks and petroleum 22 products supply both are -- BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 128 1 MR. LU: Weekly. 2 MR. FEDER: Weekly, instantaneously 3 almost. 4 MR. LU: Yes. 5 MR. FEDER: But they're more 6 volatile? 7 MR. LU: It depends. You know, 8 yeah, it's dependant on the markets. 9 Sometimes it's, yes, up and down. 10 MR. FEDER: Yeah, yeah, okay. 11 MR. LU: Yeah. In this weekly you 12 can see all those major six components. So 13 every week, which was the survey result from 14 the refinery. So it's only supply. Okay? 15 But in the other components of other oil, 16 okay, like asphalt and all those other 17 things. These are only monthly based. We 18 don't have a weekly number, right, Mike? 19 These are all monthly numbers. 20 MR. CONNER: That is true, yes. 21 MR. LU: Yeah. But the -- our 22 customer needs weekly number of this. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 129 1 Especially the weekly stocks of the, you 2 know -- and right now, you -- if you look at 3 the monthly publication you may find these 4 numbers, right? I just checked, in some of 5 them you can see, but in the others, not -- 6 MR. FEDER: Well, you used the 7 structured models of Harvey to write down 8 what you think. But of course, next to this 9 there's also the observation equation, which 10 is also always part of the modeling. And the 11 observation equation can be written to 12 reflect all the data that you have, and it 13 almost work like magic. You write down 14 everything, you crank the model, and you get 15 estimates of the state, which usually behave 16 quite well. 17 So, if you have just monthly data, 18 if you put it correctly in your model, and 19 you run the model, assuming that the model 20 performs well, you will get weekly estimates 21 from the model that should be quite good. 22 This is what I suspect. So even if those BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 130 1 data are available only monthly, since you 2 have -- also have other weekly data, I think 3 you can actually write a model to use all the 4 data and get the estimates that you want. 5 MR. LU: Yeah, but they are two 6 different approach. One is just using the 7 monthly -- predict the month's number, then 8 convert it back to the weekly. 9 MR. FEDER: Yeah. 10 MR. LU: The other one is convert 11 everything to the weekly then go ahead 12 forecast the weekly. 13 MR. FEDER: Well, it depends how 14 correctly the conversion to weekly is. I 15 prefer that the model does it. I mean, 16 rather than break it down to weekly, it's 17 like in the explanation, like what I gave 18 about using the population total to use to 19 estimate the number of households. Because 20 we know that there's this linear relationship 21 between the population total and the number 22 of households, I was able to write the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 131 1 observation equation and let the model do the 2 estimate. 3 MR. LU: Okay. 4 MR. FEDER: Same thing here. Even 5 though this is given only as a monthly 6 aggregate, you know that this is the sum of 7 the weekly numbers, you can just write it in 8 the equation and run the model and get the 9 estimates from the model rather than 10 disaggregate it like you said. 11 MR. LU: Okay. 12 MR. FEDER: Because this -- the 13 model should do it for you. I mean, the YTD 14 observation that you have in your model? 15 MR. LU: Uh-huh. 16 MR. FEDER: Can be written as the 17 weekly -- I'm sorry, the monthly can be 18 written as the weekly. 19 MR. LU: Yes, uh-huh. 20 MR. FEDER: Yeah. 21 MR. LU: Okay. 22 MR. FEDER: I mean, it requires BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 132 1 some thought but -- that's what we normally 2 do in this -- 3 MR. LU: Okay. 4 MR. HILL: All right, I think 5 we're close to 11:00, if there's -- okay, in 6 this -- let me end this session, thank you 7 very much. 8 MR. LU: Thank you. 9 MR. HILL: And we reconvene here at 10 11:15. 11 (Recess) 12 MR. HENGARTNER: All right, let's 13 get going, please. We are in our last leg 14 for this meeting, so what I'd like to do is 15 have short reports or summaries from the two 16 break-out sessions. If Moshe is ready he'll 17 be first, otherwise I'll go. Moshe is ready, 18 so excellent. 19 MR. FEDER: I want you to make the 20 last presentation of last chairmanship, so 21 I'm going to go first. Actually, we had a 22 very interesting discussion here. Let me BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 133 1 first describe the issue. There's a demand 2 for weekly figures of stocks of other oils. 3 And the data sources for the stocks in other 4 oils is a monthly basis data source and 5 there's also a time lag of about two months 6 until it's available and three months until 7 it's actually published. 8 In addition, there's weekly stocks 9 in other oils available from total petroleum 10 stocks and for petroleum product supplied 11 publications. So there was a modeling 12 attempt here to produce weekly estimates of 13 stocks of other oils that Ruey-Pyng Lu was 14 talking about. 15 And of all the methods that he 16 described, the one that looks very promising 17 is a what they call unobserved component 18 model based on these structured models that 19 have been developed and have been in use now 20 in many places. In particular, I would 21 mention the Bureau of Labor Statistics 22 estimates a lot of employment figures which BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 134 1 are now based on that. 2 Without going into specifics of the 3 model and what goes into it, roughly, the 4 assumption is that the observed data is a 5 linear function of a trend, seasonality and 6 auto regressive component, some fixed effects 7 and the random noise, and the team that's 8 been working on it used a relatively newly 9 created SAS procedure. Can someone please 10 mention the name, I forget, what's the name 11 of the SAS procedure? 12 MR. LU: UCM. 13 MR. FEDER: UCM, I guess that's an 14 acronym for Unobserved Component Model. 15 MR. LU: Yes. 16 MR. FEDER: To evaluate the method. 17 And what they did they used data beginning 18 from January 1993 up to 2002, and used a 19 fitted model to predict the figures beginning 20 from 2002 to 2004 and looked at the 21 prediction error and compared it to other 22 methods. And I hope I can speak for the BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 135 1 entire break-out session group. We felt that 2 the method looked promising. Some things 3 that we suggested to the speaker to improve 4 the model, but I think maybe it's the best 5 way to proceed. 6 There's a few things that could be 7 done to improve the model including looking 8 at the series beginning from 2003 plotting 9 the series and examination of the one step 10 ahead prediction errors to see when there's a 11 large discrepancy, try to see if there are 12 some effects that could be used to explain 13 those differences. We also talked about 14 perhaps using different seasonality model. 15 Currently, the seasonality model 16 assumes weekly effects which amount to 53 17 parameters which might be an overkill, so we 18 suggested maybe to try a seasonality based on 19 a monthly rather than weekly effects. In 20 addition, the monthly breakup currently 21 broken down into weekly data, there's another 22 way to do it, by just writing the model BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 136 1 differently without doing this, and I think 2 this is it. 3 One thought actually that I had 4 that I didn't mention is maybe also try 5 logarithmic scale just because you have data 6 of a time that may have (inaudible), so maybe 7 it's also worthwhile to try logarithmic 8 scale, although I don't know if it will make 9 big difference. I think this is it. 10 MR. HENGARTNER: Thank you very 11 much, Moshe. Okay, I have the pleasure of 12 attempting to fill in the big shoes that is 13 Mark Burton's, and summarizing our last 14 discussion downstairs as -- I kind of agree 15 with Moshe, no -- we had a very lively 16 discussion downstairs. As far as I 17 understand, we barely got to slide two or 18 three, and then all hell broke loose, and 19 that was a good thing, it was a very good 20 thing. 21 And the issue was about 22 forecasting, especially for NEMS model and BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 137 1 just to give you a context and it took me a 2 while to understand. Those NEMS models are 3 extremely large. What I mean by large is 4 they are complicated, have many bells and 5 whistles, and lo and behold making one run, 6 one prediction, can take up to eight hours. 7 And so it's not a matter of 8 fact -- matter of just making the few runs, 9 it's easy to make predictions. It actually 10 is a time consuming process it turns out that 11 the models as well as the forecasts in the 12 past -- I'm going to say 10 years, but maybe 13 its 15 years have been saved, and there is a 14 notion that if we could look in our rearview 15 mirrors we can actually learn something from 16 what we've thought about the model and about 17 reality, and it maybe how that mismatch can 18 teach us what's going on. 19 And it goes a little bit in what 20 Tom Rutherford was telling us yesterday, 21 trying to figure out what the answer is, what 22 the model is and then try to learn something, BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 138 1 and in a way that's exactly what we're doing, 2 by doing this backcasting. And the thought 3 is, at least my thought was that this is 4 extremely important in light of the need of 5 more analysis at the EIA. 6 It's very important in this 7 interplay between data collection analysis to 8 be able to improve the product, and exactly 9 it's by being able to look back and 10 understanding what comes in the future is 11 that -- that's where the sweet spot is for 12 that kind of work. 13 So clearly there is a need for this 14 and, I think everybody was in roaring 15 agreement about that. There were some 16 notions of how to do this, there was a notion 17 of a NEMS light model introduced to say, you 18 know what, we actually can get simplifying 19 models, we can get statistical models that 20 quite accurately predict the results of the 21 NEMS simulations. 22 And if we actually are able to use BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 139 1 these models to explore in a fast and dirty 2 way, and I don't want to use dirty in a bad 3 way, but at least in a fast way to be able to 4 understand what's going on then that has a 5 lot of value for us to understand how things 6 work. 7 The other advantage of having this 8 retrospective look is to actually to bring 9 together analysts and the modelers in a room 10 and say, okay, what did we do, what did we do 11 well, what we did wrong, and somehow be able 12 to keep the model alive by asking yourself, 13 did we do a good job? I mean it's easy to 14 say, well, these are the last year's numbers, 15 I kind of forget, but there is value of going 16 back and thinking and having some 17 retrospective view of things. 18 And there are lots of notions and 19 modeling issues that arose. I'm just going 20 to mention two of them. One had to do with 21 identifying missed factors or the sensitivity 22 of specific factors on the predictions. And BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 140 1 again, that seems like an obvious thing to 2 some of us, until we understand that making 3 one run takes eight hours and we don't have 4 the luxury of doing those very simple or 5 things that we statisticians think, oh, that 6 easy let's get the leverages. 7 So no, that doesn't work; we have 8 to be clever, we have to be understanding 9 about what the challenges are. And one is 10 the computational complexity of the models 11 considered. The other thing is in terms of 12 factors in the models, there's a lot of 13 factors that are included for policy reasons 14 not for modeling reasons, and being able to 15 balance these things is of interest. 16 And there's also a notion that by 17 going back, we can learn or we have the 18 opportunity to learn from what we have done 19 so far, so this was a early stage 20 presentation, and the committee felt by all 21 means it should continue, at least I'm 22 putting words in their mouth, and I hope you BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 141 1 will corroborate with me on this. 2 But that was a very worthwhile 3 endeavor, and by all means should go forward 4 even if it meant maybe battling some 5 resistance, and there was discussion of that 6 too, from -- for political reasons, let's 7 leave it that way. This is a summary, and I 8 invite anybody in my session to please 9 correct or expand my summary. 10 MR. FEDER: I was not in your 11 break-out session. 12 (Laughter) 13 MR. FEDER: I do have a comment. 14 Last week, I attended a meeting at Virginia 15 Tech where modeling was discussed. In fact, 16 there was a brief mention also of EIA, the 17 speaker was, and I hope I'm saying his name 18 right, but let me see I can e-mail it to you, 19 I think it was John Sterman, sorry -- he's 20 from MIT, and he talked about model 21 verification, model validation, an extreme 22 value test that you can apply to models. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 142 1 By that, for instance in our case, 2 if you pushed the value of a barrel of oil to 3 zero or to 200 you look at the results, and 4 see if it makes sense or not and that allows 5 you to check. So I was thinking perhaps, if 6 we could in the future, a point in time, 7 perhaps Nick you won't be in attendance, have 8 a session devoted to modeling, and maybe we 9 can have this guy or someone else. 10 Actually, I have the paper he 11 wrote, that I can also send to you, and that 12 was interesting because the discussion was 13 not just on specific models, but how do you 14 assess models in terms of verification and 15 validation. The difference between the 16 two -- perhaps I won't go into that, but 17 those are important issues, I think model 18 validation is easier than verification, but 19 both should be done. So that's all I wanted 20 to say, even though I wasn't in your 21 break-out session. 22 MS. KIRKENDALL: I'd be real BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 143 1 interested in the paper, that'll be great. 2 MR. FEDER: Yeah, I'll send it, I 3 have only hardcopy, but I'll send it to you. 4 MS. KIRKENDALL: Yeah, you could 5 fax or whatever. 6 MR. FEDER: Yeah, actually I 7 also -- if I may just take one more 8 second -- actually, I forgot to mention that 9 Ruey-Pyng and I talked about this, he 10 mentioned the fact that the models he was 11 talking about Emerging Britain by Harvey, and 12 actually Nancy, you spoke about this 13 gentleman, I think his name was Smith that 14 you once presented JSM. 15 MS. KIRKENDALL: Black Smith (?). 16 MR. FEDER: Yeah, that worked on it 17 here, I mentioned to him that actually the 18 BLS is now basing all its modeling, and I 19 mentioned the names of Richard Tiller, Stuart 20 Scott, Bill Bell who all worked on this kind 21 of models, so those are -- and I'm going to 22 send him some more information. And Carol BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 144 1 who was here before, she actually is familiar 2 with some of those publications and knows 3 about some JSM papers that have been written 4 on that, so it's much more value adopted now, 5 and there is many people in town who know 6 about it a lot that could also be -- 7 MS. KIRKENDALL: That was their own 8 software though, they aren't using the SAS 9 module. 10 MR. FEDER: No, and in fact -- I'm 11 not sure what capabilities the SAS module has 12 in terms of writing the equations that are 13 needed. Another name that I mentioned here, 14 is Paul Gilbert from Bank of Canada. He 15 created a package called, I think 16 DSN -- dynamic -- anyway he created a package 17 that goes with R and S plus which is publicly 18 available, I downloaded it, and also has this 19 capability. 20 It's been there for a while plus 21 very nice documentation as well, so 22 you -- Ruey-Pyng is going to look at it and BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 145 1 I'll send him some more including that paper 2 that I sent you Nancy, that I wrote on this 3 subject. I wrote it, I think, in 2001, it 4 was a review paper, I'll send e-mail. 5 MS. KIRKENDALL: Okay. 6 MR. FEDER: Actually, maybe I'll 7 send it to him as well as the paper that he 8 will give you. 9 MS. KIRKENDALL: That's fine. 10 MR. FEDER: Okay, all right, so 11 I'll send it to you, the paper that this MIT 12 gentleman wrote. 13 MS. KIRKENDALL: Yeah. 14 MR. FEDER: Okay. 15 MS. KIRKENDALL: Yeah, Ruey-Pyng 16 can share it. 17 MR. HENGARTNER: Any more comments 18 from Jae, Tom? 19 SPEAKER: No. 20 MR. HENGARTNER: Cutler? 21 MS. KIRKENDALL: Wherever you are. 22 (Laughter) BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 146 1 MR. HENGARTNER: Okay, so at this 2 time I'd like to invite additional comments, 3 and from the committee discussions as well as 4 from the public audience. Before 5 continuing -- public audience? Good. What 6 else is on the agenda, the agenda -- we are 7 coming to a close. There's time for comments 8 and suggestions from the committee. There's 9 also probably time to determine the date for 10 the next meeting. 11 MR. WEINIG: I'm not prepared to do 12 that. 13 MR. HENGARTNER: Not prepared to do 14 that, that's fine, putting Bill on the spot, 15 that's okay. 16 MS. KIRKENDALL: Expect e-mails. 17 MR. HENGARTNER: The last thing I 18 want to do as the outgoing chair, I'd like to 19 actually take this -- just want to thank the 20 EIA for what they have done. I mean, what I 21 found valuable here is that they've been able 22 and willing to stick out their neck and BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 147 1 present things early. And that is a big risk 2 because they are unfinished products, they 3 are products that they are in process of 4 doing. These are also the products that 5 have -- we have the most impact with our 6 comments. But it takes really courage to go 7 and present things that you are not fully 8 confident about yet. And I want to thank 9 Nancy for being able to push this forward 10 because it's not an easy thing to do. And I 11 want to thank the staff of EIA for putting up 12 with those requests, because again, it's a 13 tough thing. 14 Now it takes two to tango, and I 15 recognize the committee has been very good in 16 their positive support of this endeavor, 17 right. If we'd be overly critical, it 18 wouldn't have worked either. So it's very 19 important for us to give positive feedback 20 and good feedback, right, but there's a way 21 of couching it, and I think this committee 22 has been extremely successful in doing so. BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382 148 1 And it's this partnership that I 2 think has led to what I would qualify very 3 good meetings, and I'm just encouraging you 4 to keep on this tone. I see a bright future 5 for this committee in the future, because I 6 know the people coming on board are 7 excellent, so I'm actually going home. 8 I also want to thank the ASA even 9 if they are not here for the logistical 10 support. They've been very good at putting 11 up with our travel and hotels and whatever 12 and they should be commended for this. So, 13 thank you very much for a very productive six 14 years. I've been happy to be a member here, 15 and now it's time for me to go. Thank you. 16 MR. HENGARTNER: Okay, so, if there 17 is no other business we want to talk on the 18 record, let me suggest that we are adjourned 19 until next time. 20 (Whereupon, at 11:40 a.m., the 21 MEETING was adjourned.) 22 * * * * * BETA COURT REPORTING www.betareporting.com 202-464-2400 800-522-2382