To: Mike Buckley who wrote (32746 ) 10/3/2000 1:54:40 AM From: tekboy Read Replies (2) | Respond to of 54805 Please read the stuff at fool.com If after reading it you believe that modifications were made because of "disappointing performance" or that they were made as a result of data mining, we'll have to simply disagree. uh-oh, I think we might have to disagree. I agree there's no evidence that the modifications were made because of "disappointing performance," but it does seem to me they were not theory-driven changes but rather atheoretical tweaks made as a result of data-mining--that is, looking for statistical patterns alone. Why do I say this? Well, let's look at what the Fools themselves say about the evolution: BTD --> FF 4.0 "Ann Coleman managed to get the yearly returns for the High Yield 10 stocks and their price order from Beating the Dow co-author John Downes, and using that data, she developed a spreadsheet to test some of the questions that were plaguing the message board. O'Higgins had stated in Beating the Dow that the lowest priced stock was often a company in financial trouble but that the second lowest priced stock was frequently an outstanding performer. He even based a one-stock strategy, the PPP (Penultimate Profit Prospect) on that second lowest priced stock. Board participants wondered about this discrepancy. The first spreadsheet revealed that the lowest stock was indeed a dangerous proposition, underperforming the Dow as a whole.... Finding that the first stock was a loser and verifying the outstanding performance of the second lowest priced stock lead to the formulation of three strategies that beat Beating the Dow's BTD 5." This change--dropping the No. 1 stock and doubling up on No. 2--was driven simply by finding patterns in the data and assuming they would continue. That Higgins could come up with an explanation for the poor performance of No. 1 is unremarkable; I can come up with an explanation for just about anything after the fact. The question is, did he do so before he noticed the statistical pattern? and does his explanation account for why only the No. 1 stock should be excluded? The Fools give us no theoretical explanation at all--that is, no reason why the pattern should exist--merely confirmation, based on number-crunching, that the pattern did exist. FF4.0 --> FF4.1 "Fools on the message board had noted that the number 1 BTD stock was not always a disaster. Sometimes it performed quite well, actually. Was there any sign, they wondered, that might serve to indicate when it was a potential disaster and when it might be safe to invest in it? Robert Sheard, keeper of the virtual Foolish Four Portfolio, delved into the newly developed database and found "the sign." Almost every time that the lowest priced stock (# 1 on the BTD list) was also #1 on the high yield list (i.e., it was the highest yielding stock on the Dow), it was a disaster." For it to be truly theoretical, this evolution would have had to be based on an extension or elaboration of the theory's core premises. Instead, we are told only that after "delving" into the database a correlation was found that helped improve results most of the time. FF4.1 --> FF4.2 "Meanwhile, those busy bees on the message board were still fiddling with the spreadsheet looking for better ways to select stocks. Many strategies were proposed, some quite good, but one stood out: Foolish 4.2 - the RP variation." Again, this evolution does not seem to have come about because someone sat down and said, "how can we refine and extend Higgins' contrarian large-cap value-investing logic," but rather because people "fiddled with the spreadsheet" to see whether they could find a pattern. And in fact the Fools openly admit that "many strategies were proposed...but one stood out," which they then adopted as their latest version. What reason are we given for this evolution? simply that this particular formula, when crunched against the historical numbers, throws up a better set of returns. What reason are we given for expecting that this pattern will continue? none, apparently, except the implicit assumption that its past success could not have been due to chance alone. But why not? Where is the argument ? And where is the explanation of why, if such a mechanical strategy could actually work, hordes of people wouldn't rush in and destroy its efficacy? I realize that lots of very smart people use lots of very complicated mathematical models to move lots of money around every day, and that I don't have the skills, background, or experience to evaluate their efforts properly. But I'm familiar enough with basic social science practice to be skeptical about leaping from correlation to causality, and I think all this smacks of the kind of intellectual hubris that brought down LTCM. Speaking of which, anybody who has slogged through all this might find Roger Lowenstein's new book on that debacle a good read:amazon.com tekboy/Ares@BWTFDIK.com PS YES YES YES TM-H put it much better than I did: "the key in any test of this sort is, can I interpret this result in terms of some underlying factors, i.e., can I take the math and turn it into some kind of sense? Some people fish about, fiddling with the data, hoping that something interesting will pop out ... usually guaranteeing that anything that does pop out will be an artifact. Some construct a theory and then look to see whether it is supported, probably evolving the theory as it is tested against facts. This, it seems to me, is what Moore has done."