To: Shane M who wrote (56507 ) 1/1/2016 11:05:10 AM From: Jurgis Bekepuris Read Replies (2) | Respond to of 78760 yes. Yes. YES! :) If I could recommend this +10, I would. You are totally right. What to do about this? Read this:smile.amazon.com I am thinking about doing something based on this book. As a first step, I try to make predictions based on my investments and quantify them (i.e. "BRK book value will increase over 5% in fiscal 2016 " with 76% probability). If you read the book, this still might not work well unless you have a community of forecasters with whom you can compare your results (short explanation: you could make easy predictions and ace them, but those easy predictions don't matter, e.g. "BRK book value won't decline more than 20% in 2016 " with 99% probability - this prediction is worthless). Anyway, I'll post how this goes later. As you say, my mind wanders so I might find another shiny thing later and ditch this. :( Hopefully not. BTW though, Tetlock does not believe that superforecasting works in investing. Or at least that's what he tells in the book. Some other thoughts: Yes, investment journal might work. But it still has the problem of "good method bad result/bad method good result". Post mortems might work if you know how to do them well without biases. Some examples: Part of the reason I invested in oil this year is 2009 historical bias. Then oil went to $40 and rebounded to $80-100 really fast. I made 10x on GPOR. Clearly situation this year is different, but oh it looked similar when it started. ;) Perhaps investment in oil cos in 2009 was bad method/good result. Perhaps investing today into oil is good method/bad result. ROIQW - this was the ultimate option-like investment. Either zero or 3x. So I assign 60-70% probability to 3x and invest accordingly. The outcome is zero. Good method/bad result? Maybe. It still hurts like heck. I guess good/great poker players can deal with this. I am not sure I can. (OTOH, it might have been bad method/bad result: I possibly ignored warning signs that made this much lower probability which should have made me fold earlier. How do you know that it as good/bad method?). Now, some people will say "don't invest in zero or 3x " investments. Fine. But in some sense this only exacerbates the fuzziness of the decisions. Sure, we can take BRK as ultimate "will never go to zero " investment. But does that mean that if we buy tons of BRK and it continues to underperform the index that we made a good method/bad result decision? How do we know? Value investors try to answer this by valuation/good business/margin of safety/moat arguments. But even this goes back to the issues you raised. The risk is either to say "whatever I do is a good method because I used valuation/good business/margin of safety/moat ", which just prevents learning, since all bad results are assigned to "good method/bad result" pile. OTOH, trying to split results to "bad method/bad result" vs "good method/bad result" vs "bad method/good result" piles is hard. There's definitely strong bias to conflate goodness of method with goodness of results. Mechanical methods like magic formula try to address this by saying "the method is good, it's back tested, so all bad results are just good method/bad results ". This is fine as long as you trust that the method is good and continues to be good. But what do you do if it enters long stretch of underperformance? The article you pointed to recently that showed 10+ year periods of underperformance by P/B or P/S or P/E at different times is a good example... How do you keep the belief in good method after 10 years of underperformance? Anyway, this is getting to be a long rant. :) Food for thought for 2016.