I think there's only a handful of primary data vendors out there, so errors probably propagate from there. I by no means have investigated data sources that much - but that's my impression. The database says fundamental data is from Thompson Reuters. Earnings from I/B/E/S. DRP data from Moneypaper. And optionable data from CBOE.
Some data has been removed from the database in the past - either when vendor stopped supplying it or when either vendors changed or merged... something like that. I do remember having to mess with things a bit when that happened, but it was pretty painless for the most part. Can't even remember what it was now. The way I work adding to existing xls models is just inserting some columns in excel, and adding the new needed data to the export from the dbase, and going from there.
There will be fields that are not populated or have -99999999... that type of thing. I just have to use datatraps or macros to clean that stuff out in excel. I try to handle some of that in the screening process itself during building if it causes too much trouble in building calcs, but some of it is probably best handled in xls as it's hard to anticipate all the ways data could be weird when building screens - but it's easier to see how specific calcs can fail when you're in xls. In the building stage it's mainly an exercise of looking at how calc errors pop up, and then figuring whether I put a constraint in the screen (sometimes) or a conditional "=if(or(isnull(a1),a1<=0,isnumeric(a1)=false...)." something like that to control errors from destroying calcs. It's been a long time since I've done much of that, but that's my recollection of some of the approach. It was probably more work than I remember, but for me it was just one of those things that gets built up over longer periods of time and gradually making changes, and addressing issues as they arise, and building in conditionals for the type of things that throw the projection model off. After a few years it settles down into pretty much the type of projections that I'm wanting to see. The last big change I had to do to the Buffettology approach was after the 2008 financial crisis thing. Alot of what i did was based on predictability/stability and I reworked quite a bit of the logic to allow for larger one year disruptions. I probably should go back and reevaluate all of that again, but haven't done so in a long while.
For actual bad data - I just have to see that at the manual stage if something looks funky. I have noticed data being different from Yahoo on occassion - seems to happen more often with ADRs, maybe dividends, but I never investigated it very deeply because while I love yahoo for news, I never relied much on it for data. There's always the company websites and 10K and 10Q if I really need to know. Big one-time stuff usually shows up on the charts I look at, and I just have to go looking if I want to know what the one time stuff is. Like LHCG was one that recently that had to pay a big fine that drove their earnings negative for a year. There was no way to adjust for that than just manually tracking down what happened and manually looking at my future projections in that context. But that said, I know I don't dig down as deep into the financials as you do. |
