I'm done with the preliminary stage of paring down folder_links. It's at about 2.6M rows.
Since we'll be handling foldering on the new SI the same way it's done on iHub, I did an import last night that excluded folders 2 and 3 ("Out" and "Trash") and got 504,905 rows. Quite an improvement from 2.6M rows and a huge improvement from 19.7M! And I'm still not done. I haven't yet deleted folder_links that're pointing to deleted (or not existing for some reason) messages, or to messages not written to the person whose Inbox folder they're in. If I remember right, that encompasses tens of thousands of records.
But get this! Because I've decided we're going to handle foldering the iHub way before the site goes into production, the ending row count is going to be about 160k. That's because there won't be a Folder 1 (Inbox). This wasn't feasible with older technology, but nowadays, even as large as the message tables are, whether a post shows up in your Inbox will be controlled by a flag in each of the message tables rather than a separate entry in Folder_Links. In essence, there won't be a Folder 1 (or 2 or 3), but the functionality will be the same. Except for the Trash folder, which we don't use on iHub. It's an irrelevant setting/location in the iHub messaging/foldering model.
So, in the end, folder_links will only be used for the 160,000 messages people really have put into their own folders.
Speaking of how messaging will be handled, there are 3.1M private messages on the system, dating back nearly 5 years. I haven't decided yet what to do with these. Brad used to keep private messages purged so they'd contain only the most recent few months. Likely to make them subpoena-proof. Dunno. It's the explanation that makes the most sense aside from saving storage space, which was still a bit pricey in the old days of SI, but not prohibitively so.
I've never received a subpoena for either site asking for private messages, and in iHub's privacy policy we state that we won't deliver them without a fight.
On one hand, a case can be made for purging them (only if they haven't been foldered by sender or recipient) for a number of reasons. On the other hand, since we'll end up with the ability to search private messages (from or to the searcher) for specific text, which is terribly handy, a case can be made for keeping them.
I'll give it some thought and I invite feedback to help me decide. |
