Sector, I'm intrigued by your differentiation between files and data packets. Why the distinction? If I send a file, I send a series of "data packets." Or, depending on the platform, "blocks" of data. No?
"The fastest way to move a FILE of this size across the country (not data packets) is still to put it (the tapes or whatever) on a plane and fly it over."
I would differ with your assessment on this for some situations, for several reasons. A file rated at 300 GB, or 2.4 Tb can be sent cross country over a dedicated (or virtual) connection rated at OC-3c (a 155 Mb/s envelope with an average effective throughput of roughly 100+ Mb/s after all overheads including retransmissions of errored blocks/packets are accounted for) in under seven (7) hours from beginning to end.
A Dual OC-3 (312 raw, effective: ~225 Mb/s) in under 3.5 hours, and an OC-12 (622 raw, effective: ~500 Mb/s) in roughly an hour and 20 minutes. If GbE over Sonet were used, or an emerging variant of FC over lambda, it would be less than an hour. I wont go above these levels at this time due to I/O limitations on the memory repository hardware (the dual complexes which make up the actual sending and receiving parts).
Compare these times to the "real" time lapses involved with shipping by air. Take the scheduling of the mount/dismount/mount steps into account, the logistics of getting deliveries prepared and executed to and from data centers, etc., and you are talking twenty four hours soup to nuts, usually.
Add to this the fact that many data centers seek to eliminate all uses of tapes at some point for purposes that have to do with bulk file transfer, and the fact that many data centers are near dark room conformant and cannot accommodate manual forms of operations on a routine basis, and the case for tapes by air goes away proportionately.
The greater need for speed, however, is not simply for purposes of expediting jobs off the books. It goes beyond that. Data immediacy is required for both real time applications in remote locations where the architecture is designed for "local" serving of files to users and other applications (thousands of miles from the original source), and for the purpose of disaster readiness and business continuation in many contingency situations.
I am currently analyzing just such a set of parameters for the transference of Terabyte document imaging files between archives located in the northeast and the southwest, and while it is tempting to use fedex in order to cut down on costs, the dynamics which underlie these applications simply can't tolerate the mechanics and the delays.
There are means which are used to circumvent these I/O limitations, such as fragmenting the files and sending them individually (in parallel) over separate lines, and reassembling them after the transfers are completed. If an enterprise can tolerate the costs involved with multiple cross country OC-12 links, then that is what they would do in order to get near instantaneous transfer achieved. But last time I looked, a 3,000 mile OC-12s cost on the order of some $200,000 to $400,000 apiece, per month, depending on which carrier you selected and what the engineering particulars are.
And, of course, if such an application warranted one of these pipes, they would usually require two (one for backup). So, it gets very expensive, as you can see.
The general rule here is still to use fedex, as you have suggested, for those very large files which are a part of applications which can tolerate the delays associated with air shipment. But at the same time, more and more applications are now requiring the immediacy which I alluded to above.
Regards, Frank Coluccio |
