Pat, simple:we have an antiquated small switch/software written 20 years ago :-)) by a small co called Cascade Communications whose engineers are now all enjoying their permanent vacations in the Bahamas ( after cashing in the stock options ).
Now at LU
-nobody knows how the system works :-))
-how to fix it :-))
-where the previous engineers are
-they don't have the expertise to build a new system, GG ( that's why they bought Cascade in the first place: because they didn't know how to build an ATM switch)
so we essentially have a situation of the blind leading the blind.
Again, I go back to my old: call GAICO insurance , er, NN,
TA
PS I remember seeing pictures in 1995 of Cascade's ATM switch: it was a tiny poor little thing. The size of one of those small 1 yard refrigerators you buy at Sears. At that time ('95), also from an investors packet, I saw NN's big ATM, the size of a large meat refrigerator; awesome. And it has only gotten bigger since,
back later
TA
you said
This morning I had coffee with a telecommunications greybeard who explained the entire process of installing software upgrades. Beginning with computerized tests all the
way through fire-drill type dry-runs, there are many stages before the actual upgrade occurs. Why ASND/LU didn't find the bugs before they did would be more amazing if it
weren't for the dearth of expertise in the industry. My friend says there simply aren't enough experienced engineers and technicians to go around and MCI isn't the only carrier
impacted. Many young engineers simply haven't had hands-on experience with actual networks and those who have are hired away by the highest bidder.
At any rate, here's the latest:
<<<<
MCI assesses network outage aftermath
By Ted Smalley Bowen
InfoWorld Electric
Posted at 10:05 AM PT, Aug 16, 1999
Sifting through the damage of its recent frame-relay network outage, MCI WorldCom Monday offered its version of the events and spelled out its plans for making amends with
the customers affected.
MCI WorldCom Sunday restored service to the one of its four frame-relay networks that had experienced problems beginning Aug. 5, according to officials. The network of
about 300 switches and nodes, which serves roughly 3,000 customers, was taken offline between noon Saturday and noon Sunday, and its software rolled back to the
previous version, according to Bernard Ebbers, MCI WorldCom president and CEO.
The company is offering customers two days of free service for each of the 10 days the network was hampered, according to Ebbers, who added that further customer
concerns and individual contracts would be reviewed separately.
While criticism of MCI's handling of the outage has been stiff, Ebbers said he was not aware of any customer having dropped MCI WorldCom service as a result.
Although they have yet to pin down the exact cause of the meltdown, MCI WorldCom and Lucent, which supplies the network's hardware and software, have attributed the
problems to an upgrade of Lucent's software, and not the network blueprint or infrastructure.
MCI about a month ago began the process of upgrading to the latest version of the Lucent software in an effort to boost the network's capacity and expand the range of its
features, according to Ron Beaumont, president of MCI WorldCom's network services division.
Despite extensive testing at Lucent and MCI WorldCom, the upgrade precipitated the outage, according to the officials.
"We can't tell you that any testing procedure will be 100 percent accurate," Ebbers noted.
MCI WorldCom will conduct an audit of its test procedures, according to Beaumont.
Asked about the financial impact of the two-for-one service refund, Ebbers said it was expected to be minimal, and unlikely to alter earnings estimates for the quarter or the
year.
While the network has been restored, further adjustments to individual customers' virtual circuits will be made this week, according to Beaumont.
The software rollback still leaves MCI WorldCom with about 18 months of headroom, given the current rate of growth, according to Ebbers, who added that the company
was considering a range of possible strategies to boost volumes and features.
Ebbers declined to comment on any indemnification of MCI WorldCom by Lucent.
Among other issues, the crisis raised questions of whether software vendors are doing enough to ensure the quality and stability of their products in cases of
consolidation, according to Beaumont.
The Lucent software in question was originally written by Cascade, which was acquired by Ascend, which in turn was acquired by Lucent.
"One concern is what happened to the people who wrote the software and what happened to [the relevant documentation and expertise]," Beaumont said.
MCI WorldCom Inc., in Jackson, Miss., is at www.mciworldcom.com.
- |
