To: TD who wrote (2745 ) 10/24/1998 2:23:00 AM From: John Mansfield Read Replies (2) | Respond to of 9818
' Re: - EMBEDDED SYSTEMS ARE A BIG PROBLEM 'From: "Harlan Smith" <hwsmith.nowhere@cris.com> vr 12:36 Subject: Re: - EMBEDDED SYSTEMS ARE A BIG PROBLEM Thanks for your good comments. Mail: Y2000 infomagic com wrote in message <32E0944BBD86C989.05A545DB6AEA327D.4698E48090ACFD06@library-proxy.airnews.ne t>... >On 21 Oct 1998 11:45:24 PDT, "Harlan Smith" <hwsmith.nowhere@cris.com> >wrote: > >>What I am concerned about is the people who have given up on a remediation >>effort because they think it will be so difficult that they will never make >>any progress. >> >>I believe it's doable and most of the results from the field are supporting >>that idea. But these results are from people aggressively attacking the >>problem and I'm afraid that we don't have enough of those. > >Harlan, > >While I agree with most of what you say, I think that even you are >underestimating the inertia which is keeping these systems from being >fixed; the number which will not even be found because nobody with >enough technical insight knows they are there; and the consequences >of widespread failures of the unfixed systems. Unfortunately, I have no estimate and I don't think anybody else does either. That is definitely a big part of the problem. If we did have an intelligent estimate it might have significant impact on the way we approach the problem. >Don't forget that we have already _had_ our first major Y2K failure of >embedded systems and it took almost a year to fix, even with large >scale efforts from within a fully functional infrastructure. It even >led to the first of the Y2K liability lawsuits. I'm talking about the >failure of POS terminals to accept credit cards with an expiration >date beyond 1999. > >What's scary about this failure is the lessons to be learned. First, >the problem was completely unanticipated. The IT people at VISA and >Mastercard had _fixed_ their code, but _none_ of them even considered >the possibility of date dependent code in the terminals. Worse still, >neither had the terminal manufacturers, many of whom are located in >countries which are far behind the power curve on Y2K. That is news to me. "completely unanticipated"? I think that many of the terminal manufacturers must have known, but the logistics/economics of getting the terminals fixed and deployed to the field were immense. Same thing true for a lot of "embedded systems" vendors. I have heard of scenarios where they all sit around the conference table to discuss their problems and just hang their heads. If they were to initiate action to fix all the flawed systems they have sold and have been deployed to the field, they could probably go bankrupt many times over. I expect a high rate of attrition. That doesn't make the problem hopeless. I have been talking for a year now with someone deeply involved with "Intelligent Building Systems". He indicates that there could be two magnificent glass edifices sitting side by side and at rollover one will be just fine and the other will suffer "Building Death". (I think I coined that term). The difference would be the contractor(s) installing the systems and more accurately the manufacturers providing the elements of the systems. My assertion that the task is doable perhaps was meant to mean that half the large complex buildings will survive. This will result in huge economic impact but our society remains largely intact if we just lose half the intelligent buildings. It sure will create a lot of turmoil though and a lot of business for those that can move computers. The obvious remedy is to re-host any applications running on computers in non-compliant buildings to computers in compliant buildings. I think perhaps New York State has done some of that. It is said that after Y2K testing of applications there will be a lot of surplus computing power around. Perhaps this can be used to re-host applications running on computers in environments at risk. I would think that such actions would at least appear in contingency plans. It is not unusual to have a backup computer complex at a remote location. Will any automobile production lines remain viable. Beats me, but we can survive a long time without automobiles being produced. Again though, huge economic impact. >Second, a single failure in a particular class of systems (not even >identical units) can easily lead to large numbers of failures in many >unrelated businesses, most of whom do not have the technical expertise >to fix the problems themselves. This is analogous to a failure within >a mainframe operating system but much more widespread and much more >difficult to fix. Oh indeed, the "common mode failure" problem. Certainly more difficult to fix. The only way to counter this is pro-active effort by the "embedded systems" manufacturer to fix their systems and propagate them to the field. I don't see any such "push" acitivities going on, any remediation seems to be done on a "pull" basis, altough reading through the web pages of the various medical equipment manufacturers gives one some confidence in the idea that they at least know the status of their products and have fixes available. >Third, some of these failures propagated into the larger computer >systems to which the embedded systems were connected (e.g. the >grocery company's computer system which was the subject of the first >Y2K lawsuit). The consequences went far beyond anything we would >normally consider related to the embedded system itself. > >Fourth, although fixing the ROM code may be fairly simple (assuming >the ROM chips are still in production) fixing the individual >applications of the system takes enormous amounts of time, money and >qualified manpower. Some systems will _never_ be fixed because the >end user will go out of business before that can happen. > >Since a major failure of this type has _already_ occurred, it is >unreasonable to believe that similar widespread failures will not >occur both before, at and after the Y2K rollover. How would you deal >with only a dozen such failures when everything else is such disarray >from _other_ Y2K failures? The repair environment after rollover will likely be terrible, particularly for "embedded systems" that will likely involve parts ordering, production scheduling, manufacture, test, delivery, installation. Many roadblocks all along the way to getting any embedded system fixed. >Harlan, in spite of your knowledge of embedded systems and your belief >that fixing them is "doable", you have said nothing to make me >sanguine about that actually happening to any meaningful degree. I >really wish you could. I don't know how many are flawed, how many of those will be fixed, how many we can do without. I do expect a high rate of attrition and incredible turmoil. Harlan