More On Y2K And Utility Industry:
Sunset for the British Empire?
Source: Electric Power Research Institute Proceedings from EPRI Embedded Systems Workshop, Proceedings dated 10/4/1997
Y2K testing was conducted on a generator temperature control system at a power plant in the United Kingdom. To test for Y2K compliance, the control system clock was set to just prior to midnight, Dec. 31, 1999. Twenty seconds past midnight, the unit tripped on high generator temperature.
It turns out the process value for the control valve for generator cooling is integrated over time for smoothing and when the time moved past midnight from '99 to '00, the PV was integrated over infinity. The valve closed (fail safe), tripping the unit on high generator temperature. If this were an isolated incident, the industry would evolve through the year 2000 with little difficulty. However, the algorithms used in this control system are common throughout Europe and most systems are vulnerable to the problem. Loss of numerous generating units simultaneously in the United Kingdom could be devastating to the country.
The Bill Is In The Mail
Source:Email from company 'staffer'; Northwestern Public Service Company
ITRON meter reader decks and associated upload/download equipment won't jump over to Jan 1, 2000. ITRON told Northwestern Public Service Company the handhelds and the up/download stations won't make it. NPSC was notified of this while asking ITRON about their version of automatic meter reading (AMR).
The ITRON set-up was purchased back in the late 80's or early 1990's, and it consists of electronic handheld units and docking stations that meter readers upload in the morning and download when they come back in from reading their routes. NPSC hasn't yet decided on anything AMR, but will first need to get new handhelds and up/download stations. ITRON has indicated that the units can be updated, but the technology is old, so strategic decisions have to be made whether to fix or replace with more current technology.
When it Rains, it Pours
Source: NRC/NEI Meeting, 10/7/1997
During the major mid-west flooding a few years back, Nebraska Public Power District was forced to shutdown Cooper Nuclear Station - not because of a threat to the plant from the flooding, but because local emergency response teams would have been unable to respond to a nuclear emergency during the flood situation. Carry this out to the global Y2K implications, like loss of telecommunications and loss of offsite power. Bottom line - if local and state emergency response isn't working, eg. simple stuff like 911 plus nuclear emergency response communication et.al., the plants have to shut down.
Y2K Ugly in Paradise
Source: Hawaiian Electric Company staffer email, 10/31/1997
In 1996, HECo's Y2K project team identified significant problems with it's energy management system (EMS). EMS is the 'brains' of the power distribution system at all electric companies. This system is used to remotely control transmission system breakers, coordinate power generation schedules, compensate for large transmission line breaks, and provide protection against voltage, current and frequency transients.
Without going into a lot of technical detail, HECo and their EMS system vendor determined that EMS would crash on the date rollover to 01/01/2000. This would have, in turn, resulted in HECo's transmission network crashing, and by default, a major power outage and loss of all generating capacity. If you want a detailed technical description of HECo's findings and the Year 2000 fix that they've implemented during 1997, I've put details directly from HECo in a downloadable Word 6.0 format or you can read the online HTML version.
Going Under Down Under?
Source: Year 2000 Discussion list, John Catterall, Year 2000 Project Manager - Western Power, in response to a Los Angeles Times article from 11/3/1997.
From: jcatterall@ccmail.wpcorp.com.au Date: Thu, 06 Nov 97 17:40:01 +0800 To: year2000-discuss@year2000.com Subject: Re: Sightings: embedded systems
Automated systems are indeed widespread throughout power utilities. Exposure to the industry would in my opinion rate as extreme. At Western Power, our IT compliance project (cost around $3 million and employing 35 Full time employees) is trivial in comparison to the issues we face on the control and embedded systems front.
Just leaving aside the problems associated with ensuring supply of such items such as fuel, from the weighbridges and conveyor systems through to intelligent metering and meter reading systems dates are endemic.
Many of the control systems represented in power systems, have dates associated with them. These could be reclosers, Voltage regulators, Governors, PLCs etc. The list is endless. You then have a swathe of actual 'applications' involved in the delivery of electicity such as your Distributed Control Systems and your SCADA (System Control and DATA (eg.dates) Acquisition) systems, all of which have dates associated with them. Much of what happens throughout the process of generating and delivering electricity is 'DATE AND TIME STAMPED'.
A typical DCS at a power station will have many thousands of processors associated with it. Any one prepared to give me a guarantee that my DCS wont experience a date problem?? (I can't even get the manufacturer to give me this). Anyone prepared to run a Power Station when the DCS is throwing alarms up at you that indicates its getting no reading or a peculiar reading either from a device it is monitoring or one of its front end processors ??
But this represents just a portion of your problems. You will also need to address your security systems, your alarm systems (fire, temperature, etc.) your environmental systems - what do you do with your waste water, your emissions etc. ?? How do you monitor these and what control systems do you have in place? After all, your management (if you are managing) must know how much and over what TIME period things are occurring. Your employee protection systems - would your power station staff be happy running your power station for you when they know if they hit a panic button nothing's likely to happen?? Any suggestion that power systems don't use dates or could not be affected by the Year 2000 problem is at best ill-informed.
John Catterall Western Power - Year 2000 Project Manager (Note:these represent my views only and do not necessarily represent the views of my company). All usual disclaimers!!
Freezin' in the Dark in Nebraska Source: Internet Newsgroup comp.software.year-2000, 10/30/1997, reprinted with permission of the author.
"This is a tale of how unprepared I really was to handle no electrical power for an extended period of time. I always thought that I was a little prepared if there was an emergency, but after just completing 4 days without any electricity I now know that I was wrong.
"I lost power on Sunday morning, as part of the record breaking snow fall that Omaha and much of the Midwest experienced. For the past 4 days I kept thinking about how much worse the experience could be if my phone and natural gas was also out. I had thought I was prepared because I had a few candles and flash lights and some canned food, but believe me it isn't enough. I finally got unburied from the snow on Monday and went to look for supplies. No candles or batteries were available, any where. There has been many tales of people being rushed to the hospital because of generators being kept in the attached garages, of generators being stolen if left outside, and of house fires due of faulty wood burning fireplaces. At one point it was estimated that there were an average of 10 house-break-ins an hour during the black out. What will it be like in the bigger cities? Chicago? New York? DC? What will it be like in January in the northern Midwest and Canada?
"The power company crews that have been working around the clock to get the estimated 100,000 homes without power, back online has been an inspiration. (These outages were not just large grid problems, but they are down to about 10,000 homes that have to have lines reran to the house and reconnected). Yes, they are getting paid overtime! But we all know about those death march projects that you work around the clock to get them done, and after a point you want to say f*** the money! Let me sleep! These guys probably feel the same way!
"Upon thinking about this situation, I don't want to have to go through it again and next time compounded with no natural gas or phone service or nice neighbors that help each other out. I am fortunate that I only had myself to worry about, what about the people with little children and babies? or the elderly?
"When I started writing this, I was going to urge you all to make sure you are totally stocked with provisions, candles, generator, what ever you need, but about half way through I realized that what I really want to do is urge all of you to get on a Y2K project and work what ever it takes to get the Y2K work done. Preferably for the critical systems which does not just include the utilities, but also the manufactures, the banks, & the governments. Do we really want to see a collapse of our governments? economic systems? or our food supplies?
"This is just meant as some food for thought for everyone. Sure we can all run away and hide out on some nice tropical beach, after all we will be able to afford it by then, but what will we return to? And how will our friends and families survive? It is within our power to make things right, just ignore those pointy-heads and get the job done. If you are a geek worth your salt, you are used to ignoring the hand wavers anyway. After all, we will be getting paid to work the long hours. We can always take a year off and sleep if we need to.
"Good luck and remember they can't do it with out us!" Tempi Dickinger (tdickinger@aol.com) |