This is a report from a engineer who attended the ERPI Y2K conf.
SUBJECT Trip Report: EPRI Y2K 98 Conference on Embedded Systems
The EPRI Y2K Embedded Systems Conference in Dallas Texas, held between 5/4/98 to 5/7/98, was divided into different sections/groups. One was tailored to the technical/testing aspects with respect to the Y2K issue, while the second was geared towards the Project Management aspects which included the legal/litigation exposure. This trip report highlights the events of the technical discussions.
SUMMARY
The conference yielded several insights into the Y2K problems facing xxxxxx. It also allowed for "bench-marking" our progress against the rest of the industry. Via a show of hands at the conference, xxxxxx would rank at approximately the 33rd percentile with respect to Y2K awareness, analysis of installed components and the resultant required testing. Some industry leaders (with a large staff dedicated to the Y2K project) are 70% complete with their Y2K embedded issues. The initial good news is that no major modifications have been recognized, to date, stemming from actual Y2K testing. That is not to say there haven't been problems, see EQUIPMENT TESTING below, that required resolution to avoid a possible plant trip due to Y2K issues. It was also obvious that the consensus of the participants do not believe the majority of the original equipment manufactures are going to be forthwith with Y2K related issues due to the possible litigation ramification. It is therefore imperative that xxxxxxx utilize the EPRI database and the resultant industry contacts so as to avert any unforeseen Y2K "glitches". Utilization of the EPRI database may also corroborate that we exercised due diligence with respect to Y2K issues.
REGULATORY ISSUES
It was a consensus of many at the conference that all nuclear power plants will be required to be Y2K compliant/ready by July 1, 1999. This was confirmed by the issuance of NRC Generic Letter 98-01: "Year 2000 Readiness of Computer Systems at Nuclear Power Plants" dated May 11, 1998.
The NRC has adopted the following definitions as they relate to the Y2K issue.
ú "Y2K compliant" is defined as computer systems or applications that accurately process date/time data (including but not limited to calculating, comparing, and sequencing) from, into, and between the 20th and 21st centuries, the years 1999 and 2000, and leap-year calculations.
ú "Y2K ready" is defined as a computer system or application that has been determined to be suitable for continued use into the year 2000 even though the computer system or application is not fully Y2K compliant.
The NCR has also "strongly encourage licensees to share information regarding identified remediation and implementation activities in order to maintain the likelihood that all Y2K problems are identified". Supporting the EPRI database could be viewed as fulfilling this recommendation.
The majority also believed that there will be a Y2K NRC inspection team in place by the fall of 1998, and that ten industry leading plants will be audited as a baseline for future audits.
BACKGROUND
EPRI's Year 2000 Program for Embedded Systems complements efforts to address software-related problems. The program acts as a forum and source of practical information to meet pressing needs for energy providers. This collaborative, noncompetitive program - open to all parties able and willing to share data - consists of four major "components."
1. An electronic system for real-time data and information sharing.
2. Facilitated workshops for interactive discussion of methods and results.
3. Industry wide teams for collaborative development of critical information.
4. R&D for embedded systems.
It should be noted that the embedded system review can not address the higher level software issues which are, by default, client specific.
EPRI's Year 2000 Program is designed to reduce our exposure to Year 2000 problems by increasing the efficiency and cost-effectiveness of our in-house compliance efforts. The program's information and experiences will help xxxxxxx:
1. Identify potentially vulnerable systems, components, and interdependencies and prioritize them by building on the work from other utilities.
2. Prioritize, develop, and implement compliance testing programs utilizing the experience and lessons learned by others, not just from electric utilities.
3. Communicate with, and receive technical assistance from, system and component suppliers and vendors in a cost-effective manner.
4. Exploit approaches successfully applied elsewhere while avoiding duplication and dead-ends.
5. Gather Year 2000 compliance information from key suppliers, vendors, and customers.
6. Support due diligence in Year 2000 preparation for reporting to the financial and regulatory communities.
7. Develop contingency plans and mitigation strategies in anticipation of potential failures occurring both within and outside of our control.
STATUS OF EPRI DATABASE
Several utilities are beginning to enter their Y2K inventory into the EPRI database. A few utilities are entering preliminary test data into the database. Many utilities are only starting their testing program.
The slowness of access to the database has been fixed by EPRI whereas they have installed a new sever for database access over the WEB. The usefulness of the database cannot be realized until the utility participants begin to enter data into the database. The major hurdle, that must be overcome, is legal in nature not technical. Most vendors and several utilities are preparing a universal disclaimer to all data given to the EPRI database to forestall any future litigation due to erroneous postings/information.
EQUIPMENT TESTING
Several utilities have begun their testing of Y2K related system components. Less than a handful are well on in this phase (Palo Verde ~ 70% complete). The good news here is that Palo Verde has yet to hit a "show stopper". They were cautious to specify that the first systems tested were the "low hanging fruit".
Group Consensus (in no order nor priority)
ú The Y2K test teams should be dedicated to the project to ensure timely implementation is realized is met.
ú Most utilities are using the GM test procedure as a skeleton for their own procedures.
ú Software is available to scan most types of software for date related entries.
ú Apparently no one is tracking the individual chips (model\serial numbers) on their equipment containing integrated circuits.
ú Testing must begin as soon as possible so that there is enough time to implement a fix or work around prior to the year 2000.
ú System engineers should shoulder the responsibility and the accountability for all Y2K testing of their systems.
ú It is apparent that the original equipment manufacturers (OEMs) are not accepting responsibility for any testing related to Y2K due to liability concerns. One large OEM will not even review the clients test procedures.
ú An expert from Dallas Semiconductor argued that the "best bet" to beat the Y2K bug is to perform system integrated testing.
ú Most utilities which have begun actual plant or bench testing feel that the vendors of the applicable equipment are on the same learning curve as they are.
ú Actual Test Results (results were given by utilities without reference, due to litigation concerns)
ú Utility UU tested the Westinghouse Inadequate Core Cooling Monitor (ICCM) which failed in the rollover and would have placed their unit in a T/S action statement.
ú Utility VV tested the Bailey INFI-90 system which locked-up partial control during the 2000 rollover. This utility also could not test their ABB Advent Control System since ABB informed all their clients, with Advent, that following rollover to 2000, the system may irrevocably be damaged.
ú Utility WW runs a control algorithm on PCs, which accepted the 2000 rollover without any problems. The PCs were then cycled ON/OFF, whereupon the control systems locked-up due to error checking with the date stored in the real-time clock versus the DOS clock.
ú Utility XX tested the GE Mark V following a GE declaration that the Mark V is Y2K compliant. This utility uses the Mark V for several applications, including feedwater control. The Mark V locked-up following the roll-over to 2000 and locked in the last output signal to the feedwater regulating valves. (This would likely cause a plant shutdown)
ú Utility YY tested their energy management system (EMS) power calculation which failed during the rollover forcing the simulator to initiate a 25% down power.
ú Utility ZZ tested their DCS (Honeywell) at the component level with no notable problems. An integrated test was performed and the control function failed two weeks following the 2000 rollover due to a historian overload which was unable to write to disk on its weekly schedule.
ú Several utilities acknowledged that control devices lockup when configured with non-compliant configurators, either hand-held or PC based.
CONTINGENCY PLANS FOR YEAR 2000 NEW YEARS EVE
ú Ensure adequate staff is available to support plant problems and work arounds.
ú Setup alternative communications.
ú Use power plants in Australia and Europe as an early warning system.
ú Be ready for grid disturbances.
ú
Place older plants on spinning reserve.
Date stagger your equipment if possible to pace the operator work arounds. |
