April 2010 Vol. 237 No. 4
Features
Impact Of New PHMSA Control Room Management Rules
The Pipeline Inspection, Protection, Enforcement and Safety Act of 2006 (PIPES) mandates that the Secretary of Transportation issue regulations on control room management (CRM) requiring a human factors plan that assures control systems are matched to human capabilities and limitations, including fatigue management.(1)
Support along very similar lines for improving the safety and reliability of both liquids and gas pipelines was expressed in a letter to the Department of Transportation (DOT) from the American Gas Association (AGA), the American Petroleum Institute (API), the American Public Gas Association (APGA), the Association of Oil Pipe Lines (AOPL), and the Interstate Natural Gas Association of America (INGAA).(2)
The content of the recent amendments is a result of the National Transportation Safety Board (NTSB) investigations into real accidents and incidents that befall pipeline operations. In the process of investigating mishaps, NTSB identified key root causes and then proposed five areas for potential improvement:3 controller display graphics, alarm management, controller training, controller fatigue and leak detection.
DOT has approved and issued specific revisions to the law. The new regulation amends 49 CFR Parts 192 and 195.4 The regulation contains specific requirements for operators to design and operate their enterprise-wide SCADA to take specific account of what is needed by the controller to properly do the job of keeping things operating safely and responsibly. The amendments require the following: (a) understand and manage fatigue, (b) provide effective operating and safe maintenance procedures and training, (c) match the control room environment and equipment to the human capability, (d) provide effective controller warning and guidance of abnormal operation specifically addressed to alarm system improvements, and (e) establish and enforce an effective change management process.
Our discussion is to assist pipeline operators to focus on how each organization can best prepare for the required changes.
Notice
This article is not intended to provide legal advice or recommendations for compliance to any regulatory requirement. The purpose of this article is to provide general information to the technical public for that public’s awareness and to assist in safer and better operation of pipelining enterprises.
Applicability
As of the effective date, these regulations apply to (1) existing facilities, (2) existing facilities that undergo modification, and (3) new facilities. An exception is noted for human machine interfaces (HMI) that are not modified, replaced, or installed new. Liquefied natural gas (LNG) facilities are not covered by these amendments. Thus, there are no changes to 49 CFR Part 193.
The regulations apply to both owners and operators of liquid pipelines and gas pipelines. All control rooms are covered that contain equipment that permits the manual intervention of the operation of the pipeline. This includes local control rooms or panels to the extent that the safety effects of operational error are similar to regular control rooms. Control rooms and control areas that are “view only” are exempt. Also, gas pipelines that service less than 250,000 customers and/or lack compression equipment are subject to the requirements only for fatigue management, validation, compliance and documentation.
The rule amendment summary states: “… Under the final rule, affected pipeline operators must define the roles and responsibilities of controllers and provide controllers with the necessary information, training, and processes to fulfill these responsibilities. Operators must also implement methods to prevent controller fatigue. The final rule further requires operators to manage SCADA alarms, assure control room considerations are taken into account when changing pipeline equipment or configurations, and review reportable incidents or accidents to determine whether control room actions contributed to the event. It further requires the statutorily mandated human factors management. These regulations will enhance pipeline safety by coupling strengthened control room management with improved controller training and fatigue management.” (4)
To prepare for the referenced requirements, operators should review each aspect of the following areas of their enterprise-wide SCADA: alarm management, control room management, documentation and procedures, HMI displays, shift handover, fatigue management, change management and training.
Alarm Management. “SCADA alarms are a key tool for managing pipeline operations …. Alarm management is a significant factor in control room management …. Excessive numbers of alarms or alarms that are inaccurate or not prioritized can overwhelm a controller, resulting in a failure to take appropriate action.”(4)
Much is known about the proper design, implementation, and operation of alarm systems. (7, 8, 9, 10) The current best practices recognize the need to develop an overall alarm design basis, referred to as an alarm philosophy. From that, using good engineering alarm design principles, appropriate alarms are selected, their activation point and priority determined, and appropriate controller information and guidance produced. This process is called alarm rationalization. It is important to note that a fundamental objective requirement for alarm design is that the alarm system augments the controllers’ normal duties in a way that does not place excessive load to the point that vital alarms are missed or misinterpreted.
Alarms must be designed to appropriately announce abnormal operating situations that require explicit attention and action by controllers. Appropriately designed and implemented alarm systems provide controller notification with sufficient time to manage the situation and contain the necessary information to understand and remedy abnormal operation. Included are processes for ensuring that the controller is not overloaded by alarms and for ensuring that out-of-service alarms are tracked and managed. Specifically:
- A written alarm-management plan.
- A monthly review of alarm points that affect safety that “are not providing current data to controllers or points that may be triggering erroneous alarms4.”
- Monitor the content and volume of alarm activity directed to each controller to ensure that the controller is not overloaded and that important alarms are not missed.
- An annual review of the alarm-management plan, the alarm parameters and operation of the alarm system.
Control Room Management. CRM is a collective description of a number of the individual items already mentioned in this discussion.(11) Control room management therefore requires the following:
- effective, fully documented operating procedures (12),
- specific plans for abnormal situation management that include role modification and when the controller must stop attempting to recover good operation and begin to shut down operation,
- examination of incidents and near-misses to capture the lessons learned and incorporate them into the site infrastructure,
- explicit coordination between field personnel and the controller when emergency conditions exist or when making field changes that are likely to affect control room operations,
- specific controller training for “covered tasks” (12), and
- specific training and skills to recognize abnormal and emergency events from the indications and alarms that these events will produce through the SCADA system.
(Encouragement is given to include simulators, both electronic and table-top, for such training.) (4)
Documentation and Procedures. Develop and follow enterprise-appropriate written control room procedures that define the controller roles and responsibilities in normal, abnormal and emergency operating situations. Ensure that these plans and procedures receive sufficient review and revision on an appropriate schedule.
HMI Displays. “Pipeline controllers must have adequate and up-to-date information about the conditions and operating status of the equipment they monitor and control…. “Operators need to assure that SCADA systems perform this important function correctly, and that the information is displayed in a manner that facilitates controller understanding and recognition of abnormal operating conditions.”(4)
The failure of controllers to be able to fully understand and appreciate the larger picture has been linked to a large proportion of accidents and incidents. Operational graphic displays are the primary information and control mechanism. Electronic displays are used by controllers to view pipeline operations and to make manual adjustments where necessary. The design of the screen displays shall be clear, useful, and fully functional.
The design is expected to follow API Recommended Practice 11656 that covers the appropriate hardware to consider so that the equipment is capable of delivering the needed speed of response and providing for appropriate ways for the controller to input commands. It also lays out the proper use of color, the design of clear overviews of the process, and display navigation to enable effective controller use. Additionally, other important guides for HMI designs have been adopted by industrial users.(13, 14)
Validation of all SCADA displays is required whenever equipment is added or moved and for all other changes in the equipment or SCADA displays that might impact safe operation. Annual validation and testing is required for all backup SCADA systems and methods for manual operation in the event of failure or inability of the SCADA system.
The new guidelines are not intended to require operators to modify existing displays to comply. However, when existing displays are changed or replaced after the implementation date, the guidelines will apply.
Shift Handover. “Pipeline operators must also establish a means of recording shift changes and other situations in which responsibility for pipeline operations is handed over from one controller to another. Such changes in responsibility may occur at scheduled shift changes or within a shift, when a controller is relieved for breaks and other reasons. Handovers can also occur between control rooms, for example, where only one of multiple control rooms is used during night shifts. Pipeline operators will need to define procedures for shift changes and other circumstances in which responsibility for pipeline operation is transferred from one controller to another. The procedures must include the content of information to be exchanged during the turnover.”(4)
Fatigue Management. “Pipeline operators must implement measures to prevent fatigue that could influence a controller’s ability to perform as needed. Operators will need to schedule their shifts in a manner that allows each controller enough off-duty time to achieve eight hours of continuous sleep.
Operators must train controllers and their supervisors to recognize the effects of fatigue and in fatigue-mitigation strategies. Finally, each operator’s procedures must establish a maximum limit on the number of hours that controllers can work.”(4)
Change Management. “Pipeline operators are required to consider the effects of future changes to the pipeline on control room operations. They must involve controllers … in planning prior to implementing significant hydraulic or configuration changes that could affect control room operations.”4 (Authors’ note: “Significant” means significant in potential effects, not necessarily significant in capital scope.) This requirement includes all necessary recordkeeping demonstrating compliance and full documentation to support all site deviations from the requirements deemed necessary for safe operation. The change management process must be fully integrated into the site operations.
Training. The rule requires operators to ensure that controllers have the appropriate requisites to safely manage.(12) Specifically, controllers must:
- have the necessary knowledge, skills, and ability to perform;
- know how to utilize up-to-date condition and operating status;
- demonstrate proficiency in the procedures for appropriate operations including emergency response; and
- undergo regular, periodic evaluation of skills and knowledge to perform.
Important Perspective
The CRM and alarm-management additions to 49 CFR Parts 192 and 195 issued Dec. 3, 2009 will greatly assist the pipeline industry to understand what is needed and prepare for appropriate implementation. “This rule will increase the likelihood that pipeline controllers have the necessary knowledge, skills, and abilities to help prevent accidents. This rule will also ensure that operators provide controllers with the necessary training, tools, procedures, management support and environment where a controller’s actions can be effective in helping to assure safe operation.”(4)
The requirements identify specifically what should be considered. The technology and practices needed for compliance are well understood and widely applied in other industrial and manufacturing segments. Compliance will not only be good citizenship, it will most certainly improve the ability to operate with more safety to personnel and the community, operate with less controller stress and more effectiveness, and operate more reliably with better availability, and less service degradation.
Regulations Vs. Best Practices
Much of what is required by the amended regulations has been mandated and implemented in operating plants such as refineries and chemical facilities. As a result, a significant best practice body of work exists for control room management at operating plants. Although requirements for operating plants are somewhat different than requirements for pipeline control, much can be learned from the existing operating plant implementations. The implementation of CRM practice in operating plants has led to a number of unanticipated benefits, including higher levels of uptime, better maintenance planning, improved control, and better ability to optimize performance.
The Way Forward
The challenge for our industry as a whole is to collaborate and define best practices for CRM. The first step is to create a level of awareness around both what is in the regulations, and what other industries have already identified as contributing to best practices. From this basis, our industry will prepare itself to meet the requirements of the amended regulations. We will also be looking to our leading industry associations – ISA, API, AGA and others – to work toward publishing guidelines and best practices.
The task for individual pipeline operators is to promptly lay policy and procedures into the required plan for CRM at their individual sites. Expect the key technology players that support your pipelining activities to be providing products and services that fully align with these new challenges and opportunities.
Schedule
Effective Date:
The final rules are effective Feb. 1, 2010.
Compliance Date:
An (owner or) operator must develop control room
management procedures by Aug. 1, 2011.
Implementation Date:
The above procedures must be implemented by Feb. 1, 2013. (Notice that the original rule date of Feb. 1, 2012 was in error and DOT has issued an announcement of correction.(5))
Authors
Douglas H. Rothenberg operates D-RoTH, Inc., Shaker Heights, OH, a consulting and technology firm specializing in operator/controller support technology. He is an engineer with extensive experience in industrial controls and automation. He has done pioneering work in abnormal situation management and alarm management.
Russel W. Treat is president of EnerSys Corporation, Houston, and an instructor for the SCADA Fundamentals Certification course at the Gas Certification Institute in Houston. He has 30 years of SCADA experience and has been involved with implementing, designing, executing and operating SCADA systems for transmission, production and distribution in the energy industry since 1994.
Ardis Bartle is president of APEX Measurement and Controls, LLC, Houston. She has worked in SCADA and gas measurement for more than 15 years. As an active committee member for AGA and ISA, she has been involved in writing and reviewing standards and practices in both SCADA and gas measurement.
References
1. PIPES Act of 2006, also referred to as “Amendment of Title 49, United States Code: Public Law 109 468, 120 STAT. 3486 3501, Dec. 29, 2006.
2. Letter to DOT from AGA, API, APGA, AOPL, and INGAA: To Jeff Wiese, Associate Administrator for Pipeline Safety, USDOT, Feb. 8, 2008.
3. “Supervisory Control and Data Acquisition (SCADA) Systems in Liquid Pipelines,” Safety Study NTSB/SS 05 02, adopted Nov. 29, 2005.
4. “49 CFR Parts 192 and 195, Amended Regulations,” Pipeline and Hazardous Materials Safety Administration, Department of Transportation, Federal Register, Vol. 74, No. 231, Rules and Regulations, pages 63310 63330, U.S. Department of Transportation, Washington, DC, Dec. 3, 2009.
5. Public announcement at the weekly meeting (week of Dec. 16, 2009) of the Technical Pipeline Safety Standards Committee (TPSSC), PHMSA stated that the final implementation date for the CRM rule is corrected to be Feb. 1, 2013 (not Feb. 1, 2012, as stated in the FR published rule).
6. “Recommended Practice for Pipeline SCADA Displays,” API Recommended Practice 1165, First Edition, American Petroleum Institute, Washington, DC, January 2007.
7. Rothenberg, Douglas H., Alarm Management for Process Control – A Best Practice Guide for Design, Implementation, and Use of Industrial Alarm Systems, Momentum Press, New York, 2009.
8. “Management of Alarm Systems for the Process Industries,” ANSI/ISA 18.2 2009, [published jointly by] American National Standards Institute (Washington, D.C.) and Instrumentation and Automation Society (Research Triangle Park, NC), June 23, 2009.
9. “Pipeline Alarm Management,” API Recommended Practice 1167, American Petroleum Institute, Washington, DC, draft working document.
10. Engineering Equipment Materials Users’ Association (EEMUA), Alarm Systems A Guide to Design, Management and Procurement, EEMUA Publication 191, second edition, London, 2007.
11. “Pipeline Control Room Management,” API Recommended Practice 1168, First Edition, American Petroleum Institute, Washington, DC, XXX 2008, draft working document.
12. “Guidance Document for the Qualification of Liquid Pipeline Personnel,” API Publication 1161, First Edition, American Petroleum Institute, Washington, DC, August 2000.
13. Hollifield, B., D. Oliver, I. Nimmo, and E. Habibi, The High Performance HMI Handbook, Houston, TX; PAS, 2008.
14. Engineering Equipment Materials Users’ Association. Process Plant Control Desks Utilising Human-Computer Interfaces – A Guide to Design, Operational and Human Interface Issues, EEMUA Publication No. 201. London: EEMUA, 2002.
Comments