October 2012, Vol. 239 No. 10

Features

Measuring Control Room Mental Workload With A Multi-Dimensional Measure

This article concerns satisfying PHMSA regulatory requirements for pipeline control room management.

What causes a human factors engineer, a Ph.D. in industrial/organizational psychology and experienced pipeliners from Pipeline Performance Group (PPG) to be invited to over 20 pipeline control rooms in the 2011-2012 time period? Because companies need assistance in complying with a requirement in the PHMSA Control Room Management Rule 195.446(e)(5) and 192.631(e)(5):

“Monitor the content and volume of general activity being directed to and required of each controller at least once each calendar year, but at intervals not exceeding 15 months, that will assure controllers have sufficient time to analyze and react to incoming alarms.”

The language used in the regulation emphasizes these terms:

* Monitor – keep a close watch; perform a systematic review.
* Content and volume – identify what controllers are doing and how often they do those things.
* General activity – include all of the controller tasks even if they do not involve control and monitoring of pipelines.
* Directed to and required of – determine what the company has assigned to controllers and expects them to do well.
* Each controller – involve each of the individual controllers that work in the control room.

The stated objective of this monitoring of general activity is to “assure controllers have sufficient time to analyze and react to incoming alarms.” A starting point, but not a stopping point, is to count the number of alarms and the rate of annunciated alarms. Some companies then count other activities that are not difficult to count. A realization arises that some type of analysis of this data will involve a more comprehensive approach to workload assessment. This approach will have to be done every calendar year.

The process used by PPG is formal, structured and comprehensive. The controllers are involved in the process, which we believe is very important since controllers are the ones responding to and analyzing alarms. The method is repeatable and is designed to identify the tasks, the subtasks and the required time to do those tasks.

There are two phases:
1) Information gathering from both existing documentation and the controllers; and
2) The statistical analysis and assessment of the information to produce a report.

In the control room, our PPG personnel review documents and information related to controller tasks; conduct observations and interview controllers and support personnel; and administer a self-report workload rating scale. The self-report workload rating scale is the NASA TLX (Task Load IndeX). The second phase involves presenting the findings and recommendations and getting feedback from the operator. This feedback is incorporated into the report. The purpose of the two-phased approach is to ensure that the data is interpreted correctly and that recommendations are feasible and address the root cause of any issues.

The first time we used this method was in a products pipeline control room when automation engineers had concluded that controllers spent 60% of their time “monitoring.” To the engineers, this meant the controllers were only busy 40% of the time. To the authors, this meant there was an incomplete understanding of workload measurement and an incorrect definition of monitoring as a function that did not require much mental effort. After the use of this method, the pipeline company’s management had a more complete understanding of the controller workload and the importance of allowing adequate time for monitoring.

Workload As A Multidimensional Measure
Monitoring is an important component of the requirement that that the company must “assure controllers have sufficient time to analyze and react to incoming alarms.” This means that workload during normal conditions, excluding alarms, should not require 100% of the controller’s attentional resources. In assessing workload for controllers it is important to address qualitative and subjective issues, such as individual differences in the capabilities of controllers and individual differences in the perception of workload.

There are several workload measurement methods that are appropriate for applying to control room monitoring and control. A time and task analysis is one approach that can be performed on tasks on each console for each line. This would require 1) determination of the task complexity, 2) task goals, 3) task duration, 4) task allocation and 5) the mental and physical activities required during each task. Since the number of tasks and the duration of tasks of “general activity” appear to be important, any time pressure and the time of day tasks are performed should be considered.

The analysis would include performance of primary tasks and secondary tasks. This is important, in the context of the CRM rule statements. A primary task, for example, would be the normal startup of a pipeline or pipeline segment. A secondary task could be response to an abnormal operating condition that occurs during the normal startup. The measure of primary and secondary tasks is valuable for providing an indication of spare capacity.

The “time and tasks” observations needs to be done on shift for each line controlled from the control center. These quantitative measures of number of tasks performed by controllers, combined with the time required for each primary and secondary task, will provide data on the amount of time being used and an estimate of available time.

Our preferred method for mental or cognitive workload measurement is the NASA TLX, a multi-dimensional rating procedure that provides an overall workload score based on a weighted average of ratings on six subscales: 1) mental demand (amount of mental or perceptual activity), 2) physical demand, 3) temporal demand (time pressure), 4) effort (how hard the controller had to work), 5) performance (general concern about safety and individual job success and security) and 6) frustration level (how stressed the controller felt).The measure combines weighted ratings on the six subscales to provide one integrated workload rating. Workload is greatly influenced by the mental perception of the controller, so a self-rating method, such as the NASA TLX, is an appropriate measure.

Documentation and Task Analysis. During Phase 1, PPG collects and analyzes information to gather data on controller tasks. Task analysis is conducted in order to get a full list of the controller tasks. The task analysis is based on information that includes procedures and manuals, SCADA event logs, alarm logs, phone and fax logs, abnormal operation conditions reports, communications and any other pertinent documentation.

This information is collected and reviewed before the onsite work so that the PPG team can organize and analyze the data. This pre-work reduces the amount of disruptions during the onsite work. Scales and measures are customized based on actual controller tasks.

Observations and Interviews. A “typical period of time” is selected, after determination of times of day, week, month, or season when an average console workload exists. Using PPG methodology, observations are made on shift of each console to track amount of time for controlling, monitoring, communication and interaction with people and SCADA, etc. In addition, the data from SCADA system on number of startups, shutdowns, deliveries and alarms per line and interview results per console are analyzed. This enables an examination of the work processes.

At this time, the PPG team introduces the NASA TLX method to controllers and works with them to understand how to complete the ratings. A schedule is developed to assign controllers one specific night shift and one specific day shift to complete the NASA TLX ratings. Ratings are completed at the top of every hour during a shift. This “sampling” takes place at different times during the day, the night and the weekends in order to get representative periods of times when workload is high, low and optimal. Data sheets are completed online, using a secure site provided by PPG. Controllers complete ratings at the top of every hour of one complete day shift and one complete night shift. Along with the NASA TLX, an assessment is made of which tasks were completed during that hour.

Workload Ratings. Once all of the controllers have completed their ratings online, analysis can be conducted. Analysis of the workload data includes:

* Individual controller workload,
* Workload fluctuations during shifts,
* Workload measures for specific tasks,
* Workload differences between shifts,
* Contributors to workload,
* High workload shifts, days and hours, and
* Analysis of task contributors to workload during high workload hours.

Data are presented in both statistical format and in graphical format for clear interpretation. Table 1 and Figure 2 and Figure 3 illustrate how workload might vary among a group of controllers and across a 24 hour period.

Individual Workload Averages

Table 1: Comparison of workload scores between shifts for individual controllers.

Figure 2: Comparison of workload scores between shifts for individual controllers.

Figure 3: Workload ratings throughout a 24-hour period.

Human Factors Ratings. In addition to the workload ratings, controllers are asked to complete a human factors survey. The purpose of the survey is to provide insight on issues that can impact workload, such as communications, roles and responsibilities, safety, training, SCADA displays, procedures, shift work and fatigue. The results from this survey are presented to our customer in a separate report and are not part of the workload ratings report. The information in this report provides information and diagnostics for any existing human factors related issues.

Final Reports. After both the workload report and the human factors reports are reviewed by our customer, we consider their feedback to ensure that all the data interpretation is correct and that our recommendations are feasible. After completing the next iteration we finalize the reports.

Summary. Our total analysis combines onsite observation and input from controllers and management, information from documentation and historical archives and self-report ratings from controllers. It is a well-rounded methodology to ensure that a complete picture of controller workload is obtained.

Authors
Charles Alday has 45 years of experience in pipeline construction, operations, maintenance, management and consulting. He has a Master’s degree in business, with a focus on management. For the last six years of his 30-year career with Colonial Pipeline Co. he worked with people and teams in all parts of the organization as leader of Colonial’s Operational Excellence program – designed to eliminate pipeline leaks, spills and errors. A human factors program was a critical component of Operational Excellence. 770-856-5713, charles@pipelineperformancegroup.com.

Michele Terranova, Ph.D., is a principal of Pipeline Performance Group. She has 20 years of experience in Human Factors and User Interface Design and has worked in a vast array of environments, in nuclear, military, aviation and process control. She has held previous positions as the Director of Human Factors Research at Concord Associates, Inc. and Senior Research Scientist at The Oak Ridge National Laboratory. She has done human factors consulting and training with 16 pipeline companies. michele@pipelineperformancegroup.com.

Qualifications And Services Of Pipeline Performance Group

Pipeline Performance Group (PPG) has worked or is working with more than 25 companies in 23 control centers on control room assessments, development of Control Room Management (CRM) plans, implementation of CRM plans, new control room design, workload assessments and training for controllers and other pipeline personnel. They are working with 12 companies on fatigue risk management plans and providing fatigue management educational materials to 24 companies.

The company says it is the only human factors consulting company that applies practical pipeline shift work experience, control room management and consulting experience and doctoral qualifications to the development of control room management plans, pipeline human factors consulting and fatigue risk management programs for regulatory compliance and operational excellence.