Maintenance processes are of high importance for industrial plants. They have to be performed regularly and uninterruptedly. To assist maintenance personnel, industrial sensors monitored by distributed control systems observe and collect several machinery parameters in the cloud. Then, machine learning algorithms try to match patterns and classify abnormal behaviors.
1. Introduction
The rapid evolution of Industry 4.0
[1], accompanied by the enormous amount of data collected from various sensors, devices, machines, or embedded systems, is increasing the research and industrial communities’ needs for intelligent systems, and eventually will lead us to the arrival of the Industry 5.0 era. Until now, the ancestor of Industry 5.0, the digital Industry 4.0, has benefited from the use of the Industrial Internet of Things (IIoT), Big Data, cloud computing, and Augmented Reality, which will be followed by the exploitation of the encapsulated knowledge via Artificial Intelligence
[2] and more precisely through machine learning and deep learning techniques.
Gathering data from a set of IIoT sensors necessitates a suitable control unit. Hence, two main systems appear in the industry, decentralized control systems (DCS) and programmable logic controllers (PLC). Furthermore, the storage and analysis of the collected Big Data
[3] require distributed database management systems (DBMS) as a unified data point of origin, implementing artificially intelligent logic and cloud services. Taking into account the gathered industrial sensory data, it is unquestionable that much knowledge is encapsulated in them. The extraction of patterns, correlations, and outliers included in these collections are tasks which humans can hardly process. Consequently, automated, ingenious, and highly productive practices are in great demand to exceed human limitations while decreasing engine failure and increasing productivity.
The appearance of machinery component malfunctions and critical events are two mandatory scenarios frequently revealed in an industrial environment. Therefore, the operation status of several delicate machinery parts, such as pumps, compressors, and robotic arms, must be kept under surveillance, predominantly when they work under high temperatures, pressures, or/and strict performance indices defined by manufacturing requirements
[4]. Focusing on decision making, machine learning (ML) techniques and, more precisely, data mining
[5] and regression
[6], are broadly used
[7][8], leading to robustness in industrial maintenance, detecting the majority of possible faults through pattern recognition and triggering a proper alert.
Deep and machine learning algorithm operationalization is different from traditional algorithm deployment. Therefore, thoroughly evaluating machine learning algorithms before production is an important validation of their correct operation. Such validation includes formal reasoning over all possible inputs or property checking that all industrial responses/ behavioral requirements are captured via formal methods
[9][10], and their practical implementations over appropriate representational languages or tools
[9][11]. The verification of the strict implementation of operations and their response using validation tools should also be addressed. Model checking, model-based testing using formal operational test scenarios, and design by refinement and abstract interpretation during training and validation will lead to robust deep learning models
[11].
There are three state-of-the-art categories of algorithms for industrial maintenance and machinery operations:
Classical ML or deterministic methods: This category includes algorithms such as linear regression, fuzzy control, threshold control, proportional integral derivative (PID) control, support vector machines (SVM), decision trees, random forest, etc. These algorithms are currently in use by most modern industries and machinery maintenance software for classification and regression purposes. Nevertheless, their appliance is of a specific use and targets maintenance cases, with different hyperparameter values for each case that requires accurate calibration;
Narrow depth ML methods: This category includes ML networks of limited depth and techniques of targeted patterns detection. Gradient boosting networks such as LightGBM, and neural networks of limited and fixed depth are corresponding methods of this category. This category of algorithms focuses on the pattern detection of time-invariant decisions or specific decisions applicable to time series of measurements of minimal memory capabilities (real-time detection);
Deep learning methods: This category includes classification or regression algorithms, capable of variable patterns detection, that can apply to either time streams or irregular time intervals of sensory data and provide the detection of erratic patterns, either real-time, close-to-real-time, or periodic. This category includes convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and neural networks of variable depth based on input. This research focuses on this network category for detecting machinery operation abnormalities.
This research focuses on the oil refinery industry containing compressors and pumps processing flammable gases and liquids. Attributes influencing an engine’s proficiency are temperature, pressure, and vibrations resulting from its operation. As a result, the examined machine’s temperature for compressors and pump acceleration sensor measurements are used as data inputs. A new intelligent failure classification algorithm called the stranded-NN model is presented by the authors. This algorithm utilizes different layers of neurons based on sampling processes over the input sensory data streams. The generated model is used to detect different classes of industrial emergencies based on input time-depth of sensory measurements and can be utilized for either periodic preventive maintenance cases or real-time and close-to-real-time malfunction machinery events.
2. Deep Learning Stranded Neural Network Model for the Detection of Sensory Triggered Events
Industrial maintenance levels are mainly identified by monitoring parameters during operation using sensors. Different operation responses can be determined according to the measurement probing, variability, or limits set by the machinery manufacturers. Usually, the industrial concentrators receive a series of time-framed measurements per machine. Then, these measurements are split into measurement intervals called measurement batches and are driven to a detection model as input. Existing detection models include classifiers or regressors. The size of the measurement batches as model inputs signifies the model’s detection-monitoring granularity. For this reason, based on the data input and sensor probing intervals, there are different types of machinery monitoring cases:
- Real-time
-
critical events include sensory measurements of precise machinery operations. The sensory input in such cases is of multiple sensors of the same locality measuring different machinery parameters per second, setting the detection response interval to no more than a few minutes (1–3 min);
- Close-to-real-time
-
events include sensory measurements where the malfunction detection can be extended to a few minutes’ response due to the extended intervals of measurement acquisition or degraded sensors’ accuracy. In such cases, the detection response intervals can be of a few minutes (3–15 min);
- Periodic
-
maintenance events or checks include batches of sensory measurements of hourly or daily intervals (15 min/30 min/hourly/daily). Such detection tests can be automatically performed during operation or post-processing. These are targeted maintenance checks that detect deviations or malfunctioning patterns concerning past machinery operational behavior.
Following this categorization that requires different handling approaches per event type, the authors set real-time events of short observation intervals that require immediate actions and periodic long term observation intervals for planning future maintenance tasks. The following subsections summarize the existing methods used for each type.
2.1. Problem of Critical Events Detection
With rapid advances in information and communication technology, log files, which are time records of the occurrence of various types of events, are commonly available at both machine and system levels in industrial enterprises. For example, the service department of most manufacturers keeps a record of after-sales service provided for products during the warranty period. These records contain the occurrence of malfunction/failure events and the corresponding repair actions taken on the products over time. Many modern machines are numerically controlled by on-board computers, and various events such as machine activities, critical failures, operator/user actions, and job status are recorded in real-time during machine operation.
Information about the relationship between events, when combined with the ability to accurately predict critical events, is very useful in identifying the root causes of failures and in designing optimal preventive maintenance policies that can reduce unexpected machine downtime and maintenance costs. Thus, the establishment of a method of modeling and predicting events based on the event log can be particularly valuable in industrial practice.
Deterministic solutions for industrial maintenance and critical events detection focus on using thresholds or PID (proportional integral derivative) controllers coefficients or fuzzy rules and parameters. Unfortunately, such approximations are insufficient due to their static nature, resulting in their inability to capture patterns even among the same machines. Furthermore, fuzzy logic techniques
[12][13], e.g., using Gaussian models, are still deterministic, sometimes lacking rules and physical interpretation. However, it can perform adequately in sensory responses with no annotated output feedback
[14]. On the other hand, other methods, such as linear or non-linear regression models
[6], are limited in performance due to their absence or the static use of data processing depth and the appliance of uniform pre-processing methods towards the input sensory data or equipment
[15].
Significant progress has been made recently in machine learning and artificial intelligence
[16][17]. Many new general data-driven modeling approaches have been developed, among which deep learning methods have proven themselves quite flexible and strong. Deep learning is a general method of approximating nonlinear functions that uses a neural network framework, which can learn, from data, the relationship between high-dimensional inputs and output. The effectiveness of deep learning comes from its flexible structure. Recent advances in stochastic gradient descent (SGD) optimization and GPU-based parallel computing enable very large-scale deep learning models, thus enhancing the flexibility and efficiency of deep learning models. Narrow depth networks, specifically MLPs, are also used in industrial maintenance focusing on real-time event detection. The authors of
[18] present two MLPs of one and four hidden layers accordingly for predicting gas turbine and compressor decay states. Orru et al. present an MLP model for predicting potential machinery faults
[19]. Massaro et al. suggest an MLP model using the temperature of two milk production lines as input, providing an alerts classifier
[20]. Finally, Ullah et al. proposed an MLP classifier for thermal conditions of power substations
[21].
Towards events detection using deep learning approaches, the authors of
[22] proposed a long short-term memory (LSTM) model to achieve extreme event forecasting by working on time series to solve anomaly detection problems, budget planning, and optimal resource allocation, among others. A plethora of studies on LSTMs can be found on the web and the literature, such as
[23], a study where a method is proposed to solve the problem of predictive pump maintenance based on sensory data. Moreover, the study of
[24] describes the definition of an LSTM model for turbofan engine maintenance on NASA’s dataset. Man and Zhou
[25] presented a mixed model for hard failure predictions where both degradation signals and time-to-event data are given. The authors of
[26] also propose an IIoT framework for productive maintenance that uses LSTM networks to extract productivity and maintenance features.
Another origin of collected industrial data events is in the form of event log files, such as systems’ profiles and maintenance notes. Huang et al.
[27] proposed a Deep Learning technique based on recurrent neural networks (RNN) that are able to predict critical events trained on event logs. Yuan et al.
[28] proposed a statistical model of event logs for the problem of system failure prediction. One significant drawback of deep learning methods is that it is hard to train them on embedded systems in real-time due to underlying hardware limitations (DCS or PLCs). Nevertheless, deep learning models can be uploaded and used for prediction purposes
[29]. In this case, pre-trained models are uploaded to the cloud and are assumed to be automated periodical upgrades. Consequently, “tiny” ML
[30] methods are gaining more and more popularity due to their low needs for resources, e.g., memory.
2.2. Problem of Industrial Maintenance
As the Industrial Internet of Things (IIoT) technology develops rapidly, companies have the ability to observe the health of engine components and manufactured systems through the collection of signals from sensors. According to the results of IIoT sensors, companies can build systems to predict component conditions. Practically, the components are required to be serviced or replaced before the end of their life while performing the work assigned to them. Predicting the life state of a part is so important for industries that intend to grow in a fast-paced technological environment. Recent studies on predictive maintenance help industries to create an alert before parts fail. By detecting component failures, companies have the opportunity to keep their operations efficient while reducing their maintenance costs by repairing components in advance. Since maintenance directly affects productivity and service quality, optimized maintenance is the key factor for organizations to allow them to generate more revenue and remain competitive in the growing industrialized world. With the help of a well-designed predictive system to understand the current state of an engine, components could be taken out of service before a malfunction occurs. With the help of inspection, effective maintenance extends component life, improves equipment availability, and keeps components in good condition while reducing costs. Real-time data collected from sensors are an excellent resource for components’ live-cycle modeling. Markov chain models
[31], survival analysis for machinery lifespans
[32], ML optimization algorithms
[33], and various machine learning approaches have been applied to model predictive maintenance
[34].
For solving the prediction task, machine learning (ML) technology is increasingly being used. However, the state of recent research is not well posed and there is a lack of adoption of up-to-date models for pre-processing and training
[15]. The incorporation of the uniform use of ML algorithms for Industry 4.0 classification detection and prediction processes is also at risk. The authors of
[35] propose the use of long short term memory (LSTM) networks to predict the current state of a motor. The LSTM model deals with sequential input data. The training process of LSTM networks is performed on a large-scale data processing engine with high performance. Since the huge amount of data flow into the prediction model, Apache Spark, which offers a distributed clustering environment, has been used. The output of the LSTM network decides the current life state of the components and offers alerts for components before their end of life. To predict the current state of any system unit, condition-based maintenance (CBM) has been proposed. According to Jardine et al.
[36], CBM recommends actions based on the information collected by the system. The main objectives of CBM are to avoid unnecessary maintenance actions and to recommend maintenance actions if an anomaly is detected. Estimating the remaining useful lifetime (RUL) with high accuracy is crucial to developing an effective CBM strategy. RUL could be predicted by collecting signals with sensors located in relevant units of the system.
Furthermore, the use of deep learning RNNs and, more specifically, LSTM is considered significant for the prediction of machinery’s remaining useful life
[37]. Jain et al.
[38] developed artificial neural networks (ANN) to predict RUL under unknown initial wear. Jain et al.
[38] proposed an ANN-based approach to more accurately predict the RUL of high-speed milling cutters. The proposed model was built based on temporal statistical characteristics. Sateedh et.al proposed a new approach for RUL estimation called meta cognitive regression neural network (McRNN) for function approximation. McRNN uses extended Kalman filters (EKF) to find the optimal training of network parameters
[39]. Finally, Porotsky and Bluvband developed a new model for parameter optimization control based on the cross validation process to solve the question in the 2012 IEEE PHM Conference Challenge Competition, and their solution was awarded the “Winner from Industry”
[40]. In addition, Heimes developed a data-driven algorithm to predict RUL
[41].