1. Introduction
Artificial Intelligence (AI) is a system that can perceive its environment and take actions to maximise its chances of success
[1]. It has become an essential tool for addressing real-world problems inspired by human-logical thinking. One approach to achieving AI is through the SI field, a subfield of AI that has gained increasing attention due to its ability to solve high-complexity problems within a specific time frame
[2]. The SI concept was inspired by natural biological systems that adopt the collective behaviour of an organised group
[2]. SI is inspired by natural biological systems that use the collective behaviour of an organised group to solve complex problems
[3]. Swarms are vast numbers of simple homogenous agents that interact with their environment, exhibiting decentralised control of global behaviour
[2]. Swarm-based techniques use nature-inspired algorithms to produce fast, robust, and cost-effective solutions to complex problems
[3]. Social swarms in nature, such as honeybees, bird flocks, and ant colonies, exhibit collective behaviour that can be modelled and adapted to solve complex problems
[2][3].
Inside the SI field are examples of algorithms, such as cuckoo search, flower pollination algorithm and particle swarm, optimisation that have been included to achieve SI results
[2]. Such an example from a previous study for SI is the applications of SI towards air overpressure (AOp)
[4] from blasting and causing damage to nearby civilians. Therefore, to predict AOp accurately, ref.
[4] used the artificial neural network (ANN), and the prediction was trained using PSO to predict the AOp accuracy
[4]. PSO is a swarm-based intelligence that exploits the concept of social behaviour algorithm with a potential solution to a given problem viewed as a particle similar to a flock of birds
[3][5][6]. Each particle then combines its historical best locations and gradually approaches the objective function optimum solution
[3]. The above-mentioned advantages show that PSO is a promising development for optimising solutions to real-world problems
[7].
PSO, in computing terms, is also defined as a computational procedure to select the most effective element and optimised it based on the collection of accessible alternatives
[8]. Either with increment or minimisation, it is the true operation of the simplest state while constantly screening the input elements associated with an allowed set of accessible alternatives
[8]. In the field of structural engineering, PSO is one of the evolutionary computation (EC) techniques used by many researchers
[9]. PSO was proposed by
[5] in 1995, and it contains memory that allows each member of the swarm to remember and acquire while moving through the searching space
[9]. PSO is inspired by social problem-solving and seeks to perform a parallel search for the optimal solution with many individual searches associated with particles and collaborative influencing by joining the best performance of all searches
[10]. According to
[10], PSO is considered to be a good choice for practical solutions to optimisation problems when the problem is high-dimensional, defined by multiple criteria and potentially conflicting constraints or complex combinatorial nature
[3][11] and the results achieved are based on the particles moving as a swarm in the space of the defined problem searching for the best solution, with the likelihood of finding an optimal or near-optimal solution being high.
PSO is a versatile algorithm that has been applied to a wide range of optimisation problems, including power system management
[10] and indoor comfort and energy consumption optimisation
[11]. PSO is especially useful in practical solutions to optimisation problems that are high-dimensional, defined by multiple criteria and potentially conflicting constraints, or of complex combinatorial nature. For example, in the study by
[11], an artificial bee colony algorithm was used in conjunction with PSO to optimise the control of electrical appliances to minimise operational costs, reduce energy consumption, and maximise comfort levels in indoor spaces. In general, it is considered to be a good choice for practical solutions to optimisation problems when the problem is high-dimensional, is defined by multiple criteria and potentially conflicting constraints, or is of complex combinatorial nature. The PSO algorithm has demonstrated efficiency in various optimization-related applications, and recent technological advancements have the potential to enhance its features by integrating PSO. One such example is the ERS. ERS is a sub-part of artificial intelligence that enables a system to learn and recognise human emotions through different data modalities.
Affective computing (AC) is a related field that focuses on developing computers that can understand human behaviour, as introduced by
[12]. Over the last decade, researchers have identified the subpart of AI that can be enhanced further using the capabilities of AI machine learning. One of the main highlights that have been suggested is machines and computers identify or learn human emotions
[13][14] to improve the interaction between humans and computers and increase potential future applications. Supported by the field of AC that was introduced by
[12], in recent years, researchers identified specific applications, such as the ERS. ERS is an emerging technology that has gained attention and has become an important area of study due to the progress of technology. Commonly, ERS is an embedded technology based on AI that enables machines to recognise human emotions
[15]. The ability of machines to identify human emotions has been a focus of researchers, and the development of ERS has resulted from this body of work. ERS is still in the developing stage, and researchers and engineers are working on producing more modalities to enable more real-world applications
[16]. ERS has the potential to be a significant technology that enhances AI further in the real world. Indeed, ERS has the potential to revolutionise various sectors and bring about significant advancements, namely, towards manufacturing, healthcare, and education. A previous study suggested that the application of ERS across various industries is possible, given the innovations seen thus far
[13]. In healthcare, ERS can be used to detect and/or monitor patients’ emotional states, manage mental health conditions, and provide personalised care. For example, ERS can be used to detect depression symptoms in patients and alert healthcare providers to intervene and provide timely care
[17]. In the automotive industry, ERS can be utilised to improve driving safety by detecting and responding to the emotional state of drivers, such as detecting drowsiness or stress and alerting the driver to take necessary actions
[18]. In the education sector, ERS can be used to monitor students’ emotional states and provide personalised feedback, tailored learning materials, and interventions to improve learning outcomes
[19].
2. Particle Swarm Optimisation for Emotion Recognition Systems
2.1. Particle Swarm Optimization
As mentioned earlier in the paper, PSO was created by
[5] and used the technique that has been inspired by natural bird flocking, fish swarms, and bee colonies to optimise decision-making by nature
[5] and transform into algorithms that enable real-life metaheuristic algorithms to find optimisation
[20]. Since then, the PSO algorithm has gone through a lot of changes, and there were many extended versions have been proposed
[21]. In recent years, various dynamic and adaptive variations based on the objectives of optimising decision-making have been forwarded
[21]. With all the adaptations, variations, and innovations, some versions can be described as the ‘standard’ due to the superior outcomes of optimisation in the real-world
[21].
PSO is likened to the simulation of bird behaviour in the algorithm
[20]. This model illustrates how a swarm of birds fly around in search of food and shelter, and this behaviour is incorporated into the PSO algorithm. The movement of particles in the algorithm, analogous to the flock of birds, involves sharing the best position to efficiently identify optimal points for decision-making
[3][5][21]. In a recent study conducted by
[20], the researchers examined the behaviour of social animals using the artificial life theory to construct artificial life systems with cooperative behaviour through computer simulations. The study highlighted the importance of understanding five fundamental principles in this context. Those five principles were:
-
Proximity: The swarm will carry out simple space and time computations;
-
Quality: The swarm sense the quality change in the environment and responds to the changes;
-
Diverse Response: The swarm should not limit its way to get the resources in a narrow scope;
-
Stability: The swarm does not change its behaviour with every environmental change;
-
Adaptability: The swarm can change its behaviour when the change is worthwhile.
In PSO, particles have the ability to update their positions in response to changes in the environment
[20]. As the particles move and rotate in space, they aim to approach the optimal value by adjusting their positions relative to the best position
[3]. Additionally, ref.
[21] conducted experiments involving a small number of global optimisations based on human problems. These optimisations were conducted using different topologies, which refer to methods of organising particles to facilitate the flow of information within the population. This arrangement allows individual members of the population to be influenced by those that have achieved the best performance thus far
[21]. Although PSO and many metaheuristic algorithms have been proposed in recent years, there are many terms to solve optimisation problems that can be identified further. As one example, the Pendulum Search Algorithm (PSA)
[22] is a population-based algorithm of an optimisation problem; however, the findings of the study highlighted that PSA is outperforming PSO
[22]. The concept of PSA is a physical phenomenon mimicking the harmonic motion of a pendulum to move the search agents for the optimal solution
[22]. As suggested by
[22], the application in the real-world PSA can be applied to the vaccine distribution optimisation problem.
Over the last few years, PSO has attracted the interest of many researchers and researchers addressed many inspired nature-based algorithms rather than just flocking of birds, bee colonies, ant colonies, and fish swarms. Metaheuristic approaches, and intelligent algorithms have been identified to provide the maximisation of gain and the minimisation of loss
[11][22]. In a study by
[23], a dynamic multi-swarm (DMS-PSO) based system has been proposed to select the most suitable attributes and assist the diagnosis of heart diseases in medical analysis. The study shows that a combination of fuzzy logic and DMS-PSO can offer more effective systems of medical diagnosis with improved system accuracy. From the experimental analysis, DMS-PSO indicates a relatively higher performance when compared with existing systems in the healthcare industries and manual diagnosis, and it is expected that in real-life applications, DMS-PSO can achieve more reliable results. Furthermore, ref.
[24] showed an improved scheme of particle swarm algorithm (PSO) and Newtonian motion laws, labelled as centripetal accelerated particle swarm optimisation (CAPSO), has been proposed to accelerate the learning and convergence procedure of classifiers. The obtained results of the PSO-CAPSO identified nine medical disease diagnosis benchmarks indicating that the proposed method provides better results in terms of good convergence rate and classification accuracy.
In the PSO algorithm, each particle functions as an independent agent that explores the problem space based on its own experience and interactions with other particles, resembling the natural behaviour of animals
[20]. Therefore, it is crucial to understand the suitability of the algorithm for specific applications. The PSO algorithm offers several advantages, including robustness, adaptability to different application environments, and strong distributed abilities that enable it to quickly converge to optimal values
[20]. In recent years, advancements in computational algorithms have further enhanced the speed, quality, and robustness of the PSO algorithm. As a result, there are various potential applications that can benefit from utilising PSO. Findings on the previous studies
[23][24][25], combining PSO with several other existing methods provides positive results. There are many applications in the real world, and the existing PSO may solve complex problems and enhance features in the SI and AI field. Related to the development of ERS, ERS can be one of the future-proof technology, and the identified modalities for ERS can benefit from the embedded PSO.
2.2. Emotion Recognition System
Over the last decade, researchers have identified the subpart of AI that can be enhanced further using the capabilities of machine learning. One of the main highlights that have been suggested is machines and computers identify or learn human emotions
[26] to improve the interaction between human–computer and increase potential future applications for human–computer. Furthermore, for machines and computers to learn about human emotions, it needs a system that enables the required process to learn human emotion
[13] and is specified as an emotion recognition system (ERS). ERS enables a system that accepts various modalities of data, learns, and allows the machine to recognise the emotion of human subjects.
Previous researchers specified ERS as originally in the field of Affective Computing (AC), a field that was brought forward by
[12]. According to
[12], AC is an ability of a computer that relates to, arises from, and understands human behaviour
[12]. With that being said, AC led to further findings of emotion recognition, and it has been suggested by the previous researcher that AI learning can signifies for further findings of ERS as embedded technology in various applications
[17]. Moreover, ERS has emerged as one of the attractive areas in the field of AC and AI with the promises of what the technology can achieve
[26], such as developing robots that can interact and communicate with humans, equipped with functions for both understanding human emotions and expressing human emotions
[26].
Outcomes and expectations from the previous studies show that ERS can be an important technology in the making based on the advantages offered to individuals, society, organisations, businesses, and industry via various platforms and applications. For example, ERS in healthcare
[17], ERS in driving assistance in the car
[18], ERS in the classroom
[19] and ERS in smartwatches as the latest addition due to the modalities of smartwatches being the same ERS modalities. In existing research on ERS, researchers adopted AI algorithms, such as convolutional neural network (CNN) and deep neural network (DNN) for emotion recognition based on various data modalities, such as facial expression, voice intonation, heart signals (ECG), brain signal (EEG), and many others. It is important to look into the relationship between ERS and its significance in society and industry, specifically in the industrial revolution
[26].
One of the main highlights that benefit from ERS is in healthcare industries
[17], developed the modalities of ERS and suggested their applications in healthcare. Researchers used facial recognition combined with ECG to identify the six basic emotions
[27]. Furthermore, ERS has been suggested to provide better health services quality assurance by facilitating decision-making throughout the COVID-19 pandemic by allowing safe monitoring and higher emotional awareness among the practitioners
[26][27]. Furthermore, according to
[28], social robots have risen to facilitate limitations due to COVID restrictions. Social robots work and interact with humans; therefore, features of emotion recognition technology might enhance the interaction between social robots and humans. From a marketing perspective from retail industries, ERS helped in advertising products. One of the main examples is text sentiment analysis. Ref.
[29] suggested that text mining provides useful data in identifying emotional awareness through text. Text mining is a learning-based algorithm to describe characteristics of text, such as word expression based on human sentiments and emotions
[29].
Previous research has focused extensively on the modalities and potential applications of ERS, but there has been limited study on the readiness of the technology for adoption. A recent paper by
[30] examined the readiness of ERS from the perspective of university students to determine whether university students are aware of recent developments in ERS and whether national infrastructure should encourage the use of ERS. In a few years, ERS can be one of the significant technology that will be beneficial to industries and societies. Therefore, to contribute to the ERS development and benefits to the ERS practitioners, this study will be looking into one of the factors that enhance ERS modalities which is PSO.
SLR is a methodical and structured approach to identify, evaluate, and synthesise all relevant literature related to a specific research question or topic
[31][32]. It provides a comprehensive overview of the current state of knowledge on a particular subject, helps identify research gaps, and informs future research directions. SLR is a valuable tool for researchers as it helps them avoid bias and ensures that all relevant studies are considered, leading to more accurate and reliable research outcomes. A previous study that adopted SLR
[33][34][35] has shown significant results in identifying recent related studies and findings that help the researcher investigate the depth of the chosen field.