1. Introduction

In recent years, the advanced potential offers of understanding the physical phenomena that occurred in the brain, as well as the information technologies, make the design of brain computer interface (BCI) systems easier. Non-exhaustively, this technology is mainly used for the functional substitution and pathological analysis. the technologies which were deployed to support people with severe disabilities or doctors into two main categories with examples. There are other application domains including the video games ^[1][2][1,2], the virtual and augmented reality ^[3][4][5][3,4,5], communication ^[6][7][8][6,7,8], etc.

These 4 units are controlled by an operating protocol that defines the onset and timing of operation, the details of signal processing, the nature of the device commands, and the oversight of performance. For the other one, it contends to place sensors on the scalp and recovers cerebral activity by measuring the electrical activity (EEG), Functional magnetic resonance imaging (fMRI), and magnetic activity (MEG). The noninvasive technique based on EEG method remains the most solicited for several decades due to its ease of application, low cost, good temporal resolution and not requiring surgical operation ^[9][10]. Finally, the fourth unit is the classification which permits to discriminate between the class label of the features and convert the labels into logical control signal in order to control the artificial agents or into useful presentation for the practitioner.

The most existing BCI platforms are usually implemented in Laptops because it was difficult at the beginning to move the BCI technologies from laboratory scale experiments to the daily life of the people with a neurological disorder ^[10][11][14,15]. Furthermore, the focus of the researchers was to apply these technologies on a personal computer occupied with high computational resources. Such systems are implemented in high performance processors without taking into account the memory resource, power consumption, compactness, the volume, etc. Today, due to the wide spread of the embedded system and the computational resources that can offer, it becomes an emergency to implement the BCI technologies in embedded platforms.

Recently, many technical papers have provided short reviews of related work on BCI ^{[12][13][14][15][16][17]}[11,16,17,18,19,20]. Most, if not all of these reviews, are focused on the application domain of the BCI, the different acquisition technologies, the signal processing algorithm, and limiting the evaluation in order to compare their general advantages/disadvantages, the accuracy of the signal processing, and machine learning algorithms. To support the researchers in the embedded BCI area, as well as other interested parties including EBCI architecture, online/offline BCI, evaluation criteria of the embedded BCI systems, this paper provides a wide and deep look into state-of-the-art contributions in the field of embedded architecture of BCI systems. For this purpose, we followed a systematic approach by addressing each of the following criteria for each reviewed paper.

Concentration/Stimulation?

Some studies present hybrid systems that combined these two signals to exploit the advantage of each signal type and to get highly accurate system ^[18][22]. Despite the good performance obtained by the proposed system, this last requires the user’s disposition in front of simulation panel and needs an effort of concentration. Thus, the BCI system controlled by the evoked potential signal seems to be inappropriate for people with concentration difficulties or with sight problems when the acquisition process becomes unfeasible. Hence, in this evaluation, the system controlled by the spontaneous EEG signals will be seen as a good and an efficient system in contrast to the system controlled by the evoked potential EEG signals.

Adaptability?

For example, these artifacts are generated by the heartbeat, eye movement, muscle activation and user movement ^[19][20][21][26,27,28]. Fortunately, the methods to eliminate these artifacts are quite simple because they represent repeated morphology waves related to body member function that can be learned by the system during the training phase. On the other hand, the non-physiological artifacts are related to the electrode interface, as well as to the acquisition system and the environment in which it operates. In this review, a BCI system is classified as a good system when the EEG signal processing algorithm removes artifacts according to the dynamic approach which leads to a quasi-stationary system accuracy for all subjects.

Performance?

In the brain computer interface community, many evaluation metrics are used to measure the performance, such as accuracy of classification, Kappa coefficient, mutual information, information transfer rate (ITR), sensitivity, and specificity ^[22][31]. The most common one is the accuracy of the classification which allows for measuring of the number of trials classified correctly of total trials. For example, in the literature, the BCI systems that have an accuracy lower From this point of view, in the study, an evaluation grid based on the accuracy is defined to compare the existing BCI as presented inTable 1.

Power consumption?

The EEG signal processing algorithms are time consumed due to the huge computation and the traffic load across the whole EBCI system. This matter leads to increase the energy consumption although the EBCI power budget is still confined to a few watts. Hence, the energy consumption criterion is used in order to estimate the amount of the power used by the EBCI system to predict or to translate the brain activities to commands. According to this grid, a good EBCI system is the one which consumes less than the other ones where its power consumption does not exceed 1 W. A Fair and a poor system has a power consumption, respectively, less than 5 W and greater than 5 W.

Online/offline validation?

According to the offline approach, the BCI systems are validated using an existing benchmark. These data are recorded by research groups and shared with the community as a challenge to develop sophisticated signal processing algorithms. Often, the development of any application based on EEG signal processing starts by the validation according to offline approach in order to define the appropriate signal processing techniques. Frequently, the performance in classifying the EEG trials obtained according to the offline approach decreases significantly compared to the online ^[23][35].

InSection 2, we evaluate the existing EBCI systems according to the predefined evaluation criteria. We take a closer look into EBCI systems for functional substitution and pathological disorder analysis. InSection 3, we discuss current challenges, the different architecture of EBCI systems, the evaluation criteria of EBCI systems, and possible future research directions on embedded BCI systems. Finally, we provide concluding remarks inSection 4.

2. Review of the Embedded BCI Systems

Today, despite the tremendous interest in implementing the BCI systems into embedded platforms, there are a few embedded BCI (EBCI) systems presented in literature. In fact, at the beginning, the BCI systems are moving slightly from the laboratory to the real world in order to use these systems in the daily life of people with severe disabilities or using them to assist doctors during the analysis of pathological disorders. So, at first, there is a reason in implementing such systems in a personal computer. Today, with the tremendous advancement in the embedded platforms and especially the availability of the open source platform, it becomes too easy to implement complex signal processing algorithms, while maintaining low power consumption, price and resources, etc.

The BCI systems consist mainly of two parts, signal acquisition and translation. The embedded signal acquisition part contains electrodes, analog circuit and digital system for neurophysiological signal recording and transmission. The wireless EEG acquiring device plays a vital role in the embedded BCI systems, especially with the existence of active dry electrodes allowing a convenient installation and high-fidelity signals. Today, there are many commercial companies producing wireless acquisition systems:

In Reference ^[24][13], Chin-Teng et al. proposed an EBCI system that can acquire and analyze EEG signals in real-time to monitor human physiological, as well as cognitive states. The system is composed of a four channel physiological acquisition and an amplification unit, a wireless transmission unit, a dual core signal processing unit with multitask scheduling, a sensing real signal display and monitoring unit, and a warning device. The proposed wireless EBCI system is implemented in a dual core DSP and can predict the drowsiness state with an accuracy around 75%. In fact, authors did not take into consideration the inter-subject variability and applied the same signal processing chain for the five subjects which led to a decrease in the accuracy as the case of the 3rd subject where the obtained classification accuracy was about 58%.

proposed a novel architecture of a generalized platform that provides a set of predefined features and preprocessing steps that can be configured by a user for BCI applications ^[11][15]. The architecture integrates a power line noise cancellation and baseline removal to enhance the signal-to-noise ratio, while the feature extraction combines linear and nonlinear, univariate and bivariate measures commonly utilized in BCIs. The platform is validated by implementing a seizure detection algorithm on a epileptic seizure detection and it achieved a classification accuracy of over 96%. The advantage of such architecture is that it integrated different features extraction techniques allowing for maximization of the accuracy, while decreasing the runtime and the power consumption and allowing the user to move freely without any constraints.

In Reference ^[25][39], to detect and correct seizure, a signal processing algorithms and control circuit for patient monitoring system is presented. The EEG signal is preprocessed using the spline wavelet, to remove the baseline wander signal, and the adaptive threshold and template matching to predict seizure. The seizure detected control signals are generated and the simulator block was activated. The proposed architecture reaches a very good performance in terms of the run-time and power consumption even though the study does not report any evaluation performance in terms of the accuracy.

Table 2resumes the existing EBCI systems in literature based on a research in PubMed, IEEExplore, ScienceDirect, and Google scholar research web engines. The existing systems are evaluated using the predefined criteria.

In Reference ^[10][14], Lijun et al. proposed an embedded system to control wheelchair merely by thinking. Other EBCI systems are based on a hardware architecture and they reach a very good performance in terms of the power consumption and run-time. For example, Aravind et al. proposed an embedded system that can be used for controlling electrical devices by thinking using EEG signals ^[26][40]. The EEG signal processing chain was composed by band-pass finite impulse response filter, wavelet, and Support Vector Machine (SVM).

In Reference ^[33][47], a pure hardware system based on the FPGA for EEG-MI classification is presented. The EEG signals are processed as a series of multi-channel images in the continuous time domain showing the energy changes in the cerebral cortex during the MI of the subjects. The accuracy in classification reached 80.5% where the presented design was approximately 8 times faster than the PC in terms of the execution time and decreased the power consumption by a factor 5600 compared to a standard PC.

In Reference ^[29][43], Sawan et al. proposed an embedded Wireless Recording Systems to measure the brain activity non-invasively and send the recorded data to a host system to apply the signal processing chain. The suggested system improved the mobility of patients and is used by a doctor to predict the start of epilepsy in two patients.

proposed a wearable neurofeedback system, which supports mental status monitoring with EEG and transcranial electrical stimulation for neuromodulation ^[30][44]. The proposed architecture includes a self configured independent component analysis (ICA), implemented purely in hardware to separate the source at a low power. Based on the predefined criterion, this system can be considered as a good example of the successfully implemented EBCI system in the literature because it takes into consideration the inter-subject variability, running at a low power, and the overall time is reduced by 34% compared to the time without pipelined structure. However, the suggested architecture missed the online validation to check its effectiveness and measure the real classification accuracy.

designed a low-cost FPGA-based SSVEP multimedia control system ^[34][48]. The proposed system includes a stimulation panel to evoke the subject’s SSVEP signal. Instead of the bulky personal computer with signal processing software, the SSVEP signal processing algorithms are implemented into a cyclone FPGA by hand coding VHSIC hardware description language (VHDL) to accelerate the runtime of the algorithms. Some existing research works are only focused on the implementation of one block of the signal processing chain.

Some studies were only focused on the implementation of very used BCI algorithms. For example, in Reference ^[35][49], Plaumbo et al. implemented the spatial filter algorithm, known by ICA technique which is distinguished by its huge time consuming. The ICA algorithm is implemented on a NI CompactRIO embedded system based on an industrial 400 MHz Freescale MPC5200 processor that deterministically executes LabVIEW Real-time applications on the reliable Wind River VxWorks real-time operating system. ^[36][50], proposed a chip design using a sampling rate conversion system for BCI machine.

3. Challenges and Future Research Directions for EBCI Systems

Our vision of BCI developments that might emerge in the next 10–20 years includes a fully customized, low power, and real-time embedded BCI system. The ECBI system will be mounted on the scalp allowing in the same time to acquire the EEG signal, derived from thousands of neurons, process, and to analyze the brain activities in order to send the decision to the practitioner or the EBCI’s user. Figure 4shows an overview of the future of the BCI systems.

The advancement in the development of the acquisition BCI, such as NAUTILUS PRO from G.tech company, OpenBCI, and Emotive headset, allows us today to acquire the EEG signals using dry electrodes and send them wireless to a base station for further processing. In Reference ^[18][22], Rifai et al. proposed a hybrid BCI which senses a combination classification of mental task, SSVEP, and eyes closed detection using two EEG channels. The vision of the future BCI systems becomes too easy today, mainly with the existence of the diversity open-source library that allow for easy implementation of the EEG signal processing algorithms. In this respect, as any other expanding domains, as well as with the existing convenient infrastructures, the focus of the BCI community will go in the coming two decades to the embedded implementation of the BCI systems.

The embedded implementation of the BCI systems surely will go faster with the advancement and the capabilities of the existing embedded platforms. A first proof of concept prototype is usually validated on a desktop computer, possibly using a programming language with features that facilitates early prototyping, such as MATLAB, and relies on the emulation of external interfaces ^[37][82]. This process is guided by developers knowledge about the impact of these modifications on the final embedded BCI version. In order to simplify this review, we have divided the embedded implementation of the BCI system according to three criteria which represent the keywords to select the appropriate architecture of the EBCI systems.

Some considerations about the limitations and challenges related to the BCI usage and applications are revealed even before moving to the embedded implementation of BCI technology ^[9][38][10,83]. Such limitations we found are:Inaccuracy of the BCI in terms of prediction or classifying brain signals. The artifacts and outliers that can limit its usability and the interpretability of the extracted features which can be noise-affected due to the low signal-to-noise ratio characterizing the EEG signals. Number of ethical issues due to reading people’s inner thoughts ^[38][83].The security of personal data not being guaranteed against attackers or intruders ^[38][83].In some cases, requirement for drastic surgery.

This alternative defines the software architecture as a high level abstraction with an embedded infrastructure within which BCI application is deployed and executed ^[39][84]. For example, in Reference ^[28][42], an embedded BCI system is proposed for driver drowsiness detection. Figure 5shows the general layers of the EBCI system based on a software architecture. The top-most layer is aptly called the application layer as it implements the device’s highest level logic and glues together the rest of the components and layers.

The EEG signal processing requires advanced algorithms to filter, extract, and classify trials, where often these algorithms are based on complex mathematical operations. Today, by dint of the spread of the open source library, it becomes easy to design and deploy EEG signal processing, feature extraction techniques, and intelligent machine-learned onto resource constrained platforms and small single board computers, like FPGA, Raspberry Pi, Arduino, etc. For instance, Shark is a fast modular library, and it has overwhelming support for supervised learning algorithms, such as linear regression, neural networks, clustering, k-means, etc. These are key mathematical functions or areas that are very important when performing the EEG signals processing.

For example, there are many embedded platforms occupied with GPU, such as Nvidia Jetson Xavier, Jetson Nano, Jetson TX2, NVIDIA Clara AGX, etc. The GPU occupied with tensor flow can be a good choice to implement EEG signal processing chain which is mainly based on matrices computation. In fact, the tensor cores reduce the used cycles needed for the calculation, multiplication, and addition operations, 16-fold in my example, for a 32 × 32 matrix, from 128 cycles to 8 cycles. Furthermore, tensor cores reduce the reliance on repetitive shared memory access, thus saving additional cycles for memory access.

Co-design, HW/SW architecture, is based on the system specification, architectural design, hardware, and software partition. In this case, the critical function is exported as a custom logic instruction approach and accelerator co-processor. The top most layer is aptly called the application layer as it implements the device’s highest level logic and glues together the rest of the components and layers. These cores implement the critical parts of the software layers, while profiting from the advantages of the hardware implementation, which executes many instructions in one clock signal instead of executing one instruction in one clock signal as the case of the processor.

This architecture allows us to get at the same time high throughput and low latency. In fact, it allows us to achieve throughput using low-batch size and processes each input as soon as it is ready, resulting in low latency in contrast to the pure software architecture which allows us to get high throughput OR low latency. The pure SW architecture achieves throughput using low-batch size and processes each input as soon as it is ready, resulting in low latency. It is designed with a high efficiency and ease of use in mind to unleash the full potential of AI acceleration on Xilinx FPGAs and on adaptive compute acceleration platforms

Building an EBCI based on hardware architecture is a challenging task. Here, the key design challenge is to build an extremely powerful system (in terms of the features provided by the system so as to make it easy to use for EBCI users) at a very low power consumption, a low EBCI cost and at the same time meeting all the timing constraints ^[40][86]. This alternative consists of implementing the complete EEG signal processing chain within an FPGA using the Hardware description language (HDL) which is a specialized computer language used to program electronic and digital logic circuits. This approach can be used when the timing constraints are very stringent, and it is impossible to respect it with the pure software architecture.

In many ways, the choice of an application framework for use in an embedded platform is the most important design decision, at least in the case of the embedded systems with human users, such as mobile handsets. In fact, most contemporary users seem to exhibit strong opinions about which application environment their chosen handset supports.

In addition to guidelines, our survey also enabled us to identify a common classification criterion that must be used to compare the EBCI systems. The suggested evaluation criterion takes into consideration the suggested criteria defined in the introduction. So far, the majority of the BCIs has been evaluated based on the accuracy criteria, which computes the percentage of the trial classified correctly ^[22][31]. TheEcis suggested to evaluate the EBCI systems in the future by taking into consideration the predefined criteria.

4. Conclusions

This paper is an attempt to summarize the last two decades of embedded Brain-Computer Interface mostly because of the electroencephalography influence on these systems. Their main purpose was to assist practitioner to analyze some pathological disorders and to enable direct non-muscular communication for people with severe disabilities. This resulted in issuing various inexpensive, consumer-grade headsets. In this paper, we have delimited our review in the application related to the functional substitution and pathological disorder analysis only.

Six important criteria are defined to evaluate the existing EBCI systems and are represented as a unified ruler applied during our evaluation. The adaptability criterion allows for identification of the EBCI systems that have taken into consideration the inter-subject variability. The runtime and the power consumption reflect the performance of the used embedded platform. The existing EBCI systems are evaluated according to these criteria and allow us to define a few indexes to estimate the EBCI technology’s performance.

In the future, we plan to perform a quantitative analysis of the optimization approaches for EBCI system using machine learning and deep learning algorithms.

The following abbreviations are used in this manuscript: