1. Introduction

From the viewpoint of nonlinear dynamics, brain is a very complex dynamical system, containing around 86 billion neurons ^[1]. The neurons are connected by synapses, thus forming a complex network with nodes and links represented respectively by neurons and synapses. Specific features of time-spatial activity of the neural network in the brain cortex and cooperative dynamics of different brain areas (e.g., event-related synchronization/desynchronization) on a sensor level provide important information about the current state of the nervous system and cognitive brain ability ^[2]. Particular brain states are associated with motor brain activity during either real or imaginary movement.

Revealing specific features of spatial brain cortex activity related to real motions and motor imagery of different limbs can be essential not only for basic research in neuroscience, but also for applications in medicine to improve the quality of life of post-traumatic and post-stroke patients using brain-computer interfaces (BCI) for rehabilitation ^[3][4][5] or to control prostheses and exoskeletons ^[6]. One of the important BCI functions is online detection of specific features of electromagnetic brain activity using electroencephalography (EEG) or magnetoencephalography (MEG), and transformation of certain patterns into control commands to perform specific actions in the environment without the need of “classical” methods of human–machine interaction.

Apart from EEG and MEG, other methods are also used to acquire information about brain states. In particular, functional near-infrared spectroscopy (fNIRS) ^[7][8] is a powerful tool of noninvasive optical imaging successfully used in BCI for registration of brain activity and control command formation ^[9][10]. Control commands for this kind of BCI should not be affected by any muscular activity ^[11]. Therefore, a study of brain states related to motor imagery is very important for designing such BCI ^[12][13]. Motor imagery is a mental process by which a person rehearses or simulates a given action with no real motor activity. Some researchers treat motor imagery as a conscious application of unconscious preparation for real motor activity ^[14]. A number of studies have highlighted common features for real and imaginary motor activity ^[15][16][17]. One of the common features, important for the BCI development, is that the cortical layout in the primary motor cortex M1 is quite similar between motor execution and motor imagery.

M2. Paterials and Methodrticipants

Participants

Twelve healthy volunteers (age: 22–38 years, gender: 7 men and 5 women), right-handed, amateur practitioners of physical exercises, non-smokers participated in the experiment. None of the subjects had diagnosed diseases of the musculoskeletal system and neurological diseases and did not take medications. Every participant was asked to maintain a healthy lifestyle with 8-hours night rest for 48 h before starting the experiment.

Each participant provided informed written consent before participating in the experiment. The experimental procedure was performed in accordance with the Helsinki’s Declaration and approved by the local Ethics Committee of the Innopolis University. 

Experimental Equipment

3. Experimental Equipment

The experiment was designed to record a hemodynamic neuronal response in the motor cortex using fNIRS which records fast changes in the brain activity. The fNIRS signals were acquired by the NIRScout device manufactured by the NIRx Company (Germany). The NIRScout system has a 7.8125-Hz resolution and contains 8 sources and 8 detectors placed on the subject’s scalp in the primary motor cortex area (M1) as shown in Figure 1a. Each pair “source–detector” was placed close enough to each other (about 3 cm) to form a fNIRS channel. In our experiments, we used 20 fNIRS channels as illustrated in Figure 1b.

Experimental Procedure

4. Experimental Procedure

The experiment was performed as follows. The subjects were sitting on a comfortable chair while performing motor actions or motor imaginary of left and right hands according to the corresponding text command on a computer monitor placed in front of the subject’s eyes at a distance of 70–80 cm.

The experimental design is schematically shown in Figure 2. Each experiment began and ended with a 3-min recording of background brain activity, during which the subject were instructed to relax and make no hand movements. The experiment included two sessions (Figure 2a). In the first session, the subject was asked to perform real movements with left or right hand according to the corresponding command on the screen. Then, after a short break, in the second session the subject was asked to imagine the same type of movement according to the corresponding command on the screen. Each fNIRS trial during every session consisted of the text command presentation indicating the type of motor activity (the subject was given 15 s to perform required movement) and the rest interval (15 s from the end of motor activity till the next command). There were 10 trials for each type of motor activity. Hand movement consisted of repeated bending/unbending of fingers to the center of the palm (similar to clenching of imaginary ball), as illustrated in (Figure 2b). The repeated movements were performed at the pace comfortable for the subject. 

Data Acquisition and Pre-Processing

5. Data Acquisition and Pre-Processing

The fNIRS data acquisition and pre-processing procedure were performed with software NIRScout. It is well-known that experimental fNIRS data are often affected by side physiological noises and artifacts, whose characteristic frequencies are in the fNIRS frequency band, including Mayer wave (with a typical frequency close to 0.1 Hz), respiration (close to 0.25 Hz), and heartbeat (close to 1 Hz). As was mentioned in the review paper ^[18], in many cases the band-pass filtering is mostly sufficient for removing low-frequency physiological noise in fNIRS data. According to this observation, we also applied the 0.01–0.1 Hz band-pass filter to the fNIRS signals using NIRScout to prevent the effect of side physiological activities.

Algorithm for Classification of Brain Activity During Real and Imagery Motor Executions

6. Algorithm for Classification of Brain Activity During Real and Imagery Motor Executions

Let us now consider the online algorithm based on the construction of a decision tree for binary classification of brain activity during real and imagery motor executions. The tree-like model of decisions was created as a result of empirical analysis of the experimental data. The proposed algorithm for processing fNIRS data is illustrated in Figure 3 in the form of a flowchat which contains the following main steps.

Figure 3. Flowchart of online classification algorithm for fNIRS signals corresponding to left/right-hand movement. The main advantage of the algorithm is that it can be used for both real and imaginary movements.

For each considered channel i and type of motor activity (right/left hand, execution/imagery) we subtract spatial oxyhemoglobin (HbO) () and deoxyhemoglobin (HbR) () distributions for the right hemisphere (, fNIRS channels j of interest from right hemisphere) from the corresponding distribution for the left hemisphere (, symmetrical channels i from left hemisphere). We calculate differences for individual symmetrical channels in the left and right hemispheres as 

1. For each considered channel i and type of motor activity (right/left hand, execution/imagery) we subtract spatial oxyhemoglobin (HbO) () and deoxyhemoglobin (HbR) () distributions for the right hemisphere (, fNIRS channels j of interest from right hemisphere) from the corresponding distribution for the left hemisphere (, symmetrical channels i from left hemisphere). We calculate differences for individual symmetrical channels in the left and right hemispheres as 
   
   
   
   
2. Then, we average  and  over the time interval corresponding to motor activity  s to find  and  as
   
   
   
   
3. For each separate fNIRS signal trial, we calculate characteristics C_R and C_L taking into account the following criteria for each considered symmetric fNIRS channels in the left and right hemispheres.
   
   
   
   (i)  If  and  is true for one of the channels i, then C_R:= C_R + 1 (value C_R takes discrete values, minimal value is C_R =0 and peak value is equal to the number of considered fNIRS channels of interest in one of the hemispheres).
   
   (ii) If  and  is true for one of the channels i, then C_L:= C_L + 1 (value C_R takes discrete values, minimal value is C_L =0 and peak value is the same as peak value of C_R)
   
   
   
   
4. Finally, we make a decision according to the following criteria.
   
   
   
   (i)   If C_R > C_L, then right-hand (real or imaginary) motor activity takes place.
   
   (ii)  If C_R < C_L, then left-hand (real or imaginary) activity takes place.
   
   (iii) If C_R = C_L, then the type of activity is uncertain. 

Then, we average  and  over the time interval corresponding to motor activity  s to find  and  as

For each separate fNIRS signal trial, we calculate characteristics C_R and C_L taking into account the following criteria for each considered symmetric fNIRS channels in the left and right hemispheres.

(i)  If  and  is true for one of the channels i, then C_R:= C_R + 1 (value C_R takes discrete values, minimal value is C_R =0 and peak value is equal to the number of considered fNIRS channels of interest in one of the hemispheres).

(ii) If  and  is true for one of the channels i, then C_L:= C_L + 1 (value C_R takes discrete values, minimal value is C_L =0 and peak value is the same as peak value of C_R)

Finally, we make a decision according to the following criteria.

(i)   If C_R > C_L, then right-hand (real or imaginary) motor activity takes place.

(ii)  If C_R < C_L, then left-hand (real or imaginary) activity takes place.

(iii) If C_R = C_L, then the type of activity is uncertain. 

7. Results

We implemented the classification algorithm described above in our fNIRS-based experimental system for online classification of real and imaginary motor actions. We used the proposed classifier with six fNIRS channels i = {2, 7, 8} in the left hemisphere and j = {12, 17, 18} symmetric channels in the right hemisphere. Notably, in the majority of cases the type of motor action (both real and imagery) in all subjects was correctly identified by the data from only three fNIRS channels. Table 1 shows the results of automatic classification of left/right hand real and imaginary movements, as well as statistical analysis of true positive fraction, true negative fraction, and false positive fraction.

Table 1.

The results of automatic classification of different types of motor action (mean ± S.D.) using

i

= {2, 7, 8} and

j = {12, 17, 18} fNIRS channels in the left and right hemisphere, respectively.

= {12, 17, 18} fNIRS channels in the left and right hemisphere, respectively.

The proposed fNIRS-based method provides close to 100% recognition accuracy in the detection of real movements, while the classification accuracy of motor imagery is a little smaller and reach 90%.

The important advantage of the proposed method is the possibility to efficiently classify different types of movement, both real and imaginary, without recalculation of the system parameters. This essential feature of the developed sensor results from pronounced laterality of the hemodynamic brain response to motor activity.

The knowledge of the hemodynamic behavior in the motor cortex during real and imaginary motor activity along with approaches for its detection can be helpful not only for fundamental studies on human motor-related tasks but also for the development of fNIRS-based BCIs.