A non-orthogonal multiple access (NOMA) system has been classified as promoting a multiple access structure for future wireless systems to boost system throughput and spectral efficacy. NOMA could utilize the present resources more effectively by resourcefully benefiting from the users’ channel environments and providing numerous users with distinct quality of service (QoS) demands. NOMA enables several users to achieve simultaneous arrival to the same time-frequency block by superpositioning them in the code or power domains [1]. The concept of NOMA is established on that the user with a weak channel condition can be combined with the user with a good channel condition in the same time slot and on the same allocated subcarrier to ensure that the bandwidth block could be effectively exploited [2].

In the NOMA scheme, receiver equipment will receive the multiplexing of symbols from different users in the system; therefore, elimination of interference from other users is necessary for coordinated decoding.

In [6], the authors introduced DL-based detector for the multiuser downlink OFDM-NOMA system. The authors mainly depended on pilot signals for the channel information, and according to these pilot responses, a DL based joint channel estimation and symbol detection was achieved without additional processing for channel estimation. The simulation’s outcomes revealed that the proposed DL scheme outperforms the conventional SIC-based detector. On the other hand, the proposed scheme needed to be initially trained offline for different channel conditions and the simulation results were presented in terms of BER only.

In [7], the authors suggested a deep learning framework to perform signal recovery in the MIMO-NOMA system when the Rayleigh fading channel is considered. The proposed technique can simultaneously carry out the channel estimation process and signal detection. Simulations were conducted for the proposed DL scheme, and the results were compared with the conventional SIC procedure in terms of the symbol error rate (SER) and throughput. According to the simulation results, the proposed DL scheme can address channel impairment, but the examined NOMA cell was limited for two users and an offline training stage was also required. Also, the DNN training phase needed two components, the received signal, and the labels, which were used as supervised data to help the DNN to optimize the parameters.

In [8], the authors proposed a data-driven deep learning estimator for time- and frequency-selective channels. The proposed algorithm was designed such that a pre-training scheme and pilot symbols were utilized as inputs for the DNN to attain a desired initialization, which can further enhance the performance of the DL estimator. The DNN was trained offline in both the pre-training and training stages, while in the testing stage, the channels could be dynamically tracked by the DNN with only pilots identified, and then the transmitted symbols were detected. The performance of a DL estimator with different numbers of layers was investigated and the numerical results demonstrated that the proposed DL estimator outperformed the standard channel estimator in terms of efficiency and robustness.

In [9], a deep learning approach was employed to estimate the downlink channel and to reduce the training overhead in a fog radio access network. The Gated Recurrent Unit (GRU) was utilized to learn the hidden correlations among the channel matrices from different users, and a bidirectional GRU was also employed to further improve the estimation performance. Simulation results were provided to demonstrate the performance gains, but the examined performance metrics were limited to the loss function and mean square error.

Based on the deep learning (DL) algorithm, the authors in [10] introduced a sliding window Gated Recurrent Unit (GRU) channel estimator to acquire knowledge for the time-varying Rayleigh fading channel. Interleaver and channel coding schemes were merged with the proposed sliding window estimator to further enhance system performance. The simulation results proved the ability of the suggested procedure to follow the channel in a reliable way and achieve better mean square error (MSE). Moreover, the sliding window-based GRU estimator was examined with different numbers of pilot symbols, and the robustness against the variations in the channel characteristics was analyzed.

In [11], the authors went to conclude that DL algorithm can be utilized in signal detection for uplink analysis in NOMA network. The authors proposed a DL approach to characterize the complex channel parameters, where restricted Boltzmann machines (RBM) were implemented as a pre-training phase for the original input sequence for the network. The proposed learning scenario based on LSTM layer could track the environment statistics automatically via offline learning and an iterative support detection procedure was suggested to identify the transmitted symbols. Performance analysis for the proposed DL scheme was evaluated merely in terms of the sum data rate and block error rate.

In [12], a pilot-aided receiver structure was presented for an uplink single input, multi output (SIMO) NOMA system, which incorporated a combined channel estimation and signal detection scheme for random channels. The authors brought together a deep learning model with SIC detection structure to minimize the learnable parameters. Furthermore, signal detection accuracy improvement and noise interference reduction were achieved by adding noise and interference elimination factors at the SIC detection stage. The simulation results indicated that BER performance based on the proposed DL scheme was more acceptable than the traditional MMSE procedure and the complexity of the receiver was diminished.

In [13], the authors proposed a semi-blind mutual detection scheme-based DL to distinguish users’ symbols in the co-operative NOMA system. The proposed method was capable of detecting the signal without the need for further channel estimation process since it could achieve a simultaneous detection on the basis of pilot responses. The DL model was trained offline over a Rayleigh fading channel and the trained network was deployed in the online detection phase. In addition, the trained model was inspected using Rician and Na-agami-m fading channels and simulation outcomes proved the capability of the proposed scheme in outperforming conventional detectors.

In [14], the authors examined deep neural network (DNN) for combined channel estimation and signal detection in an OFDM system. This approach considered OFDM system and fading channel as a black box and the presented DNN network is trained offline using simulated data. The simulation results revealed that the proposed DL approach had the capability to learn and investigate the complicated attributes of the wireless channels. In addition, the results of the DL approach proved its dominance over conventional methods when fewer pilot symbols were utilized, and cyclic prefix was ignored.