Hot Most Recent
|Ver.||Summary||Created by||Modification||Content Size||Created at||Operation|
|1||+ 2510 word(s)||2510||2020-10-06 07:45:52||||
|2||-62 word(s)||2448||2020-10-21 11:57:33||| ||
|3||Meta information modification||2448||2020-10-22 08:28:56||| ||
|4||Meta information modification||2448||2020-10-26 09:47:57||||
The popularity of deep reinforcement learning (DRL) applications in economics has increased exponentially. DRL, through a wide range of capabilities from reinforcement learning (RL) to deep learning (DL), offers vast opportunities for handling sophisticated dynamic economics systems. DRL is characterized by scalability with the potential to be applied to high-dimensional problems in conjunction with noisy and nonlinear patterns of economic data. In this paper, we initially consider a brief review of DL, RL, and deep RL methods in diverse applications in economics, providing an in-depth insight into the state-of-the-art. Furthermore, the architecture of DRL applied to economic applications is investigated in order to highlight the complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. The survey results indicate that DRL can provide better performance and higher efficiency as compared to the traditional algorithms while facing real economic problems in the presence of risk parameters and the ever-increasing uncertainties. View Full-Text
Deep learning (DL) techniques are based on the use of multi-neurons that rely on the multi-layer architectures to accomplish a learning task. In DL, the neurons are linked to the input data in conjunction with a loss function for the purpose of updating their weights and maximizing the fitting to the inbound data . In the structure of a multi-layer, every node takes the outputs of all the prior layers in order to represent outputs set by diminishing the approximation of the primary input data, while multi-neurons learn various weights for the same data at the same time. There is a great demand for the appropriate mechanisms to improve productivity and product quality in the current market development. DL enables predicting and investigating complicated market trends compared to the traditional algorithms in ML. DL presents great potential to provide powerful tools to learn from stochastic data arising from multiple sources that can efficiently extract complicated relationships and features from the given data. DL is reported as an efficient predictive tool to analyze the market . Additionally, compared to the traditional algorithms, DL is able to prevent the over-fitting problem, to provide more efficient sample fitting associated with complicated interactions, and to outstretch input data to cover all the essential features of the relevant problem .
Reinforcement learning (RL)  is a powerful mathematical framework for experience-driven autonomous learning . In RL, the agents interact directly with the environment by taking actions to enhance its efficiency by trial-and-error to optimize the cumulative reward without requiring labeled data. Policy search and value function approximation are critical tools of autonomous learning. The search policy of RL is to detect an optimal (stochastic) policy applying gradient-based or gradient-free approaches dealing with both continuous and discrete state-action settings . The value function strategy is to estimate the expected return in order to find the optimal policy dealing with all possible actions based on the given state. While considering an economic problem, despite traditional approaches , reinforcement learning methods prevent suboptimal performance, namely, by imposing significant market constraints that lead to finding an optimal strategy in terms of market analysis and forecast . Despite RL successes in recent years , these results suffer the lack of scalability and cannot manage high dimensional problems. The DRL technique, by combining both RL and DL methods, where DL is equipped with the vigorous function approximation, representation learning properties of deep neural networks (DNN), and handling complex and nonlinear patterns of economic data, can efficiently overcome these problems . Ultimately, the purpose of this paper is to comprehensively provide an overview of the state-of-the-art in the application of both DL and DRL approaches in economics. However, in this paper, we focus on the state-of-the-art papers that employ DL, RL, and DRL methods in economics issues. The main contributions of this paper can be summarized as follows:
Classification of the existing DL, RL, and DRL approaches in economics.
Providing extensive insights into the accuracy and applicability of DL-, RL-, and DRL-based economic models.
Discussing the core technologies and architecture of DRL in economic technologies.
Proposing a general architecture of DRL in economics.
Presenting open issues and challenges in current deep reinforcement learning models in economics.
This section discusses an overview of various interesting uses of both DL and deep RL approaches in economics.
The recent attractive application of deep learning in a variety of economics domains is discussed in this section.
From an economic point of view, the stock market value and its development are essential to business growth. In the current economic situation, there are many investors around the world that are interested in the stock market in order to receive quick and better return compared to other sectors. The presence of uncertainty and risk in the forecasting of stock pricing bring challenges to the researcher to design a market model for prediction. Despite all advances to develop mathematical models for forecasting, they are still not that successful . The deep learning topic attracts scientists and practitioners as it is useful for high revenue while enhancing the prediction accuracy with DL methods. Table 1 presents recent research.
|||Two-Streamed gated recurrent unit network||Deep learning framework for stock value prediction|
|||Filtering methods||Novel filtering approach|
|||Pattern techniques||Pattern matching algorithm for forecasting the stock value|
|||Multilayer deep Approach||Advanced DL framework for the stock value price|
Another application of DL methods is the insurance sector. One of the challenges of insurance companies is to efficiently manage fraud detection (see Table 2). In recent years, ML techniques have been widely used to develop practical algorithms in this field due to the high market demand for new approaches compared with traditional methods to practically measure all types of risks (Brockett et al. 2002; Pathak et al. 2005, Derrig, 2002). For instance, there are many demands for car insurance that forces companies to find novel strategies in order to meliorate and upgrade their system. Table 2 summarizes the most notable studies for the application of DL techniques in insurance.
|||Cycling algorithms||Fraud detection in car insurance|
|||LDA-based appraoch||Insurance fraud|
|||Autoencoder technique||Evaluation of risk in car insurance|
Auction design has a major importance in practice that allows the organizations to present better services to their customers. A great challenge to learn a trustable auction is that its bidders require optimal strategy for maximizing profit. In this direction, Myerson designed an optimal auction with only a single item . There are many works with results for single bidders but most often with partial optimality . Table 3 presents the notable studies developed by DL techniques in Auction Mechanisms.
|||Augmented Lagrangian Technique||Optimal auction design|
|||Extended RegretNet method||Maximized return in auction|
|||Data-Driven Method||Mechanism design in auction|
|||Multi-layer neural Network method||Auction in mobile networks|
In current technology improvement, fraud detection is a challenging application of deep learning, namely, in online shopping and credit cards. There is a high market demand to construct an efficient system for fraud detection in order to keep the involved system safe (see Table 4).
|||AE||Fraud detection in unbalanced datasets|
|||Network topology||credit card transactions|
|||Natural language Processing||Anti-money laundering detection|
|||AE and RBM architecture||Fraud detection in credit cards|
Macroeconomic prediction approaches have gained much interest in recent years, which are helpful for investigating economics growth and business changes . There are many proposed methods that can forecast macroeconomic indicators, but these approaches require huge amounts of data and suffer from model dependency. Table 5 shows the recent results which are more acceptable than the previous ones.
|||Backpropagation Approach||Forecasting inflation|
|||Feed-Forward neural Network||Asset allocation|
In financial markets, it is crucial to efficiently handle the risk arising from credits. Due to recent advance in big data technology, DL models can design a reliable financial model in order to forecast credit risk in banking systems (see Table 6).
|||Binary Classification Technique||Loan pricing|
|||Feature selection||Credit risk analysis|
|||Likelihood Esrtimation||Mortgage risk|
Financial problems generally need to be analyzed in terms of datasets from multiple sources. Thus, it is substantial to construct a reliable model for handling unusual interactions and features from the data for efficient forecasting. Table 7 comprises the recent results of using deep learning approaches in financial investment.
|||LSTM and AE||Market investment|
|||Hyper-parameter||Option pricing in finance|
|||LSTM and SVR||Quantitative strategy in investment|
|||R-NN and genetic method||Smart financial investment|
|||Augmented reality and image classification||Improving shopping in retail markets|
|||DNN methods||Sale prediction|
|||CNN||Investigation in retail stores|
|||Adaptable CNN||Validation in the food industry|
|||MLP||BI with client data|
|||MLS and SAE||Feature selection in market data|
|||RNN||Information detection in business data|
|||RNN||Predicting procedure in business|
|||DDPG||Dynamic stock market|
|||Adaptive DDPG||Stock portfolio strategy|
|||DQN methods||Efficient market strategy|
|||Model-less CNN||Financial portfolio algorithm|
|||Model-free||Advanced strategy in portfolio trading|
|||Model-based||Dynamic portfolio optimization|
|||Actor–critic method||Recommendation architecture|
|||SS-RTB method||Bidding optimization in advertising|
|||DDPG and DQN||Pricing algorithm for online market|
|||DQN scheme||Online news recommendation|
In the current fast economics and market growth, there is a high demand for the appropriate mechanisms in order to considerably enhance the productivity and quality of the product. Thus, DL can contribute to effectively forecast and detect complex market trends, as compared to the traditional ML algorithms, with the major advantage of a high-level feature extraction property and proficiency of the problem solver methods. Furthermore, reinforcement learning enables us to construct more efficient frameworks regarding the integration of the prediction problem with the portfolio structure task, considering crucial market constraints and better performance, while using deep reinforcement learning architecture and the combination of both DL and RL approaches, for RL to resolve the problem of scalability and to be applied to the high-dimensional problems as desired in real-world market settings. Several DL and deep RL approaches, such as DNN, Autoencoder, RBM, LSTM-SVR, CNN, RNN, DDPG, DQN, and a few others, were reviewed in the various application of economic and market domains, where the advanced models improved prediction to extract better information and to find the optimal strategy mostly in complicated and dynamic market conditions. This brief work represents the basic issue that all proposed approaches are mainly to fairly deal with the model complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. Practitioners can employ a variety of both DL and deep RL techniques, with the relevant strengths and weaknesses, that serve in economic problems to enable the machine to detect the optimal strategy associated with the market. Recent works showed that the novel techniques in DNN, and recent interaction with reinforcement learning, so-called deep RL, have the potential to considerably enhance the model performance and accuracy while handling real-world economic problems. We mention that our work indicates the recent approaches in both DL and deep RL perform better than the classical ML approaches. Significant progress can be obtained by designing more efficient novel algorithms using deep neural architectures in conjunction with reinforcement learning concepts to detect the optimal strategy, namely, optimize the profit and minimize the loss while considering the risk parameters in a highly competitive market. For future research, a survey of machine learning and deep learning in 5G, cloud computing, cellular network, and COVID-19 outbreak is suggested.