LSTM-Based Stock Prediction and Selection

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Abdellilah nafia	--	2449	2023-12-04 09:24:51

This entry is adapted from the peer-reviewed paper 10.3390/ijfs11020057

A great deal of attention has been devoted to the use of neural networks in portfolio management, particularly in the prediction of stock prices. First, data from historical quotes and technical and fundamental indicators are used in the long short-term memory (LSTM) network to provide good predictions. Second, the EMN strategy allows for the funding of long-position stocks by short-sell-position stocks, thus hedging the market risk.

stock prediction stock selection long short-term memory (LSTM)

1. Introduction

According to the market efficiency hypothesis developed by Fama (1970), it is impossible to make accurate predictions about stock prices in the future, because the current prices of financial assets reflect all the information that is available, and thus there is no such thing as an undervalued or overvalued stock. Nonetheless, many empirical studies have debunked this hypothesis, and have shown that, with some methods and techniques, it is possible to make good predictions about future stock prices. Most of these techniques use historical stock prices and/or financial information from the issuing companies, as part of two well-known types of stock analysis: fundamental analysis and technical analysis (Carhart 1997).

Since the foundational work of Markowitz (1952), which established the mathematical foundations of portfolio construction, many statistical and econometric models have been developed in order to predict the future prices and returns of financial assets, such as the Capital Asset Pricing Model (CAPM) in 1961, the Three Factor Model by Fama and French (1993), the Four Factor Model by Carhart (1997), the Autoregressive Model (AR) by Yule (1926), the Moving Average process (MA) by Wold in 1938 (Neyman 1939), the ARMA model by Box and Jenkins (1970), the ARIMA in 1976, the ARCH by Engle (1982), and the GARCH by Bollerslev (1986). All of these statistical models are based on assumptions relating to data, such as normality and stationarity.

With the technological and algorithmic evolution, rapid advances in artificial intelligence technologies, the development of processors with high computing capacity, large size disks, and digital platforms with high connectivity and automatic trading systems, portfolio managers are increasingly turning to automatic learning techniques, or machine learning (ML), in their investment decision. These techniques allow managers to benefit from opportunities by predicting future stock prices and increasing prediction accuracy. As a matter of fact, the techniques capture complex patterns in the data and provide quick executions, allowing for large-size data processing. Additionally, these technical solutions have revolutionized automatic trading and greatly reduced the impact of behavioral biases.

Many studies have used machine learning techniques to predict future stock prices/returns. Among these techniques are Logistic Regression (LR), the Support Vector Machine (SVM), Random Forest (RF), and Adaboost. Other studies have used neural networks to predict future stock prices and returns. Neural networks perform better than classic ML methods due to their ability to learn complex non-linear functions with significant accuracy, and to process a wide range of data (Nafia et al. 2022).

Since the appearance of the first Perceptron, made by Frank Rosenblatt, in 1957, the structure of neural networks has continued to evolve (Rosenblatt 1957). The Multilayer Perceptron (MLP) is an improved version of the Perceptron. It is considered the most simplified version of a deep neural network (DNN), and is composed of an input layer, several hidden layers, and an output layer. Other powerful and sophisticated DNN models have been developed over recent years, including Convolutional Neural Networks (CNN) that process visual data such as images and videos. Recurrent Neural Networks (RNN) have attracted a great deal of attention, especially in handling modeling problems related to time series, such as stock prices and returns, and sequential data in general, such as speech recognition, language modeling, and translation. Long Short-Term Memory (LSTM) neural networks have been devised to resolve the problem of the vanishing gradient, which is the problem encountered by RNNs when dealing with long-term data sequences. Gated Recurrent Unit (GRU) neural networks were introduced recently; they have a similar design to LTSMs, but with fewer parameters.

Designing a strategy to predict stock returns is nonetheless a demanding task that involves many challenges, since financial markets are complex, unstable, and evolving environments. Moreover, the portfolio manager not only needs to identify the model that best predicts stock returns, but also faces the challenge of building a high-performance portfolio by implementing a promising strategy.

Even if the stocks in a “Buy and Hold” portfolio are selected meticulously, the portfolio may still be exposed to high levels of systematic risk and be affected by market risk. Therefore, investment strategy is paramount when building a portfolio. The “Equity-Market-Neutral” strategy, designed and applied for the first time by Edward Thorp between 1979 and 1980, is widely used by hedge funds. It allows users to build a market-neutral portfolio with market exposure close to zero, while generating high returns. This strategy is an alternative to classic investment strategies that generate returns independent of market fluctuations. It essentially relies on the portfolio manager’s ability to pick stocks. In fact, when the market rises, the short positions’ losses are partially offset by the long positions’ gains, and when the market falls, the short positions provide a hedge against the long positions’ losses (Jacobs and Levy 2005; Ganchev 2022).

Thanks to their ability to accurately predict time series, LSTM neural networks are often used for stock price and return prediction. However, the utilization of LSTMs in constructing a profitable portfolio using the EMN strategy is a topic that has seldom been explored in the scientific literature.

2. LSTM-Based Stock Prediction and Selection

Since machine learning (ML) was introduced to the finance world 40 years ago, neural networks in different form have risen to prominence in many research areas and fields of application, including portfolio management, scam detection, the evaluation of financial assets and derivatives, trading algorithms, studies of blockchains and cryptocurrency, feelings analysis and behavioral finance, and text-mining in finance. With the arrival of recurrent neuronal networks (RNN) and their improved version, long short-term memory (LSTM), which process time series and other sequential data, many studies have attempted to apply these techniques to portfolio management, particularly stock price predictions.

In the literature review of research related to the implementation of deep learning (DL) in finance during the last five years, Ozbayoglu et al. (2020) identified LSTM as the predominant process in the research in terms of the number of uses. The reason for this is that the LSTM structure is more able to adapt to financial time series. Jiang (2021) also conducted a literature review on the application of DL in stock prediction by studying more than 120 research papers from 2017, 2018, and 2019. He found that RNN models, including LSTM, are more commonly used than other models. Not only did he reveal that LSTM is popular in financial stock predictions, but he also demonstrated its predicting power. In their literature review of Forex and the prediction of stock prices, Hu et al. (2021) used data from the DBLP database and Microsoft Academic between 2015 and 2021, and found that all 27 papers that used LSTM agreed that LSTM neural networks outperform other models, or that it is at least capable of obtaining good prediction results.

Many research articles have used LSTM neural networks in applications related to stocks. Most of these studies apply LSTM to the prediction of stock prices with different study characteristics, such as the learning data period, the prediction horizon, the number of variables in the study, the frequency of the historical data used (intraday, daily, weekly, or monthly), the nature of the used variables (OCHLV prices (Open, Close, High, Low, and Volume), technical, fundamental, feeling analysis or macroeconomics), different hyperparameters, and different LSTM networks settings (Naik and Mohan 2019; Qiu et al. 2020; Ding and Qin 2020; Ghosh et al. 2019). Other research uses LSTM networks in the prediction of index prices, since they are less volatile than stocks and constitute a set of structurally linked stocks (in terms of sector, industry, size, etc.) (Michańków et al. 2022; Tfaily and Fouad 2022). The application of LSTM networks is not limited to the prediction of financial asset prices, but it is also used in the prediction of the direction of price trends. In fact, several studies have used LSTM to predict the rise or the fall of stock prices by transforming the regression problem to a classification problem with other metrics for performance measurement (Patel et al. 2015; Yao et al. 2018).

However, less research has focused on portfolio construction and asset allocation methods that use LSTM neural networks. Indeed, the real challenge for portfolio managers is to figure out the best investment strategy for building a profitable portfolio with less risk. The stocks need to be selected in such a way as to ensure optimal capital allocation. Managers not only strive for accuracy in their stock price predictions in order to build their portfolios, but also need to account for other considerations, such as the number of stocks their portfolios must contain to diversify away idiosyncratic risk, and questions such as how to hedge against systemic market risk, how to allocate capital among the stocks on the portfolio to maximize its profitability, and how to fund the acquisition of long positions.

Chaweewanchon and Chaysiri (2022) proposed a hybrid model, R-CNN-BiLSTM (BiLSTM is an improved version of LSTM), to build a mean-variance (MV) optimal portfolio containing stocks that obtained the best predicted returns. CNN networks are used to extract the data’s important characteristics and the BiLSTM networks are used to predict prices. The model these authors propose is compared to other reference models that use mean-variance optimization or equal weights to allocate capital on one side, and either LSTM or BiLSTM to select stocks on the other side. The authors used the following metrics to evaluate the portfolio performance: the mean return, the standard deviation, and the Sharpe ratio. Their experiments on the SET50 index of Thailand’s stock exchange between 2015 and 2020 demonstrate that BiLSTM outperforms other techniques. They also demonstrated that models that use “robust” inputs (i.e., those undergoing raw prices transformations) outperform those that directly use closing prices.

Sen et al. (2021) built portfolios containing five stocks each from the seven sectors that are part of the National Stock Exchange (NSE) in India. To achieve this, they used the OCHLV historical prices of the chosen stocks for the previous five years (from 2016 to 2020) to train the LSTM neural networks, and implemented a test period from 1 January to 1 June 2021. Two portfolios were built for each sector: a minimum risk portfolio and an optimal risk portfolio, according to Markowitz minimum variance optimization. LSTM networks were used to predict stock prices that were then used to calculate portfolios returns. The results demonstrated that LSTM performed well when the actual returns were compared to the predicted returns.

Touzani and Douzi (2021) proposed a trading strategy for some stocks in the Moroccan stock exchange using LSTM and GRU in the short term and the long term. To overcome the liquidity problem, the small number of listed stocks (76 stocks), and the low volume negotiated in the Moroccan market, the authors trained the model on data from the S&P500 index and the CAC40 index in the French stock exchange. Validation and other processes were performed on data from the Moroccan stock exchange. The trading strategy involved buying or selling a stock depending on how a function of the predicted price and the actual price compare to a certain calibrated threshold. Finally, two stocks were chosen to construct a portfolio and assess its performance during a test period from March 2019 to March 2021. Their results showed that the portfolio they built generated an annual return of 27.13%, and thus outperformed all the utilized benchmarks, except for the “Software and IT services” index, which achieved a high return during the COVID-19 period.

Liu et al. (2017) presented a trading strategy based on a hybrid model combining CNN and LSTM. CNN was used to select stocks and LSTM was used to manage the timing of opening or closing a position as part of a long-short strategy. To achieve this, the authors used OCHLV prices and returns data related to stocks in the Chinese Exchange. The training period ran from 2007 to 2013, and the test period from January 2014 to March 2017. They found that their strategy was more profitable than the benchmark and a simple momentum-based strategy (which stipulates that the stocks that performed best in the last three to twelve months will continue to perform well for the next few months, and that the reverse is also true).

Hou et al. (2020) proposed a hybrid LSTM-DNN model by integrating 18 monthly returns in LSTM and 19 fundamental variables in DNN to build a portfolio with a long–short strategy. The authors tested the model on 1398 stocks listed on NYSE, AMEX, and NASDAQ from 1977 to 2018. The portfolio was rebalanced in each period by buying the stocks with the highest predicted returns in the top decile and selling those in the lowest decile of the predicted returns ranking. To assess the model’s performance, two metrics were used: the average monthly return and the Sharpe ratio. The results demonstrated that this model outperformed other OLS and DNN reference models.

Cipiloglu Yildiz and Yildiz (2022) used LSTM to predict the prices of stocks in the Turkish BIST30 using monthly OCHLV data from May 2000 to June 2019. They calculated the predicted returns to infer price trends. Portfolios were built using stocks with predicted returns above a certain threshold. Among the five methods used for weighting, equal weighting and minimum variance were used. The metrics used to evaluate the portfolios’ performance were the Sharpe ratio, maximum drawdown, and conditional VaR. The results show that portfolios using LSTM outperformed the other portfolios and the benchmarks.

Yi et al. (2022) proposed a model named “IntelliPortfolio”, which is geared toward building a portfolio within the framework of Enhanced Index Tracking (EIT). The portfolio is constructed in two steps: the first step involves stock selection using principal component analysis (PCA) and the k-means clustering algorithm, and the second step comprises weight calculation using LSTM neural networks. Testing was performed on daily prices and some fundamental indicators of five stock exchange indexes from 2009 to 2018. The model was tested with four performance indicators (the tracking error, excess return, information ratio, and Sharpe Ratio) over the last 60 days of the sample. The model was compared to five existing models in the literature and the results show that it outperformed them.

The literature offers many studies focused on the prediction of prices and the direction of change or stocks returns, but the integration of predictions in portfolio construction is a research subject that has not yet been adequately explored. Furthermore, research that uses predictions as part of equity-market-neutral (EMN) alternative investment strategies is quite rare. Hence, it would be beneficial to have a comprehensive framework combining prediction, stock selection, and capital allocation to build a portfolio with an EMN strategy and offer a detailed performance analysis.

References

Fama, Eugene F. 1970. Efficient capital markets: A review of theory and empirical work. The Journal of Finance 25: 383–417.
Carhart, Mark M. 1997. On Persistence in Mutual Fund Performance. The Journal of Finance 52: 57–82.
Markowitz, Harry. 1952. Portfolio Selection. The Journal of Finance 7: 77–91.
Fama, Eugene F., and Kenneth R. French. 1993. Common risk factors in the returns on stocks and bonds. Journal of Financial Economics 33: 3–56.
Yule, G. Udny. 1926. Why Do We Sometimes Get Nonsense-Correlations between Time-Series? A Study in Sampling and the Nature of Time-Series. Journal of the Royal Statistical Society 89: 1–63.
Neyman, J. 1939. Review of A Study in Analysis of Stationary Time Series, by Herman Wold. Journal of the Royal Statistical Society 102: 295.
Box, George E. P., and Gwilym M. Jenkins. 1970. Time Series Analysis, Forecasting and Control. San Francisco: Holden-Day.
Engle, Robert F. 1982. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation. Econometrica: Journal of the Econometric Society 50: 987–1007.
Bollerslev, Tim. 1986. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics 31: 307–27.
Nafia, Abdellilah, Abdellah Youssefi, and Abdellah Echaoui. 2022. Modèles classiques et de datamining les plus utilisés en évaluation et en prédiction de la performance des actions. Revue des Etudes Multidisciplinaires en Sciences Economiques et Sociales 7: 231–68.
Rosenblatt, Frank. 1957. The Perceptron, a Perceiving and Recognizing Automaton. Buffalo: Cornell Aeronautical Laboratory.
Jacobs, Bruce I., and Kenneth N. Levy. 2005. Market Neutral Strategies, Frank J. Fabozzi Series. New York: John Wiley & Sons.
Ganchev, Alexander. 2022. The Performance of Hedge Fund Industry during the COVID-19 Crisis—Theoretical Characteristics and Empirical Aspects. Economic Studies (Ikonomicheski Izsledvania) 31: 18–37.
Ozbayoglu, Ahmet Murat, Mehmet Ugur Gudelek, and Omer Berat Sezer. 2020. Deep learning for financial applications: A survey. Applied Soft Computing 93: 106384.
Jiang, Weiwei. 2021. Applications of deep learning in stock market prediction: Recent progress. Expert Systems with Applications 184: 115537.
Hu, Zexin, Yiqi Zhao, and Matloob Khushi. 2021. A survey of forex and stock price prediction using deep learning. Applied System Innovation 4: 9.
Naik, Nagaraj, and Biju R. Mohan. 2019. Study of stock return predictions using recurrent neural networks with LSTM. In International Conference on Engineering Applications of Neural Networks. Cham: Springer, pp. 453–59.
Qiu, Jiayu, Bin Wang, and Changjun Zhou. 2020. Forecasting stock prices with long-short term memory neural network based on attention mechanism. PLoS ONE 15: e0227222.
Ding, Guangyu, and Liangxi Qin. 2020. Study on the prediction of stock price based on the associated network model of LSTM. International Journal of Machine Learning and Cybernetics 11: 1307–17.
Ghosh, Achyut, Soumik Bose, Giridhar Maji, Narayan Debnath, and Soumya Sen. 2019. Stock price prediction using LSTM on Indian Share Market. Paper presented at the 32nd International Conference on Computer Applicationsin Industry and Engineering, San Diego, CA, USA, September 30–October 2; vol. 63, pp. 101–10.
Michańków, Jakub, Paweł Sakowski, and Robert Ślepaczuk. 2022. LSTM in Algorithmic Investment Strategies on BTC and S&P500 Index. Sensors 22: 917.
Tfaily, Fatima, and Mohamad M. Fouad. 2022. Multi-level stacking of LSTM recurrent models for predicting stock-market indices. Data Science in Finance and Economics 2: 147–62.
Patel, Jigar, Sahil Shah, Priyank Thakkar, and K Kotecha. 2015. Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Expert Systems with Applications 42: 259–68.
Yao, Siyu, Linkai Luo, and Hong Peng. 2018. High-frequency stock trend forecast using lstm model. Paper presented at the 2018 13th International Conference on Computer Science & Education (ICCSE), Colombo, Sri Lanka, August 8–11; pp. 1–4.
Chaweewanchon, Apichat, and Rujira Chaysiri. 2022. Markowitz Mean-Variance Portfolio Optimization with Predictive Stock Selection Using Machine Learning. International Journal of Financial Studies 10: 64.
Sen, Jaydip, Sidra Mehtab, Abhishek Dutta, and Saikat Mondal. 2021. Precise Stock Price Prediction for Optimized Portfolio Design Using an LSTM Model. Paper presented at the 2021 19th OITS International Conference on Information Technology (OCIT), Bhubaneswar, India, December 16–18; pp. 210–15.
Touzani, Yassine, and Khadija Douzi. 2021. An LSTM and GRU based trading strategy adapted to the Moroccan market. Journal of Big Data 8: 126.
Liu, Shuanglong, Chao Zhang, and Jinwen Ma. 2017. CNN-LSTM neural network model for quantitative strategy analysis in stock markets. In International Conference on Neural Information Processing. Cham: Springer, pp. 198–206.
Hou, Xiurui, Kai Wang, Jie Zhang, and Zhi Wei. 2020. An enriched time-series forecasting framework for long-short portfolio strategy. IEEE Access 8: 31992–2002.
Cipiloglu Yildiz, Zeynep, and Selim Baha Yildiz. 2022. A portfolio construction framework using LSTM-based stock markets forecasting. International Journal of Finance & Economics 27: 2356–66.
Yi, Kan, Jin Yang, Shuangling Wang, Zhengtong Zhang, Jing Zhang, Jinqiu Song, and Xiao Ren. 2022. IntelliPortfolio: Intelligent Portfolio for Enhanced Index Tracking Using Clustering and LSTM. Mathematical Problems in Engineering 2022: 3751452.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Business, Finance

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Abdellilah Nafia

Abdellah Yousfi

Abdellah Echaoui

View Times: 218

Update Date: 04 Dec 2023

Table of Contents

Video Upload Options

Confirm

1. Introduction

2. LSTM-Based Stock Prediction and Selection

References