Portfolio Management: a Deep Reinforcement Learning Approach

Portfolio management is the task of constantly redistributing the wealth of an investor into different financial products known as assets in order to maximize the overall profit during the trading period while maintaining an acceptable level of risk. Traditionally, this problem has been heavily studied using a static approach, leading to the creation of what is known as the Modern Portfolio Theory by economist H. Markowitz in 1952. However, more recent developments in the field of optimization of dynamical systems, especially via reinforcement learning, have sparked a growing interest in applying these new techniques to the portfolio optimization task. In this thesis, we represent the trading environment as a discrete-time stochastic dynamical system and apply deep learning based reinforcement learning algorithms to train an agent to learn a trading strategy that optimizes a given objective. In particular, we assume that the agent has information only on the daily open, high, low and close prices and the trading volumes and, using these values, the trading agent has to decide, at the end of every trading day, what portfolio to build for the next day, keeping into account that all transactions are penalized due to the presence of a transaction cost equal to a small percentage of the total transaction value. Regarding the reinforcement learning algorithms employed, we focused only on actor-critic methods using LSTM-based architectures to represent the value and policy functions. These techniques are further improved via a pre-training step by initializing the networks weights using the optimal weights obtained by training a network with the same structure to forecast portfolio returns. Multiple experiments done using real market data consisting of a small set of stocks chosen from those included in the S&P500 index show the validity of applying the reinforcement learning framework to the task of portfolio optimization.