A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market
Reinforcement learning (RL) has been applied to financial portfolio management in recent years. Current studies mostly focus on profit accumulation without much consideration of risk. Some risk-return balanced studies extract features from price and volume data only, which is highly correlated and m...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-01-01
|
| Series: | IET Signal Processing |
| Online Access: | http://dx.doi.org/10.1049/2024/5399392 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849395666287067136 |
|---|---|
| author | Ruidan Su Chun Chi Shikui Tu Lei Xu |
| author_facet | Ruidan Su Chun Chi Shikui Tu Lei Xu |
| author_sort | Ruidan Su |
| collection | DOAJ |
| description | Reinforcement learning (RL) has been applied to financial portfolio management in recent years. Current studies mostly focus on profit accumulation without much consideration of risk. Some risk-return balanced studies extract features from price and volume data only, which is highly correlated and missing representation of risk features. To tackle these problems, we propose a weight control unit (WCU) to effectively manage the position of portfolio management in different market statuses. A loss penalty term is also designed in the reward function to prevent sharp drawdown during trading. Moreover, stock spatial interrelation representing the correlation between two different stocks is captured by a graph convolution network based on fundamental data. Temporal interrelation is also captured by a temporal convolutional network based on new factors designed with price and volume data. Both spatial and temporal interrelation work for better feature extraction from historical data and also make the model more interpretable. Finally, a deep deterministic policy gradient actor–critic RL is applied to explore optimal policy in portfolio management. We conduct our approach in a challenging non-short-selling market, and the experiment results show that our method outperforms the state-of-the-art methods in both profit and risk criteria. Specifically, with 6.72% improvement on an annualized rate of return, 7.72% decrease in maximum drawdown, and a better annualized Sharpe ratio of 0.112. Also, the loss penalty and WCU provide new aspects for future work in risk control. |
| format | Article |
| id | doaj-art-a8d012ce33524c98934d9eee53a31402 |
| institution | Kabale University |
| issn | 1751-9683 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Signal Processing |
| spelling | doaj-art-a8d012ce33524c98934d9eee53a314022025-08-20T03:39:32ZengWileyIET Signal Processing1751-96832024-01-01202410.1049/2024/5399392A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling MarketRuidan Su0Chun Chi1Shikui Tu2Lei Xu3Department of Computer Science and EngineeringDepartment of Computer Science and EngineeringDepartment of Computer Science and EngineeringDepartment of Computer Science and EngineeringReinforcement learning (RL) has been applied to financial portfolio management in recent years. Current studies mostly focus on profit accumulation without much consideration of risk. Some risk-return balanced studies extract features from price and volume data only, which is highly correlated and missing representation of risk features. To tackle these problems, we propose a weight control unit (WCU) to effectively manage the position of portfolio management in different market statuses. A loss penalty term is also designed in the reward function to prevent sharp drawdown during trading. Moreover, stock spatial interrelation representing the correlation between two different stocks is captured by a graph convolution network based on fundamental data. Temporal interrelation is also captured by a temporal convolutional network based on new factors designed with price and volume data. Both spatial and temporal interrelation work for better feature extraction from historical data and also make the model more interpretable. Finally, a deep deterministic policy gradient actor–critic RL is applied to explore optimal policy in portfolio management. We conduct our approach in a challenging non-short-selling market, and the experiment results show that our method outperforms the state-of-the-art methods in both profit and risk criteria. Specifically, with 6.72% improvement on an annualized rate of return, 7.72% decrease in maximum drawdown, and a better annualized Sharpe ratio of 0.112. Also, the loss penalty and WCU provide new aspects for future work in risk control.http://dx.doi.org/10.1049/2024/5399392 |
| spellingShingle | Ruidan Su Chun Chi Shikui Tu Lei Xu A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market IET Signal Processing |
| title | A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market |
| title_full | A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market |
| title_fullStr | A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market |
| title_full_unstemmed | A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market |
| title_short | A Deep Reinforcement Learning Approach for Portfolio Management in Non-Short-Selling Market |
| title_sort | deep reinforcement learning approach for portfolio management in non short selling market |
| url | http://dx.doi.org/10.1049/2024/5399392 |
| work_keys_str_mv | AT ruidansu adeepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT chunchi adeepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT shikuitu adeepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT leixu adeepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT ruidansu deepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT chunchi deepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT shikuitu deepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket AT leixu deepreinforcementlearningapproachforportfoliomanagementinnonshortsellingmarket |