Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies

In today’s modern financial landscape, where accuracy and speed of prediction are increasingly critical, machine learning techniques play a vital role in stock price forecasting. This study evaluates the effectiveness of two deep learning models—Long Short-Term Memory (LSTM) and Transformer—in pred...

Full description

Saved in:
Bibliographic Details
Main Authors: Muhammad Irfan Abdul Gani, Putry Wahyu Setyaningsih
Format: Article
Language:Indonesian
Published: Islamic University of Indragiri 2025-09-01
Series:Sistemasi: Jurnal Sistem Informasi
Subjects:
Online Access:https://sistemasi.ftik.unisi.ac.id/index.php/stmsi/article/view/5445
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1850044543442878464
author Muhammad Irfan Abdul Gani
Putry Wahyu Setyaningsih
author_facet Muhammad Irfan Abdul Gani
Putry Wahyu Setyaningsih
author_sort Muhammad Irfan Abdul Gani
collection DOAJ
description In today’s modern financial landscape, where accuracy and speed of prediction are increasingly critical, machine learning techniques play a vital role in stock price forecasting. This study evaluates the effectiveness of two deep learning models—Long Short-Term Memory (LSTM) and Transformer—in predicting NVIDIA (NVDA) stock prices using historical data from June 7, 2021 to June 7, 2025, with an 80% training and 20% testing data split. The results show that the LSTM model achieved a Root Mean Squared Error (RMSE) of 2.7703 on the training data and 7.3796 on the testing data, while the Transformer model produced an RMSE of 5.3573 (training) and 10.8563 (testing). A hybrid model demonstrated improved prediction accuracy with an RMSE of 3.5643 (training) and 8.6727 (testing), although it still did not outperform LSTM. The model also indicated a moderately declining trend in stock prices over the projected 30-day period. Gaussian noise augmentation was applied during training to improve model generalization. This study also explores investment strategy development by analyzing rule-based trading signals, generating buy (long) and sell (short) signals based on predicted price movements. Additionally, risks such as market volatility and potential overfitting were evaluated, alongside the influence of non-technical factors such as market sentiment. The primary focus of the research is to compare the performance of the LSTM and Transformer models in forecasting NVIDIA’s closing stock prices and applying a simple rule-based trading strategy. For future work, the use of methods such as Prophet, ARIMA, and hybrid ensemble approaches is recommended to enhance prediction accuracy, improve market adaptability, and deliver a more robust stock forecasting system leveraging advanced machine learning techniques for more optimal investment decisions.
format Article
id doaj-art-d24bc124ea32482283dd043ea126211a
institution DOAJ
issn 2302-8149
2540-9719
language Indonesian
publishDate 2025-09-01
publisher Islamic University of Indragiri
record_format Article
series Sistemasi: Jurnal Sistem Informasi
spelling doaj-art-d24bc124ea32482283dd043ea126211a2025-08-20T02:54:54ZindIslamic University of IndragiriSistemasi: Jurnal Sistem Informasi2302-81492540-97192025-09-011452424243710.32520/stmsi.v14i5.54451197Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading StrategiesMuhammad Irfan Abdul Gani0Putry Wahyu Setyaningsih1Universitas MercuBuana YogyakartaUniversitas Mercu Buana YogyakartaIn today’s modern financial landscape, where accuracy and speed of prediction are increasingly critical, machine learning techniques play a vital role in stock price forecasting. This study evaluates the effectiveness of two deep learning models—Long Short-Term Memory (LSTM) and Transformer—in predicting NVIDIA (NVDA) stock prices using historical data from June 7, 2021 to June 7, 2025, with an 80% training and 20% testing data split. The results show that the LSTM model achieved a Root Mean Squared Error (RMSE) of 2.7703 on the training data and 7.3796 on the testing data, while the Transformer model produced an RMSE of 5.3573 (training) and 10.8563 (testing). A hybrid model demonstrated improved prediction accuracy with an RMSE of 3.5643 (training) and 8.6727 (testing), although it still did not outperform LSTM. The model also indicated a moderately declining trend in stock prices over the projected 30-day period. Gaussian noise augmentation was applied during training to improve model generalization. This study also explores investment strategy development by analyzing rule-based trading signals, generating buy (long) and sell (short) signals based on predicted price movements. Additionally, risks such as market volatility and potential overfitting were evaluated, alongside the influence of non-technical factors such as market sentiment. The primary focus of the research is to compare the performance of the LSTM and Transformer models in forecasting NVIDIA’s closing stock prices and applying a simple rule-based trading strategy. For future work, the use of methods such as Prophet, ARIMA, and hybrid ensemble approaches is recommended to enhance prediction accuracy, improve market adaptability, and deliver a more robust stock forecasting system leveraging advanced machine learning techniques for more optimal investment decisions.https://sistemasi.ftik.unisi.ac.id/index.php/stmsi/article/view/5445stock price predictionlong short term memory (ltsm)transformercombined forecastlong-short trading strategy
spellingShingle Muhammad Irfan Abdul Gani
Putry Wahyu Setyaningsih
Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
Sistemasi: Jurnal Sistem Informasi
stock price prediction
long short term memory (ltsm)
transformer
combined forecast
long-short trading strategy
title Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
title_full Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
title_fullStr Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
title_full_unstemmed Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
title_short Comparison of LSTM and Transformer Models in Predicting NVIDIA Stock Closing Prices and the Application of Rule-based Trading Strategies
title_sort comparison of lstm and transformer models in predicting nvidia stock closing prices and the application of rule based trading strategies
topic stock price prediction
long short term memory (ltsm)
transformer
combined forecast
long-short trading strategy
url https://sistemasi.ftik.unisi.ac.id/index.php/stmsi/article/view/5445
work_keys_str_mv AT muhammadirfanabdulgani comparisonoflstmandtransformermodelsinpredictingnvidiastockclosingpricesandtheapplicationofrulebasedtradingstrategies
AT putrywahyusetyaningsih comparisonoflstmandtransformermodelsinpredictingnvidiastockclosingpricesandtheapplicationofrulebasedtradingstrategies