Replacing Backpropagation with the Forward-Forward (FF) Algorithm in Transformer Models: A Theoretical and Empirical Study on Scalable and Efficient Gradient-Free Training

Replacing Backpropagation with the Forward-Forward (FF) Algorithm in Transformer Models: A Theoretical and Empirical Study on Scalable and Efficient Gradient-Free Training

This study proposes a novel integration of the Forward-Forward (FF) algorithm into Transformer architectures as an efficient and gradient-free alternative to Backpropagation (BP). Motivated by the computational limitations of BP-such as high memory usage and gradient instability-we aim to examine wh...

Full description

Saved in:
Bibliographic Details
Main Authors: Hyun Jung Kim, Sang Hyun Yoo
Format: Article
Language:English
Published: University North 2025-01-01
Series:Tehnički Glasnik
Subjects:
Backpropagation Alternative
Computational Efficiency
Efficient AI Training
Forward-Forward (FF) Algorithm
Training Stability
Transformer Models
Online Access:https://hrcak.srce.hr/file/480463
Tags: Add Tag
No Tags, Be the first to tag this record!

Internet

https://hrcak.srce.hr/file/480463

Similar Items