Integrating structured and unstructured data for predicting emergency severity: an association and predictive study using transformer-based natural language processing models by Xingyu Zhang, Yanshan Wang, Yun Jiang, Charissa B. Pacella, Wenbin Zhang

“…Unstructured data, including chief complaints and reasons for visit, were processed using a Bidirectional Encoder Representations from Transformers (BERT) model. …”
Get full text

Overview of deep learning and large language models in machine translation: a special perspective on the Arabic language by Sanaa Abou Elhamayed, Mohamed Nour

“…The bidirectional-encoder-representation from transformer (BERT) and LLMs are presented to utilize the big amount of textual data to learn translation patterns. …”
Get full text

A deep learning model for prediction of lysine crotonylation sites by fusing multi-features based on multi-head self-attention mechanism by Yunyun Liang, Minwei Li

“…Multiple features are extracted from natural language processing features and hand-crafted features, where natural language processing features include token embedding and positional embedding encoded by transformer, and hand-crafted features include one-hot, amino acid index and position-weighted amino acid composition, and encoded by bidirectional long short-term memory network. …”
Get full text

Acoustic Event Detection in Vehicles: A Multi-Label Classification Approach by Anaswara Antony, Wolfgang Theimer, Giovanni Grossetti, Christoph M. Friedrich

“…The proposed detection methodology uses the pre-trained network Bidirectional Encoder representation from Audio Transformers (BEATs) and a single-layer neural network trained on the database of real audio recordings collected from different cars. …”
Get full text

From Extractive to Generative: An Analysis of Automatic Text Summarization Techniques by Liu Zixu

“…The review highlights significant milestones in the development of summarization algorithms, including the emergence of Transformer-based models like Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), which have significantly improved the quality and coherence of generated summaries. …”
Get full text

Multi-Head Graph Attention Adversarial Autoencoder Network for Unsupervised Change Detection Using Heterogeneous Remote Sensing Images by Meng Jia, Xiangyu Lou, Zhiqiang Zhao, Xiaofeng Lu, Zhenghao Shi

“…The MHGAN employs a bidirectional adversarial convolutional autoencoder network to reconstruct and perform style transformation of heterogeneous images. …”
Get full text

Information extraction from green channel textual records on expressways using hybrid deep learning by Jiaona Chen, Jing Zhang, Weijun Tao, Yinli Jin, Heng Fan

“…Eight entities are designed and proposed in the NER processing for the expressway green channel. three typical pre-trained natural language processing models are utilized and compared to recognize entities and obtain feature vectors, including bidirectional encoder representations from transformer (BERT), ALBERT, and RoBERTa. …”
Get full text

Identifying Non-Functional Requirements From Unconstrained Documents Using Natural Language Processing and Machine Learning Approaches by Qais A. Shreda, Abualsoud A. Hanani

“…In our approach, features were extracted from the requirement sentences using four different natural language processing methods including statistical and state-of-the-art semantic analysis presented by Google word2vec and bidirectional encoder representations from transformers models. …”
Get full text

A fake news detection model using the integration of multimodal attention mechanism and residual convolutional network by Ying Lu, Naiwei Yao

“…Baseline models used for comparison include Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized Bidirectional Encoder Representations from Transformers Approach (RoBERTa), Generalized Autoregressive Pretraining for Language Understanding (XLNet), Enhanced Representation through Knowledge Integration (ERNIE), and Generative Pre-trained Transformer 3.5 (GPT-3.5). …”
Get full text

Rethinking Technological Investment and Cost-Benefit: A Software Requirements Dependency Extraction Case Study by Gouri Ginde, Guenther Ruhe, Chad Saunders

“…Specifically, we extract dependencies from textual descriptions of software requirements and analyze the performance of two state-of-the-art ML techniques: Random Forest and Bidirectional Encoder Representations from Transformers (BERT), a encoder only Large Language Model. …”
Get full text

A Hybrid Deep Learning Approach for Cotton Plant Disease Detection Using BERT-ResNet-PSO by Chetanpal Singh, Santoso Wibowo, Srimannarayana Grandhi

“…It is, therefore, crucial to accurately identify leaf diseases in cotton plants to prevent any negative effects on yield. This paper presents a hybrid deep learning approach based on Bidirectional Encoder Representations from Transformers with Residual network and particle swarm optimization (BERT-ResNet-PSO) for detecting cotton plant diseases. …”
Get full text

EYE-Llama, an in-domain large language model for ophthalmology by Tania Haghighi, Sina Gholami, Jared Todd Sokol, Enaika Kishnani, Adnan Ahsaniyan, Holakou Rahmanian, Fares Hedayati, Theodore Leng, Minhaj Nur Alam

“…We evaluated EYE-Llama against Llama 2, Llama 3, Meditron, ChatDoctor, ChatGPT, and several other LLMs. Using BERT (Bidirectional Encoder Representations from Transformers) score, BART (Bidirectional and Auto-Regressive Transformer) score, and BLEU (Bilingual Evaluation Understudy) metrics, EYE-Llama achieved superior scores. …”
Get full text

Tackling misinformation in mobile social networks a BERT-LSTM approach for enhancing digital literacy by Jun Wang, Xiulai Wang, Airong Yu

“…Early detection of misinformation is essential yet challenging, particularly in contexts where initial content propagation lacks user feedback and engagement data. This study presents a novel hybrid model that combines Bidirectional Encoder Representations from Transformers (BERT) with Long Short-Term Memory (LSTM) networks to enhance the detection of misinformation using only textual content. …”
Get full text

Needle in a haystack: Harnessing AI in drug patent searches and prediction. by Leonardo Costa Ribeiro, Valbona Muzaka

“…The aim is primarily that of demonstrating how the proverbial needle in a haystack was identified, namely through leveraging the superb pattern-recognition abilities of the BERT (Bidirectional Encoder Representations from Transformers) algorithm. …”
Get full text

Sentiment Analysis of X Users Toward Electric Motorcycles Using SVM and BERT Algorithms by Calvin Adiwinata, Afiyati Afiyati

“…This study presents a comparative analysis of Support Vector Machine (SVM) and Bidirectional Encoder Representations from Transformers (BERT) for sentiment analysis on electric motorcycles in Indonesia using data from the social media platform X, formerly known as Twitter. …”
Get full text

Comparison of Deep Learning Sentiment Analysis Methods, Including LSTM and Machine Learning by Jean Max T. Habib, A. A. Poguda

“…In this case, it is crucial for researchers to explore the possibilities of updating certain tools, either to combine them or to develop them to adapt them to modern tasks in order to provide a clearer understanding of the results of their treatment. We present a comparison of several deep learning models, including convolutional neural networks, recurrent neural networks, and long-term and shortterm bidirectional memory, evaluated using different approaches to word integration, including Bidirectional Encoder Representations from Transformers (BERT) and its variants, FastText and Word2Vec. …”
Get full text

Twitter User Account Classification to Gain Insights into Communication Dynamics and Public Awareness During Tampa Bay's Red Tide Events by Andrey Skripnikov, Tania Roy, Fehmi Neffati, Melvin Adkins, Marcus Beck

“…Having used several text classification algorithms and feature preprocessing approaches, Support Vector Machine with Bidirectional Encoder Representations from Transformers (BERT) yielded the best cross-validation performance in both accuracy (90%) and versatility (unweighted F1 score of 0.67). …”
Get full text

Sporting a virtual future: exploring sports and virtual reality patents using deep learning-based analysis by Jea Woog Lee, Sangmin Song, JungMin Yun, Doug Hyun Han, YoungBin Kim

“…Using patent big data, we introduce SportsBERT, a bidirectional encoder representation from transformers (BERT)-based algorithm tailored for enhanced natural language processing in sports-related knowledge-based documents. …”
Get full text

Rumor detection using dual embeddings and text-based graph convolutional network by Barsha Pattanaik, Sourav Mandal, Rudra M. Tripathy, Arif Ahmed Sekh

“…This model uses dual embedding from two pre-trained transformer models: generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT). …”
Get full text

A Deep Learning Model for Automatic Citation Document Recommendation in Non-Obviousness Judgment: Using BERT-for-patents and Contrastive Learning by Dongkun Yoo, Jiheon Han

“…Six models were trained based on the bidirectional encoder representations from transformers (BERT), and the performances were compared. …”
Get full text