Confidence-Based Knowledge Distillation to Reduce Training Costs and Carbon Footprint for Low-Resource Neural Machine Translation by Maria Zafar, Patrick J. Wall, Souhail Bakkali, Rejwanul Haque

“…Large-scale pretrained transformer models produce state-of-the-art performance across a wide range of MT tasks for many languages. However, such deep neural network (NN) models are often data-, compute-, space-, power-, and energy-hungry, typically requiring powerful GPUs or large-scale clusters to train and deploy. …”
Get full text

A pre-trained deep potential model for sulfide solid electrolytes with broad coverage and high accuracy by Ruoyu Wang, Mingyu Guo, Yuxiang Gao, Xiaoxu Wang, Yuzhi Zhang, Bin Deng, Mengchao Shi, Linfeng Zhang, Zhicheng Zhong

“…Here, we propose a pre-trained deep potential model purpose-built for sulfide solid electrolytes with attention mechanism, known as DPA-SSE. …”
Get full text

Design of intelligent optimization of sports strategy and training decision support system based on deep reinforcement learning by Hua Xu, Bing Lin, Long Liu

“…Abstract Existing training decision support systems mostly rely on preset rules or frameworks based on prior knowledge. …”
Get full text

A Knowledge-Enhanced Object Detection for Sustainable Agriculture by Youcef Djenouri, Ahmed Nabil Belbachir, Tomasz Michalak, Asma Belhadi, Gautam Srivastava

“…Our framework uses a knowledge base of visual features and loss values from multiple deep-learning models during the training phase to choose the most effective model for the testing phase. …”
Get full text

Enhancement and evaluation for deep learning-based classification of volumetric neuroimaging with 3D-to-2D knowledge distillation by Hyemin Yoon, Do-Young Kang, Sangjin Kim

“…This framework is designed to employ volumetric prior knowledge in training 2D CNNs. Our proposed method includes three modules: (i) a 3D teacher network that encodes volumetric prior knowledge from the 3D dataset, (ii) a 2D student network that encodes partial volumetric information from the 2D dataset, and aims to develop an understanding of the original volumetric imaging, and (iii) a distillation loss introduced to reduce the gap in the graph representation expressing the relationship between data in the feature embedding spaces of (i) and (ii), thereby enhancing the final performance. …”
Get full text

Transient Overvoltage Prediction Method for Renewable Energy Stations via Knowledge-Embedded Enhanced Deep Neural Network by Guangyao Wang, Jun Liu, Jiacheng Liu, Yuting Li, Tianxiao Mo, Sheng Ju

“…Building on this, a knowledge-embedded enhanced deep neural network (KEDNN) approach is proposed for predicting the RES’s TOV for complex power systems. …”
Get full text

Knowledge Graph–Enhanced Deep Learning Model (H-SYSTEM) for Hypertensive Intracerebral Hemorrhage: Model Development and Validation by Yulong Xia, Jie Li, Bo Deng, Qilin Huang, Fenglin Cai, Yanfeng Xie, Xiaochuan Sun, Quanhong Shi, Wei Dan, Yan Zhan, Li Jiang

“…Therefore, it has the potential to provide neurosurgeons with rapid and reliable decision support, especially in emergency conditions. The knowledge graph–enhanced deep-learning model exhibited excellent performance in the clinical practice tasks.…”
Get full text

Recognition of building group patterns using GCN and knowledge graph by Tao Liu, Ziqiang Zhang, Ping Du, Wenning Wang, Haowen Yan, Bo Qiang, Shenglu Xu

Get full text

KITE-DDI: A Knowledge Graph Integrated Transformer Model for Accurately Predicting Drug-Drug Interaction Events From Drug SMILES and Biomedical Knowledge Graph by Azwad Tamir, Jiann-Shiun Yuan

“…Most contemporary research for predicting DDI events relies on either information from Biomedical Knowledge graphs (KG) or drug SMILES, with very few managing to merge data from both to make predictions, while others use heuristic algorithms to extract features from SMILES and KGs, which are then fed into a Deep Learning framework to generate output. …”
Get full text

A NOVEL DEEP LEARNING APPROACHES FOR MULTI-CLASS HISTOPATHOLOGICAL SUB-IMAGE CLASSIFICATION USING PRIOR KNOWLEDGE by Riyam Ali Yassin, Morteza Valizadeh, Alaa Hussein Abdulaal

“…The study evaluates various pre-trained deep neural networks, including Inception V3, VGG19, GoogleNet, ResNet 101, and NASNet. …”
Get full text

A Few-Shot Learning Framework for Depth Completion Based on Self-Training with Noise and Pixel-Wise Knowledge Distillation by Shijie Zhang, Shengjie Zhao, Jin Zeng, Hao Deng

“…Depth completion generates a comprehensive depth map by utilizing sparse depth data inputs, supplemented by guidance provided by an RGB image. Deep neural network models depend on annotated datasets for optimal training. …”
Get full text

Large Scale Mowing Event Detection on Dense Time Series Data Using Deep Learning Methods and Knowledge Distillation by T. Moumouris, V. Tsironis, A. Psalta, K. Karantzalos

“…To address data scarcity, we employed knowledge distillation, pre-training models on pseudo-labeled data derived from a dataset in Germany. …”
Get full text

A Dual-Encoder Contrastive Learning Model for Knowledge Tracing by Yanhong Bai, Xingjiao Wu, Tingjiang Wei, Liang He

Get full text

Chinese medical named entity recognition integrating adversarial training and feature enhancement by Xu Zhang, Youchen Kao, Shengbing Che, Juan Yan, Sha Zhou, Shenyi Guo, Wanqin Wang

“…Firstly, the model integrates various advanced technologies, such as Bidirectional Long Short-Term Memory networks (BiLSTM), Iterative Deep Convolutional Neural Networks (IDCNN), and Conditional Random Fields (CRF), to improve the accuracy of named entity recognition. …”
Get full text

De-speckling of medical ultrasound image using metric-optimized knowledge distillation by Mostafa Khalifa, Hanaa M. Hamza, Khalid M. Hosny

“…We introduce the Metric-Optimized Knowledge Distillation (MK) model, a deep-learning approach that utilizes Knowledge Distillation (KD) for denoising ultrasound images. …”
Get full text

GPPK4PCM: pest classification model integrating growth period prior knowledge by Jianhua Zheng, Junde Lu, Yusha Fu, Ruolin Zhao, Jinfang Liu, Zhaoxi Luo, Zhijie Luo

“…To address this issue, we propose a Pest Classification Model Integrating Growth Period Prior Knowledge. The model is composed of three sub-modules where: i) A deep learning network first identifies the growth periods of pests, and this prior knowledge is then used to guide the text encoder of the CLIP pre-trained model in generating period-specific textual features. ii) A parallel deep learning network extracts visual features from pest images. iii) An efficient low-rank multimodal fusion module integrates textual and visual features through parameter-optimized tensor decomposition, significantly improving classification accuracy across pest developmental phases. …”
Get full text

Federated Subgraph Learning via Global-Knowledge-Guided Node Generation by Yuxuan Liu, Zhiming He, Shuang Wang, Yangyang Wang, Peichao Wang, Zhangshen Huang, Qi Sun

Get full text

Efficient and assured reinforcement learning-based building HVAC control with heterogeneous expert-guided training by Shichao Xu, Yangyang Fu, Yixuan Wang, Zhuoran Yang, Chao Huang, Zheng O’Neill, Zhaoran Wang, Qi Zhu

“…In this work, we present a systematic approach to accelerate online reinforcement learning for HVAC control by taking full advantage of the knowledge from domain experts in various forms. Specifically, the algorithm stages include learning expert functions from existing abstract physical models and from historical data via offline reinforcement learning, integrating the expert functions with rule-based guidelines, conducting training guided by the integrated expert function and performing policy initialization from distilled expert function. …”
Get full text

Knowledge, interest and perspectives on Artificial Intelligence in Neurosurgery. A global survey by A. Boaro, E. Mezzalira, F. Siddi, C. Bagattini, N. Gabrovsky, N. Marchesini, M. Broekman, F. Sala, N. Gabrovsky, Marcel Ivanov, Florian Ringel, Enrico Tessitore, Nicolas Sampron, Alessandro Boaro, Victor E. Staartjes

“…The correct definition of ‘Machine Learning’, ‘Deep Learning’ and main Big Data features were identified by respectively 42%, 23% and 23% of the respondents. …”
Get full text

High efficiency classification of thyroid cytopathological images based on knowledge distillation and vision transformer by Jiazhe Zhang, Haolin Zhang, Peng Jiang, Qin Huang, Guangya Zhu, Jingjing Chen, Yingling Cheng, Shu Ran, Fusong Jiang

Get full text