KAFA-Merge: Model Merging Method With Layer-Based Bayes+Linear Search

KAFA-Merge: Model Merging Method With Layer-Based Bayes+Linear Search

In this paper, we consider the model merging process for large language models (LLMs) under a two-stage optimization framework. Traditional merging methods usually apply fixed blending rates to all layers, which ignores the different levels of contribution of layers to the output. To overcome this l...

Full description

Saved in:
Bibliographic Details
Main Authors: Muzaffer Kaan Yuce, Mehmet Fatih Amasyali
Format: Article
Language:English
Published: IEEE 2025-01-01
Series:IEEE Access
Subjects:
Bayesian optimization
large language models
layer-based model merging
linear optimization
model blending
weight merging
Online Access:https://ieeexplore.ieee.org/document/11048544/
Tags: Add Tag
No Tags, Be the first to tag this record!

Internet

https://ieeexplore.ieee.org/document/11048544/

Similar Items