DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning
Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tas...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2025-01-01
|
| Series: | AI Open |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666651025000075 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849390541840580608 |
|---|---|
| author | Zhi Chen Da Ma Hanqi Li Lu Chen Jiabao Ji Yuncong Liu Bei Chen Mengyue Wu Su Zhu Xin Dong Fujiang Ge Qingliang Miao Jian-Guang Lou Shuai Fan Kai Yu |
| author_facet | Zhi Chen Da Ma Hanqi Li Lu Chen Jiabao Ji Yuncong Liu Bei Chen Mengyue Wu Su Zhu Xin Dong Fujiang Ge Qingliang Miao Jian-Guang Lou Shuai Fan Kai Yu |
| author_sort | Zhi Chen |
| collection | DOAJ |
| description | Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve competitive performance on very rich cross-domain downstream dialogue tasks. Furthermore, when scaling to large language models, DFM remains effective. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model. |
| format | Article |
| id | doaj-art-6ff577a130624a829f4c1f5251b61916 |
| institution | Kabale University |
| issn | 2666-6510 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | AI Open |
| spelling | doaj-art-6ff577a130624a829f4c1f5251b619162025-08-20T03:41:34ZengKeAi Communications Co. Ltd.AI Open2666-65102025-01-01610811710.1016/j.aiopen.2025.04.001DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learningZhi Chen0Da Ma1Hanqi Li2Lu Chen3Jiabao Ji4Yuncong Liu5Bei Chen6Mengyue Wu7Su Zhu8Xin Dong9Fujiang Ge10Qingliang Miao11Jian-Guang Lou12Shuai Fan13Kai Yu14X-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, China; Corresponding authors.X-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaMicrosoft Research Asia, Beijing, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, ChinaAISpeech Co., Ltd., Suzhou, ChinaAISpeech Co., Ltd., Suzhou, ChinaAISpeech Co., Ltd., Suzhou, ChinaAISpeech Co., Ltd., Suzhou, ChinaMicrosoft Research Asia, Beijing, ChinaAISpeech Co., Ltd., Suzhou, ChinaX-LANCE Lab, School of Computer Science MoE Key Lab of Artificial Intelligence, SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, China; Corresponding authors.Building a universal conversational agent has been a long-standing goal of the dialogue research community. Most previous works only focus on a small set of dialogue tasks. In this work, we aim to build a unified dialogue foundation model (DFM) which can be used to solve massive diverse dialogue tasks. To achieve this goal, a large-scale well-annotated dialogue dataset with rich task diversity (DialogZoo) is collected. We introduce a framework to unify all dialogue tasks and propose novel auxiliary self-supervised tasks to achieve stable training of DFM on the highly diverse large scale DialogZoo corpus. Experiments show that, compared with models of the same size, DFM can achieve competitive performance on very rich cross-domain downstream dialogue tasks. Furthermore, when scaling to large language models, DFM remains effective. This demonstrates that DFM largely extends the ability of unified dialogue pre-trained model.http://www.sciencedirect.com/science/article/pii/S2666651025000075Dialogue foundation modelPre-trained language modelMultitask learningKnowledge-grounded dialogue |
| spellingShingle | Zhi Chen Da Ma Hanqi Li Lu Chen Jiabao Ji Yuncong Liu Bei Chen Mengyue Wu Su Zhu Xin Dong Fujiang Ge Qingliang Miao Jian-Guang Lou Shuai Fan Kai Yu DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning AI Open Dialogue foundation model Pre-trained language model Multitask learning Knowledge-grounded dialogue |
| title | DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning |
| title_full | DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning |
| title_fullStr | DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning |
| title_full_unstemmed | DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning |
| title_short | DFM: Dialogue foundation model for universal large-scale dialogue-oriented task learning |
| title_sort | dfm dialogue foundation model for universal large scale dialogue oriented task learning |
| topic | Dialogue foundation model Pre-trained language model Multitask learning Knowledge-grounded dialogue |
| url | http://www.sciencedirect.com/science/article/pii/S2666651025000075 |
| work_keys_str_mv | AT zhichen dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT dama dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT hanqili dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT luchen dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT jiabaoji dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT yuncongliu dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT beichen dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT mengyuewu dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT suzhu dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT xindong dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT fujiangge dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT qingliangmiao dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT jianguanglou dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT shuaifan dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning AT kaiyu dfmdialoguefoundationmodelforuniversallargescaledialogueorientedtasklearning |