AICB: A benchmark for evaluating the communication subsystem of LLM training clusters
AICB (Artificial Intelligence Communication Benchmark) is a benchmark for evaluating the communication subsystem of GPU clusters, which includes representative workloads in the fields of Large Language Model (LLM) training. Guided by the theories and methodologies of Evaluatology, we simplified the...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2025-03-01
|
| Series: | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2772485925000250 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849319608168742912 |
|---|---|
| author | Xinyue Li Heyang Zhou Qingxu Li Sen Zhang Gang Lu |
| author_facet | Xinyue Li Heyang Zhou Qingxu Li Sen Zhang Gang Lu |
| author_sort | Xinyue Li |
| collection | DOAJ |
| description | AICB (Artificial Intelligence Communication Benchmark) is a benchmark for evaluating the communication subsystem of GPU clusters, which includes representative workloads in the fields of Large Language Model (LLM) training. Guided by the theories and methodologies of Evaluatology, we simplified the real-workload LLM training systems through AICB that maintain good representativeness and usability. AICB bridges the gap between application benchmarks and microbenchmarks in the scope of LLM training. In addition, we constructed a new GPU-free evaluation system that helps researchers evaluate the communication system of the LLM training systems. To help the urgent demand on this evaluation subject, we open-source AICB and make it available at https://github.com/aliyun/aicb. |
| format | Article |
| id | doaj-art-4b3e69b5f7f44103bbda4403d00a0548 |
| institution | Kabale University |
| issn | 2772-4859 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| spelling | doaj-art-4b3e69b5f7f44103bbda4403d00a05482025-08-20T03:50:22ZengKeAi Communications Co. Ltd.BenchCouncil Transactions on Benchmarks, Standards and Evaluations2772-48592025-03-015110021210.1016/j.tbench.2025.100212AICB: A benchmark for evaluating the communication subsystem of LLM training clustersXinyue Li0Heyang Zhou1Qingxu Li2Sen Zhang3Gang Lu4Corresponding author.; Alibaba Cloud, Beijing, 100124, ChinaAlibaba Cloud, Beijing, 100124, ChinaAlibaba Cloud, Beijing, 100124, ChinaAlibaba Cloud, Beijing, 100124, ChinaAlibaba Cloud, Beijing, 100124, ChinaAICB (Artificial Intelligence Communication Benchmark) is a benchmark for evaluating the communication subsystem of GPU clusters, which includes representative workloads in the fields of Large Language Model (LLM) training. Guided by the theories and methodologies of Evaluatology, we simplified the real-workload LLM training systems through AICB that maintain good representativeness and usability. AICB bridges the gap between application benchmarks and microbenchmarks in the scope of LLM training. In addition, we constructed a new GPU-free evaluation system that helps researchers evaluate the communication system of the LLM training systems. To help the urgent demand on this evaluation subject, we open-source AICB and make it available at https://github.com/aliyun/aicb.http://www.sciencedirect.com/science/article/pii/S2772485925000250LLM training clusterBenchmarkCollective communicationDistribuited training |
| spellingShingle | Xinyue Li Heyang Zhou Qingxu Li Sen Zhang Gang Lu AICB: A benchmark for evaluating the communication subsystem of LLM training clusters BenchCouncil Transactions on Benchmarks, Standards and Evaluations LLM training cluster Benchmark Collective communication Distribuited training |
| title | AICB: A benchmark for evaluating the communication subsystem of LLM training clusters |
| title_full | AICB: A benchmark for evaluating the communication subsystem of LLM training clusters |
| title_fullStr | AICB: A benchmark for evaluating the communication subsystem of LLM training clusters |
| title_full_unstemmed | AICB: A benchmark for evaluating the communication subsystem of LLM training clusters |
| title_short | AICB: A benchmark for evaluating the communication subsystem of LLM training clusters |
| title_sort | aicb a benchmark for evaluating the communication subsystem of llm training clusters |
| topic | LLM training cluster Benchmark Collective communication Distribuited training |
| url | http://www.sciencedirect.com/science/article/pii/S2772485925000250 |
| work_keys_str_mv | AT xinyueli aicbabenchmarkforevaluatingthecommunicationsubsystemofllmtrainingclusters AT heyangzhou aicbabenchmarkforevaluatingthecommunicationsubsystemofllmtrainingclusters AT qingxuli aicbabenchmarkforevaluatingthecommunicationsubsystemofllmtrainingclusters AT senzhang aicbabenchmarkforevaluatingthecommunicationsubsystemofllmtrainingclusters AT ganglu aicbabenchmarkforevaluatingthecommunicationsubsystemofllmtrainingclusters |