LogProb: Online Parsing Evolving Logs With Complex Parameters
To cope with the massive volume of log messages in complex systems, effective and accurate log parsers are crucial for system maintenance. However, existing log parsers have accuracy issues, particularly when handling evolving logs with complex structures. To address this, this paper proposes LogPro...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11003143/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849328577336573952 |
|---|---|
| author | Yan Li Ziying Wang Jiaming Mao Zhimin Gu Haitao Jiang |
| author_facet | Yan Li Ziying Wang Jiaming Mao Zhimin Gu Haitao Jiang |
| author_sort | Yan Li |
| collection | DOAJ |
| description | To cope with the massive volume of log messages in complex systems, effective and accurate log parsers are crucial for system maintenance. However, existing log parsers have accuracy issues, particularly when handling evolving logs with complex structures. To address this, this paper proposes LogProb, an online log parsing method that incorporates a token state prediction component and a search tree-based template extraction component. The token state prediction component determines whether each token in a log message is static (template word) or dynamic (parameter value). The token sequence and predicted token states are parsed by the template extraction component to generate event templates. The extraction process includes two stages: identifying candidate templates with predicted static tokens, and using a fast matching algorithm to determine the final template. Experiments on 16 benchmark datasets show that LogProb outperforms state-of-the-art methods in accuracy while maintaining comparable efficiency. |
| format | Article |
| id | doaj-art-35acce2f32f042be8022a0dda65a47f2 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-35acce2f32f042be8022a0dda65a47f22025-08-20T03:47:33ZengIEEEIEEE Access2169-35362025-01-0113848448485410.1109/ACCESS.2025.356954811003143LogProb: Online Parsing Evolving Logs With Complex ParametersYan Li0https://orcid.org/0009-0001-0524-1992Ziying Wang1Jiaming Mao2Zhimin Gu3Haitao Jiang4Electric Power Research Institute, State Grid Jiangsu Electric Power Company Ltd., Nanjing, Jiangsu, ChinaElectric Power Research Institute, State Grid Jiangsu Electric Power Company Ltd., Nanjing, Jiangsu, ChinaElectric Power Research Institute, State Grid Jiangsu Electric Power Company Ltd., Nanjing, Jiangsu, ChinaElectric Power Research Institute, State Grid Jiangsu Electric Power Company Ltd., Nanjing, Jiangsu, ChinaElectric Power Research Institute, State Grid Jiangsu Electric Power Company Ltd., Nanjing, Jiangsu, ChinaTo cope with the massive volume of log messages in complex systems, effective and accurate log parsers are crucial for system maintenance. However, existing log parsers have accuracy issues, particularly when handling evolving logs with complex structures. To address this, this paper proposes LogProb, an online log parsing method that incorporates a token state prediction component and a search tree-based template extraction component. The token state prediction component determines whether each token in a log message is static (template word) or dynamic (parameter value). The token sequence and predicted token states are parsed by the template extraction component to generate event templates. The extraction process includes two stages: identifying candidate templates with predicted static tokens, and using a fast matching algorithm to determine the final template. Experiments on 16 benchmark datasets show that LogProb outperforms state-of-the-art methods in accuracy while maintaining comparable efficiency.https://ieeexplore.ieee.org/document/11003143/Log analysislog parsingsimilarity metricsystem maintenance |
| spellingShingle | Yan Li Ziying Wang Jiaming Mao Zhimin Gu Haitao Jiang LogProb: Online Parsing Evolving Logs With Complex Parameters IEEE Access Log analysis log parsing similarity metric system maintenance |
| title | LogProb: Online Parsing Evolving Logs With Complex Parameters |
| title_full | LogProb: Online Parsing Evolving Logs With Complex Parameters |
| title_fullStr | LogProb: Online Parsing Evolving Logs With Complex Parameters |
| title_full_unstemmed | LogProb: Online Parsing Evolving Logs With Complex Parameters |
| title_short | LogProb: Online Parsing Evolving Logs With Complex Parameters |
| title_sort | logprob online parsing evolving logs with complex parameters |
| topic | Log analysis log parsing similarity metric system maintenance |
| url | https://ieeexplore.ieee.org/document/11003143/ |
| work_keys_str_mv | AT yanli logprobonlineparsingevolvinglogswithcomplexparameters AT ziyingwang logprobonlineparsingevolvinglogswithcomplexparameters AT jiamingmao logprobonlineparsingevolvinglogswithcomplexparameters AT zhimingu logprobonlineparsingevolvinglogswithcomplexparameters AT haitaojiang logprobonlineparsingevolvinglogswithcomplexparameters |