IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions

The internet key exchange (IKE) protocol, integral to the authentication and key negotiation process within the Internet Protocol Security (IPSec) framework, has been utilized for the protection of IP communications. Given the complex protocol logic, security vulnerabilities in the implementation of...

Full description

Saved in:
Bibliographic Details
Main Authors: ZHENG Yonghui, ZHAO Dongliang, GU Chunxiang, ZHANG Xieli
Format: Article
Language:English
Published: POSTS&TELECOM PRESS Co., LTD 2024-08-01
Series:网络与信息安全学报
Subjects:
Online Access:http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2024057
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1841530443456839680
author ZHENG Yonghui
ZHAO Dongliang
GU Chunxiang
ZHANG Xieli
author_facet ZHENG Yonghui
ZHAO Dongliang
GU Chunxiang
ZHANG Xieli
author_sort ZHENG Yonghui
collection DOAJ
description The internet key exchange (IKE) protocol, integral to the authentication and key negotiation process within the Internet Protocol Security (IPSec) framework, has been utilized for the protection of IP communications. Given the complex protocol logic, security vulnerabilities in the implementation of the IKE protocol are inevitably present. Fuzz testing, recognized as an effective means of detecting potential vulnerabilities in protocol implementations, has been conventionally applied. However, the direct application of existing fuzzing tools to the IKE protocol has been found to present limitations, such as the generation of low-quality test cases and difficulty in exploring deep states. To address these issues, a mutation strategy based on the grammar of the IKE protocol was designed, aiming to reduce the generation of invalid test cases while increasing the diversity of generated test cases. Additionally, an evolutionary strategy-based mutation scheduling scheme was introduced, which automatically optimized the probability distribution of mutation operators, further increasing the likelihood of generating high-quality test cases. A message handler was designed to maintain protocol interaction context information and perform cryptographic operations, thereby supporting testing of the IKE protocol under black-box conditions. This enabled the exploration of deep protocol interaction behavior and state space. Utilizing the aforementioned methods, an IKE protocol stateful fuzz testing tool named IKEChecker was implemented, supporting testing of both IKEv1 and IKEv2 protocols. Testing was conducted on two widely used open-source IKE protocol implementations, strongSwan and Libreswan, resulting in the revelation of 4 undisclosed vulnerabilities. By comparing IKEChecker with other fuzz testing tools, its efficiency in vulnerability detection was evaluated.
format Article
id doaj-art-54c9ff7142784a018682e4e20a2396ad
institution Kabale University
issn 2096-109X
language English
publishDate 2024-08-01
publisher POSTS&TELECOM PRESS Co., LTD
record_format Article
series 网络与信息安全学报
spelling doaj-art-54c9ff7142784a018682e4e20a2396ad2025-01-15T03:04:14ZengPOSTS&TELECOM PRESS Co., LTD网络与信息安全学报2096-109X2024-08-011010912270108372IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementionsZHENG YonghuiZHAO DongliangGU ChunxiangZHANG XieliThe internet key exchange (IKE) protocol, integral to the authentication and key negotiation process within the Internet Protocol Security (IPSec) framework, has been utilized for the protection of IP communications. Given the complex protocol logic, security vulnerabilities in the implementation of the IKE protocol are inevitably present. Fuzz testing, recognized as an effective means of detecting potential vulnerabilities in protocol implementations, has been conventionally applied. However, the direct application of existing fuzzing tools to the IKE protocol has been found to present limitations, such as the generation of low-quality test cases and difficulty in exploring deep states. To address these issues, a mutation strategy based on the grammar of the IKE protocol was designed, aiming to reduce the generation of invalid test cases while increasing the diversity of generated test cases. Additionally, an evolutionary strategy-based mutation scheduling scheme was introduced, which automatically optimized the probability distribution of mutation operators, further increasing the likelihood of generating high-quality test cases. A message handler was designed to maintain protocol interaction context information and perform cryptographic operations, thereby supporting testing of the IKE protocol under black-box conditions. This enabled the exploration of deep protocol interaction behavior and state space. Utilizing the aforementioned methods, an IKE protocol stateful fuzz testing tool named IKEChecker was implemented, supporting testing of both IKEv1 and IKEv2 protocols. Testing was conducted on two widely used open-source IKE protocol implementations, strongSwan and Libreswan, resulting in the revelation of 4 undisclosed vulnerabilities. By comparing IKEChecker with other fuzz testing tools, its efficiency in vulnerability detection was evaluated.http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2024057security protocolfuzz testingsoftware testingIKE protocol
spellingShingle ZHENG Yonghui
ZHAO Dongliang
GU Chunxiang
ZHANG Xieli
IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
网络与信息安全学报
security protocol
fuzz testing
software testing
IKE protocol
title IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
title_full IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
title_fullStr IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
title_full_unstemmed IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
title_short IKEChecker: grammar-guided stateful fuzzer for IKE protocol implementions
title_sort ikechecker grammar guided stateful fuzzer for ike protocol implementions
topic security protocol
fuzz testing
software testing
IKE protocol
url http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2024057
work_keys_str_mv AT zhengyonghui ikecheckergrammarguidedstatefulfuzzerforikeprotocolimplementions
AT zhaodongliang ikecheckergrammarguidedstatefulfuzzerforikeprotocolimplementions
AT guchunxiang ikecheckergrammarguidedstatefulfuzzerforikeprotocolimplementions
AT zhangxieli ikecheckergrammarguidedstatefulfuzzerforikeprotocolimplementions