On the Adversarial Robustness of Decision Trees and a Symmetry Defense

Gradient-boosting decision tree classifiers (GBDTs) are susceptible to adversarial perturbation attacks that change inputs slightly to cause misclassification. GBDTs are customarily used on non-image datasets that lack inherent symmetries, which might be why data symmetry in the context of GBDT clas...

Full description

Saved in:
Bibliographic Details
Main Author: Blerta Lindqvist
Format: Article
Language:English
Published: IEEE 2025-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/10843676/
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1832583226362691584
author Blerta Lindqvist
author_facet Blerta Lindqvist
author_sort Blerta Lindqvist
collection DOAJ
description Gradient-boosting decision tree classifiers (GBDTs) are susceptible to adversarial perturbation attacks that change inputs slightly to cause misclassification. GBDTs are customarily used on non-image datasets that lack inherent symmetries, which might be why data symmetry in the context of GBDT classifiers has not received much attention. In this paper, we show that GBDTs can classify symmetric samples differently, which means that GBDTs lack invariance with respect to symmetry. Based on this, we defend GBDTs against adversarial perturbation attacks using symmetric adversarial samples in order to obtain correct classification. We apply and evaluate the symmetry defense against six adversarial perturbation attacks on the GBDT classifiers of nine datasets with a threat model that ranges from zero-knowledge to perfect-knowledge adversaries. Against zero-knowledge adversaries, we use the feature inversion symmetry and exceed the accuracies of default and robust classifiers by up to 100% points. Against perfect-knowledge adversaries for the GBDT classifier of the F-MNIST dataset, we use the feature inversion and horizontal flip symmetries and exceed the accuracies of default and robust classifiers by up to 96% points. Finally, we show that the current definition of adversarial robustness based on the minimum perturbation values of misclassifying adversarial samples might be inadequate for two reasons. First, this definition assumes that attacks mostly succeed, failing to consider the case when attacks are unable to construct misclassifying adversarial samples against a classifier. Second, GBDT adversarial robustness as currently defined can decrease by training with additional samples, even training samples, which counters the common wisdom that more training samples should increase robustness. With the current definition of GBDT adversarial robustness, we can make GBDTs more adversarially robust by training them with fewer samples! The code is publicly available at <uri>https://github.com/blertal/xgboost-symmetry-defense</uri>.
format Article
id doaj-art-5b20c987f69445b582f0fdf5b6792e74
institution Kabale University
issn 2169-3536
language English
publishDate 2025-01-01
publisher IEEE
record_format Article
series IEEE Access
spelling doaj-art-5b20c987f69445b582f0fdf5b6792e742025-01-29T00:01:18ZengIEEEIEEE Access2169-35362025-01-0113161201613210.1109/ACCESS.2025.353069510843676On the Adversarial Robustness of Decision Trees and a Symmetry DefenseBlerta Lindqvist0https://orcid.org/0000-0002-4950-2250Department of Computer Science, Aalto University, Espoo, FinlandGradient-boosting decision tree classifiers (GBDTs) are susceptible to adversarial perturbation attacks that change inputs slightly to cause misclassification. GBDTs are customarily used on non-image datasets that lack inherent symmetries, which might be why data symmetry in the context of GBDT classifiers has not received much attention. In this paper, we show that GBDTs can classify symmetric samples differently, which means that GBDTs lack invariance with respect to symmetry. Based on this, we defend GBDTs against adversarial perturbation attacks using symmetric adversarial samples in order to obtain correct classification. We apply and evaluate the symmetry defense against six adversarial perturbation attacks on the GBDT classifiers of nine datasets with a threat model that ranges from zero-knowledge to perfect-knowledge adversaries. Against zero-knowledge adversaries, we use the feature inversion symmetry and exceed the accuracies of default and robust classifiers by up to 100% points. Against perfect-knowledge adversaries for the GBDT classifier of the F-MNIST dataset, we use the feature inversion and horizontal flip symmetries and exceed the accuracies of default and robust classifiers by up to 96% points. Finally, we show that the current definition of adversarial robustness based on the minimum perturbation values of misclassifying adversarial samples might be inadequate for two reasons. First, this definition assumes that attacks mostly succeed, failing to consider the case when attacks are unable to construct misclassifying adversarial samples against a classifier. Second, GBDT adversarial robustness as currently defined can decrease by training with additional samples, even training samples, which counters the common wisdom that more training samples should increase robustness. With the current definition of GBDT adversarial robustness, we can make GBDTs more adversarially robust by training them with fewer samples! The code is publicly available at <uri>https://github.com/blertal/xgboost-symmetry-defense</uri>.https://ieeexplore.ieee.org/document/10843676/Adversarial perturbation attacksadversarial robustnessequivariancegradient-boosting decision treesinvariancesymmetry defense
spellingShingle Blerta Lindqvist
On the Adversarial Robustness of Decision Trees and a Symmetry Defense
IEEE Access
Adversarial perturbation attacks
adversarial robustness
equivariance
gradient-boosting decision trees
invariance
symmetry defense
title On the Adversarial Robustness of Decision Trees and a Symmetry Defense
title_full On the Adversarial Robustness of Decision Trees and a Symmetry Defense
title_fullStr On the Adversarial Robustness of Decision Trees and a Symmetry Defense
title_full_unstemmed On the Adversarial Robustness of Decision Trees and a Symmetry Defense
title_short On the Adversarial Robustness of Decision Trees and a Symmetry Defense
title_sort on the adversarial robustness of decision trees and a symmetry defense
topic Adversarial perturbation attacks
adversarial robustness
equivariance
gradient-boosting decision trees
invariance
symmetry defense
url https://ieeexplore.ieee.org/document/10843676/
work_keys_str_mv AT blertalindqvist ontheadversarialrobustnessofdecisiontreesandasymmetrydefense