ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Regularizer to mitigate gradient masking effect during single-step adversarial training

Vivek, BS and Baburaj, A and Babu, RV (2019) Regularizer to mitigate gradient masking effect during single-step adversarial training. In: 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2019, 16 June 2019 - 20 June 2019, Long Beach, pp. 66-73.

[img] PDF
CVPRW_2019.pdf - Published Version
Restricted to Registered users only

Download (473kB) | Request a copy
Official URL: https://doi.org/10.1109/CVPRW.2019.00014


Neural networks are susceptible to adversarial samples: samples with imperceptible noise, crafted to manipulate network's prediction. In order to learn robust models, a training procedure, called Adversarial Training has been introduced. During adversarial training, models are trained with mini-batch containing adversarial samples. In order to scale adversarial training for large datasets and networks, fast and simple methods (e.g., FGSM:Fast Gradient Sign Method) of generating adversarial samples are used while training. It has been shown that models trained using single-step adversarial training methods (i.e., adversarial samples generated using non-iterative methods such as FGSM) are not robust, instead they learn to generate weaker adversaries by masking the gradients. In this work, we propose a regularization term in the training loss, to mitigate the effect of gradient masking during single-step adversarial training. The proposed regularization term causes training loss to increase when the distance between logits (i.e., pre-softmax output of a classifier) for FGSM and R-FGSM (small random noise is added to the clean sample before computing its FGSM sample) adversaries of a clean sample becomes large. The proposed single-step adversarial training is faster than computationally expensive state-of-the-art PGD adversarial training method, and also achieves on par results.

Item Type: Conference Paper
Publication: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Publisher: IEEE Computer Society
Additional Information: The copyright for this article belongs to IEEE Computer Society
Keywords: Computer vision; Large dataset, Large datasets; Non-iterative method; Regularization terms; Robust models; SIMPLE method; State of the art; Training methods; Training procedures, Iterative methods
Department/Centre: Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited: 23 Dec 2022 05:28
Last Modified: 23 Dec 2022 05:28
URI: https://eprints.iisc.ac.in/id/eprint/78525

Actions (login required)

View Item View Item