Regularizer to mitigate gradient masking effect during single-step adversarial training

Vivek, BS and Baburaj, A and Babu, RV (2019) Regularizer to mitigate gradient masking effect during single-step adversarial training. In: 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2019, 16 June 2019 - 20 June 2019, Long Beach, pp. 66-73.

PDF CVPRW_2019.pdf - Published Version Restricted to Registered users only Download (473kB) | Request a copy

Abstract

Neural networks are susceptible to adversarial samples: samples with imperceptible noise, crafted to manipulate network's prediction. In order to learn robust models, a training procedure, called Adversarial Training has been introduced. During adversarial training, models are trained with mini-batch containing adversarial samples. In order to scale adversarial training for large datasets and networks, fast and simple methods (e.g., FGSM:Fast Gradient Sign Method) of generating adversarial samples are used while training. It has been shown that models trained using single-step adversarial training methods (i.e., adversarial samples generated using non-iterative methods such as FGSM) are not robust, instead they learn to generate weaker adversaries by masking the gradients. In this work, we propose a regularization term in the training loss, to mitigate the effect of gradient masking during single-step adversarial training. The proposed regularization term causes training loss to increase when the distance between logits (i.e., pre-softmax output of a classifier) for FGSM and R-FGSM (small random noise is added to the clean sample before computing its FGSM sample) adversaries of a clean sample becomes large. The proposed single-step adversarial training is faster than computationally expensive state-of-the-art PGD adversarial training method, and also achieves on par results.

Item Type:	Conference Paper
Publication:	IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Publisher:	IEEE Computer Society
Additional Information:	The copyright for this article belongs to IEEE Computer Society
Keywords:	Computer vision; Large dataset, Large datasets; Non-iterative method; Regularization terms; Robust models; SIMPLE method; State of the art; Training methods; Training procedures, Iterative methods
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	23 Dec 2022 05:28
Last Modified:	23 Dec 2022 05:28
URI:	https://eprints.iisc.ac.in/id/eprint/78525

Actions (login required)

View Item