Robust loss functions under label noise for deep neural networks

Ghosh, A and Kumar, H and Sastry, PS (2017) Robust loss functions under label noise for deep neural networks. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017, 4 - 10 February 2017, San Francisco, pp. 1919-1925.

PDF AAAI_2017- 1919 _1925-2017.pdf - Published Version Restricted to Registered users only Download (871kB) | Request a copy

Abstract

In many applications of classifier learning, training data suffers from label noise. Deep networks are learned using huge training data where the problem of noisy labels is particularly relevant. The current techniques proposed for learning deep networks under label noise focus on modifying the network architecture and on algorithms for estimating true labels from noisy labels. An alternate approach would be to look for loss functions that are inherently noise-tolerant. For binary classification there exist theoretical results on loss functions that are robust to label noise. In this paper, we provide some sufficient conditions on a loss function so that risk minimization under that loss function would be inherently tolerant to label noise for multiclass classification problems. These results generalize the existing results on noise-tolerant loss functions for binary classification. We study some of the widely used loss functions in deep networks and show that the loss function based on mean absolute value of error is inherently robust to label noise. Thus standard back propagation is enough to learn the true classifier even under label noise. Through experiments, we illustrate the robustness of risk minimization with such loss functions for learning neural networks. © Copyright 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org).

Item Type:	Conference Paper
Publication:	31st AAAI Conference on Artificial Intelligence, AAAI 2017
Publisher:	AAAI press
Additional Information:	The copyright for this article belongs to AAAI press.
Keywords:	Artificial intelligence; Backpropagation; Bins; Deep learning; Deep neural networks; Network architecture, Absolute values; Alternate approaches; Binary classification; Classifier learning; Learning neural networks; Loss functions; Multiclass classification problems; Risk minimization, Classification (of information)
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	22 Jul 2022 11:37
Last Modified:	22 Jul 2022 11:37
URI:	https://eprints.iisc.ac.in/id/eprint/74740

Actions (login required)

View Item