Robust Loss Functions for Learning Multi-class Classifiers

Kumar, H and Sastry, PS (2019) Robust Loss Functions for Learning Multi-class Classifiers. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018, 10 October 2018, Miyazaki; Japan, pp. 687-692.

PDF Int_Con_Sys_man_Cyb_2018_687_2018.pdf - Published Version Restricted to Registered users only Download (524kB) | Request a copy

Abstract

Robust learning in presence of label noise is an important problem of current interest. Training data often has label noise due to subjective biases of experts, crowd-sourced labelling or other automatic labelling processes. Recently, some sufficient conditions on a loss function are proposed so that risk minimization under such loss functions is provably tolerant to label noise. The standard loss functions such as cross-entropy or mean-squared error, used for learning neural network classifiers, do not satisfy these conditions. It was shown that a loss function based on mean absolute value of error satisfies the conditions and is also empirically seen to be robust to label noise. However, minimizing absolute value of error is a difficult optimization problem. In this paper we propose a new loss function, called robust log loss and show that it satisfies the sufficient conditions for robustness. The resulting optimization problem of minimizing empirical risk is well behaved. Through extensive empirical results we show that, in terms of accuracy and learning rate, the proposed loss function is as good as cross-entropy loss for learning neural network classifiers when there is no label noise and that it is better when the training data has label noise. © 2018 IEEE.

Item Type:	Conference Proceedings
Publication:	Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	Copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	Cybernetics; Entropy; Errors; Mean square error; Optimization, Automatic labelling; Learning neural networks; Loss functions; Mean squared error; Multi-Class; Multi-class classifier; Optimization problems; Risk minimization, Classification (of information)
Department/Centre:	Division of Electrical Sciences > Electrical Engineering
Date Deposited:	08 Apr 2019 11:47
Last Modified:	08 Apr 2019 11:47
URI:	http://eprints.iisc.ac.in/id/eprint/62021

Actions (login required)

View Item