Gorantla, S and Bhansali, E and Deshpande, A and Louis, A (2024) Optimizing Learning-to-Rank Models for Ex-Post Fair Relevance. In: 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, 14 July 2024through 18 July 2024, Washington, USA, pp. 1525-1534.
![]() |
PDF
SIGIR_2024_10_jul_2024.pdf - Published Version Restricted to Registered users only Download (6MB) | Request a copy |
Abstract
Learning-to-rank (LTR) models rank items based on specific features, aiming to maximize ranking utility by prioritizing highly relevant items. However, optimizing only for ranking utility can lead to representational harm and may fail to address implicit bias in relevance scores. Prior studies introduced algorithms to train stochastic ranking models, such as the Plackett-Luce ranking model, that maximize expected ranking utility while achieving fairness in expectation (ex-ante fairness). Still, every sampled ranking may not satisfy group fairness (ex-post fairness). Post-processing methods ensure ex-post fairness; however, the LTR model lacks awareness of this step, creating a mismatch between the objective function the LTR model optimizes and the one it is supposed to optimize. In this paper, we first propose a novel objective where the relevance (or the expected ranking utility) is computed over only those rankings that satisfy given representation constraints for groups of items. We call this the ex-post fair relevance. We then give a framework for training Group-Fair LTR models to maximize our proposed ranking objective. Leveraging an efficient sampler for ex-post group-fair rankings and efficient algorithms to train the Plackett-Luce LTR model, we demonstrate their use in training the Group-Fair Plackett-Luce model in our framework. Experiments on MovieLens and Kiva datasets reveal improved fairness and relevance with our group-fair Plackett-Luce model compared to post-processing. In scenarios with implicit bias, our algorithm generally outperforms existing LTR baselines in both fairness and relevance. © 2024 Owner/Author.
Item Type: | Conference Paper |
---|---|
Publication: | SIGIR 2024 - Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval |
Publisher: | Association for Computing Machinery, Inc |
Additional Information: | The copyright for this article belongs to the publisher. |
Keywords: | Learning systems; Stochastic systems, Ex antes; Group-fairness; Item-based; Movielens; Objective functions; Postprocessing methods; Rank modeling; Ranking model; Relevance score; Stochastic ranking, Stochastic models |
Department/Centre: | Division of Electrical Sciences > Computer Science & Automation |
Date Deposited: | 28 Aug 2024 07:02 |
Last Modified: | 28 Aug 2024 07:02 |
URI: | http://eprints.iisc.ac.in/id/eprint/86015 |
Actions (login required)
![]() |
View Item |