Karmali, T and Parihar, R and Agrawal, S and Rangwani, H and Jampani, V and Singh, M and Babu, RV (2022) Hierarchical Semantic Regularization of Latent Spaces in StyleGANs. In: 17th European Conference on Computer Vision, ECCV 2022, 23 - 27 October 2022, Tel Aviv, pp. 443-459.
|
PDF
ECCV_2022.pdf - Published Version Download (13MB) | Preview |
Abstract
Progress in GANs has enabled the generation of high-resolution photorealistic images of astonishing quality. StyleGANs allow for compelling attribute modification on such images via mathematical operations on the latent style vectors in the W/ W+ space that effectively modulate the rich hierarchical representations of the generator. Such operations have recently been generalized beyond mere attribute swapping in the original StyleGAN paper to include interpolations. In spite of many significant improvements in StyleGANs, they are still seen to generate unnatural images. The quality of the generated images is predicated on two assumptions; (a) The richness of the hierarchical representations learnt by the generator, and, (b) The linearity and smoothness of the style spaces. In this work, we propose a Hierarchical Semantic Regularizer (HSR) (Project Page and code: https://sites.google.com/view/hsr-eccv22 ) which aligns the hierarchical representations learnt by the generator to corresponding powerful features learnt by pretrained networks on large amounts of data. HSR is shown to not only improve generator representations but also the linearity and smoothness of the latent style spaces, leading to the generation of more natural-looking style-edited images. To demonstrate improved linearity, we propose a novel metric - Attribute Linearity Score (ALS). A significant reduction in the generation of unnatural images is corroborated by improvement in the Perceptual Path Length (PPL) metric by 16.19 averaged across different standard datasets while simultaneously improving the linearity of attribute-change in the attribute editing tasks. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
| Item Type: | Conference Paper |
|---|---|
| Publication: | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
| Publisher: | Springer Science and Business Media Deutschland GmbH |
| Additional Information: | The copyright for this article belongs to the Author(S). |
| Keywords: | Image enhancement; Vector spaces, reductions; Hierarchical representation; High resolution; Large amounts of data; Learn+; Mathematical operations; Pathlengths; Photorealistic images; Regularisation; Regularizer, Semantics |
| Department/Centre: | Division of Interdisciplinary Sciences > Computational and Data Sciences |
| Date Deposited: | 09 Jan 2023 09:00 |
| Last Modified: | 09 Jan 2023 09:00 |
| URI: | https://eprints.iisc.ac.in/id/eprint/78930 |
Actions (login required)
 |
View Item |
