Cycle consistent twin energy-based models for image-to-image translation

Tiwary, P and Bhattacharyya, K and Prathosh, AP (2024) Cycle consistent twin energy-based models for image-to-image translation. In: Medical Image Analysis, 91 .

PDF med_img_ana_91_2024.pdf - Published Version Restricted to Registered users only Download (3MB)

Abstract

Domain shift refers to change of distributional characteristics between the training (source) and the testing (target) datasets of a learning task, leading to performance drop. For tasks involving medical images, domain shift may be caused because of several factors such as change in underlying imaging modalities, measuring devices and staining mechanisms. Recent approaches address this issue via generative models based on the principles of adversarial learning albeit they suffer from issues such as difficulty in training and lack of diversity. Motivated by the aforementioned observations, we adapt an alternative class of deep generative models called the Energy-Based Models (EBMs) for the task of unpaired image-to-image translation of medical images. Specifically, we propose a novel method called the Cycle Consistent Twin EBMs (CCT-EBM) which employs a pair of EBMs in the latent space of an Auto-Encoder trained on the source data. While one of the EBMs translates the source to the target domain the other does vice-versa along with a novel consistency loss, ensuring translation symmetry and coupling between the domains. We theoretically analyze the proposed method and show that our design leads to better translation between the domains with reduced langevin mixing steps. We demonstrate the efficacy of our method through detailed quantitative and qualitative experiments on image segmentation tasks on three different datasets vis-a-vis state-of-the-art methods. © 2023 Elsevier B.V.

Item Type:	Journal Article
Publication:	Medical Image Analysis
Publisher:	Elsevier B.V.
Additional Information:	The copyright for this article belongs to elsevier publishers.
Keywords:	Medical imaging, Energy-based models; Generative model; Image domain; Image translation; Imaging modality; Learning tasks; Measuring device; Medical image segmentation; Model-based OPC; Performance, Image segmentation, article; autoencoder; diagnosis; energy; human; image segmentation; learning; staining; twins
Department/Centre:	Division of Electrical Sciences > Electrical Communication Engineering
Date Deposited:	29 Feb 2024 05:40
Last Modified:	29 Feb 2024 05:40
URI:	https://eprints.iisc.ac.in/id/eprint/83718

Actions (login required)

View Item