Learning of discrete graphical models with neural networks

Abhijith, J and Lokhov, AY and Misra, S and Vuffray, M (2020) Learning of discrete graphical models with neural networks. In: 34th Conference on Neural Information Processing Systems, NeurIPS 2020, 6-12 Dec., 2020, virtual.

PDF NeurIPS_2020_Abhijith etal.pdf - Published Version Restricted to Registered users only Download (731kB) | Request a copy

Abstract

Graphical models are widely used in science to represent joint probability distributions with an underlying conditional dependence structure. The inverse problem of learning a discrete graphical model given i.i.d samples from its joint distribution can be solved with near-optimal sample complexity using a convex optimization method known as Generalized Regularized Interaction Screening Estimator (GRISE). But the computational cost of GRISE becomes prohibitive when the energy function of the true graphical model has higher order terms. We introduce NeurISE, a neural net based algorithm for graphical model learning, to tackle this limitation of GRISE. We use neural nets as function approximators in an Interaction Screening objective function. The optimization of this objective then produces a neural-net representation for the conditionals of the graphical model. NeurISE algorithm is seen to be a better alternative to GRISE when the energy function of the true model has a high order with a high degree of symmetry. In these cases NeurISE is able to find the correct parsimonious representation for the conditionals without being fed any prior information about the true model. NeurISE can also be used to learn the underlying structure of the true model with some simple modifications to its training procedure. In addition, we also show a variant of NeurISE that can be used to learn a neural net representation for the full energy function of the true model. © 2020 Neural information processing systems foundation. All rights reserved.

Item Type:	Conference Paper
Publication:	Advances in Neural Information Processing Systems
Publisher:	Neural information processing systems foundation
Additional Information:	The copyright for this article belongs to Neural information processing systems foundation
Keywords:	Convex optimization; Graphic methods; Inverse problems; Learning systems; Probability distributions; Screening, Computational costs; Conditional dependence; Convex optimization methods; Function approximators; Joint distributions; Joint probability distributions; Simple modifications; Training procedures, Neural networks
Department/Centre:	Division of Physical & Mathematical Sciences > Centre for High Energy Physics
Date Deposited:	06 Aug 2021 10:21
Last Modified:	06 Aug 2021 10:21
URI:	http://eprints.iisc.ac.in/id/eprint/69041

Actions (login required)

View Item