Upadhya, V and Sastry, PS (2017) Learning RBM with a DC programming Approach. In: 9th Asian Conference on Machine Learning, ACML 2017, 15 - 17 November 2017, Seoul, pp. 498-513.
![]() |
PDF
jou_mac_lea_res_77_498-513_2017.pdf - Published Version Restricted to Registered users only Download (476kB) | Request a copy |
Abstract
By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of the above formulation and the hyperparameters of the two algorithms can be chosen such that the amount of computation per mini-batch is identical. We show that for a given computational budget the proposed algorithm almost always reaches a higher log-likelihood more rapidly, compared to the standard contrastive divergence algorithm. Further, we modify this algorithm to use the centered gradients and show that it is more efficient and effective compared to the standard centered gradient algorithm on benchmark datasets.
Item Type: | Conference Paper |
---|---|
Publication: | Journal of Machine Learning Research |
Publisher: | Microtome Publishing |
Additional Information: | The copyright for this article belongs to the Microtome Publishing. |
Keywords: | Budget control; Machine learning; Maximum likelihood; Stochastic systems, Benchmark datasets; Computational budget; Contrastive divergence; D-C programming; Difference of convex functions; Gradient algorithm; Log-likelihood functions; Maximum likelihood learning, Functions |
Department/Centre: | Division of Electrical Sciences > Electrical Engineering |
Date Deposited: | 22 Jul 2022 10:44 |
Last Modified: | 22 Jul 2022 10:44 |
URI: | https://eprints.iisc.ac.in/id/eprint/74665 |
Actions (login required)
![]() |
View Item |