Allerton 2015 Paper Abstract


Paper ThB2.3

Makur, Anuran (Massachusetts Institute of Technology), Kozynski, Fabián (Massachusetts Institute of Technology), Huang, Shao-Lun (Massachusetts Institute of Technology), Zheng, Lizhong (Massachusetts Institute of Technology)

An Efficient Algorithm for Information Decomposition and Extraction

Scheduled for presentation during the Invited Session "Information Theory and Applications" (ThB2), Thursday, October 1, 2015, 11:10−11:30, Solarium

53rd Annual Allerton Conference on Communication, Control, and Computing, Sept 29-Oct 2, 2015, Allerton Park and Retreat Center, Monticello, IL, USA

This information is tentative and subject to change. Compiled on December 5, 2021

Keywords Statistical Signal Processing, Information Theory


The Hirschfeld-Gebelein-Rényi maximal correlation is a well-known measure of statistical dependence between two (possibly categorical) random variables. In inference problems, the maximal correlation functions can be viewed as so called features of observed data that carry the largest amount of information about some latent variables. These features are in general non-linear functions, and are particularly useful in processing high-dimensional observed data. The alternating conditional expectations (ACE) algorithm is an efficient way to compute these maximal correlation functions. In this paper, we use an information theoretic approach to interpret the ACE algorithm as computing the singular value decomposition of a linear map between spaces of probability distributions. With this approach, we demonstrate the information theoretic optimality of the ACE algorithm, analyze its convergence rate and sample complexity, and finally, generalize it to compute multiple pairs of correlation functions from samples.



All Content © PaperCept, Inc..

This site is protected by copyright and trademark laws under US and International law.
All rights reserved. © 2002-2021 PaperCept, Inc.
Page generated 2021-12-05  08:55:48 PST  Terms of use