Sparsity vs. statistical independence from a best-basis viewpoint, (with B. M. Larson and B. Benichou), Wavelet Applications in Signal and Image Processing VIII (A. Aldroubi, A. F. Laine, and M. A. Unser, eds.), Proc. SPIE 4119, pp.474-486, 2000. Invited Paper.


We examine the similarity and difference between sparsity and statistical independence in image representations in a very concrete setting: use the best basis algorithm to select the sparsest basis and the least statistically-dependent basis from basis dictionaries for a given dataset. In order to understand their relationship, we use synthetic stochastic processes (e.g., spike, ramp, and generalized Gaussian processes) as well as the image patches of natural scenes. Our experiments and analysis so far suggest the following: 1) Both sparsity and statistical independence criteria selected similar bases for most of our examples with minor differences; 2) Sparsity is more computationally and conceptually feasible as a basis selection criterion than the statistical independence, particularly for data compression; 3) The sparsity criterion can and should be adapted to individual realization rather than for the whole collection of the realizations to achieve the maximum performance; 4) The importance of orientation selectivity of the local Fourier and brushlet dictionaries was not clearly demonstrated due to the boundary effect caused by the folding and local periodization. These observations seem to encourage the pursuit of sparse representations rather than that of statistically independent representations.

  • Get the full paper: PDF file.
  • Get the official version via doi:10.1117/12.408635.

  • Please email me if you have any comments or questions!
    Go back to Naoki's Publication Page