Image approximation and modeling via least statistically-dependent bases, Pattern Recognition, vol. 34, no. 9, pp. 1765-1784, 2001.


Statistical independence is one of the most desirable properties of a coordinate system for representing and modeling images. In reality, however, truly independent coordinates may not exist for a given set of images, or it may be too difficult to compute them in practice. Therefore, it makes sense to obtain the least statistically-dependent coordinate system efficiently. To achieve this goal, we use the best-basis algorithm with new criterion that can rapidly select the least statistically-dependent basis (LSDB) from a basis dictionary (e.g., the local cosine or wavelet packet dictionaries) containing a huge number of orthonormal (or biorthogonal) bases. Our new basis selection criterion is minimization of the mutual information of the distributions of the basis coefficients as a measure of statistical dependence, which in turn is equivalent to minimization of the sum of the differential entropy of each coordinate in the basis dictionary. We show that this criterion, combined with the best-basis algorithm, can find the coordinates closest to the statistical independence from all possible bases searchable in a basis dictionary with O(n [log n]^p), where n is the dimensionality of the image (the number of pixels in each image), and p=1 for the wavelet packet dictionaries, and p=2 for the local cosine/sine dictionaries. In this sense, we can view this LSDB algorithm as the best-basis version of the Independent Component Analysis (ICA), which is increasingly gaining popularity. This criterion is different from that of the Joint Best Basis (JBB) proposed by Wickerhauser, which can be viewed as the best-basis version of the Karhunen-Loeve basis (KLB).

We demonstrate the application of the LSDB to image approximation and modeling and compare its performance with that of KLB and JBB using a collection of real geophysical acoustic waveforms and an image database of human faces. For these datasets, the LSDB provides the best approximation in terms of the average relative l^2 errors among various bases including the KLB, JBB, DCT, and wavelet basis. For image modeling, we propose two simple stochastic models for a given class of signals or images based on the LSDB coordinates. The first model is to assume the statistical independence among the LSDB coordinates, which allows us to sample typical coefficients of each coordinate separately using the empirical distribution estimated from the available training coefficients of that coordinate, which in turn easily allows us to simulate new images at our disposal. For the geophysical acoustic waveforms, this first model turned out to be good enough. The second model is based on the "second rotation" by the KLB computed from the top m LSDB coordinates. This model gives us the decorrelated coordinates built on top of the LSDB coordinates. The simulation results on the human face database using the second model suggest that this second rotation can further reduce the statistical dependency among the coordinates, and allows better modeling for a class of complicated images.

  • Get the full paper: PDF file.
  • Get the official version via doi:10.1016/S0031-3203(00)00116-3.

  • Please email me if you have any comments or questions!
    Go back to Naoki's Publication Page