Computational Harmonic Analysis References Page (Winter,
2002)
Course: MAT 280
CRN: 61380
Title: Computational Harmonic Analysis
Class: MW 5:40pm-7:00pm, 693 Kerr
Instructor: Naoki Saito
Office: 675 Kerr
Phone: 754-2121
Email:saito@math.ucdavis.edu
Office Hours: TTh 2:00pm-3:00pm or by appointment via
email
The following references are useful and contains much
more details of the topics covered or referred to in my lectures.
I strongly encourage you to take a look at some
of them.
Lecture 1: Overture and Motivation
Lecture 2: What is a Signal? Basics of the
Fourier Transforms
- H. Dym & H. McKean: Fourier Series and Integrals
Fourier Series and Wavelets , Academic Press, 1972. Chap.
2.
- G. B. Folland: Fourier Analysis and Its Applications
, Wadsworth & Brooks/Cole, 1992. Chap. 7.
For quantization, which will not be discussed in this course,
see
R. M. Gray and D. L. Neuhoff: "Quantization," IEEE
Trans. Inform. Theory , vol. 44, no.6, pp.2325-2383, 1998.
Lecture 3: Basics of the Fourier Transforms II:
L2 Theory, The Heisenberg Uncertainty Principles
Basics:
Details of L2 theory:
- G. B. Folland: Real Analysis, 2nd Ed., Wiley
Interscience, 1999. Chap. 8.
- E. M. Stein & G. L. Weiss: Introduction to Fourier
Analysis on Euclidean Spaces, Princeton Univ. Press, 1970. Chap.
1.
Survey on the uncertainty principle (advanced):
- G. B. Folland & A. Sitaram: "The uncertainty principle:
A mathematical survey," Journal of Fourier Analysis and Applications
, vol.3, no.3, pp.207-238, 1999.
Lecture 4: Bandlimited Functions, Prolate Spheroidal
Wave Functions, and Sampling Theorems
Prolate
Spheroidal Wave Functions:
-
D. Slepian: "Some comments on Fourier analysis, uncertainty
and modeling," SIAM Review, vol. 25, pp.379-393, 1983.
- H. J. Landau: "An overview of time and frequency limiting,"
in Fourier Techniques and Applications ( J. F. Price, ed.), pp.201-220,
Plenum Press, New York, 1985.
- D. Slepian: "On bandwidth," Proc. IEEE, vol.63,
no.4, pp.292-300, 1976.
- D. Slepian & H. O. Pollak: "Prolate spheroidal
wave functions, Fourier analysis and uncertainty-I," Bell Syst.
Tech. J., vol.40, pp.43-63, 1961.
- H. J. Landau & H. O. Pollak: "Prolate spheroidal
wave functions, Fourier analysis and uncertainty-II," Bell Syst.
Tech. J. , vol.40, pp.65-83, 1961.
- H. J. Landau & H. O. Pollak: "Prolate spheroidal
wave functions, Fourier analysis and uncertainty-III: The dimension
of
the space of essentially time- and band-limited signals," Bell
Syst. Tech. J., vol.41, pp.1295-1336, 1962.
- D. Slepian: "Prolate spheroidal wave functions, Fourier
analysis and uncertainty-IV: Extensions to many
dimensions; generalized prolate spheroidal functions," Bell
Syst. Tech. J., vol.43, pp.3009-3057, 1964.
- D. Slepian: "Prolate spheroidal wave functions, Fourier
analysis and uncertainty-V: The discrete case," Bell Syst. Tech.
J., vol.57, pp.1371-1430, 1978.
Sampling Theorems:
- R. N. Bracewell: The Fourier Transform and Its Applications
, 2nd Ed., Revised, McGraw-Hill, 1987. Chap. 10.
- W. L. Briggs & V. E. Henson: The DFT: An Owner's
Manual for the Discrete Fourier Transform, SIAM, 1995. Sec. 3.4,
Chap. 6.
For more details on Sampling Theorems and Non-Uniform Sampling
Schemes, see:
- H. J. Landau: "Sampling, data transmission, and the
Nyquist rate," Proc. IEEE, vol.55, no.10, pp.1701-1706,
1967.
-
A. Aldroubi and K. Gröchenig, "Nonuniform sampling and
reconstruction in shift-invariant spaces," SIAM Review, vol.43,
no.4, pp.585-620, 2001.
For the historical articles on the sampling theorems, see:
- E. T. Whittaker: "On the functions which are represented
by the expansions of the interpolation-theory," Proc. Royal Soc.
Edinburgh , Sec. A, vol.35, pp.181-194, 1915.
- C. E. Shannon: "Communication in the presence of
noise," Proc. IRE, vol.37, pp.10-21, 1949.
Lecture 5: Fourier Series; Periodization vs
Sampling
Basic Fourier Series Theory:
- Dym & McKean, Chap. 1.
- Folland: Fourier Analysis, Chap. 2.
Generalized Functions (Distributions):
For the other stuff I menioned in the class, the details
can be found as follows:
Spherical Harmonics:
Orthogonal Polynomials:
- G. Szegö: Orthogonal Polynomials, 4th
Ed., AMS, 1975.
- Folland: Fourier Analysis, Chap. 6.
Lecture 6: Discrete Fourier Transform
For advanced topics on DFT, check out the following articles
and book:
- J. H. McClellan ad T. W. Parks: "Eigenvalue and eigenvector
decomposition of the discrete Fourier transform," IEEE Trans. Audio
and Electroacoustics, vol.AU-20, no.1, pp.66-74, 1972 (with comments,
vol.AU-21, pp.65, 1973).
- L. Auslander and R. Tolimieri: "Is computing with
the finite Fourier transform pure or applied mathematics?" Bull.
AMS , vol.1, no.6, pp.847-897, 1979.
- A. Terras: Fourier Analysis on Finite Groups and
Applications , London Mathematical Society Student Texts vol.43,
Cambridge Univ. Press, 1999.
Lecture 7: Fast Fourier Transform (FFT)
There are many references on FFT, but the following are particularly
useful:
For advanced and thought-provoking aspects of FFT, read:
The history of the FFT is another very interesting subject. See
e.g.,
- M. T. Heideman, D. H. Johnson, and C. S. Burrus: "Gauss
and the history of the fast Fourier transform," Arch. Hist. Exact
Sciences , vol.34, pp.265-277, 1985.
Lecture 8: Basic Sturm-Liouville Theory
- Folland: Fourier Analysis: Sec.3.5, 3.6, 7.4.
- Dym & McKean: Sec. 1.7, 1.9.
For more details, see e.g.,
- R. Courant & D. Hilbert: Methods of Mathematical
Physics, Vol. I, First English Edition, John Wiley & Sons, 1953.
Republished as Wiley Classics Library in 1989. See Chap. V
in particular.
Lecture 9: Discrete Cosine &
Sine Transforms
- N. Ahmed, T. Natarayan, and K. R. Rao: "Discrete cosine
transform," IEEE Trans. Comput., vol.COM-23, pp.90-93,
1974.
- K. R. Rao and P. Yip: Discrete Cosine Transform: Algorithms,
Advantages, and Applications, Academic Press, 1990.
-
G. Strang: "The discrete cosine transform," SIAM Review,
vol.41, no.1, pp.135-147, 1999.
- M. V. Wickerhauser: Adapted Wavelet Analysis from Theory
to Software, A K Peters, Ltd., 1994. Chap. 3.
Lecture 10: Karhunen-Loè
ve Expansion
Discrete version (aka Principal Component Analysis [PCA]):
- K. Fukunaga: Introduction to Statistical Pattern Recognition
, 2nd Edition, Academic Press, 1990. Chap. 9 & Appendix
A.
- K. V. Madia, J. T. Kent, and J. M. Bibby: Multivariate
Analysis , Academic Press, 1979. Chap. 8.
-
N. Saito: "Image approximation and modeling via least statistically
dependent bases, Pattern Recognition, vol.34, no.9, pp.1765-1784, 2001.
- S. Watanabe: "Karhunen-Loève expansion and factor
analysis: Theoretical remarks and applications," Trans. 4th Prague
Conf. Inform. Theory, Statist. Decision Functions, Random Processes
, Publishing House of the Czechoslovak Academy of Sciences, Prague,
pp.635-660, 1965.
Continuous version:
- U. Grenander: Stochastic processes and statistical inference,
Arkiv för Matematik, vol.1, pp.195-277, 1950.
- W. B. Davenport and W. L. Root: An Introduction to the
Theory of Random Signals and Noise, McGraw Hill, 1958, republished by IEEE
Press, 1987. Chap. 6.
- W. D. Ray and R. M. Driver: "Further decomposition of
the Karhunen-Loève series representation of a stationary random
process," IEEE Trans. Inform. Theory, vol.IT-15, no.6,
pp.663-668, 1970.
See also:
- F. Riesz and B. Sz.-Nagy: Functional Analysis,
Frederic Ungar, 1950, republished by Dover, 1990. Chap. VI.
- R. Courant & D. Hilbert: Methods of Mathematical
Physics, Vol. I, First English Edition, John Wiley & Sons, 1953.
Republished as Wiley Classics Library in 1989. Chap. 3.
Applications (too numerous to list all):
Signals:
Images:
Lecture 11: KLT and DCT
Relationship with DCT:
- R. J. Clarke: "Relation between the Karhunen Loève
and cosine transforms," IEE Proc., vol.128, Part F, no.6, pp.359-360,
1981.
On Toeplitz matrices:
- U. Grenander and G. Szegö: Toeplitz Forms and Their
Applications , 2nd Ed., AMS-Chelsea, 1984.
-
R. M. Gray: Toeplitz and circulant matrices: A review, Technical Report,
Information Systems Laboratory, Department of Electrical Engineering,
Stanford University, 2000.
- H. Widom: "Toeplitz matrices," in Studies in Real and
Complex Analysis (I. I. Hirschman, Jr. ed.), MAA Studies in Mathematics,
1965.
Prolate Spheroidal Wave Functions:
- D. Slepian: "Estimation of signal parameters in the presence
of noise," Trans. IRE Professional Group on Information Theory
, PGIT-3, pp.68-89, 1954.
Ramp Process:
- Y. Meyer: Oscillating Patterns in Image Processing and
Nonlinear Evolution Equations, University Lecture Series, vol.22,
AMS, 2001, Sec.1.8-1.10.
Manifold Learning:
Lecture 12: Time-Frequency Analysis and Synthesis
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd
Ed., Academic Press, 1999. Chap. 4.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.
Chap. 2.
Some historical papers:
- D. Gabor: "Theory of communication," J. IEE (London),
vol.93, pp.429-457, 1946.
- J. Ville: "Théorie et applications de la notion de
signal analytique," Cables et Transmissions, 2ème A, no.1,
pp.61-74, 1948.
Lecture 13: Windowed (or Short-Time) Fourier Transform
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd
Ed., Academic Press, 1999. Chap. 4.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.
Chap. 2.
-
I. Daubechies: "The wavelet-transform, time-frequency localization and
signal analysis," IEEE Trans. Inform. Theory, vol.36, pp.961-1005,
1990.
Lecture 14: Introductory Frame Theory and the Balian-Low
Theorem
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd
Ed., Academic Press, 1999. Chap. 5.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.
Chap. 3, 4.
- J.-P. Kahan and P.-G. Lemarié-Rieusset: Fourier Series
and Wavelets, Studies in the Development of Modern Mathematcis, Vol.
3, Gordon and Breach Publishers, 1995. Chap. 1 of the Wavelet portion.
-
Course handout on the Balian-Low theorem.
See also the following original papers:
- R. Balian: "Un principe d'incertitude fort en théorie
du signal ou en mécanique quantique," C. R. Acad. Sci. Paris
, vol.292, pp.1357-1362, 1981.
- M. J. Bastiaans: "Gabor's signal expansion and degrees of freedom
of a signal," Proc. IEEE, vol.68, pp.538-539, 1980.
- G. Battle: "Heisenberg proof of the Balian-Low theorem,"
Lett. Math. Phys., vol.15, pp.175-177, 1988.
-
I. Daubechies: "The wavelet transform, time-frequency localization and
signal analysis," IEEE Trans. Inform. Theory, vol.36, pp.961-1005,
1990.
-
I. Daubechies and A. J. E. M. Janssen: "Two theorems on lattice expansions,"
IEEE Trans. Inform. Theory, vol.39, pp.3-6, 1993.
Lecture 15: Continuous Wavelet Transform
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed.,
Academic Press, 1999. Chap. 4, 5.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.
Chap. 3, 4.
- M. Holschneider: Wavelets: An Analysis Tool, Clarendon Press,
Oxford, 1995. Chap. 1.
Lecture 16: Wavelet Frames; Discrete Wavelet Transform
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed.,
Academic Press, 1999. Chap. 4, 5.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992. Chap.
3, 4.
- M. Holschneider: Wavelets: An Analysis Tool, Clarendon Press,
Oxford, 1995. Chap. 1.
- J.-P. Kahan and P.-G. Lemarié-Rieusset: Fourier Series
and Wavelets, Studies in the Development of Modern Mathematcis, Vol.
3, Gordon and Breach Publishers, 1995. Chap. 2 of the Wavelet portion.
Analytic Signals:
- A. Papoulis: Signal Analysis, McGraw-Hill, 1977. Sec.4-2.
Lecture 17: Discrete Wavelet Transform II; Local Trigonometric
Transform
Discrete Wavelet Transforms, Multiresolution Analysis:
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed.,
Academic Press, 1999. Chap. 7.
- I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992. Chap.
4, 5.
- J.-P. Kahan and P.-G. Lemarié-Rieusset: Fourier Series
and Wavelets, Studies in the Development of Modern Mathematcis, Vol.
3, Gordon and Breach Publishers, 1995. Chap.3-5 of the Wavelet portion.
Local Trigonometric Transform:
- S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed.,
Academic Press, 1999. Chap. 8.
- M. V. Wickerhauser: Adapted Wavelet Analysis from Theory
to Software, A K Peters, Ltd., 1994. Chap. 4.
-
P. Auscher and G. Weiss and M. V. Wickerhauser: "Local sine and cosine bases
of Coifman and Meyer and the construction of smooth wavelets", in Wavelets:
A Tutorial in Theory and Applications (C. K. Chui, ed.), Academic Press,
1992, pp.237-256.
Please email
me
if you have any comments or questions!
Go back to Computational Harmonic Analysis home page