Applied & Computational Harmonic Analysis:
Comments, Handouts, and References Page (Winter 2010)
Course: MAT 271
CRN: 63579
Title: Applied & Computational Harmonic Analysis
Class: MW 5:10pm-6:30pm, 2112 Math. Sci. Bldg.
Instructor: Naoki Saito
Office: 2142 MSB
Phone: 754-2121
Email:saito@math.ucdavis.edu
Office Hours: MW 3:10pm-4:00pm, or by appointment
The following references are useful and contains much more
details of the topics covered or referred to in my lectures.
I strongly encourage you to take a look at some of them.
Lecture 1: Overture and Motivation;
What is a Signal?
Good references on basic Fourier analysis, in particular, the Fourier transforms:
- H. Dym & H. McKean: Fourier Series and Integrals, Academic Press, 1972. Chap. 2.
- G. B. Folland: Fourier Analysis and Its Applications,
AMS, 1992. Chap. 7.
- M. A. Pinsky: Introduction to Fourier Analysis and Wavelets,
AMS, 2002. Chap. 2.
For quantization, which will not be discussed in this course due to the
time constraint, see: - R. M. Gray and D. L. Neuhoff: "Quantization," IEEE Trans. Inform.
Theory, vol. 44, no.6, pp.2325-2383, 1998.
- K. Sayood: Introduction to Data Compression, 3rd Ed., Morgan
Kaufmann Publ., 2006. In particular, Chapters 8 & 9.
If you are interested in vision science, I would strongly recommend the following books:
-
D. H. Hubel: Eye, Brain, and Vision, Scientific American Library, 1995.
-
B. A. Wandell: Foundation of Vision, Sinauer Associates, Inc., 1995.
Lecture 2: Basics of the Fourier Transforms
Denoising Enrico Caruso's recording (through the courtesy of Maxim Goldberg) we listened in the class was due to the following paper:
-
J. Berger, M. Goldberg, & R. Coifman:
"Removing Noise from Music Using Local Trigonometric Bases and Wavelet Packets,"
Journal of the Audio Engineering Society, vol.42, no.10, pp.808-818, 1994.
Lecture 3: Fourier Transforms in L2; The Heisenberg Uncertainty Principle
Details of the L2 theory:
- Dym & McKean: Sec. 2.3-2.5.
- Pinsky: Sec. 2.4.
- G. B. Folland: Real Analysis, 2nd Ed., Wiley
Interscience, 1999. Sec. 8.3.
- E. M. Stein & G. L. Weiss: Introduction to Fourier
Analysis on Euclidean Spaces, Princeton Univ. Press, 1970. Sec. 1.2.
Basic references on the Heisenberg inequality/uncertainty principle:
- Dym & McKean: Sec. 2.8.
- Folland: Fourier Analysis, Sec. 7.3.
- Pinsky: Sec. 2.4.3.
Lecture 4: Bandlimited Functions and Sampling Theorems; Periodization vs Sampling; Fourier Series
Sampling Theorems: - R. N. Bracewell: The Fourier
Transform and Its Applications, 2nd Ed., Revised, McGraw-Hill,
1987. Chap. 10.
- W. L. Briggs & V. E. Henson: The DFT: An Owner's Manual
for the Discrete Fourier Transform, SIAM, 1995. Sec. 3.4, Chap. 6.
- Pinsky: Chap.4.
- See also a nice interactive java demonstration by Matt Herman on sampling and aliasing.
For more details on Sampling Theorems and Non-Uniform Sampling Schemes,
see: - H. J. Landau: "Sampling, data transmission, and the Nyquist rate," Proc. IEEE, vol.55, no.10, pp.1701-1706, 1967.
- A. Aldroubi and K. Gröchenig, "Nonuniform sampling and reconstruction in shift-invariant spaces," SIAM Review, vol.43, no.4, pp.585-620, 2001.
For the historical articles on the sampling theorems, see: - E. T.
Whittaker: "On the functions which are represented by the expansions of
the interpolation-theory," Proc. Royal Soc. Edinburgh, Sec. A,
vol.35, pp.181-194, 1915.
- C. E. Shannon: "Communication in the presence of noise," Proc.
IRE, vol.37, pp.10-21, 1949.
- E.Meijering: "A chronology of interpolation: From ancient astronomy to modern signal and image processing," Proc. IEEE, vol.90, no.3, pp.319-342, 2002.
For the other stuff I menioned in the class, the details
can be found as follows:
Orthogonal Polynomials:
- G. Szegö: Orthogonal Polynomials, 4th
Ed., AMS, 1975.
- Folland: Fourier Analysis, Chap. 6.
Lecture 5: Smoothness of Functions and Decay Rate of the
Fourier Coefficients; Functions of Bounded Variation
On the Basel Problem &sum k-2 =
&pi2/6, Euler, different proofs, etc.
- W. Dunham: Euler: The Master of Us All, Math. Assoc. Amer., 1999.
Chap.3.
- R. Ayoub: "Euler and the zeta function," Amer. Math. Monthly, vol.81, no.10, pp.1067-1086, 1974.
- Dan Kalman: "Six ways to sum a series," College Math. Jour., vol.24, no.5, pp.402-421, 1993.
Smoothness Class Hierarchy: - P. J. Davis and P.
Rabinowitz: Methods of Numerical
Integration, Academic Press, 1984 (Reprinted by Dover Pub. Inc., 2007)
Sec. 1.9.
The definition of BV in higher dimensions can be found in: - L. C.
Evans and R. F. Gariepy: Measure Theory and Fine
Properties of
Functions, CRC Press, 1992, Chap.5.
Lecture 6: Functions of Bounded Variations and the Decay Rate of the
Fourier Coefficients II; Fourier Series on Intervals; Discrete Fourier Transform I
Functions of Bounded Variation, the Fourier Coefficients:
- C. Lanczos: Discourse on Fourier Series, Hafner
Publishing Co., New York, 1966. Sec 2. This is the best book on 1D
Fourier series from the applied perspective. Unfortunately, this book
is out of print.
- V. I. Smirnov: A Course of Higher Mathematics, Vol. V,
Pergamon Press, 1964, Chap. 1. This is out of print too.
- M.Taibleson: "Fourier coefficients of functions of bounded variation," Proc. Amer. Math. Soc., vol.18, pp.766, 1967.
Fourier Series on Intervals, Fourier Cosine and Sine Series: -
Folland: Fourier Analysis, Sec. 2.4.
- C. Lanczos: Applied Analysis, Prentice-Hall, Inc., 1956,
Reprinted by Dover, 1988, Sec. 4.5. This book is still in print. I
strongly urge you to buy this book and read it from cover to cover!
Lecture 7: Discrete Fourier Transform II
For advanced topics on DFT, check out the following articles and book:
- J. H. McClellan ad T. W. Parks: "Eigenvalue and eigenvector decomposition of the discrete Fourier transform," IEEE Trans. Audio and
Electroacoustics, vol.AU-20, no.1, pp.66-74, 1972 (with comments, vol.AU-21, no.1, pp.65, 1973).
-
L. Auslander and R. Tolimieri: "Is computing with the finite Fourier transform pure or applied mathematics?" Bull. AMS, vol.1, no.6, pp.847-897, 1979.
- A. Terras: Fourier Analysis on Finite Groups and Applications,
London Mathematical Society Student Texts vol.43, Cambridge Univ.Press,
1999.
Relationship between Fourier Series, DFT, and Trigonometric
Interpolation: - C. Lanczos: Discourse on Fourier Series,
Hafner Publishing Co., New York, 1966. Sec 17.
- I. P. Natanson: "On the convergence of trigonometrical interpolation at
equi-distant knots," Annals of Math., vol.45, no.3, pp.457-471, 1944.
- J. P. Boyd: Chebyshev and Fourier Spectral Methods, 2nd Ed., Dover, 2001. Chapter 4, in particular, Sec.4.5.
- D. M. Young and R. T. Gregory: A Survey of Numerical
Mathematics, Vol. I, Addison-Wesley, 1972, pp.329-339.
Lecture 8: Fast Fourier Transform (FFT)
There are many references on FFT, but the following are particularly
useful: -
J. W. Cooley and J. W. Tukey: "An algorithm for the machine
calculation of complex Fourier series," Math. Comput., vol.19,
pp.297-301, 1965.
- Briggs & Henson: Chap. 10.
- C. F. Van Loan: Computational Frameworks for the Fast Fourier
Transform, SIAM, 1992.
- W. T. Cochran et al.: "What is the Fast Fourier Transform?", Proc. IEEE, vol.55, no.10, pp.1664-1674, 1967.
Perhaps, the best FFT public software package is: - FFTW, which also includes
higher dimensional FFT as well as DCT/DST.
Lecture 9: Discrete Cosine & Sine Transforms (DCT/DST)
For basic and important references on DCT/DST are:
- N. Ahmed, T. Natarayan, and K. R. Rao: "Discrete cosine transform," IEEE Trans. Comput., vol.COM-23, pp.90-93, 1974.
- K. R. Rao and P. Yip: Discrete Cosine Transform: Algorithms,
Advantages, and Applications, Academic Press, 1990.
- G. Strang: "The discrete cosine transform," SIAM Review, vol.41, no.1, pp.135-147, 1999.
- M. V. Wickerhauser: Adapted Wavelet Analysis from Theory to
Software, A K Peters, Ltd., 1994. Chap. 3.
For the Sturm-Liouville Theory I referred to in today's lecture,
the following are nice references:
- Folland: Fourier Analysis: Sec.3.5, 3.6, 7.4.
- Dym & McKean: Sec. 1.7, 1.9.
- R. Courant & D. Hilbert: Methods of Mathematical Physics,
Vol. I, First English Edition, John Wiley & Sons, 1953.
Republished as Wiley Classics Library in 1989. See Chap. V
in particular.
Lecture 10: Karhunen-Loève Transform (KLT)
Discrete version (aka Principal Component Analysis [PCA]): -
K. Fukunaga: Introduction to Statistical Pattern Recognition,
2nd Edition, Academic Press, 1990. Chap. 9, Appendix A.
- K. V. Madia, J. T. Kent, and J. M. Bibby: Multivariate
Analysis, Academic Press, 1979. Chap. 8.
- S. Watanabe: "Karhunen-Loève expansion and factor
analysis: Theoretical remarks and applications," Trans. 4th Prague
Conf. Inform. Theory, Statist. Decision Functions, Random Processes,
Publishing House of the Czechoslovak Academy of Sciences, Prague,
pp.635-660, 1965.
- R. Kannan and S. Vempala:
"Spectral algorithms," Foudations and Trends in Theoretical Computer Science, vol.4, nos.3-4, pp.157-288, 2008.
We only discussed the discrete version in the class, but KLT has its
continuous version. The following are some references: - U.
Grenander: Stochastic processes and statistical inference,
Arkiv för Matematik, vol.1, pp.195-277, 1950.
- W. B. Davenport and W. L. Root: An Introduction to the Theory of
Random Signals and Noise, McGraw Hill, 1958, republished by IEEE
Press, 1987. Chap. 6.
- W. D. Ray and R. M. Driver: "Further decomposition of
the Karhunen-Loève series representation of a stationary random process," IEEE Trans. Inform. Theory, vol.IT-16, no.6, pp.663-668, 1970.
See also: - F. Riesz and B. Sz.-Nagy: Functional Analysis,
Frederic Ungar, 1950, republished by Dover, 1990. Chap. VI.
- Courant & Hilbert: Vol.I, Chap. III.
Please email
me if you have any comments or questions!
Go
back to Applied & Computational Harmonic Analysis home page