Mathematical Foundations for Data Science (Spring 2019)

• Lecture 0 (Introduction).
• Lectures 1 and 2 (Curses, Blessings, Surprises, basic probability) here and here.
• Lectures 3 and 4: (PCA and SVD) here.
• Lectures 5, 6, and 7: (Deep Learning) A nice introduction to Deep Learning, well suited for applied mathematicians.
• Lectures 8, 9, and 10: (Clustering, Diffusion maps, PageRank) here and here.
• Lectures 11 and 12: (Johnson-Lindenstrauss and variations) here
• Lectures 13: (Randomized algorithms) here
• Lectures 14, 15 and 16: (Compressive sensing and sparsity) here

Homework:

• Homework 1 .
• Solutions to Homework 1 .
  Files needed for this homework:
  • File with face images.
  • showfaces.m.
  • acropolis.mat.

• Homework 2 .

• Homework 3.
  
  Files needed for this homework:
  • iris.data.txt
  • crescents.mat
  • Trajectories.tar.gz
• Homework 4 .

Relevant papers, software, and interesting links:

Data Sets:
• The UC Irvine Machine Learning Repository is a good source for interesting datasets.

Artificial Intelligence: Issues and Challenges:
• AI will cause some major changes in our society. To get a better understanding of these potential changes, their challenges and prospects, I highly recommend this book The Second Machine Age, by Erik Brynjolfsson and Andrew McAfee (Chapter 1 is available for free on their website) and the article Big other: surveillance capitalism and the prospects of an information civilization, by Shoshana Zuboff.

General Data Science:
• D. Donoho: High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality.
• D. Donoho: 50 years of Data Science.
• An interesting article by Michael Jordan on the state of AI.
• A list of Big Data Startups
• Kaggle: A website for data science competitions.

Probability, concentration of measure, random matrices:
• High-dimensional Probability, an excellent book by Roman Vershynin. You can download a draft from this link for free.

Deep Learning/Classification:
• A review paper about Deep Learning by Turing Prize Winners Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.
• A review paper about Deep Learning by Juergen Schmidhuber. (He and Sepp Hochreiter also belong to the pioneers of deep learning.)
• A nice introduction to Deep Learning, well suited for applied mathematicians.

Clustering
• A review of k-means and other clustering algorithms.
• A tutorial on spectral clustering. Diffusion maps.
• Here is a software package, called cvx, to solve convex (including SDP) problems, written by Boyd and Vanderberghe. It is not suitable for large scale problems, but on the upside it is very, very easy to use. Here is the Python version and here is the Matlab version.

Linear dimension reduction:
• Fast Johnson-Lindenstrauss projections
• Johnson-Lindenstrauss projections and RIP

Randomized Algorithms:
• A review of some algorithms from randomized linear algebra.
• Another review of some algorithms from randomized linear algebra.

Compressive sensing:
• A paper by E.Candes that proves recovery of sparse signals with l1 minimization and RIP for noisy and approximately sparse signals.
• A very nice talk by E.Candes about compressive sensing.
• A paper by Bandeira et al. on the difficulty of constructing good deterministic RIP matrices.
• A paper about constructing optimal incoherent line sets and associated theoretical bounds.

Low-rank matrix recovery and variations:
• A nice paper on low-rank matrix completion.
• Another paper on low-rank matrix completion.
• Yet another paper on low-rank matrix completion.
• Robust PCA.