MAT 135B: Stochastic Processes (Winter 2023)
Course materials

  • Complete lecture notes. Please let me know of any mistakes.

    Sample exams:

  • 2009/10 exams, with solutions: midterm 1, midterm 2, final.
  • 2010/11 exams, with solutions: midterm 1, midterm 2, final.
    MATLAB scripts: runs, empty boxes, Central Limit Theorem, large deviations, gambler's ruin, bold play, n-step transition probabilities,
    division into classes, classes in a random binary matrix, simulation of branching processes, applications of mgf to branching processes,
    invariant distribution, MCMC solution to the knapsack problem, MCMC solution to the traveling salesman problem,
    patterns in coin tosses, problem 2 from HW8, convergence to Poisson process.
    If you are interested in conversion of these to Python scripts, here is the link to the Github site. (Thank you, Justin!)
  • Discussion on Thu., Jan 12, will be a review of probability. Here is a final exam for MAT 135A that can be used as a review; such problems will not be on the exams for this class.
  • Homework 1 (Due Fri., Jan. 20, 2pm, in the Assignments module in Canvas).
    Jan. 17, noon: typo in Problem 2(c) corrected.
    Homework 1 solutions.
  • Homework 2 (Due Fri., Jan. 27, 2pm, in the Assignments module in Canvas).
    Homework 2 solutions.
  • Homework 3 (Due Fri., Feb. 3, 2pm, in the Assignments module in Canvas).
    Homework 3 solutions.
  • Midterm 1. Time and place: Fri., Feb. 3, 2023, in class. Bring a pencil and your university ID.
    This exam covers chapters 9, 10, and 11 of the lecture notes and the first three homework assignments. Topics: indicator trick (i.e., writing a random variable as a sum of indicators), variance-covariance formula, convergence in probability, moment generating functions (incl. large deviation bounds but no central limit theorem), conditional distributions, expectations, computing probabilities and expectations by conditioning (incl. sums with random number of terms). For practice, solve this sample Midterm 1 on your own, then look at the solutions and solve it again. For more practice, repeat this procedure with another sample Midterm 1.

    While you will not be surprised by anything on exams in this course, I recommend you challenge yourself during preparation. Most probability problems are not solvable by following an automatic procedure without any thinking. No interpretation questions will be allowed during exams: in this subject, a proper interpretation of a word problem is a necessary skill.
    Solutions to Midterm 1.

  • Homework 4 (Due Fri., Feb. 10, 2pm, in the Assignments module in Canvas).
    Homework 4 solutions.
    Feb. 7, noon: typo in P of Problem 2, and end of solution to Problem 3 corrected. Feb. 8, 3pm: solution to Problem 2(a) corrected. Feb. 9, 11pm: solution to Problem 3 corrected.
  • Homework 5 (Due Fri., Feb. 17, 2pm, in the Assignments module in Canvas).
    Homework 5 solutions.
    Feb. 17, 2:30pm: solution to Problem 3 corrected (2-n-1 was incorrect, 2-n+1 is correct).
  • Homework 6 (Due Fri., Feb. 24, 2pm, in the Assignments module in Canvas).
    Homework 6 solutions.
    Feb. 21, 11:30am: mistake at the end of solution to Problem 1(a) corrected. Feb. 23, 7:40pm: mistake in Problem 2(b)(c) corrected; the number of dimes received is not X_6 but X_7/5. Thanks to Chengyang for pointing this out!
  • Homework 7 (Due Fri., Mar. 3, 2pm, in the Assignments module in Canvas).
    Homework 7 solutions.
  • Midterm 2. Time and place: Fri., Mar. 3, 2023, in class. It covers chapters 12, 13, 14, and 15 of the book and homework assignments 4, 5, 6, and 7. Topics: Markov chains, transition matrix, n-step transition probabilities, classes, recurrence, transience, aperiodicity, invariant distributions, limiting probabilities, branching processes. For practice, solve this sample Midterm 2 on your own, then look at the solutions and solve it again. For more practice, repeat this procedure with another sample Midterm 2.

    The general guidelines on how to prepare for a probability exam remain the same as for Midterm 1, and the exam administration will also be the same.
    Solutions to Midterm 2

  • Homework 8 (Due Fri., Mar. 10, 2pm, in the Assignments module in Canvas).
    Homework 8 solutions.
  • Final Exam. Time and place: Fri., Mar. 24, 2023, 3:30-5:30, in 146 Olson (the same room where our class meets). Bring a pencil and your university ID.

    Review Session: Thu, Mar. 23, 1:10-3pm, also in 146 Olson.

    Finals Week Office Hours: Wed., Mar. 22, 2:10-3pm, and Thu, Mar. 23, 12:10-1pm, both in my office (3210 Math).

    Material covered on the final: Topics covered by Midterm 1 (convergence in probability, computing probabilities and expectations by conditioning (incl. sums with random number of terms)) and Midterm 2 (Markov chains, transition matrix, n-step transition probabilities, classes, recurrence, transience, aperiodicity, limiting distributions, branching processes), reversible Markov chains, renewal theorem, Poisson process. For practice, solve this sample Final on your own, then look at the solutions and solve it again. For more practice, repeat this procedure with another sample Final.

    The exam will have six problems of the same type and in the same order as on the sample Final, so you will again not be surprised by anything, but general guidelines on challenging yourself during preparation still apply.
    Solutions to Final Exam