# Table of Contents

## Machine Learning/AI

### General AI

### Useful Math for Machine Learning

- Online Courses
- Statistical Thinking and Inference
- Introduction to Computational Thinking and Data Science - edX - MIT
- Computational Probability and Inference - edX - MIT

- Matrix Methods: Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares
- The Matrix Calculus You Need For Deep Learning by Jeremy Howard of Fast.AI
- Nonlinear Dynamics and Chaos by Steven Strogatz of Cornell - Good if want to study RNNs

- Statistical Thinking and Inference
- Books
- Probability Theory: The Logic of Science by E. T. Jaynes
- Information Theory, Inference, and Learning Algorithms by David Mackay
**Online Book: Mathematical for Machine Learning**(By Professors @ Imperial College London)**Online Book: Optimization for Machine Learning**(By Elad Hazan, Professor @ Princeton, Google Brain Research Scientist)

### General Machine Learning

- Online Courses:
- Google Machine Learning Crash Course - 15 hours
**Stanford Course - notes, review, problem sets w/ solutions - RECOMMENDED**- Oxford - 16 lectures. Slides+Video. Coding+Math HW - (Nando Freitas)

- Book:
- [2102.05242] Patterns, predictions, and actions: A story about machine learning - Graduate ML Textbook by Ben Recht
**Kevin Murphy: Machine Learning - RECOMMENDED**- Patterns, predictions, and actions: A story about machine learning - Moritz Hardt, Benjamin Recht.
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- Pattern Recognition and Machine Learning: Ch 1-4 (Christopher Bishop)
- Probabilistic Graphical Models (Daphne Koller)

- Software
**Coding Tutorials**- Google Machine Learning Crash Course - 15 hours
- MILA PyTorch Tutorials - Introductory Machine Learning Tutorials from Yoshua Bengio’s Lab

### Deep Learning

**Book: Deep Learning Book - RECOMMENDED**- Courses:
**CS231n Convolutional Neural Networks for Visual Recognition - RECOMMENDED**- Stanford: probabilistic graphical models by Stefano Ermon (builds up to first-principles introduction of VAE)
- Deep Learning Specialization - deeplearning.ai - less technical but good introduction to Deep Learning from Andrew Ng’s Deep Learning Specialization course

- Useful Notes:
- Optimization for deep learning: theory and algorithms
- MIT Deep Learning by Lex Friedman - seems like a good resources with code tutorials and video lectures that cover what’s current and the tricks to get things working.

### Reinforcement Learning

- Deep RL Bootcamp 2017 (12 lectures)
- Courses:
- Books
**Richard S. Sutton: Reinforcement Learning: An Introduction - Bible of RL - RECOMMENDED**- 1st Edition, 1998
- 2nd Edition, 2018, complete draft available!
- recommended for deep RL: (1, 3, 4, 6, 9, 10, 11, 13)
- Suggested follow-up: Algorithms for Reinforcement Learning by Csaba Szepesvari

- Dimitri Bertsekas: Neuro-dynamic programming - short and math-focused

### Generative Models

- Books:
- Online Courses:
- Differentiable Inference and Generative Models - covers topics such as VAEs, GANs, Invertible Density Estimation, Autoregressive Models - lists papers and motivations for them for each topic
- Probabilistic Learning and Reasoning - how to build, fit, and do inference with probabilistic models
- Learning Discrete Latent Structure - how to learn model structure and represent data using mixed discrete and continuous data structures such as lists of vectors, graphs, or even programs (seems useful for probabilistic programming) - @Toronto by Professor David Duvenaud
- CS294-158 Deep Unsupervised Learning Spring 2019 - Berkeley Course by Pieter Abbeel

### Probabilistic Programming

- Books:
- Causal Inference in Statistics: A Primer 1st Edition. (Introduction to causality)

- Papers/Tutorials/Reviews:
- Variational Inference: A Review for Statisticians - by David Blei

- also see Cognitive Science/Neuroscience page - many of those use/teach probabilistic programming
- Software:
- Pyro by Uber - probabilistic programming atop pytorch

### Misc.

```
1. Graph Neural Neworks: [Geometric Deep Learning](https://geometricdeeplearning.com/) by Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković
2. [Machine Learning on Graphs: A Model and Comprehensive Taxonomy](https://arxiv.org/pdf/2005.03675.pdf)
2. [Meta-Learning Reading List](https://docs.google.com/document/d/1TT-rTb6WWB0hN5coRnGFWvSytk3P912O6jJC5T0Mbn4/edit)
```

## External Sources

- Machine Learning Tutorials by Professor Yanjun Qi of UVA
- Collection of Tutorials on Bayesian Nonparametrics by Professor Peter Orbanz of Columbia
- Machine Learning Videos by Dustrin Tran of Google
- Deep Learning Reading List
- AI Reading List. includes ML, philosophy, logic, etc
- Fast.ai Deep Learning: Part 1
- Fast.ai Deep Learning: Part 2