Table of Contents

Machine Learning/AI

General AI

• MIT - Artificial General Intelligence

Useful Math for Machine Learning

• Online Courses
  • Statistical Thinking and Inference
    • Introduction to Computational Thinking and Data Science - edX - MIT
    • Computational Probability and Inference - edX - MIT
  • Matrix Methods: Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares
    • Courses: EE103 (Stanford) and EE133A (UCLA)
  • The Matrix Calculus You Need For Deep Learning by Jeremy Howard of Fast.AI
  • Nonlinear Dynamics and Chaos by Steven Strogatz of Cornell - Good if want to study RNNs
• Books
  • Probability Theory: The Logic of Science by E. T. Jaynes
  • Information Theory, Inference, and Learning Algorithms by David Mackay
  • Online Book: Mathematical for Machine Learning (By Professors @ Imperial College London)
  • Online Book: Optimization for Machine Learning (By Elad Hazan, Professor @ Princeton, Google Brain Research Scientist)

General Machine Learning

• Online Courses:
  1. Google Machine Learning Crash Course - 15 hours
  2. Stanford Course  - notes, review, problem sets w/ solutions - RECOMMENDED
  3. Oxford - 16 lectures. Slides+Video. Coding+Math HW - (Nando Freitas)
• Book:
  • [2102.05242] Patterns, predictions, and actions: A story about machine learning - Graduate ML Textbook by Ben Recht
  • Kevin Murphy: Machine Learning - RECOMMENDED
  • Patterns, predictions, and actions: A story about machine learning - Moritz Hardt, Benjamin Recht.
  • The Elements of Statistical Learning: Data Mining, Inference, and Prediction
  • Pattern Recognition and Machine Learning: Ch 1-4 (Christopher Bishop)
  • Probabilistic Graphical Models (Daphne Koller)
• Software
  • Pytorch
  • Tensorflow
• Coding Tutorials
  • Google Machine Learning Crash Course - 15 hours
  • MILA PyTorch Tutorials - Introductory Machine Learning Tutorials from Yoshua Bengio’s Lab

Deep Learning

• Book: Deep Learning Book - RECOMMENDED
• Courses:
  1. CS231n Convolutional Neural Networks for Visual Recognition - RECOMMENDED
  2. Stanford: probabilistic graphical models by Stefano Ermon (builds up to first-principles introduction of VAE)
  3. Deep Learning Specialization - deeplearning.ai - less technical but good introduction to Deep Learning from Andrew Ng’s Deep Learning Specialization course
• Useful Notes:
  1. Optimization for deep learning: theory and algorithms
  2. MIT Deep Learning by Lex Friedman - seems like a good resources with code tutorials and video lectures that cover what’s current and the tricks to get things working.

Reinforcement Learning

• Deep RL Bootcamp 2017 (12 lectures)
• Courses:
  • Berkeley CS 294: Deep Reinforcement Learning
  • David Silver’s RL Class
• Books
  1. Richard S. Sutton: Reinforcement Learning: An Introduction - Bible of RL - RECOMMENDED
     • 1st Edition, 1998
     • 2nd Edition, 2018, complete draft available!
     • recommended for deep RL: (1, 3, 4, 6, 9, 10, 11, 13)
     • Suggested follow-up: Algorithms for Reinforcement Learning by Csaba Szepesvari
  2. Dimitri Bertsekas: Neuro-dynamic programming - short and math-focused

Generative Models

• Books:
  • Bayesian Reasoning and Machine Learning
  • Pattern Recognition and Machine Learning
  • Deep Learning Book: Part III
  • Elements of Causal Inference
• Online Courses:
  • Differentiable Inference and Generative Models - covers topics such as VAEs, GANs, Invertible Density Estimation, Autoregressive Models - lists papers and motivations for them for each topic
  • Probabilistic Learning and Reasoning - how to build, fit, and do inference with probabilistic models
  • Learning Discrete Latent Structure - how to learn model structure and represent data using mixed discrete and continuous data structures such as lists of vectors, graphs, or even programs (seems useful for probabilistic programming) - @Toronto by Professor David Duvenaud
  • CS294-158 Deep Unsupervised Learning Spring 2019 - Berkeley Course by Pieter Abbeel

Probabilistic Programming

• Books:
  • Causal Inference in Statistics: A Primer 1st Edition. (Introduction to causality)
• Papers/Tutorials/Reviews:
  • Variational Inference: A Review for Statisticians - by David Blei
• also see Cognitive Science/Neuroscience page - many of those use/teach probabilistic programming
• Software:
  • Pyro by Uber - probabilistic programming atop pytorch

Misc.

1. Graph Neural Neworks: [Geometric Deep Learning](https://geometricdeeplearning.com/) by Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković
2. [Machine Learning on Graphs: A Model and Comprehensive Taxonomy](https://arxiv.org/pdf/2005.03675.pdf)
2. [Meta-Learning Reading List](https://docs.google.com/document/d/1TT-rTb6WWB0hN5coRnGFWvSytk3P912O6jJC5T0Mbn4/edit)

External Sources

• Machine Learning Tutorials by Professor Yanjun Qi of UVA
• Collection of Tutorials on Bayesian Nonparametrics by Professor Peter Orbanz of Columbia
• Machine Learning Videos by Dustrin Tran of Google
• Deep Learning Reading List
• AI Reading List. includes ML, philosophy, logic, etc
• Fast.ai Deep Learning: Part 1
• Fast.ai Deep Learning: Part 2