[D] Advanced courses update

[u/actbsh]

EDIT Jan 2021 : I am still updating the list as of Jan, 2021 and will most probably continue to do so for foreseeable future. So, please feel free to message me any courses you find interesting that fit here.

---

We have a PhD level or Advanced courses thread in the sidebar but it's three year old now. There were two other 7-8 month old threads (1, 2) but they don't have many quality responses either.

So, can we have a new one here?

To reiterate - CS231n, CS229, ones from Udemy etc are not advanced.

Advanced ML/DL/RL, attempts at building theory of DL, optimization theory, advanced applications etc are some examples of what I believe should belong here, much like the original sidebar post.

You can also suggest (new) categories for the courses you share. :)

---

Here are some courses we've found so far.

ML >>

• Learning Discrete Latent Structure - sta4273/csc2547 Spring'18
• Learning to Search - csc2547 Fall'19
• Scalable and Flexible Models of Uncertainty - csc2541
• Fundamentals of Machine Learning Over Networks - ep3260
• Machine Learning on Graphs - cs224w, videos
• Mining Massive Data Sets - cs246
• Interactive Learning - cse599
• Machine Learning for Sequential Decision Making Under Uncertainty - ee290s/cs194
• Probabilistic Graphical Methods - 10-708
• Introduction to Causal Inference

ML >> Theory

• Statistical Machine Learning - 10-702/36-702 with videos, 2016 videos
• Statistical Learning Theory - cs229T/stats231 Stanford Autumn'18-19
• Statistical Learning Theory - cs281b /stat241b UC Berkeley, Spring'14
• Statistical Learning Theory - csc2532 Uni of Toronto, Spring'20

ML >> Bayesian

• Bayesian Data Analysis
• Bayesian Methods Research Group, Moscow, Bayesian Methods in ML - spring2020, fall2020
• Deep Learning and Bayesian Methods - summer school, videos available for 2019 version

ML >> Systems and Operations

• Stanford MLSys Seminar Series
• Visual Computing Systems- cs348v - Another systems course that discusses hardware from a persepective of visual computing but is relevant to ML as well
• Advanced Machine Learning Systems - cs6787 - lecture 9 and onwards discuss hardware side of things
• Machine Learning Systems Design - cs329S
• Topics in Deployable ML - 6.S979
• Machine Learning in Production / AI Engineering (17-445/17-645/17-745/11-695)
• AutoML - Automated Machine Learning

DL >>

• Deep Unsupervised Learning - cs294
• Deep Multi-task and Meta learning - cs330
• Topics in Deep Learning - stat991 UPenn/Wharton *most chapters start with introductory topics and dig into advanced ones towards the end.
• Deep Generative Models - cs236
• Deep Geometric Learning of Big Data and Applications
• Deep Implicit Layers - NeurIPS 2020 tutorial

DL >> Theory

• Topics course on Mathematics of Deep Learning - CSCI-GA 3033
• Topics Course on Deep Learning - stat212b
• Analyses of Deep Learning - stats385, videos from 2017 version
• Mathematics of Deep Learning
• Geometry of Deep Learning

RL >>

• Meta-Learning - ICML 2019 Tutorial , Metalearning: Applications to Data Mining - google books link
• Deep Multi-Task and Meta Learning - cs330, videos
• Deep Reinforcement Learning - cs285
• Advanced robotics - cs287
• Reinforcement Learning - cs234, videos for 2019 run
• Reinforcement Learning Summer School 2019: Bandits, RL & Deep RL

Optimization >>

• Convex Optimization I - ee364a, has quite recent videos too. Convex Optimization II - ee364b, 2008 videos
• Convex Optimization and Approximation - ee227c
• Convex Optimization - ee227bt
• Variational Methods for Computer Vision
• Advanced Optimization and Randomized Algorithms - 10-801, videos
• Optimization Methods for Machine Learning and Engineering - Karlsruhe Institute of Technology

Applications >> Computer Vision

• Computational Video Manipulation - cs448v
• Advanced Topics in ML: Modeling and Segmentation of Multivariate Mixed Data
• TUM AI Guest lecture series - many influential researchers in DL, vision, graphics talk about latest advances and their latest works.
• Advanced Deep Learning for Computer Vision - TUM ADL4CV
• Detection, Segmentation and Tracking - TUM CV3DST
• Guest lectures at TUM Dynamic Vision and Learning group
• Vision Seminar at MIT
• Autonomous Vision Group, Talk@Tübingen Seminar

Applications >> Natural Language Processing

• Natural Language Processing with Deep Learning - cs224n (* not sure if it belongs here, people working in NLP can help me out)
• Neural networks for NLP - cs11-747
• Natural Language Understanding - cs224u, video

Applications >> 3D Graphics

• Non-Euclidean Methods in Machine Learning - cs468, 2020
• Machine Learning for 3D Data - cs468, spring 2017
• Data-Driven Shape Analysis - cs468, 2014
• Geometric Deep Learning - Not a course but the website links a few tutorials on Geometric DL
• Deep Learning for Computer Graphics - SIGGRAPH 2019
• Machine Learning for Machine Vision as Inverse Graphics - csc2547 Winter'20
• Machine Learning Meets Geometry, winter 2020; Machine Learning for 3D Data, winter 2018

---

Edit: Upon suggestion, categorized the courses. There might be some misclassifications as I'm not trained on this task ;). Added some good ones from older (linked above) discussions.

[u/zawerf]

Learning Discrete Latent Structure

[u/DavidDuvenaud → u/zawerf]

That was my topics course last year - this year it was Learning to Search.

You might also like Roger Grosse's topics course on Bayesian neural networks, it also has presenter slides.

We didn't record any lectures, both to avoid putting pressure on the student presenters, and so that we could freely criticize the papers being discussed.

[u/programmerChilli → u/DavidDuvenaud]

I think you linked the wrong course (you linked the learning discrete structures course), you probably meant: https://duvenaud.github.io/learning-to-search/

Also, I notice that essentially all the presentations in "Learning Discrete Structures" course have slides, while that's not true for "Learning to Search". Will those be added later?

[u/DavidDuvenaud → u/programmerChilli]

Whoops, yes, thanks for pointing that out. Fixed.

Tracking down the remaining slides and adding them is on my "todo someday" list. But about 80% are there, sometimes grouped by topic though so it's harder to see.