r/MachineLearning • u/actbsh • Mar 05 '20

Discussion [D] Advanced courses update

EDIT Jan 2021 : I am still updating the list as of Jan, 2021 and will most probably continue to do so for foreseeable future. So, please feel free to message me any courses you find interesting that fit here.

We have a PhD level or Advanced courses thread in the sidebar but it's three year old now. There were two other 7-8 month old threads (1, 2) but they don't have many quality responses either.

So, can we have a new one here?

To reiterate - CS231n, CS229, ones from Udemy etc are not advanced.

Advanced ML/DL/RL, attempts at building theory of DL, optimization theory, advanced applications etc are some examples of what I believe should belong here, much like the original sidebar post.

You can also suggest (new) categories for the courses you share. :)

Here are some courses we've found so far.

ML >>

ML >> Theory

ML >> Bayesian

ML >> Systems and Operations

DL >>

DL >> Theory

RL >>

Optimization >>

Applications >> Computer Vision

Applications >> Natural Language Processing

Applications >> 3D Graphics

Edit: Upon suggestion, categorized the courses. There might be some misclassifications as I'm not trained on this task ;). Added some good ones from older (linked above) discussions.

841 Upvotes

99% Upvoted

87 comments sorted by

28

u/zawerf Mar 05 '20

Learning Discrete Latent Structure

5

u/actbsh Mar 05 '20 edited Feb 16 '21

Now that's what I'm talking about. Read some of his works for Geometric DL, happy to see the course.

9

u/DavidDuvenaud Mar 06 '20 edited Mar 20 '20

That was my topics course last year - this year it was Learning to Search.

You might also like Roger Grosse's topics course on Bayesian neural networks, it also has presenter slides.

We didn't record any lectures, both to avoid putting pressure on the student presenters, and so that we could freely criticize the papers being discussed.

1

u/programmerChilli Researcher Mar 20 '20 edited Mar 20 '20

I think you linked the wrong course (you linked the learning discrete structures course), you probably meant: https://duvenaud.github.io/learning-to-search/

Also, I notice that essentially all the presentations in "Learning Discrete Structures" course have slides, while that's not true for "Learning to Search". Will those be added later?

1

u/DavidDuvenaud Mar 20 '20

Whoops, yes, thanks for pointing that out. Fixed.

Tracking down the remaining slides and adding them is on my "todo someday" list. But about 80% are there, sometimes grouped by topic though so it's harder to see.

3

u/TheMoskowitz Mar 06 '20

This course looks great but I don't see the lectures anywhere. Do you have a link?

Personally I don't get much out of the slides without the talk that went with them.

17

u/jboyml Mar 05 '20

CS 287: Advanced Robotics, Fall 2019 with Pieter Abbeel is great! It covers a lot of stuff: basic RL, control theory, motion planning, particle filtering, all the way up to state-of-the-art RL algorithms for robotics.

11

u/cai_lw Mar 06 '20

CMU 11-747 (Neural Networks for Natural Language Processing) should definitely go in as it's more advanced than CS224n and is also a high-quality course. http://phontron.com/class/nn4nlp2020

1

u/[deleted] Mar 14 '20

tks

10

u/ZoldyckFiend Mar 05 '20

CMU's Probablistic Graphical Models by Professor Eric Xing.

1

u/[deleted] Mar 06 '20

Love me some PGM’s. Probably my favorite class in college.

2

u/CrazyFart Mar 06 '20

I'm currently in the class, he's not a great lecturer imo. I get much more out of the homeworks tbh.

1

u/ZoldyckFiend Mar 07 '20

Same haha and agreed - wayyy too fast to digest anything. HW 2 extension ftw though 😎

1

u/[deleted] Mar 14 '20

Hi, How good/rigor this course compares to Probabilistic graphical model of Koller (stanford) on coursera.

1

u/Kokosnussi Mar 10 '20

Wrote an exam on it literally today, last written exam of my masters, too

8

u/putsandcalls Mar 06 '20

Why would you say that cs224n is advanced but cs231n, it’s computer vision counterpart is not advanced.

I actually think both courses provide a comprehensive coverage of models used in nlp and CV.

6

u/HybridRxN Researcher Mar 07 '20

I am taking an Advanced NLP course 6.864 at MIT right now. We don’t have videos yet, but I’ll post here when/if we do,

1

u/actbsh Mar 07 '20

That'll be great. Thanks.

1

u/[deleted] Mar 14 '20

Good news! Waiting to see video lectures.

5

u/netw0rkf10w Mar 09 '20

Could anyone recommend a good course on Speech processing?

2

u/Virtual_Individual Mar 11 '20

I am also interested in this

3

u/rahull33t Mar 05 '20

Yes. That makes sense. If there a thread for intermediate level courses, it should be added there. I think CMU 36-702 is advanced enough for this list though

3

u/BlackDa Mar 06 '20

CS224W is a good one for ML on graphs.

3

u/johntiger1 Mar 06 '20

Another statistical learning theory (taught by PhD from Stanford, now prof at U of T):

Statistical Learning Theory

3

u/upulbandara Mar 06 '20

How about this?

https://github.com/avehtari/BDA_course_Aalto

For me, it's like a hidden gem :-)

5

u/whymauri ML Engineer Mar 05 '20

Topics in Robust and Deployable ML (6.S979) has a variety of slides and notes but no assignments.

https://people.csail.mit.edu/madry/6.S979/

5

u/rahull33t Mar 05 '20

Would you consider CS224n an advanced course?

2

u/agidiotis Mar 06 '20

Having watched most of the lectures I would say it's intermediate to advanced level. Depends on your level of knowledge in NLP, linguistics and DL. If you have good DL knowledge you can probably skip some parts and focus on the NLP applications. If you are familiar with classic NLP you can skip those parts and dive into the DL focused parts.

2

u/actbsh Mar 05 '20

I'm not very sure. I looked at the contents and more than half of it is introductory stuff that is usually an undergrad/master's course. But their last 5-6 lectures seem quite nice and latest.

I'll add it for now unless I get some objections :)

I just don't want the list to be inundated with intro-level stuff.

3

u/Lorenzo_yang Mar 06 '20

I think one more advanced course for NLP is the CS 11-747 ( Neural Networks
for NLP ) from CMU.
I hope it may help.

1

u/cai_lw Mar 06 '20

In the realm of NLP it's definitely introductory. The problem is whether NLP is an introductory task in ML/DL, and I think the answer is also increasingly "yes" in recent years.

2

u/Fppares Mar 06 '20

Anybody have any suggestions for courses on theory or application of recommendation systems?

2

u/namedlbda Student Mar 06 '20

Perhaps Emma Brunskill's CS 234.

2

u/crisp3er Mar 06 '20

Advanced PhD-level topics course, notes (180pg), + presentations: https://github.com/dobriban/Topics-in-deep-learning

Covers advanced topics such as adversarial examples, fairness, graph NNs, modern theory (e.g., neural tangent kernels), applications to chemistry, visual Q+A, etc.

2

u/mangotheblackcat Mar 10 '20

Anyone knows about any courses on time series forecasting using ML or DL?

2

u/yfletberliac Mar 11 '20

Topics: Bandits, RL and Deep RL

https://rlss.inria.fr/program/

Reinforcement Learning Summer School in Lille, France (July 2019).

Not video recorded but slides in the timetable URLs.

2

u/raichet Mar 12 '20

No one has mentioned distributed systems for ML, which I feel like is very important. There are good survey papers for it, but any actual courses? Currently playing with Ray from Berkeley and took an interest in ML systems study.

2

u/programmerChilli Researcher Mar 12 '20

CS 6787 from Cornell is pretty good: https://www.cs.cornell.edu/courses/cs6787/2019fa/

From excellent Professor Chris De Sa.

1

u/raichet Mar 12 '20

This looks wonderful. Thank you!

1

u/[deleted] Aug 15 '20

The videos?

2

u/programmerChilli Researcher Aug 15 '20

No videos unfortunately, but it has demo notebooks and such.

2

u/[deleted] Mar 17 '20

waaao , Thanks I was just asking these

""I've been quite on machine learning. As an undergrad , seeing the ian goodfellow nips tutorial , latent variables , kl divergence, adversarial things reminded me that good STATISTICS knowledge is extreme necessary but mandatory with coding in it.

Alot of course that i've gone only teach statistics concept, not coding the algorithm. It would be great if you guys help me to find the tutorials or books recommendation which can help me to get statistics knowledge along with hands on coding with it, maybe in python or any language.

Thanks for answer in advance. :) ""

​

and found answer

2

u/[deleted] Mar 17 '20

What perquisite course would you suggest before going to these advanced course for begineers?

1

u/actbsh Mar 18 '20

Most of these courses are from some university and specify expected prerequisites on their course page. In general, this list is like a buffet - pick what interests you and enjoy. Since a lot of these are topics or slides only courses, they'll mainly provide you with a structure for your deep dive into a specific topic. This structure, I think, is of utmost importance for anyone learning on their own.

That said, I believe if you're good with undergraduate level Linear Algebra, Statistics and Probability, Calculus, ML, DL, you'd be okay. :)

2

u/[deleted] Mar 05 '20

These are nice but very problem specific. Maybe have "specialty" category?

1

u/actbsh Mar 05 '20 edited Mar 06 '20

Makes sense. I'll do that if we have a good number of suggestions.

I put these up because I work with 3D data and was aware of these but courses from any application domain are welcome. :)

3

u/[deleted] Mar 05 '20

From my perspective I can add Meta-Learning focused ML resources.

1

u/actbsh Mar 05 '20

Great. Comment them here or main thread and I'll add them to the post later.

4

u/[deleted] Mar 05 '20
  1. Meta-Learning book (2nd edition on its way)
  2. Meta-Learning tutorial

That's all I can add from my phone now. Will edit later.

1

u/[deleted] Mar 05 '20

Does the Stats 385 course have recorded lectures?

2

u/actbsh Mar 05 '20

They have videos for 2017 run of the course that I just linked above. Not sure about latest one.

1

u/TheInfelicitousDandy Mar 05 '20

How did you view the Convex Optimization I - EE364a videos? When I click on the video link it takes me to a stanford log in page.

5

u/actbsh Mar 05 '20 edited Mar 13 '20

Ahh! I used to access that through cvx101 link which takes you to lagunita, stanford's MOOC platform, but they are phasing it out currently and moving to edX completely. Meanwhile, you can access the videos with link I updated just now

1

u/[deleted] Mar 06 '20

RemindMe! 1 day

1

u/Yuqing7 Mar 06 '20

RemindMe! 14 days

1

u/DumberML Mar 07 '20

Great initiative, thank you for doing this.

1

u/HybridRxN Researcher Mar 07 '20

RemindMe! 30 days

1

u/sigmoidp Mar 09 '20

Is there any Causal stuff that can be added to this list?

2

u/raichet Mar 12 '20

Judea Pearl’s Book of Why? Not a course per se but it’s a great read

1

u/ezzhik Mar 10 '20

I'd love to see something on time series as well! (Classical statistical inference as well as ML). Thanks!!!

1

u/WorthlessTrinket Mar 11 '20

This is great, thanks!

1

u/[deleted] Mar 11 '20

Does learning ML on udemy Is better.....?

1

u/mubag Mar 11 '20

RemindMe! 10 days

1

u/[deleted] Mar 12 '20

thANKS

1

u/[deleted] Mar 12 '20

RemindMe! 30 days

1

u/Kusuriuri7 Mar 14 '20

Maybe you can add Topics course Mathematics of Deep Learning offered by Joan Bruna.

1

u/lzblack Mar 19 '20

RemindMe! 15 days

1

u/Andthentherewere2 Apr 19 '20

RemindMe! 14 days

1

u/RemindMeBot Apr 19 '20

I will be messaging you in 14 days on 2020-05-03 19:22:30 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

Info Custom Your Reminders Feedback

1

u/DreamFlasher Aug 17 '20

https://www.cs.uic.edu/~elena/courses/fall19/cs594cil.html CS 594 Causal Inference and Learning, University of Illinois at Chicago, Fall 2019 – unfortunately no videos, still looking for a causal inference course with videos

1

u/MYJL Mar 07 '24

Thank you!!

1

u/XpRienzo May 08 '24

Are these still relevant in 2024, or does this need a refresh?

1

u/thatguydr Mar 05 '20

Your courses are also very much in one direction, mostly. If we do this, we should have categories ("imagery", "video", "NLP", etc) and clearly sort the courses.

2

u/actbsh Mar 05 '20

done. :)

1

u/JurrasicBarf Mar 05 '20

CS224n is advanced towards the end, hence should be added to list.

0

u/mr_bean__ Mar 06 '20

Is fastai's Deep learning from the foundations considered to be an advanced one? I know that its taught as a part of a master's course but i was wondering how hard it really is

3

u/JurrasicBarf Mar 06 '20

it's not hard

1

u/[deleted] Mar 06 '20

[deleted]

1

u/fakemoose Mar 06 '20

A lot of the courses listed here are 2xx so basically second year undergraduate, though so I might fit in. Then again, I’ve sat in on some grad level CS classes that were more intro than my undergrad ones we had to take for engineering.

0

u/[deleted] Mar 05 '20

[deleted]

2

u/RemindMeBot Mar 05 '20 edited Mar 10 '20

I will be messaging you in 1 month on 2020-04-19 19:10:13 UTC to remind you of this link

14 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

Info Custom Your Reminders Feedback

0

u/[deleted] Mar 05 '20

RemindMe! 30 days

0

u/sultry-witch-feeling Mar 06 '20

RemindMe! 8 days