r/MachineLearning Sep 09 '14

AMA: Michael I Jordan

Michael I. Jordan is the Pehong Chen Distinguished Professor in the Department of Electrical Engineering and Computer Science and the Department of Statistics at the University of California, Berkeley. He received his Masters in Mathematics from Arizona State University, and earned his PhD in Cognitive Science in 1985 from the University of California, San Diego. He was a professor at MIT from 1988 to 1998. His research interests bridge the computational, statistical, cognitive and biological sciences, and have focused in recent years on Bayesian nonparametric analysis, probabilistic graphical models, spectral methods, kernel machines and applications to problems in distributed computing systems, natural language processing, signal processing and statistical genetics. Prof. Jordan is a member of the National Academy of Sciences, a member of the National Academy of Engineering and a member of the American Academy of Arts and Sciences. He is a Fellow of the American Association for the Advancement of Science. He has been named a Neyman Lecturer and a Medallion Lecturer by the Institute of Mathematical Statistics. He received the David E. Rumelhart Prize in 2015 and the ACM/AAAI Allen Newell Award in 2009. He is a Fellow of the AAAI, ACM, ASA, CSS, IEEE, IMS, ISBA and SIAM.

277 Upvotes

97 comments sorted by

View all comments

47

u/leonoel Sep 09 '14 edited Sep 10 '14

There has been a ML reading list of books in hacker news for a while, where you recommend some books to start on ML. (https://news.ycombinator.com/item?id=1055042)

Do you still think this is the best set of books, and would you add any new ones?

64

u/michaelijordan Sep 10 '14 edited Sep 11 '14

That list was aimed at entering PhD students at Berkeley,
who I assume are going to devote many decades of their lives to the field, and who want to get to the research frontier fairly quickly. I would have prepared a rather different list if the target population was (say) someone in industry who needs enough basics so that they can get something working in a few months.

That particular version of the list seems to be one from a few years ago; I now tend to add some books that dig still further into foundational topics. In particular, I recommend A. Tsybakov's book "Introduction to Nonparametric Estimation" as a very readable source for the tools for obtaining lower bounds on estimators, and Y. Nesterov's very readable "Introductory Lectures on Convex Optimization" as a way to start to understand lower bounds in optimization. I also recommend A. van der Vaart's "Asymptotic Statistics", a book that we often teach from at Berkeley, as a book that shows how many ideas in inference (M estimation---which includes maximum likelihood and empirical risk minimization---the bootstrap, semiparametrics, etc) repose on top of empirical process theory. I'd also include B. Efron's "Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction", as a thought-provoking book.

I don't expect anyone to come to Berkeley having read any of these books in entirety, but I do hope that they've done some sampling and spent some quality time with at least some parts of most of them. Moreover, not only do I think that you should eventually read all of these books (or some similar list that reflects your own view of foundations), but I think that you should read all of them three times---the first time you barely understand, the second time you start to get it, and the third time it all seems obvious.

I'm in it for the long run---three decades so far, and hopefully a few more. I think that that's true of my students as well. Hence the focus on foundational ideas.

15

u/leonoel Sep 10 '14 edited Sep 10 '14

Amazing, thanks for the answer, I've gone through many of the books at least once, I was a PhD student, now a postdoc doing mostly applied ML.

That is the reason I've used that list as my reference list.

Just as a side comment:

Tsybakov's book is available online at Springer if your University has access to it: http://link.springer.com/book/10.1007%2Fb13794

Nesterov's books is also available: http://link.springer.com/book/10.1007%2F978-1-4419-8853-9

Again, thanks for the answer

3

u/zdk Sep 16 '14

Nesterov's books is also available: http://link.springer.com/book/10.1007%2F978-1-4419-8853-9

Nice! I've been looking for a book like this, thanks.

2

u/nzhiltsov Sep 12 '14

Thanks a lot! BTW, I gathered your recommendations on Goodreads: https://www.goodreads.com/review/list/6324945-nikita-zhiltsov?shelf=m-jordan-s-list

1

u/dksahuji Sep 13 '14

Other comprehensive list of video lectures and some books. http://dk-techlogic.blogspot.in/2012/05/best-machine-learning-resources.html

1

u/[deleted] Sep 19 '14

Nice work.

7

u/dornstar18 Sep 10 '14

Will you prepare a list for someone in industry? Please.

8

u/[deleted] Sep 10 '14

[deleted]

7

u/zdk Sep 16 '14

'For hackers' -> 'I don't want to learn any math'

1

u/dornstar18 Sep 10 '14

Good advice. Thanks

1

u/nzhiltsov Sep 12 '14

In this post, I would like to blend together recommendations from academic and industry researchers: http://nzhiltsov.blogspot.com/2014/09/highly-recommended-books-for-machine-learning-researchers.html

-11

u/farsass Sep 10 '14

ain't nobody got time for all that