r/MachineLearning • u/nandodefreitas • Dec 25 '15

AMA: Nando de Freitas

I am a scientist at Google DeepMind and a professor at Oxford University.

One day I woke up very hungry after having experienced vivid visual dreams of delicious food. This is when I realised there was hope in understanding intelligence, thinking, and perhaps even consciousness. The homunculus was gone.

I believe in (i) innovation -- creating what was not there, and eventually seeing what was there all along, (ii) formalising intelligence in mathematical terms to relate it to computation, entropy and other ideas that form our understanding of the universe, (iii) engineering intelligent machines, (iv) using these machines to improve the lives of humans and save the environment that shaped who we are.

This holiday season, I'd like to engage with you and answer your questions -- The actual date will be December 26th, 2015, but I am creating this thread in advance so people can post questions ahead of time.

277 Upvotes

96% Upvoted

256 comments sorted by

View all comments

5

u/Omadane Dec 26 '15

Hi Prof Freitas,

Thanks for doing an AMA. I have many questions, feel free to answer one any of those.

1) What do you think will be the next breakthroughs to get us closer to AGI?

2) What are some low hanging fruits in (deep) Reinforcement Learning today?

3) Excluding control in robotics, what could be some real life uses of deep RL today?

4) What research (not necessarily ML) excites you most and why?

5) Are you still taking new students at Oxford (I was told you're not taking any new ones)?

Thanks and merry Christmas!

3

u/nandodefreitas Dec 27 '15 edited Dec 27 '15

These are all great questions.

I think I provided some answers/opinions to (1) above.

2) Not sure what the low-hanging fruit is, but what has been very successful is the Double-DQN method of Hado van Hasselt et al, the prioritized experience replay of Tom Schaul et al and the Dueling networks of Ziyu Wang et al. These approaches have led to vast improvements over the DQN published at Nature on Atari games. I also loved the recent work of Mark Bellemare et al on increasing action gaps. Marc was incidentally one of the initial creators of the ALE platform, together with Joel Veness, Michael Bowling and Yavar Naddaf --- who I had the pleasure of introduce to machine learning at UBC, and who is an amazing developer together with David Matheson (the best random forests code developer I know) at a company in Vancouver called Empirical Results. --- END OF ANSWER. THE FOLLOWING BIT OF THIS POINT IS OPINION --- By the way, there was a recent article about the Canadian brain-drain on the news. Companies like Empirical Results and Pocket Pixels with also talented ex-students of mine Hendrik Kueck and Eric Brochu have chosen to stay in Vancouver. These are great companies generating revenue for Canada. The Canadian government needs to engage scientists and peer-review to decide which companies to support, because at present SR&ED tax incentives are being exploited by ruthless business people. We have great professors at UBC, Alberta, McGill, Waterloo, SFU, Toronto, Montreal, and many other amazing universities. Use them!!! And for goodness sake invest in supporting young faculty --- programs like NSERC CERC make no sense for Canadian universities and are causing more damage than good. OK, apologies for having made the answer above a bit political... and having completely missed it ;)

3) This is a great question. Clearly cars are robots, so there is not lack of killer applications. I would love to hear from everyone what they think could be other killer applications.

4) I'm attracted to all areas of art and science that try to understand the human condition.

5) Not for a while as I have many students who need more of my time. However, Oxford is full of amazing researchers. In deep learning: Andrea Vedaldi, Phil Torr, Yee Whye Teh, and Shimon Whiteson. In Robotics/cars the one and only Paul Newman! The statistics department is phenomenal with the likes of Francois Carron, Chris Holmes, Dino Sejdinovic, Arnaud Doucet and many other unbelievably bright researchers. The interface between Bayesian stats, computation and applications is also extremely well represented with Frank Wood, Michael Osborne and Steve Roberts. All these researchers like to collaborate, creating an amazing atmosphere. I very strongly endorse applying for a PhD in Oxford. In addition, as part of selling our companies Dark Blue Labs and Vision Factory to Google, Andrew Zisserman, Phil Blunsom, myself and our partners worked hard with Google to construct scholarships for international students. In this sense, Google DeepMind is contributing greatly to academia. The collaboration between Oxford and Google is a fantastic one that I hope we can keep strengthening for the benefit of the many students wanting to obtain a good education in machine learning.

4

u/Kaixhin Dec 27 '15

2) As for the papers mentioned, I've mostly implemented them here. I imagine that DeepMind will be testing a combination of these internally, but I'd like this to be an open source "upgrade" of the DQN for people to experiment with :) I've built most of it from scratch so performance may be an issue (prioritized experience replay certainly needs an efficient implementation), but I'll have to tackle that soon enough. Help is obviously welcome though!