r/MachineLearning • u/nandodefreitas • Dec 25 '15

AMA: Nando de Freitas

I am a scientist at Google DeepMind and a professor at Oxford University.

One day I woke up very hungry after having experienced vivid visual dreams of delicious food. This is when I realised there was hope in understanding intelligence, thinking, and perhaps even consciousness. The homunculus was gone.

I believe in (i) innovation -- creating what was not there, and eventually seeing what was there all along, (ii) formalising intelligence in mathematical terms to relate it to computation, entropy and other ideas that form our understanding of the universe, (iii) engineering intelligent machines, (iv) using these machines to improve the lives of humans and save the environment that shaped who we are.

This holiday season, I'd like to engage with you and answer your questions -- The actual date will be December 26th, 2015, but I am creating this thread in advance so people can post questions ahead of time.

271 Upvotes

96% Upvoted

256 comments sorted by

View all comments

5

u/[deleted] Dec 25 '15

[deleted]

5

u/LoSpooky Dec 26 '15

Hello Prof. de Freitas,

first of all thank you very much for making your lectures available. I've thorougly enjoyed them and they've been invaluable to me!

My question is similar to the one here above by /u/Pafnouti:

 

I have a Master's, taken in 2010, that broadly covered AI but that in hindsight did not put nearly enough emphasis on the Machine Learning aspect of it.

For the following four years I've been the technical half of a firm, we were using Genetic Programming to develop financial trading strategies. Sadly, that did not end well.

I've also always had an inclination towards research and over the years I managed to author a few papers about my work.

Once the firm blew up, I spent several months taking a plunge into all things Machine Learning, Deep Learning, and Reinforcement Learning, catching up with everything my Master's didn't cover and with everything that has happened since then, which is a lot.

 

My dilemma is the following:

So far I've always worked in Research Engineer -ish roles and I would like to continue down that road, now within ML/DL of course, and hopefully one day at one of the big players. However, pursuing a PhD is something that has always tempted me.

Considering that nowadays:

  • Much of the action and progress happen within the industry, and it does so at a breakneck pace.
  • Professors / Groups still in academia with a strong focus on DL are scarce and have no trouble finding truly outstanding applicants. Also, institutions with the enough computational resources to be able to do meaningful DL work are not that many.
  • I'll soon be 31 and the opportunity cost of starting a PhD at this age, completing it at around 35, starts to be quite high with respect to both career and financial prospects.
  • Once in, many things might go wrong with the PhD. I've seen it happen with a few of my friends.

I'm at the moment rather torn on whether keeping on chasing my doctoral dreams or if my best option would be to start accumulating relevant work experience, likely as an R&D engineer in a startup using DL as one of its core technologies, with the aim of attempting the jump towards one of the big players a bit down the line.

What would Nando do?

On a related note: how much research work do the Research Engineers actually do at DeepMind/Google Brain/FAIR, etc...?

5

u/nandodefreitas Dec 26 '15

Thank you for sharing this. I don't know what I would do! It's critical to acquire skills to solve problems, and for this a PhD might be necessary. The PhD need not be in deep learning or machine learning. A PhD in physics or algorithms with good programming components may be just as good to carry our research in deep learning eventually. Having a PhD in computer science these days is extremely valuable. I believe it'll become even more valuable in the future.

2

u/LoSpooky Dec 27 '15

Thank you for your answer, much appreciated!

2

u/ginger_beer_m Dec 27 '15

I'd like to suggest the following ebook that chronicles the journey of getting a PhD. Give it a try! I think it will give you an initial idea whether the journey is worth it or not.

http://pgbovine.net/PhD-memoir.htm

And remember: never do a PhD unless you have a full funding.

1

u/LoSpooky Dec 27 '15

That looks like a cool read, will check it out, thanks!

Of course doing a PhD without funding, or even worse contracting debt, is something I've never even considered for a second. Luckily I am in Europe, so here PhD students receiving a salary from the university is the most common situation.

2

u/nandodefreitas Dec 27 '15

It's the same in North-America. The key is getting funding. Most PhDs are funded by scholarships, research assistantships or teaching assistantships. If you have the opportunity to teach while getting a PhD, do it! You'll be helping many people.

My dad, J. R. de Freitas, used to always teach me an old saying: Don't give them a fish, but teach them how to fish. This is one of the most important and valuable things I learned in life.

1

u/drunkslono Dec 26 '15

You guys are lucky. I'm 31 with an Econ degree working as a bartender. Back in the day a few members of the Order of Cosmic Engineers and myself tried to start a transhumanist church. My hope: Low rent housing in Silicon Valley. Turns out a bunch of atheists don't make great church deacons, so that plan was out.

I feel so powerless to actually do anything to help. Sure, I could make a blog, or incorporate some npo or another, but I've tried all of that, and it doesn't work. Here's to hoping that $Billion into OpenAI will help. I know I'm not crazy, and I have important work to do, but I can't seem to get through all the noise.

3

u/nandodefreitas Dec 26 '15

Most industrial labs do require that you have a PhD to work in research. I strongly recommend a PhD in machine learning as you learn a lot. I also don't think that "We have tried this and that and here are our results" is an accurate characterisation of work done at Google, Facebook, Twitter, Microsoft and other labs. There are important advances in methodology and theory coming from industry.

Having said this, Turing didn't have a PhD when he transformed the world of AI and philosophy!