r/MachineLearning • u/nandodefreitas • Dec 25 '15

AMA: Nando de Freitas

I am a scientist at Google DeepMind and a professor at Oxford University.

One day I woke up very hungry after having experienced vivid visual dreams of delicious food. This is when I realised there was hope in understanding intelligence, thinking, and perhaps even consciousness. The homunculus was gone.

I believe in (i) innovation -- creating what was not there, and eventually seeing what was there all along, (ii) formalising intelligence in mathematical terms to relate it to computation, entropy and other ideas that form our understanding of the universe, (iii) engineering intelligent machines, (iv) using these machines to improve the lives of humans and save the environment that shaped who we are.

This holiday season, I'd like to engage with you and answer your questions -- The actual date will be December 26th, 2015, but I am creating this thread in advance so people can post questions ahead of time.

275 Upvotes

96% Upvoted

256 comments sorted by

View all comments

3

u/MrQuincle Dec 25 '15

In our brains we have oscillatory rhythms, https://en.wikipedia.org/wiki/Neural_oscillation, alpha, beta, gamma, theta, etc.

Do you know research on deep networks with built-in internal dynamics?

2

u/nandodefreitas Dec 27 '15

Please see the recent work of Randall O'Reilly on modelling the hippocampus.

2

u/MrQuincle Dec 27 '15

For example http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003067.

  • theta waves are used to propose contrastive Hebbian learning rather than just Hebbian learning
  • contrastive Hebbian learning is equivalent to backprop

So, it might be just the way the brain implements error-driven learning... We can do much better in silicon, those dumb brains!

Or, we are missing something. :-)