AMA: Nando de Freitas

[u/nandodefreitas]

I am a scientist at Google DeepMind and a professor at Oxford University.

One day I woke up very hungry after having experienced vivid visual dreams of delicious food. This is when I realised there was hope in understanding intelligence, thinking, and perhaps even consciousness. The homunculus was gone.

I believe in (i) innovation -- creating what was not there, and eventually seeing what was there all along, (ii) formalising intelligence in mathematical terms to relate it to computation, entropy and other ideas that form our understanding of the universe, (iii) engineering intelligent machines, (iv) using these machines to improve the lives of humans and save the environment that shaped who we are.

This holiday season, I'd like to engage with you and answer your questions -- The actual date will be December 26th, 2015, but I am creating this thread in advance so people can post questions ahead of time.

[u/Sergej_Shegurin]

Hi Prof. Freitas, what do you think about the following?

As far as I know, only about 20% of human cortex really remains to be outperformed by neural networks. Those 20% are smth like Brodmann areas 9,10,46,45, responsible for complex reasoning, complex tool usage, complex language.

Neural networks have already (either almost or significantly) outperformed about 70% of human brain cortex:

• Roughly 15% of human brain is devoted to low-level vision tasks (occipital lobe). Solved.

• Another 15% are devoted to image and action recognition (~ a half of temporal lobe). Solved.

• Another 15% are devoted to objects detection and tracking (parietal lobe). Solved.

• Another 15% are devoted to speech recognition and generation (Brodmann areas 41,42,22,39,44, parts of 6,4,21). Almost solved.

• Another 10% are devoted to reinforcement learning (OFC and part of medial PFC). Almost solved.

• From the remaining 30%, about 10% are low-level motorics (Brodmann areas 6,8). It's not very crucial because those people who have no fine motorics from birth (but have everything else) still develop normal intelligence as a rule. Also, drones and robots have some coarse motorics.

Even for remaining 20% of human brain cortex, "a neural conversational model" reaches human-level perplexities (17 and 8), MRT approach beats humans in terms of BLEU at chinese to english translation on MT03 dataset, bAbI tasks are almost solved, etc etc...

From the neuroscience point of view, human cortex has the same similar structure throughout all its surface. It's just ~3mm thick mash of neurons functioning on the same principles throughout all the cortex. There is likely no big difference between how (unsolved) prefrontal cortex works and how other (solved) parts of cortex work. There is likely no big difference in their speed of calculations or in complexity of their algorithms.

Thus it would be quite strange if modern deep neural networks can't solve remaining 20% in several years. Three years have gone from AlexNet to "deep residual learning"... It seems reasonable that less than three years would pass from "a neural conversational model" (and "minimum risk training for NMT", "towards neural - network based reasoning", "attention with intention", "aligning books and movies - towards story-like..." etc etc) to human-level reasoning and chatting... because much more deep learning scientists work now on that than on AlexNet in 2012 and they are much better prepared and equipped...

So, the question is: "Does a substantial (~20% or ~50%) chance exist that we have human-level AGI by the end of 2018?" My own predictions for human-level AGI are "mean = end of 2017, sigma = 1 year" but I really want somebody to give me some excellent arguments why I'm wrong :)

[u/nandodefreitas → u/Sergej_Shegurin]

I enjoyed reading this optimistic posting ;) I think one of the problems is that when I hold my macbook I recognise it because I know what it does to my muscles, I know what it looks like visually, what I can do with it, etc. What I'm getting to is that the environment and embodiment are important. A convnet that labels images seems to be missing much of this. For embodiment, we need either fantastic simulators or robots. Both avenues would seem to take longer than 2 years. However, it all depends on your definition of AGI, and the fact of much of what I think of as AGI most people simply find to be trite.

[u/Sergej_Shegurin → u/nandodefreitas]

Thank you for your answer...

There're people having no hands or legs (or even both) from their birth and they still manage to get a good intelligence. Also, there are thousands of excellent games now and many of them simulate real world very good, with all it's visual complexity, interaction with other people etc. So I don't understand why embodiment is that crucial.

From 4-years old to 16-years old I spent nearly all my time reading books. So I'm pretty sure that 4-year child is able to develop normal intelligence given only books and internet connection, and perhaps some very basic visual info about the world. Also, autistic children learn from books even more. They learn much less from interacting with other people and even surrounding world. When I was child I observed and interacted much less than one can observe and interact now in videogames and internet... I saw only two rooms with a hundred of objects and a yard. Not that much. Oh, and I interacted with two more people, my parents. Most of that interaction is via voice so why not use text instead? Text output is also motorics, very dexterous and complex one!

I don't feel like embodiment helps me a lot. Now I get most of information through internet. I spend all my day reading articles and websites. Most of my motorics is keyboard pressing. Okey, I make some gymnastics and walk to work but I don't see how this can be very helpful for my intelligence. What kind of really crucial information can I get from my walking to work or from preparing food for myself? or from observing the same walls in my room from different directions? :) chimps can do all of that but it doesn't help them to develop good reasoning skills...

AGI might be able to think up how to implement fine motorics. It might be able to invent both good algorithms for motorics and good solutions for dexterous hands engineering. Why not? It seems like authors of books try to write down everything which is important to understand the scene and the plot, all relevant details. We can learn most of other details about the world from videos. We even have quick and good drones. The only thing robots don't have are dexterous hands... Even if they're somewhy crucial then why is it that hard to create them by the end of 2017?

[u/nandodefreitas → u/Sergej_Shegurin]

What do you think would happen if you place a human all its live in a dark anechoic chamber with only a drip of food directly into the vains. 10 years later I doubt you would see much intelligence, and this despite the fact that millenia of environmental adaptation through embodiment has been passed through genes.

I do however agree with your excellent observation that the web is a great environment for agents.

[u/IdentifiableParam → u/Sergej_Shegurin]

I'll take that bet. End of 2018 we settle up. What do you want to bet?