Deep learning, the main innovation that has renewed interest in artificial intelligence (AI) in the past years, has helped solve many critical problems in computer vision, natural language processing, and speech recognition. However, as deep learning matures and moves from hype peak to its trough of disillusionment, it is becoming clear that it is missing some fundamental components.
This is a reality that many of the pioneers of deep learning and its main component, artificial neural networks, have acknowledged in various AI conferences in the past year. Geoffrey Hinton, Yann LeCun, and Yoshua Bengio, the three “godfathers of deep learning,” have all spoken about the limits of neural networks.
The question is, what is the path forward?
At NeurIPS 2019, Bengio discussed System 2 deep learning, a new generation of neural networks that can handle compositionality, out of order distribution, and causal structures. At the AAAI 2020 Conference, Hinton discussed the shortcomings of convolutional neural networks (CNN) and the need to move toward capsule networks.
But, for cognitive scientist Gary Marcus, the solution lies in developing hybrid models that combine neural networks with symbolic AI, the branch of AI that dominated the field before the rise of deep learning. In a paper titled “The Next Decade in AI: Four Steps Toward Robust Artificial Intelligence,” Marcus discusses how hybrid artificial intelligence can solve some of the fundamental problems deep learning faces today.
Connectionists, the proponents of pure neural-network-based approaches, reject any return to symbolic AI. Hinton has compared hybrid AI to combining electric motors and internal combustion engines. Bengio has also shunned the idea of hybrid artificial intelligence on several occasions.
But Marcus believes the path forward lies in putting aside old rivalries and bringing together the best of both worlds.