Until recently I agreed with David Deutsch's view that we can't program an AGI until we understand the nature of human intelligence & creativity. But I'm starting to change my mind. This thread is worth a read:
Recent language models have shown greater reach beyond just language - in robotics, protein folding, etc. Perhaps there's something special about the knowledge of language that allows a jump to universality.
I don’t know exactly what that is, but Deutsch himself says that the crucial thing is not to understand every detail, but to have a good explanation of *why*. So here’s the case for why the knowledge of language led to human intelligence and could lead to AGI.
Universal Explainers
What makes us human? As Brett Hall puts it:
People are universal explainers. We explain stuff. We explain our lives, science, how things work. We create new explanations: new theories. Creativity is what we have, and what animals (and computers!) lack. We just don’t understand it. We have no “theory” of creativity and we know this because we have never programmed a creative computer.
Since humans are universal explainers and our closest ape relatives are not, what happened? Perhaps the evolution of language is key. Imagine simple ape gestures representing a limited vocabulary evolving into a greater repertoire (as seen between Bonobos and Chimps). As the hardware of the brain & vocal systems evolved alongside the software of memes, at a certain point a jump to language universality occurred and the repertoire of human language becomes infinite. The broad structure of that jump includes:
All human languages rely on combining sounds or ‘phones’ to make words.
Different languages seem to structure the world semantically in similar ways.
All human languages structure words into sentences.
Explanatory ability could be infused into this structure. We often believe that language allows us to communicate meaning & logic. But perhaps it’s the other way around - meaning & logic are dependent upon language abilities in the first place.
The relationship between language and universal explainers is a close one explored by Bruce Nielson:
Popper unintentionally took a stance on one of the requirements for being a universal explainer (or General Intelligence). Specifically, he argued that language was a requirement for his epistemology to work and thus a requirement for general intelligence.
Voila. Language abilities & universal explanatory abilities could be the same thing. That this statement is rather vague should not be seen as a problem.
Universal Writing Systems
Another crucial jump to universality occurred with modern writing systems that have the reach to represent any word. Again, this probably occurred by chance (The Beginning of Infinity ch6):
And indeed it seems to be a recurring theme in the early history of many fields that universality, when it was achieved, was not the primary objective, if it was an objective at all. A small change in a system to meet a parochial purpose just happened to make the system universal as well. This is the jump to universality.
So here we have a universal system created accidentally - writing - based upon another accidental universal system - language. Is it surprising that machine learning models parochially trained upon a massive dataset of writing also display signs of universality accidentally, without its creators understanding the details?
Language Models and the Brain
I’m not an expert in machine learning but have a general understanding. Attention-based models assign vectors representing the importance of various words in a dataset to each other. A higher-level symbolic representation of the language dataset is created. What I'd proposing is a similarity between that symbolic representation and the structure of the knowledge of language in our brains. And this inherently includes the universal explanatory abilities of AGI.
We don’t need to understand the specifics - they’re emergent in this process. Evolution didn't have an understanding of what language really means, evolution didn't understand the nature of creativity, it just selected for increasing language abilities until one day those abilities leapt to become universal. Why should our artificial attempts be any different?
David Deutsch says that human creativity and AGI are “the only kind of universality capable of transcending its parochial origins”. Our AI attempts to this point have been parochial, but we may be on the cusp of leaping beyond their origins into something general, simply by improving our language models & the knowledge within them. Understanding exactly how this happens may only be known in retrospect, but perhaps the above begins to explain the "why".
Many thanks to @bnielson for his feedback and criticism of the above.