“Brilliant,” “powerful,” and “cutting-edge” are words frequently used to describe Open AI’s bot ChatGPT.
Is it the first of its kind?
Yes and no, depending on who you ask. On the one hand, ChatGPT is one of the most widespread, quickly accessible AI tools of its caliber, creating computer code, poems, research copy, marketing tools, and more with customization features to boot.
NY Times tech columnist Kevin Roose thinks the world isn’t ready for the future of AI — aka the bot’s evolution to ChatGPT4 (currently 3). He also describes it as the first tool so readily available to the public with a “free, easy-to-use web interface.”
But Meta’s AI chief Yann LeCun put some degree of brakes on the whole conversation, explaining during a talk hosted by interactive speaker series Collective[i] Forecast, that ChatGPT isn’t the first of its kind. The deep learning expert credits Open AI for their strong engineering of ChatGPT.
Still, he points out that the self-supervising, Transformer architectures have been around for over 20 years, with similar techniques being developed in other labs around the world.
Here are a few more highlights from LeCun’s recent speaking points on ChatGPT:
On ChatGPT’s innovation (or lack thereof)
“You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner. Self-supervised-learning is something I’ve been advocating for a long time, even before OpenAI existed.”
”Large language models, the first neural net language model — at the time, it was large, by today’s standards, it’s tiny — was by Yoshua Bengio, about 20 years ago.”
“It’s nothing revolutionary, although that’s the way it’s perceived in the public,” said LeCun. “It’s just that, you know, it’s well put together, it’s nicely done.”
“It’s not only just Google and Meta, but there are half a dozen startups that basically have very similar technology to it. I don’t want to say it’s not rocket science, but it’s really shared, there’s no secret behind it, if you will.”
On AI and science
“One of the things I find pretty promising about AI is the use of the AI in science and medicine at the moment” to better people’s lives.”
“We’re going to need this because we need to solve climate change, so, we need to be able to have high-capacity batteries that don’t cost a fortune, and don’t require you to use exotic materials that we can only find in one place.” [on AI’s potential for materials science]
On AI and intelligent behavior
“They are completely reactive. You give them a context of a few thousand words, and then from that, the system just generates the next token, completely reactively. There’s no planning ahead or decomposition of a complex task into simpler ones, it’s just reactive.”
“The question is, how do we get from systems that generate code that sometimes runs but sometimes doesn’t. And the answer to this is all of those systems today are not capable of planning; they are completely reactive.”
“And this is not what you need for intelligent behavior.”
“If you want intelligent behavior, you need a system that is capable of anticipating the effect of its own actions,” and “some sort of internal world model, a mental model of how the world is going to change as a consequence of its own actions.”
Learn more about this session, and explore additional talks from Collective[i] Forecast.
