They’re all stating this to hide from the reality that llm development has nearly hit the upper limits of what these models can do. The hope was that big data would indefinitely sustain the progress of these models but we are seeing diminishing returns. They projected exponential returns back in 2020 but the reality is that these are hill curves. We’ll continue to see incremental progress with llms but academic AI progress will come from exploration of other novel techniques.
Yes because they have no choice: the technology has plateaued and is beyond the point of diminishing returns.
This isn’t about making things safe; it’s about getting “ahead” of the stories of them failing to meet the crazy expectations from the market bubble. They are managing expectations as the bubble is bursting and this is a stratagem to try and talk them out of this mess. But they can’t change reality by just saying stuff, as much as them seem to believe. Reality is catching up with them.
the technology has plateaued and is beyond the point of diminishing returns.
The technology is still advancing at a good pace and there’s no sign they’ve plateaued, especially in math and “reasoning”. For example the frontier math benchmark is showing big strides for the frontier models. A year ago they were only getting <10% of questions, now sol from openai is scoring 89% and they’re reporting the unreleased astra is at 98% .
They even had to create a new frontier math benchmark called erdos that all previous models scored 0% on and astra got 3% , which may not look like much but is a huge relative jump.
you know, i smoked away my memory in college, but i swear people in media have goldfish brain. these companies have been talking about this for a decade. remember when they were hesitant to release GPT-2 because of how dangerous it was?
this is just more marketing. “oh no my products are so powerful we’re just way too awesome at making stuff uwu.” shovel salesman says there’s too much gold in the hills.
If they’re saying “we need to regulate new development”.
Then potentially they realized that the tech their entire scam is based on has hit its cap so now they’re pulling the ladder up behind them.
Pause Giant AI Experiments: An Open Letter is the title of a letter published by the Future of Life Institute in March 2023. The letter calls “all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4”, citing risks such as AI-generated propaganda, extreme automation of jobs, human obsolescence, and a society-wide loss of control.[1] It received more than 30,000 signatures, including academic AI researchers and industry CEOs such as Yoshua Bengio, Stuart Russell, Elon Musk, Steve Wozniak and Yuval Noah Harari.
literally Elon signs this before creating xAI and Grok.
https://en.wikipedia.org/wiki/Pause_Giant_AI_Experiments:_An_Open_Letter
people have been talking about a plateau since 2020 as well (basically since implementations of Attention is All You Need ie transformer architecture has been popular). but it’s hard to know for sure, since what used to be public (you might call it “Open”) scientific research is now closely guarded trade secrets.
“hold me back, look how much I can do. Investors take notes”
This is nukes all over again
Nukes are actually dangerous. AI is mostly just wasteful, fraudulent, and stupid.
AI doesn’t work like advertised and investors will lynch them so they’re seeking government intervention as protection.
There is more to this story, of course:
https://apnews.com/article/china-anthropic-ai-us-amodei-3da458d2c078da3e60900728d59f1ae8
it’s not that he is concerned about AI threat in general, he is mostly concerned about being trampled over by evil foreign AI.
Anthropic = Good, Qwen and DeepSeek = Eveeeeel!
Text book the answer to a headline in the form of a question is always ‘no’.
Short answer: No
Long answer: Officially they will, but they’ll continue behind closed doors - as will all their competition.
The chance to stop this globally came and went when we were all laughing about Will Smith eating spaghetti. All these companies are far too invested in this to stop now even if their founders actually wanted them to.
They will talk wide and loud about it, in the hopes that their competitors fall for it, they will continue in secret
no
CEOs tell what people want to hear. Populism is their job. Most people haven’t the slightest idea how LLMs work and are too stupid to even imagine any effective usage of “AI” therefore declaring LLMs evil. So CEOs repeat that too. All are happy; spice must flow.
