Came here to say this exactly. AI companies are not going to deliver what they promised, they can’t hide it any longer, and now they’re looking for any excuse other than admitting that it was a sham all along.
The pace of advancement is still going strong, especially in math and “reasoning”. For example the frontier math benchmark is showing big strides for the frontier models. A year ago they were only getting <10% of questions, now sol from openai is scoring 89% and they’re reporting the unreleased astra is at 98% .
They even had to create a new frontier math benchmark called erdos that all previous models scored 0% on and astra got 3% , which may not look like much but is a huge relative jump.
I’m aware of what the pace is. You might want to check up on the current controversy surrounding OpenAI’s math “achievements”.
By my reckoning, the difference between Mythos and Opus is smaller than the difference between Opus and Sonnet. Same with the difference between GPT 5.5 to 5.6 is smaller than the difference between GPT 4 to GPT 5.
The size of improvements over time is diminishing. We’re not in “big bang” territory anymore and we’re about two years into the “incremental refinement” period. We’re about to enter the next AI Winter unless somebody comes up with a new architectural component as revolutionary as transformers have been for ML models.
The core problem is that LLMs do not create. Full stop. All creativity is borne by the human inputs. Until that changes - until the model gains the capability to truly create new information, we’ve hit the limits in raw capability.
The size of improvements over time is diminishing. We’re not in “big bang” territory anymore and we’re about two years into the “incremental refinement” period. We’re about to enter the next AI Winter unless somebody comes up with a new architectural component as revolutionary as transformers have been for ML models.
The models are also getting extremely big as well, since the big improvement currently seems to largely be stuffing the model with more parameters, and making that work.
I can only imagine that the training cost has also been skyrocketing.
The core problem is that LLMs do not create. Full stop. All creativity is borne by the human inputs. Until that changes - until the model gains the capability to truly create new information, we’ve hit the limits in raw capability.
The models suppress outliers by design. Statistically speaking, the most novel thing is a garbled mess of random words, but random noise is useless, so it ends up being suppressed. You can see by fiddling the samplers, or increasing the temperature.
The Library of Babel is the most creative thing in the world, containing every possible combination of English words and letters. You can basically act like an LLM by trying to find a new coherent sentence in it, but also one that hasn’t been said before. It’s basically impossible.
But that is what an improvement is supposed to be. Compare that to finding a sentence that has been said, or something close to it.
Recursive self-improving AI isn’t happening like they expected. AGI is nowhere in sight.
The pace of advancement is slowing down and they need a cover story for why they’re not living up to their own hype.
Came here to say this exactly. AI companies are not going to deliver what they promised, they can’t hide it any longer, and now they’re looking for any excuse other than admitting that it was a sham all along.
The pace of advancement is still going strong, especially in math and “reasoning”. For example the frontier math benchmark is showing big strides for the frontier models. A year ago they were only getting <10% of questions, now sol from openai is scoring 89% and they’re reporting the unreleased astra is at 98% .
They even had to create a new frontier math benchmark called erdos that all previous models scored 0% on and astra got 3% , which may not look like much but is a huge relative jump.
I’m aware of what the pace is. You might want to check up on the current controversy surrounding OpenAI’s math “achievements”.
By my reckoning, the difference between Mythos and Opus is smaller than the difference between Opus and Sonnet. Same with the difference between GPT 5.5 to 5.6 is smaller than the difference between GPT 4 to GPT 5.
The size of improvements over time is diminishing. We’re not in “big bang” territory anymore and we’re about two years into the “incremental refinement” period. We’re about to enter the next AI Winter unless somebody comes up with a new architectural component as revolutionary as transformers have been for ML models.
The core problem is that LLMs do not create. Full stop. All creativity is borne by the human inputs. Until that changes - until the model gains the capability to truly create new information, we’ve hit the limits in raw capability.
The models are also getting extremely big as well, since the big improvement currently seems to largely be stuffing the model with more parameters, and making that work.
I can only imagine that the training cost has also been skyrocketing.
The models suppress outliers by design. Statistically speaking, the most novel thing is a garbled mess of random words, but random noise is useless, so it ends up being suppressed. You can see by fiddling the samplers, or increasing the temperature.
The Library of Babel is the most creative thing in the world, containing every possible combination of English words and letters. You can basically act like an LLM by trying to find a new coherent sentence in it, but also one that hasn’t been said before. It’s basically impossible.
But that is what an improvement is supposed to be. Compare that to finding a sentence that has been said, or something close to it.