One of the wonderful things about AI is how it's made idiots out of us, and I don't mean just our ability to do maths problems. It's changed so fast, and in such unpredictable ways, that confident predictions made just a few years ago have turned out to be way, way off the mark. Its amusing, and embarrassing ... and revealing.
In a 2022 exercise organised by the Forecasting Research Institute, AI experts predicted that machines would reach gold-medal standard at the International Mathematical Olympiad in 2030. Professional superforecasters, whose speciality is predicting the future, were even more pessimistic: 2035. In July 2025, Google's Gemini achieved the required standard, a decade ahead of the superforecasters' expectations.
In late 2025, a panel of experts was asked how well AI would perform on a set of fiendishly difficult programming puzzles, the sort designed to test the world's best competitive programmers. These aren't questions about how to write a spreadsheet formula; they require devising ingenious mathematical shortcuts to solve problems that would otherwise take computers an eternity. The experts predicted AI would solve 14% by the end of 2026. By May, it was already solving 53.8%.
These are not trivial mistakes. They are the forecasting equivalent of predicting that your children will learn to walk sometime around their 18th birthday, only to discover them running a marathon at six.
But the experts have also been magnificently wrong in the opposite direction.
In 2016, Geoffrey Hinton, one of the godfathers of AI and a subsequent Nobel laureate, famously advised people to stop training radiologists because AI would be better at reading medical images within five years. Ten years later, radiologists are employed in greater numbers than ever since AI makes medical imaging more useful and therefore in greater demand. American radiologists earned an average of about $571,000 in 2025.
The reason is wonderfully ironic: better and cheaper image analysis makes medical imaging more useful, encouraging doctors to order more scans. Meanwhile, someone still has to interpret the results, make clinical decisions and take responsibility when things go wrong. Hinton understood the technology rather better than he understood the economics. As it turns out, replacing a task is not quite the same thing as replacing a job.
If this is what being replaced by a machine looks like, sign me up.
All this strikes me as evidence of an odd virtue in bad predictions. Looking back at them tells us something about the assumptions and prejudices we held when we made them. It's rather like discovering your high-school yearbook photograph. The haircut is embarrassing, obvs, but what's really disturbing is the expression of absolute confidence with which you wore it.
One of my favourite YouTubers, the German theoretical physicist Sabine Hossenfelder, recently examined the dismal record of AI predictions and arrived at an interesting conclusion.
We have wildly underestimated how quickly computers would master intellectual tasks such as mathematics, coding, and scientific reasoning, but overestimated how quickly they would transform the physical world. The distinction, she argues, is between software, which can advance extraordinarily quickly, and hardware, which must contend with manufacturing, infrastructure, regulation, and the tiresome properties of physical objects.
It's much easier to teach a computer to solve a mathematical theorem than to persuade a robot to empty the dishwasher without breaking the wedding china.
Her conclusion about the reliability of expert predictions is suitably cheerful: "Don't listen to the experts, follow me for more uninformed opinions." I very much concur with that.
But the point is more serious than the entertaining spectacle of experts being publicly humiliated. A failed forecast can reveal the assumptions people were making, the forces they overlooked, and sometimes the interests they were protecting. In that sense, the errors may contain more information than the forecasts themselves.
Just to bring it back home quickly, consider South Africa's electricity planning. The 2010 Integrated Resource Plan assumed the country would require about 296 terawatt-hours of electricity by 2016. South Africa spent years planning for an electricity boom that never arrived. By 2016, demand had fallen 18% below the level anticipated in the country's 2010 electricity plan. Fifteen years later, total electricity demand remains well below its 2010 level.
The planners had assumed a growing economy with steadily expanding industrial electricity demand. What they got was stagnant industrial growth, rising electricity prices, and businesses increasingly keen to generate their own power. Electricity demand was responding to the very problems of the electricity system.
The forecast was ostensibly about electricity consumption. Its failure told us rather more about the dismal trajectory of the South African economy.
Then there's the 2008 financial crisis. Even in October of that year, after Lehman Brothers had collapsed, the International Monetary Fund was predicting global economic growth of 3% in 2009. The world economy subsequently contracted.
What the mistakes exposed was the inadequacy of economic models that failed to appreciate how financial leverage, liquidity shortages, and interconnected banks could turn a housing-market problem into an international catastrophe. The experience was enormously educational, although the course fees were rather steep.
The opposite happened with solar energy. For years, energy models repeatedly underestimated the growth of solar power. A 2017 paper in Nature Energy examined why. Among the problems was an inadequate appreciation of the relationship between increasing production, technological innovation, and falling costs.
As solar became cheaper, demand increased, encouraging further investment that helped make it cheaper still. The forecasts failed to appreciate the feedback loop. It's an interesting parallel with AI, where improvements in capability encourage investment that helps produce further improvements.
These examples suggest that what forecasters most often get wrong is their understanding of how things change. They assume existing relationships will continue, that technological improvements will proceed at familiar rates, or that people will behave tomorrow much as they behave today. Sometimes they confuse what is technically possible with what is commercially practical.
There is also the delicate question of incentives. A technology entrepreneur seeking billions in investment has an understandable preference for announcing that the future is imminent. An electricity planner seeking approval for a power station has every reason to believe demand will justify it. Neither is necessarily being dishonest; being rewarded for optimism can make optimism feel remarkably like objective analysis.
And there's another statistical complication. A prediction that something has a 70% probability of happening isn't wrong simply because it doesn't happen. The remaining 30% is there for a reason, although television commentators tend to regard it as an administrative technicality.
Nevertheless, there is a powerful case for taking a much closer look at old predictions before placing too much faith in new ones.
Ultimately, the problem with predictions is that they are made by people who know the present and are trying to imagine a future in which some of the things they currently know will no longer be true. And, as AI is demonstrating with almost indecent enthusiasm, change doesn't necessarily arrive in the order we expect.
Perhaps every forecast should therefore be accompanied by a list of assumptions, an explanation of what might invalidate them, and a date on which the forecaster agrees to return and explain the result.
I confidently predict that economists, technology gurus, and management consultants will be deeply unenthusiastic about this proposal.
But given the record of predictions, there's every possibility I'm wrong. Which would make the proposal even more valuable. 💥
From the department of absolutely missing the point ...
Marcia Dunn and Associated Press
From the department of unforgivable gender discrimination ...
From the department of the really shocking ...
Kevin Mileham MP - DA Spokesperson On Electricity and Energy
Thanks for reading - please share if you have a friend (or enemy!) you think would value this blog and ask them to add their email in the block below - it's free for the time being. If the sign-up link doesn't appear, you'll find it on the site. And my book - based on these columns - has just been published, get it here!
By all means respond in the space below or email me directly.
Till next time. 💥
