<a href="https://archive.is/dFnBu" rel="nofollow">https://archive.is/dFnBu</a>
> We asked it [GPT-5.5] to assess whether each leader had made a falsifiable claim about the future as part of its main thesis. About 1,400 did. We then extracted those predictions, and asked the AI to mark out of ten both how contrarian the leader’s outlook was at the time and how accurate the prediction turned out to be. We ran those queries several times and took an average.<p>I understand why it wouldn’t be feasible for a human to do this, but I’m quite sceptical about an AI assessing how accurate predictions turned out to be/how contrarian they were at the time. It seems like that would depend a lot on what sources it chooses, be liable to hallucination or getting poisoned by bad sources, etc. They don’t mention whether they used independent queries for each prediction either, or whether it was doing multiple sequentially.<p>Given that LLMs can’t really distinguish prompt from instructions etc, I’m sceptical that they can reason particularly well about things like how contrarian a view was at a particular point in time.
I thought the same while reading this - I can easily imagine the AI using hindsight to affect "how contrarian the leader's outlook was at the time", in a way that's similar to how we often do the same.<p>Would be interesting to do a similar analysis but maybe pushing the AI to search the web for articles/other reporting written during that time, to at least correct for that bias a little bit.
you can still judge if it was correct or not, irrespective of contrarianness, still a good signal to measure
One should do this analysis for various politicians, political parties, TV and news media and so on.<p>How do we know how good or bad the Economist is if we have no other source to compare against? Presumably it isn't trivial to predict the outcome of millions of people's reactions to all the complicated things that are happening to them.
They should have checked if they are more accurate than the consensus, the data doesn't show this.
to quote Taleb: those who can, do (trade). those who can't, write about it (financial press)
I don’t see any other publication doing this kind of analysis of themselves, pointing out the mistakes they’ve made in the past.<p>Even in this article they mention “The Economist was convinced by the false claim that Saddam Hussein was hiding weapons of mass destruction”, something they remind the reader of each time the Iraq War is discussed. I remember an article from 2017 that said the Economist got it wrong, while a less honest publication wouldn’t have.<p>It’s more remarkable considering that the economist’s articles and opinions are published without the authors details. So when an article gets it wrong, it reflects poorly on the economist as a whole. They can’t simply blame the previous guy, it’s their fault.<p>They’ll be talking about how they got the Iraq War wrong as long as someone mentions it even in passing 50 years from now. That’s candour I appreciate.
That's a disappointing chart from the economist. When you have a lot of points, you can't show them as dots that occlude other dots, or the reader can't get much more of a sense than 'none / some / lots'. What you need to do is bucket by region and do a background heatmap with small dots added, or use a different level of opacity on the dots.
The internet of 2026.<p>Betteridge's law wrapping a paywall wrapping an AI piece masquerading as domain-specific analysis.
I really like the economist, they are quite centric in their views but don't hide a certain bias for strong representative democracies, deregulation, globalism and open (publicly traded) markets.<p>Their writing style is probably the cleanest in the industry and they always point out potential conflict of interest even though I've never seen the write anything really positive about the companies Exor controls (Stellantis, Philips, etc).
The Economist Group is largely owned by the Agnelli family ("Italy's Kennedys"), Canadian billionaire Stephen Smith, the Rothschilds and others, which is likely a significant factor in those biases.
If you find a topic where reality lies outside of the overton window for western elites <i>at the time</i> theyll reliably get it wrong.<p>There are a number of examples of this (e.g. the Afghanistan and Iraq wars).<p>They usually use a lot of hedging language to cover their behinds when they make predictions, though, so their predictions are usually more nudges and winks than they are commitments.
It’s always wrong when it really matters.
rare example of a question in the headline where the answer is yes
"We investigated ourselves and found we did nothing wrong"