The AI Hype: Take 2

Hal Swerissen

It would be hard not noticing the hype — and the real and spectacular development — of AI over the three years since I wrote my first piece on the subject. We continue to live in an era dominated by proclamations of an impending technological utopia or doom. Headlines regularly warn, or promise, that Artificial General Intelligence is just around the corner, ready to upend human labour, solve our deepest scientific mysteries, and usher in a hyper-advanced post-work society, or run rogue and end life as we know it.

Where you sit on the continuum of utopia versus dystopia depends on your conception of AI. On one side are those who see it as a useful set of tools to enhance the ongoing pursuit of economic, social and environmental ends. On the other, are those who see it as the emergence of a completely new form of consciousness with its own motivations and goals that will ultimately either coexist with humans or replace us.

The pace of change over the past three years makes it risky to reach hard and fast conclusions, but the evidence is moving more in the direction of AI as a very useful set of tools with some real risks that will need to be managed. The short to medium term risk is that the value of the tool set is being over sold as a byproduct of cut throat Silicon Valley competition and myopic investor greed.

The Long Road of Technical Friction

Breakthroughs in digital intelligence do not happen overnight. They are the product of decades of grinding, incremental engineering.

Take speech recognition. Bell Labs built Audrey, which could recognise spoken digits, in 1952. DARPA ran a major Speech Understanding Research programme through the 1970s. Today speech recognition is pretty good, although still not perfect. But it’s easy to forget that seventy years of work separates that beginning from the fluid conversational interaction we now take for granted. And the last vital ten per cent is always the hard bit. The same is true for AI.

Fully self-driving cars make the point in the present tense. Compare Waymo and Tesla. Waymo digitises and standardises each localised driving environment to minimise risk. Tesla, and now the Chinese entrant XPeng, are attempting to bypass the need for detailed localised information by developing a a kind of generalised artificial intelligence for autonomous driving on any road, anywhere.

Waymo runs cameras, lidar and radar with detailed mapping and localised training for each new city.Waymo’s approach works. It now operates in eleven US metropolitan areas with roughly 3,000 vehicles, providing around 500,000 paid rides a week. In July 2026 it moved San Diego, Las Vegas, Tampa and Denver towards commercial service, with London flagged as its first international market. Anyone claiming this is a technology stuck in the lab is not paying attention. Waymo has developed a detailed tool set and development system for a localised rollout of fully autonomous driving. It depends on highly accurate localised spatial data, full sensor redundancy and extensive local AI training. Even then software updates, changes to the environment and rare edge cases can disrupt the Waymo model. This is the classic example of AI as an extraordinarily useful tool set.

Tesla’s position is instructive for the same reason. Tesla and XPeny are betting on the use of camera only technology combined with hyper-scaled onboard processing and extensive training to develop a generalised transport AI that can be used virtually anywhere. The problem is that works very well to enhance human driving, but not well enough to be fully autonomous, despite the processing of vast amounts of training data. Teslas paid robotaxi service launched in Austin in June 2025 but still runs with a human safety monitor in the front passenger seat. The system is genuinely impressive and a very useful enhancement of human driving. It is also not trusted to be out on its own. Whether it ever will be remains an open question, one that mirrors the debate about whether hyper scaling will produce genuine artificial general intelligence or just very good pattern recognition and prediction.

Current systems are marvels of statistical pattern matching, but they lack genuine comprehension, agency or consciousness. They predict the next most likely token in a sequence drawn from vast oceans of human text; they do not know what they are saying. Expecting them to transition smoothly into autonomous entities operating safely in complex, unpredictable environments without continuous human oversight ignores decades of hard-earned computer science history.

The Enterprise Productivity Puzzle

Currently, the best bet is that AI is a very useful tool set that can fully automate predictable systems and processes and enhance human decision action and decision making in complex and unpredictable environments. The evidence about the real world limits of AI applications, even with the most sophisticated models is beginning to emerge and it’s a mixed bag.

There are now innumerable examples of AI automating predictable and verifiable tasks with spectacular productivity savings. Most of this came from the application of classical machine learning rather than generative AI. Applications in warehouse management, transport, industrial processing, mining and a range of other settings are now well established and validated. The hope is that new generative AI models will stretch these gains to unpredictable, complex and hard to verify settings. Here the results are much less promising.

As with autonomous driving, the evidence is that large scale complex, unpredictable and hard to verify systems like education delivery, clinical assessment, aged care classification, child protection, courts and sentencing cannot be automated without significant human intervention and oversight. Doing so risks real harm to individuals.

Even systems that are already heavily automated, such as energy grids, complex logistics, traffic management and customer relations management require overarching human intervention and oversight. The balance of automation and human oversight for high risk, complex systems that have the potential to cause serious harm to individuals will require detailed, contextualised, evidence based development which is likely to take extended periods of time. The removal of the human override in the Integrated Assessment Tool for aged care is a good example of a high stakes system being put at risk for a significant number of older people through automation over reach.

Nor is it clear that AI will enhance individual worker productivity without careful management. The most rigorous study we have is a pre-registered randomised trial of 758 Boston Consulting Group consultants, run with Harvard researchers and now published in Organization Science. On tasks inside the AI’s capability frontier, consultants using GPT-4 completed 12.2 per cent more tasks, worked 25.1 per cent faster, and produced work rated more than 40 per cent higher in quality. Those are large, genuine numbers.

But on tasks that fell outside that frontier, the same consultants using the same tool were 19 percentage points less likely to reach a correct answer than colleagues working without it.

The authors called this the “jagged technological frontier”. The boundary between what AI does brilliantly and what it does badly is irregular and cannot be predicted by intuition. Tasks that look hard may sit inside it; tasks that look trivial may sit outside. Worse, the failures are quiet — the output looks plausible and reads well and it is difficult to tell where the line is.

In July 2025 the research group METR ran a randomised controlled trial with experienced open-source developers working on repositories they knew well. Developers using AI tools took 19 per cent longer to complete their tasks because they spent time on evaluating, waiting for responses, rejecting and correcting AI code. And yet having just been slowed down, they estimated that AI had made them 20 per cent faster because it reduced the psychological burden of having to produce code – much like an author facing a blank page.

These results are contested by later research on productivity, particularly for novice operators with unfamiliar content. But the gap between actual productivity and user perceptions is durable, thereby making practitioner self reports unreliable. Effectively, use of AI makes people feel like they are more productive, but that perception has to be validated with actual performance metrics.

This goes some of the way in explaining why large corporations are not mass-retrenching staff despite three years of transformation rhetoric. The tools are cheap enough that the licence cost barely registers. The gains are real but unevenly distributed, hard to verify, and structurally difficult to convert into either headcount reduction or output growth.

The Architectural Question

Behind the enterprise productivity puzzle sits a deeper one.

Errors, hallucinations and context drift are usually discussed as bugs awaiting a patch. They may be closer to fundamental properties of the current models. Large language models operate by statistical pattern matching over token sequences, and that approach appears to have inherent limits in maintaining factual coherence and logical consistency across long, complex chains of reasoning. Scale may not lead to integrated, coherent symbolic reasoning and attention management that characterises human consciousness. This may be an inherent design limitation.

If that is right, more scale will not fix it. Reaching reliability sufficient for genuinely autonomous operation may require hybrid architectures that combine symbolic reasoning with neural learning — approaches structurally distinct from the current scaling paradigm. Whether such systems can deliver dependable performance even in narrow domains, let alone approach general capability, remains a serious open question.

But I recognise this is a contested claim rather than a settled one. The models keep getting better, and confident predictions about limits have a poor track record. But the failure mode documented in the BCG study — plausible, fluent, confidently wrong output at the edge of competence — is exactly what you would expect if the limitation is architectural rather than incidental. Current models are great for well defined domains (tasks) and as assistants for humans doing complex tasks. They are not so great for complex, unpredictable environments and tasks that have to be managed over extended time periods without human oversight.

The Dark Side: Industrialised Crime

While businesses debate fractional productivity gains, malicious actors face an entirely different economic equation.

Sophisticated fraud, automated phishing, deepfake extortion and malicious code generation can now be executed at near-zero marginal cost. Bad actors no longer need elite technical syndicates to mount convincing, multi-layered attacks. Generative models let them scale hyper-personalised scams and continuous network probing automatically.

Note the asymmetry with the enterprise picture. Criminal applications have no verification tax. A phishing campaign that works one time in a thousand is profitable; nobody needs to check the other 999 for hallucinations. The jagged frontier does not constrain you if you are indifferent to quality.

Overbuilding and the Financial Cliff

But the most precarious aspect of the current boom is the infrastructure race. Hyperscale technology companies are pouring unprecedented capital into data centres, specialised chips and dedicated water and power supply, on the assumption that brute-force scaling will eventually unlock general intelligence.

That expenditure increasingly relies on corporate debt and structured external financing rather than organic cash flow, which is what makes it fragile. If productivity gains stay stubbornly hard to bank, and enterprise customers decline to renew, the correction will be quick.

The physical infrastructure also faces early obsolescence from two directions. Algorithmic efficiency gains could make today’s compute footprint unnecessary faster than expected. And chip architecture moves quickly enough that multi-billion-dollar facilities filled with current accelerators risk becoming power-hungry stranded assets before depreciation cycles complete. And then there are the concerns about environmental and social impact that are rapidly gaining pace leading to tighter regulation and increased cost.

Worryingly for investors, this month the Chinese company Moonshot AI released Kimi K3 – a release that nicely demonstrates the potential risks. It performs as well as if not better than the best US models. You can download K3, modify it and run it on your own servers. And most importantly, it cost considerably less to develop. Unsurprisingly quick market correction followed the release.

The real world is that there will be a general expansion of AI licenses into administrative and knowledge work and in automation of a range of relatively straightforward applications. But large scale automation of complex, unpredictable systems that lead to vast productivity improvements (and mass human redundancy) don’t look likely in the near term at least. Of course, that doesn’t mean some organisations won’t use AI as an excuse for lay offs.

More importantly, whether the massive investment in infrastructure will result in a sufficient return on investment remains to be seen. The next two years are likely to tell the tale.

The Australian Picture

Australia’s position in all this is easy to assert and harder to evidence, but there is some data.

Widespread systemic adoption of generative AI to restructure business systems remains low and most use of AI focuses on providing individual workers with AI tools. The Australian Bureau of Statistics found that around 12 per cent of Australian businesses reported using AI in 2024–25, up from one per cent in 2022–23. Forty-six per cent said they were pursuing savings and efficiencies through innovation and new technologies. Adoption rises steeply with size, and concentrates in information, media and telecommunications (38 per cent), followed by professional, scientific and technical services and financial services (24 per cent each). Among small businesses, those already classified as innovation-active adopted at 19 per cent — almost five times the rate of those doing no innovation activity at all.

Two things follow.

First, the definitional chaos is itself the story. Depending on the survey, Australian AI adoption sits anywhere between 12 and 68 per cent — the CSIRO puts it near 68, the Ai Group at 52, the National AI Centre’s tracker at 37, MYOB at 29, the ABS at 12. When we cannot agree on how many organisations are using the technology, claims about what it is delivering deserve heavy discounting. The ABS will add further AI content to the 2026–27 survey, which should help.

Second, the adoption gradient by firm size matters more than any national average. Large enterprises have the technical teams, data infrastructure and organisational capacity to run pilots, measure results and iterate — which is precisely what the BCG evidence says you need to stay on the productive side of the jagged frontier. Smaller organisations, which are most of the Australian economy, are adopting the same tools without any of that scaffolding. They get the failure modes without the machinery for detecting them.

For public sector and community organisations, that is the practical warning. The licence is affordable. The capability to know whether it is working is not, and it is the part nobody budgets for.

A Sober Re-appraisal

Artificial intelligence is neither a herald of superintelligence nor a fraud. It is a genuinely useful and rapidly improving toolset that delivers real value under disciplined oversight. But, as Karen Hao’s book, Empire of AI demonstrates, the cut throat race to build it (between companies and the nations) has led to unregulated human and environmental exploitation, extravagant claims and truly remarkable levels of speculative investment.

The tools are remarkably cheap, but the industry has built an enormous physical and financial edifice on the assumption that capability translates directly into captured value, and the evidence for that translation remains thin. Freed hours are absorbed. Gains are jagged and quietly reversible. Practitioners cannot reliably tell when they are being helped. And an open-weights model from Beijing has just demonstrated how quickly the frontier commoditises. Ever increasing scale may not produce the predicted AGI models.

Navigating this requires giving up the fantasy of frictionless transformation, and taking seriously a harder question: not what the technology can do, but what any given organisation can actually verify, absorb and bank.