AI: The good, the bad, and the ugly

Hal Swerissen

At this point in time, Waymo has rolled out driverless taxies in five cities in the United States. These vehicles use radar, lidar, cameras and audio to ‘see’ the road. The data from these sensors is processed by an artificial intelligence system that makes sense of what is seen and translates the data for acceleration, steering, signalling and braking to drive the car. 

The Waymo cars have to learn how to operate in each city through an initial mapping process that is then kept up to date as the cars go about day-to-day journeys. This is a kind of crowd sourcing that is sent up to the cloud and then back down to the fleet.

Waymo nicely illustrates the key elements of artificial intelligence. You need a set of sensory information that is digitised into data and then analysed to make predictions that can be used to generate useful responses. The difference between traditional programmed computing and AI systems is that AI systems learn from the data and program themselves. The most recent versions do this continually.

The development of artificial intelligence, particularly generative artificial intelligence is arguably the third wave of digital technology to fundamentally alter economic and social life. The first was the development of the general programable computer. It evolved from valves to transistors and then solid-state chips with an incredible increase in calculation capacity. 

The second wave was the development of computing interconnectivity through the internet, the world wide web and cloud computing. That led to the aggregation of massive data sets from billions of users. Effectively, all the information in the world has become accessible to everyone, at least notionally.

And now artificial intelligence means self-programming computers that learn from the data they process can automate human thinking and decision making for an increasingly vast range of ever more complex tasks that previously could only be done by people. 

AI has been around in one form or another for over 50 years, from early systems that learned how to play digital ping pong to simple chatbots that kept people engaged by general encouragement and open-ended questions. 

By the early part of this century, AI was already widely employed in manufacturing robotics, warehouse logistics, inventory management and modelling of complex systems. These ‘narrow’ applications are extremely effective of predicting and managing routine tasks based on unambiguous and reliable data within a well-defined domain.

The spectacular breakthrough in AI came when it became possible to generate predictions and responses based on hugely complex patterns of often ambiguous information like that involved in language systems. 

That happened about 10 years ago when AI systems moved from serial to parallel processing and then transformer chip architecture. Clever maths and redesigned chips made it possible to use context to decide what to pay attention to in processing huge data sets, like all the text on the World Wide Web.  Think of it as the difference between ‘bat’ in the phrase, ‘bat and ball’ and the phrase, ‘bat the ball’. Context and attention are everything. The result is enormously powerful systems that we can interact with as if they are human, like ChatGPT, Copilot, Gemini, Claude and Anthropic.

It only became possible to process the trillions of calculations necessary to find the patterns and generate the responses these AI systems produce by using graphic processing units or chips that had been developed to handle the huge demands of video gaming. As a result, Nvidia, initially a gaming chip maker has become the world’s most valuable company.

These new AI models generate novel responses in text, images and video based on what they have learned and what you ask them to do. They do so, in much the same way as human brains, using artificial neural networks.

Originally, my interest was in how humans processed information, particularly how children first learn basic number facts and then use them for much more complex arithmetic calculations. Our brains are made up of about 80 billion neurons with probably 100 trillion synaptic connections between them. When we learn something, a pattern of connections is consolidated in response to particular sensory input, be that what we see hear, smell, touch or experience from bodily sensations. These patterns are neural networks made up of millions of neurons firing together. This is what we record on electro encephalographs and MRI and PET scans of brain function. It is literally what occurs when we think. Our brains are electro chemical computers.

Generative artificial intelligence is based on the development of artificial neural networks that mimic our brains. These networks are developed by strengthening connections between large numbers of digital switches organised as a series of layers between the input signals from the sensors, and the responses or outcomes. Current models, like ChatGPT have 1.8 trillion parameters or connections organised in 120 layers, giving them enormous processing power for large data sets. Still not as sophisticated as our brains, but getting there.

The new GAI models are rapidly evolving. Currently they are widely used to enhance complex tasks, including data management, analysis and summary, coding and query responses, particularly using chatbot functions. Chatbots are probably responding to about 4 to 5 billion human prompts per day at the moment, and growing rapidly.

The emerging extension of GAI is the development of AI agents capable of managing a complex of inputs and carrying out a set of autonomous tasks without human prompting. In health settings, this might include AI monitoring patient recall systems, ordering relevant tests, reviewing the results and alerting staff for follow up activities where required. Companies like Oracle are rolling these systems out.

It is unlikely, that current GAI models will completely replace professional expertise, but it is almost certain that GAI will automate a range of complex tasks that are part of professional practice. That has happened many times before.

When I first taught statistics to university research students, they had to learn to convert theoretical models to calculation formulae so that they could do their own calculations. Today all these calculations are automated. When I was at the Grattan Institute we ran the statistical modelling for the spread of COVID every night for the whole Australian population. Something that would have been impossible when I first taught these techniques.

In the same way, GAI will replace professional activity to summarise a range of diagnostic, triage, clinical monitoring, medication management and staff alert functions. It is already being used to summarise clinician-patient interactions, analyse scans and synthesise information for differential diagnoses. Professional activity will shift from these tasks to a greater focus on action and responses.

At the moment most GAI use is still prompt driven. Agentic systems are mainly in the ‘proof of concept’ phase. Large scale roll-outs are few and far between, particularly in health where the risks are high. And there are real risks.

There have been significant disasters trying agentic systems too quickly. MacDonalds tried to use agentic AI in drive through ordering only to discover that that the system couldn’t handle the variety of customers and ordering requests accurately and reliably in the noisy and chaotic environment of the drive through lane. Tesla, which doggedly relies only on camera input for self-driving, has found itself in the ‘uncanny valley’ where self-driving is very good, but occasionally makes weird and frightening decisions, sometimes killing the occupants. This includes ‘long tail’ errors like confusing a shadow with a deep pothole and braking sharply in the middle of busy traffic.

In health settings, IBM’s attempt to create a ‘super doctor’ for cancer treatment was a $5 billion failure. Watson couldn’t manage the variability of data from actual clinical practice. Instead, trained on a relatively small set of synthetic, curated data if failed miserably in the messy real world of oncology, agreeing with clinicians and guidelines only about 30% of the time. It mostly told clinicians what they already knew and often suggested unrealistic treatment options that didn’t exist in local settings.

For high-stakes settings like health decisions, we will continue to need legally enforceable ‘human in the loop’ controls, even when GAI systems are more accurate than humans on average. When very accurate systems produce very low probability errors that a human would never make (the ‘uncanny valley problem), trust in automated systems is undermined. Imagine, for example, a zero-shot clinical error like suggesting high-dose opioids for a paediatric case or missing a critical negative finding in a pathology report. Trust built up over hundreds of correct decisions is smashed.

Careful consideration will have to be given to how these controls are put in place. Highly accurate agentic AI can easily lull busy decision-makers into taking their hands off the steering wheel. Automated systems will need to include mandated fail-safe review and sign-off steps.

In the immediate future AI systems will replace a range of relatively low risk dull, dirty, dangerous and difficult tasks that humans don’t like doing and don’t do well. They will assist and enhance humans to do more complex tasks more efficiently and effectively. There will be mandated, risk-based requirements for humans to be in control when decisions are risky. These trends are already happening and will only accelerate.

Most large organisations are now developing their AI plans in a world where there are any number of AI platforms, products, services and consultants on the market. Not surprisingly, decision-making then becomes messy, and for smaller organisations, there is probably no reason to rush.  Being in the sensible second tier of adoption avoids the risks, costs and errors of being on the bleeding edge. Those of us old enough to have been around when the internet came into being know that you really don’t want to have invested in Netscape Navigator as your web browser or Looksmart as your search engine.

Despite not rushing, it’s difficult to escape the need to develop policies and procedures for the use of AI. Amongst other issues that have to be addressed, there are privacy concerns about using in-house data with public chatbots; there are approved and unapproved enterprise-level tools; there are rules for compliance with data protection; there are intellectual property issues; there are verification requirements; and there are rules about when it’s sensible and not sensible to use AI tools.

There are broader societal costs and risks too. AI requires huge data centres which take up large volumes of water and energy. They already consume about 1.5% of global electricity and that is likely to grow rapidly putting pressure on emissions and climate change.

 A range of routine and entry-level jobs will likely disappear in the near future. This includes data entry staff, customer service staff, administrative and research assistants, graphic designers, data analysts, shop assistants, factory workers and drivers. That means there are concerns about the future of work and a ‘just transition’ for those who are displaced.

Technology control will become more centralised and dominated by a limited number of national and corporate players. Organisations will be locked into technology ecosystems. Nations are likely to treat AI capabilities as a strategic asset. Trade wars, intellectual property theft, restrictions and access barriers are likely. We have already seen this in the hyper-partisan contest between US and Chinese technology, complete with security fear-mongering.

Worse, AI has a genuinely ugly and exploitive side. It is increasingly being used to scam, extort and kill people. Bad actors are using deep fakes to mislead. Cybercrime and fraud are on the rise. Identities are being stolen and children are being groomed. Military applications have developed autonomous drones capable of deciding when to kill people.

These are challenges for governments and governance. Legislation, regulations and oversight are not keeping up with the development and implementation of the technology. The most significant risk is that poorly designed and controlled, scaled up use of generative AI to manage large, complex, high risk systems including transport, logistics, communication and energy infrastructure could lead to catastrophic failure if the ‘uncanny valley’ errors happen at scale or if there are unintended consequences because AI goals are misaligned or incomplete.

On the other hand, the doomsday scenario of a fully conscious, universal artificial general intelligence developing its own motivation to eliminate humans is unlikely. Projecting human emotions and motivations to AI is a conceptually risky business. Machines do not have our evolutionary history and inbuilt drives for reproduction and survival. They are not embodied and do not process or experience the world in the same way as we do. 

The more likely scenario is that generative AI will continue to progressively evolve and that we will see much more personalised, responsive systems develop, including personalised health care where AI manages and coordinates routine tasks, personalises answers to questions, provides tailored advice and, as robotics evolves, carries out a range of activities for us at home and work.

In the short term, GAI is probably overhyped. Like self-driving cars, there are signs of progress but it’s taking much longer than expected with more limited results than hoped for. Wide spread scaled up use that dramatically improves effectiveness and efficiency across complex tasks and systems has yet to emerge.  

But there is a longer term, think 50 or 100 years, and, in the longer term, the impact of AI on the future of industry, the environment, work, health and human services, our relationships with one another and technology, and our views about thinking, consciousness and being human are likely to be very different from today. AI is a fascinating technology with enormous potential for both good and bad.

Hal Swerissen is emeritus professor of public health at La Trobe University. He has a longstanding interest in cognitive science, learning theory and artificial intelligence and its alternatives.