Is AI really the problem?

Leading cyber security experts say that the anodyne antics of agentic AI are a distraction from the more serious issue of data breaches and cyber intrusions. 

In the space of a week, Google admitted its Gemini AI model hacked three companies, OpenAI failed (again) to stop its AI agents hacking Medicare, the icon of Australia’s public medical infrastructure, and all the while, a global hacking collective known as ‘ShinyHunters’, responsible for the Canvas hack in mid-May, claimed it hacked into the United States’ Federal Bureau of Investigation (FBI) and stole agent details. Reuters reports that highly sensitive information was taken, including real agent names, addresses, surveillance targets and other highly sensitive information. The Dutch police recently arrested a suspect for the alleged FBI hack. 

Back in July, as OpenAI’s agents were exploiting flaws in Hugging Face, and Anthropic’s agents were doing the same with other companies’ systems, another group of hackers, thought to be state-sponsored, attacked the operational technology of several water utilities across the United States, causing loss of water pressure and flooding, which can result in untreated ground water seeping into pipes. 

Professor Ciaran Martin, the founding CEO of the UK’s National Cyber Security Centre, says the cause of data breaches and cyber intrusions relates to the underlying cyber security being so weak “hackers have no need of expensive new techniques. AI doesn’t yet give most hackers magical new powers.” Professor Martin believes¹⁰ that while the world was preoccupied with OpenAI’s flawed testing environment, critical water infrastructure in the United States was being attacked by alleged Iran-sponsored hackers using “using crude techniques exploiting basic security failures, not advanced AI.”

Australian cyber security commentators, James Wilson and Tom Uren, writing for Risky.Biz, regard the activities of ShinyHunters as 'real and very serious'. In May this year, ShinyHunters hacked into the Canvas learning management system, causing widespread interruptions to the operations of schools, universities and other educational institutions worldwide. 

While the Medicare incident is but the latest example of an agentic AI system going beyond the limitations set by its human operators, euphemistically described as ‘misalignment’, and while the dossier of harms from agentic AI systems grows, it is important for governments, legislators and policymakers not to lose sight of the cyber security issues facing government departments and entities, corporations, small businesses and other organisations. As recent events demonstrate, all remain vulnerable to cyber-attacks, with or without the assistance of AI. 

Australia continues to implement the Australian Cyber Security Strategy 2023-2030 with the commencement of Horizon 2 this year. Horizon 2 includes commitments to strengthening “public, private and international collaboration on AI and quantum security threats, including leadership of the Five Country Ministerial AI Working Group and improved crisis management frameworks for major AI incidents,” among other things. 

In May, around the same time as the Canvas hack, the Department of Home Affairs released a policy advisory for government departments and entities to start getting ready for the frontier AI era. Also in May, Australia’s lead agencies for cyber security, the Australian Signals Directorate (ASD) and the Australian Cyber Security Centre (ACSC), along with their Five Eyes partners in cyber, advised that careful adoption of agentic AI services is essential. Following the hack on Medicare, the Secretary of the Department of Home Affairs directed government departments and agencies to review their ‘legacy technology system'. A week before the Medicare hack was made public, Abigail Bradshaw, the head of ASD, warned that "AI will either do one or the other: it’ll be a catalyst for decisions about replacing legacy technology; or, if we don’t move fast enough, almost certainly the legacy technology will be the first to be compromised in a major AI-enabled attack." 

The Security of Critical Infrastructure Act 2018 (Cth) (SOCI) already places extensive cyber security obligations and responsibility on the owners and operators of critical infrastructure, such as water utilities, banks and telecommunications, to secure their systems from unauthorised access, including access orchestrated by humans and AI agents working together or alone. The Cyber Security Act 2024 (Cth) provides the regulatory framework for managing and responding to cyber security risks, including mandatory reporting of ransomware attacks, information sharing in relation to major cyber security incidents and enabling post-incident review of major cyber security incidents. The government is currently considering proposed amendments to SOCI to plug potential gaps relating to AI and other emerging technologies. 

The Medicare incident was not an aberration. It was not accidental. It was a known risk of agentic AI systems. The Prime Minister made in-roads on AI regulation and governance in his address to the United Nations during his recent tour of the United States. Mr Albanese also had strong words with OpenAI’s Sam Altman over the Medicare intrusion. The creation of the Office for AI within the Prime Minister’s department and the establishment of the AI Safety Institute will go some way to maintaining oversight of developments in artificial intelligence. 

But the big question from the last couple of months is this: is Australia doing enough?