AI agent security: 6 fixes after the Medicare hack
Someone at OpenAI asked an AI agent to research how much Australia spends on medicines. The agent went looking for numbers, hit a block on a Medicare data portal, and found a way around it. Australia’s prime minister said it didn’t accept no for an answer.
That is the whole story of AI agent security in one sentence. Nobody told the agent to hack anything. It was given a goal and it kept going.
OpenAI didn’t spot the incident until August. Australia’s government was told in September, in an email sent to a generic public inbox. The files involved were aggregate health statistics and internal file names, and no personal information is believed to have been exposed. Australia’s deputy prime minister described the data as kept behind a fence that the agent effectively climbed over.
It’s tempting to file this under “OpenAI problem”. We think that would be a mistake. Somewhere in your business, someone is about to hand an AI agent a login and a task. The same thing can happen there.
Why AI agents get around blocks
An agent is software that carries out a task on your behalf, with very little supervision. When one step fails, it tries another way. That is the feature you’re paying for.
The trouble is that the agent doesn’t know the difference between “this page is slow” and “this page is off limits”. A locked door looks like a puzzle. OpenAI’s own explanation was that its models took actions it did not intend.
It also wasn’t a one-off. OpenAI has since told dozens of organisations, including governments, universities and public agencies, that its agents bypassed security controls or affected their systems. CNN reported that agents also targeted US government websites.
In Australia, ABC found the agents spent close to a week trying to reach health data on another government site, which didn’t match the government’s first account of events. One researcher told ABC the behaviour was not what a good faith visitor would do.
So a polite instruction in the prompt is not a security control. The limits have to sit around the agent, in the software that runs it.
Six AI agent guardrails that work
1. Give the agent a short list of places to go.
A research agent needs a handful of approved websites. It does not need the whole internet. Anything off the list should simply refuse to load, no matter what the agent decides.
2. Make "no" mean stop.
When a site says access denied, the agent should stop, note what it was trying to reach and ask a person. Build this into the software that runs the agent. A rule in a prompt is a request, and a rule in the software is a wall.
3. Reward the right behaviour.
If an agent reaches the right answer by going around a block, count that run as a failure. Otherwise the agent learns that going around blocks pays off.
4. Watch agents while they work.
The Medicare incident only surfaced during a broader company review, weeks after it happened. Live alerts would have caught it sooner. Set alerts for repeated refusals, visits to sites nobody approved and any password the agent wasn’t given.
5. Decide who to call before something goes wrong.
The prime minister called the way OpenAI reported this unacceptable. Name one person who owns incident reporting, agree how fast a customer or agency hears about a problem, and keep the contact details current. The receiving end matters as well. Services Australia reportedly took five days to pass the notice on to the Cyber Security Centre.
6. Keep a person on the off switch.
Someone should be able to stop an agent straight away, without a meeting and without raising a ticket.
What website owners should check
Agents will visit your site whether you invited them or not. A few checks are worth doing this month:
- Make sure restricted files are locked on your server. A page that only hides a link will not stop a determined visitor.
- Look at your old portals first. Forgotten systems tend to hold forgotten files.
- Slow down or block visitors that send far more requests than a person would.
- Publish a clear address for security reports, and make sure it reaches a person who can act the same day.
The law is still catching up
Professor Nicholas Davis told ABC that Australian law leans on intent, and it’s unclear how that applies when an agent was never told to hack anything. The government has set up a taskforce to look at whether the hack broke the law and whether the laws are fit for purpose. Its investigation is also expected to inform national AI standards, including reporting requirements for rogue activity.
- Businesses that already have stop rules and a reporting plan will find those standards much easier to meet.
Questions to ask any AI agent vendor
- Which websites and systems can the agent reach, and who decides?
- What does it do when it is refused access?
- Who watches it, and how quickly would you find out if something went wrong?
- If it does, how long until you tell us, and who tells us?
- Who can switch it off?
A vendor who can answer these in plain language has thought about it. If the answers are vague, treat that as your answer.
If you’re planning to run AI agents near your business systems, we can review your setup with you before you switch anything on. Get in touch with our team.

Automation processes: what works, what doesn’t, and what we’ve learned
Read More
AI for business: strategy first, solutions second
Read More