Prime Minister Anthony Albanese today announced a new set of mandatory standards for AI companies following a review of an agent-led infiltration earlier this year.

Mr Albanese says the Australian AI Standards legislation will provide “world-leading” protection, which will be enforced by a new artificial intelligence safety regulator.

The regulator will enforce “significant penalties” for AI companies that fail to comply with the obligations.

AI companies will have to: maintain security controls across development and deployment; identify and manage serious risks associated with their models; allow independent evaluation; and provide mandatory notification of incidents.

“Artificial intelligence presents enormous opportunities for Australia, but we must also be prepared for the new risks that come with rapidly advancing technology,” Mr Albanese said.

“These protections will ensure we keep Australians safe and secure our national interests, while seizing the generational opportunities that AI offers.”

A rapid review of the OpenAI agent infiltration of a government health website earlier this year has produced 18 recommendations, which the government says it accepts and will implement.

In August, regulators urged insurers to take “decisive action” as new AI tools increase the cyber threat.

More details on the legislation will become available as the government completes further work and consultation.