The authors of a report testing the know-how underpinning Australia’s social media ban have conceded ChatGPT was utilized in enhancing, however denied plenty of quotation errors within the report had been attributable to AI hallucinations.
The $3.48m age assurance know-how trial, run by the UK-based Age Verify Certification Scheme (ACCS) final 12 months, examined varied varieties of know-how that could possibly be utilized by social media platforms as a part of Australia’s under-16 social media ban.
The communications minister, Anika Wells, heralded the report as displaying “many efficient choices” for checking folks’s ages, and it paved the best way for the ban to come back into impact in December final 12 months.
Many of the report is predicated on checks the authors did of the assorted age-verification applied sciences at the moment out there, however one chapter covers “rising applied sciences” and cites plenty of journal papers about potential new methods to examine ages.
An ongoing Senate inquiry in search of to additional strengthen the social media ban regulation obtained a submission highlighting a minimum of two quotation errors within the report. The “citations … seem like AI hallucinated, slightly than being primarily based on actual sources,” it stated.
When requested concerning the allegation by Guardian Australia this month, a spokesperson for ACCS initially denied using AI within the report, however later conceded that AI had been used to rewrite paragraphs extra succinctly.
“We didn’t use AI within the era of the report or the cited supplies … Every considered one of them had been checked as real hyperlinks and experiences and that they had been related to the particular difficulty being cited,” they stated.
Nonetheless, Guardian Australia’s personal evaluation of the report recognized six references in the identical part that contained errors.
The errors embody: Digital Object Identifiers (DOIs – distinctive combos of letters and numbers that are related to particular tutorial articles) that hyperlink to papers that don’t exist; DOIs that time to incorrect papers; creator identify, journal and publication 12 months combos that don’t match any identified references; and DOI hyperlinks to papers that didn’t say what the report instructed they stated.
The spokesperson later offered clarifications and new references to the analysis ACCS had relied on, however this response too contained errors.
A listing of journal articles meant to appropriate these within the report included years, authors and journal titles not matching the unique citations within the report.
One article ACCS cited was apparently accessed in March 2025. Nonetheless, one of many lead authors of this research confirmed to the Guardian that their article was not publicly accessible till it was printed in June 2025. In addition they confirmed that the particular person initially cited by ACCS because the paper’s lead creator, “Jamil”, had by no means been on the paper, and that the ACCS report had summarised their paper incorrectly.
The ACCS spokesperson offered additional clarification relating to one other reference that would not be discovered within the journal cited – “Weber et al. 2011” – by saying it was cited in a research by Monash College, which they offered to the Guardian. Nonetheless there was no reference attributed to Weber within the Monash paper ACCS despatched.
The spokesperson solely conceded AI had been used after the Guardian recognized 4 hyperlinks partly E and half Okay of the report that contained metadata figuring out that the supply of the hyperlink was ChatGPT.
They maintained that by together with the metadata of ChatGPT within the hyperlinks the organisation had disclosed its use, and given the hyperlinks had been appropriate, they didn’t see that as an issue.
They stated AI had been used to rewrite some paragraphs extra succinctly, however was not used for originating analysis or for the era of the report.
“All the hyperlinks had been checked and verified and, while I apologise if there may be an inadvertent error, this was all finished by human verification.”
after publication promotion
The federal authorities has beforehand taken a dim view of contractors utilizing AI in experiences to authorities. Deloitte final 12 months refunded a part of its $440,000 contract with the federal government after errors had been recognized, and the advisor agency admitted to utilizing AI.
Inquiries to Wells’ workplace had been redirected to her division.
“The division is analyzing the issues raised beneath and can have interaction with ACCS as acceptable,” a spokesperson for the division stated final Friday.
In a listening to for the Senate inquiry on Friday, division officers stated they met with ACCS, who claimed that the errors had been attributable to hyperlinks breaking that had beforehand labored. The officers haven’t independently verified this, stating it might be “very tough” to return and examine.
They stated it was a “handful of errors” in 26 pages of citations for a 1,000-page report.
“[ACCS] assured us that the hyperlinks had been all checked on the time that the doc was printed, and so they all labored then,” the division’s first assistant secretary, Sarah Vandenbroek, stated. “So, one thing could have modified within the interim. [They] did additionally guarantee us that whereas a few of the hyperlinks could have been damaged, although the supply paperwork do nonetheless exist, they’re nonetheless legitimate.”
Prof Christian Downie, from the varsity of regulation and international governance on the Australian Nationwide College, who has beforehand examined submissions to authorities that contained AI hallucinations, stated if experiences or submissions to authorities are discovered to have false citations in them – AI or in any other case – it might result in dangerous choices, and “erode public confidence and belief” in establishments.
“One would assume the federal government is already serious about this, and that’s going to must require contractual preparations and presumably penalties to discourage one of these behaviour,” he stated.
Fatima Payman, an impartial senator, stated: “The saga of final 12 months’s Deloitte AI slop report ought to have marked the top of slack referencing in experiences by authorities contractors.”
“The federal government should demand an apology and refund from the contractor behind the report,” she stated.
