🔗 Share this article British Tech Companies and Child Safety Officials to Test AI's Ability to Generate Abuse Images Technology companies and child protection organizations will be granted authority to evaluate whether AI systems can generate child abuse images under recently introduced UK laws. Significant Rise in AI-Generated Harmful Material The declaration came as revelations from a protection monitoring body showing that reports of AI-generated child sexual abuse material have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025. New Legal Structure Under the amendments, the government will allow approved AI developers and child safety organizations to inspect AI models – the foundational technology for chatbots and visual AI tools – and ensure they have sufficient safeguards to stop them from producing images of child exploitation. "Ultimately about stopping exploitation before it happens," stated the minister for AI and online safety, noting: "Specialists, under rigorous protocols, can now identify the risk in AI systems early." Addressing Legal Challenges The amendments have been implemented because it is illegal to produce and own CSAM, meaning that AI developers and others cannot generate such images as part of a evaluation process. Previously, authorities had to delay action until AI-generated CSAM was uploaded online before addressing it. This legislation is aimed at averting that issue by helping to stop the creation of those materials at their origin. Legislative Framework The amendments are being introduced by the authorities as modifications to the crime and policing bill, which is also establishing a ban on owning, producing or sharing AI models developed to generate child sexual abuse material. Real-World Consequences This week, the official visited the London headquarters of a children's helpline and heard a mock-up conversation to counsellors featuring a report of AI-based abuse. The call portrayed a teenager requesting help after facing extortion using a sexualised deepfake of themselves, created using AI. "When I hear about young people facing blackmail online, it is a cause of extreme frustration in me and rightful concern amongst families," he stated. Alarming Statistics A prominent internet monitoring foundation stated that instances of AI-generated abuse content – such as webpages that may contain numerous images – had significantly increased so far this year. Instances of category A material – the most serious form of exploitation – rose from 2,621 visual files to 3,086. Female children were predominantly victimized, making up 94% of prohibited AI depictions in 2025 Depictions of infants to toddlers rose from five in 2024 to 92 in 2025 Sector Response The law change could "constitute a crucial step to guarantee AI products are safe before they are released," commented the head of the internet monitoring organization. "AI tools have made it so victims can be victimised all over again with just a few clicks, providing criminals the capability to make potentially limitless quantities of advanced, lifelike child sexual abuse material," she continued. "Material which further exploits victims' trauma, and makes young people, particularly female children, more vulnerable both online and offline." Support Session Information Childline also released information of counselling sessions where AI has been mentioned. AI-related risks mentioned in the sessions comprise: Using AI to rate weight, physique and looks Chatbots discouraging young people from talking to trusted adults about abuse Being bullied online with AI-generated content Digital extortion using AI-manipulated pictures Between April and September this year, the helpline conducted 367 support interactions where AI, chatbots and related topics were mentioned, four times as many as in the equivalent timeframe last year. Fifty percent of the references of AI in the 2025 interactions were connected with mental health and wellness, encompassing utilizing AI assistants for assistance and AI therapy applications.