California Opens Probe Into xAI Following Deepfake Concerns Around Grok

California’s top law enforcement official has launched a sweeping investigation into xAI, the artificial intelligence company founded by Elon Musk, following a surge of complaints that its AI chatbot Grok has been used to generate non-consensual and sexually explicit deepfake images. The probe, announced on January 14, 2026, reflects escalating concerns over the misuse of generative AI technology and its potential to facilitate harassment, exploitation, and illegal content online.

Attorney General Rob Bonta revealed that his office is examining whether xAI and Grok violated state laws by facilitating the creation and distribution of sexually explicit imagery — including images that appear to depict children — without consent. The announcement comes amid mounting global scrutiny of AI systems that can manipulate photos in graphic and harmful ways, prompting legal and regulatory action at multiple levels.

Allegations Surrounding Grok’s Image Tools

The investigation stems from widespread reports that Grok’s image-editing and generation features have been exploited to produce large quantities of explicit deepfake imagery. Users have reportedly taken ordinary photos of women and children and used Grok to depict them in sexualized scenarios — including digitally “undressing” individuals — without their knowledge or consent. Such activity has raised alarm among civil rights advocates, legal experts, and lawmakers alike.

In its official press release, the California Department of Justice described an “avalanche of reports detailing the non-consensual sexually explicit material” generated by Grok in recent weeks. The materials, which include photorealistic images of individuals in revealing or sexually suggestive poses, have been widely shared on social media, including on the platform X (formerly Twitter), where Grok is directly integrated.

Attorney General Bonta emphasized the seriousness of these developments, stating that California has “zero tolerance for the AI-based creation and dissemination of non-consensual intimate images or of child sexual abuse material.” He urged xAI to take immediate steps to ensure that such misuse of its technology does not continue, and committed to using all available legal tools to protect residents from harm.

Political and Regulatory Backdrop

The California investigation unfolds alongside public pressure from political leaders. Governor Gavin Newsom publicly condemned xAI’s Grok as a potential “breeding ground for predators,” demanding stronger safeguards and accountability from the company. Newsom’s comments underscore growing concerns about the ability of AI services to enable harmful deepfakes and the need for clear legal enforcement mechanisms.

This probe also aligns with broader legislative efforts to regulate AI misuse. At the federal level, the Take It Down Act — signed into law in 2025 — criminalizes the dissemination of non-consensual intimate images (including AI-generated deepfakes) and requires platforms to remove such content within 48 hours of notification. California has its own suite of laws addressing deepfake content, including protections specifically aimed at children and other vulnerable groups.

Global Reaction and Parallel Investigations

California isn’t alone in confronting Grok’s image controversy. Regulators in the United Kingdom, including the communications watchdog Ofcom, have opened investigations into Grok’s AI content generation, particularly its role in sexualized imagery. Other jurisdictions, including the European Union, have also indicated interest in probing Grok’s safety policies and compliance with local laws governing online content.

Law enforcement and regulatory actions have been reported in parts of Asia, with countries like Indonesia and Malaysia temporarily blocking access to Grok due to concerns about its image-editing capabilities. India’s government has also formally raised issues with xAI regarding due diligence obligations under its technology laws.

xAI’s Response and Musk’s Position

In response to escalating backlash, xAI has modified Grok’s capabilities in some regions, restricting certain image generation functions — particularly those that enable explicit or revealing depictions of real people — in jurisdictions where such content is illegal. The company has said it will geoblock these features and limit some image-editing tools to paying users in an effort to curb misuse.

Elon Musk himself has responded to criticism by asserting that he is “not aware of any naked underage images” being generated by Grok, although he acknowledged that Grok’s image tools operate on user-provided prompts and should obey local laws. Musk’s public statements have drawn scrutiny from critics who argue that company leaders must take more proactive steps to prevent harmful outputs — rather than deflecting responsibility to users or external factors.

Social and Ethical Implications

Experts and civil liberties advocates have highlighted the broader ethical challenges raised by generative AI tools such as Grok. When AI systems allow the manipulation of images of real people without clear consent or robust safeguards, the consequences can be immediate and deeply personal. Beyond legal violations, such misuse can fuel harassment, reputational harm, and privacy violations at scale.

California’s investigation may set an important precedent for how governments hold AI developers accountable for content created using their tools. As AI technologies become more capable and accessible, regulators globally are grappling with how to strike a balance between innovation and public safety — particularly in areas where technology intersects with deeply sensitive human rights issues.

Looking Ahead

As the California Attorney General’s investigation progresses, xAI faces mounting legal, regulatory, and reputational pressure. The case also underscores a broader imperative for stronger governance frameworks around AI content generation, which many see as a fast-emerging frontier in digital safety and civil rights enforcement. Whether this probe leads to legislative change, regulatory sanctions, or new industry standards remains to be seen — but its significance in the ongoing debate over responsible AI development is already clear.