Apple has removed three apps from the iPhone's App Store after discovering that their AI image generation feature could be used to create non-consensual nude images. The move comes amid rumors that Apple is developing new generative AI features of its own, which will likely be showcased in iOS 18 at its WWDC event in June.
The app was originally discovered earlier this week, and Apple appears to have only removed the app after it was reported online. In fact, according to a report detailing the news, Apple was unable to find the app in question and needed help identifying it before removing it.
It's unlikely that any of the generative AI features Apple is rumored to be working on will be able to perform the same functions that these apps do, but they still pose an interesting conundrum for Apple. How do you market AI capabilities, especially in a world where public trust in them is waning?
removed
404 Media reports that it was able to find the app after discovering it in Meta's ad library, a feature that archives ads available on the platform. Two of the ads found were web-based, while three were for apps that can be downloaded from the App Store. Meta removed the ad after it was made aware, according to the report. However, 404 Media said Apple “initially did not respond to a request for comment on the story, but contacted me requesting more information after the article was published.” Then, a day later, Apple confirmed that it had removed three apps from the App Store.
The report also stated that “removal occurred only after we provided the company with links to specific apps and associated ads, indicating that the company was unable to find apps that violated the policy itself.”
Apps similar to the one Apple removed use generative AI to undress people by using AI to manipulate existing photos to make it look like someone is nude. The report notes that these apps and the images they generate have already made their way into schools across the country. Some students said they found the apps they used on TikTok, but other social networks also show ads for similar apps, 404 Media's report notes.
As is often the case with new technologies, the world is currently struggling with an influx of new AI tools and their capabilities. Sometimes these features can be amazing, but sometimes they can be used to cause harm, as is the case with these apps. Apple will no doubt do its best to prevent similar apps from ever entering the App Store again. But it will certainly raise questions about how the app got into the store in the first place.