NYC Mayor Casually Announces Deepfaking, Experts Scared

Mayor Eric Adams of New York City has been creating audio deepfakes of himself to communicate with the diverse population of the city in various languages.

During a press conference on Monday, Adams revealed that his office has been utilizing robo-calling technology to reach out to New Yorkers in various languages that the mayor does not speak. This algorithmically-generated version of his voice was accidentally disclosed while briefing reporters on a recent “chatbot forum” held at City Hall.

During the Monday press conference, the mayor made an announcement about the implementation of AI-powered voice chatbots that will communicate with residents in their preferred language when they call the city’s 311 or other services. However, he also revealed that the technology has already been in use for other purposes, and New Yorkers may not be aware of the extent of his language capabilities.

During the news conference, Adams mentioned that his team has initiated robocalls with his voice in various languages. He shared that individuals frequently recognize him on the street and express their surprise by saying, “I didn’t know you speak Mandarin.”

According to him, they are using robocalls in various languages to cater to the diverse population of New York.

According to an email sent to Motherboard, the city has been utilizing AI robocalls since March 2022. These calls have been sent out in various languages, including 5,000 in Spanish, 250 in Yiddish, 160 in Mandarin, 89 in Cantonese, and 23 in Haitian Creole. The purpose of the calls is to inform New Yorkers about hiring events and a free concert series.

Unfortunately, the robocalls fail to mention that they are automated, leading many New Yorkers to believe that Mayor Adams is capable of speaking Spanish, Mandarin, and Yiddish.

The Surveillance Technology Oversight Project (STOP) released a statement following the press conference on Monday, denouncing the robocalls and labeling them as deceptive. Executive Director Albert Fox Cahn expressed his disapproval, stating that this act is highly unethical, especially when using taxpayer money. He criticized the use of AI technology to mislead New Yorkers into believing that the mayor can speak languages that he cannot, deeming it a deeply Orwellian tactic. While acknowledging the need for announcements in all native languages, Cahn believes that the use of deep fakes is nothing more than a disturbing vanity project.

In a statement, STOP expressed its criticism towards the use of AI and related technologies, such as facial recognition, by the city. The organization deemed the Mayor’s actions as enabling some of the most severe AI abuses in the nation and accused him of hiding behind the guise of AI responsibility.

There is growing concern over the use of algorithmically generated images, videos, and voices in political campaigns. The main issue is that these technologies can be misleading, leading to calls for mandates to require disclosure to constituents about their use. A bill in the New York State Assembly aims to do just that for political ads. However, there is currently no similar legislation or regulation that mandates disclosure in other forms of government communication, such as robocalls or 311 requests.

According to the city, the robocalls were created using Voice Lab, a well-known solution developed by Eleven Labs. This company was established by former employees of Google and Palantir and enables users to generate a replica of their own voice by simply uploading an audio sample to the website.

In January, it was discovered by Motherboard that a group of 4chan users had been utilizing a beta version of the service to create the voices of public figures like Joe Rogan, Ben Shapiro, and Emma Watson. Unfortunately, these generated voices were being used to make derogatory and offensive remarks about marginalized communities. The company responded by acknowledging the “misuse cases” and promised to implement additional safeguards. Then, in February, a Motherboard journalist used the same service to bypass voice authentication and successfully gained access to his own bank account. Despite this security breach, the company did not provide a comment in response to the incident.

Mayor Adams promotes the ethical utilization of AI and positions himself as a tech-savvy individual.

During the press conference, Adams emphasized that their approach towards AI is not about avoiding it, but rather ensuring responsible usage through proper governance. He pointed out that anything can be used or abused and refraining from progress due to fear of misuse would hinder productivity. “You can use or abuse anything, and if we stay away from moving forward because we’re afraid someone is going to abuse it, you won’t get anything done,” Adams stated.

Also Read: