India’s politics becomes an AI arms race as deepfakes threaten elections – and the risks are being felt globally

Rao’s party ultimately lost the election, with Congress going on to form a new state government.

In one doctored clip, Bachchan can be seen asking a contestant about a loan-waiver scheme for farmers in Madhya Pradesh, with the ensuing response and explanation casting the state’s then-Chief Minister Shivraj Singh Chouhan in a negative light and heaping praise on political rival Kamal Nath. But the exchange is a complete fabrication.

Other uses of AI during the state elections included the chief ministerial candidate for Congress in Rajasthan, Ashok Gehlot, having his voice cloned to send personalised WhatsApp messages addressing each voter by name, and in Tamil Nadu, the ruling Dravida Munnetra Kazhagam party using the technology to ‘resurrect’ former chief minister M. Karunanidhi, who died in 2018, so he could address the crowd at public events.

‘Everyone looked real’: deepfake conference call scam hits multinational in Hong Kong

The government issued an advisory in December to all social-media platforms calling on them to comply with the Ministry of Electronics and Information Technology’s rules on prohibited content, especially the ones related to material intended to deceive, mislead or impersonate another person.

Another advisory issued by the ministry earlier this month urged tech companies to obtain approval before the public release of generative AI models that are “unreliable” or still being tested, warning against responses that could threaten the integrity of the electoral process.

Despite the regulations, however, many experts believe it’s not possible to stop the spread of AI-generated content, especially deepfakes.

Part of the difficulty stems from the fact that India has yet to formulate any laws regulating deepfakes, which may prove impossible anyway, according to tech lawyer and policy researcher Anushka Jain.

“Labelling deepfakes is enforced by major companies like Google and Facebook,” said Jain, who is a research associate at technology research network Digital Futures Lab.

“However, the content creators may shift towards open-source generators that don’t require such labelling and are freely available in the market. So, technically it is not feasible to regulate deepfakes in India.”

‘A global problem’

The sudden emergence of fast-evolving generative AI has left many countries scrambling to legislate for the threats that deepfakes can pose to elections.

Some US states have already enacted specific laws to deal with the issue, and a bill to ban the use of materially deceptive AI-generated content from elections – known as the Protect Elections from Deceptive AI Act – was introduced to Congress in September.

Britain’s recently introduced Online Safety Act also aims to regulate some elements of AI, such as making the sharing of non-consensual deepfakes an offence.

Prateek Waghre, executive director of Indian digital rights organisation the Internet Freedom Foundation, said the group had published an open letter urging electoral candidates and parties to refrain from using deepfake technology in India’s coming elections.

“Political parties have the most power in the elections to decide on the use of deepfakes, everyone else is sort of playing catch up. But how realistic these parties can be, is another question,” he said.

Waghre says the biggest challenge is detecting deepfakes – then making sure they are appropriately dealt with.

“If authorities are not going to respond to deepfakes by criminalising them in some form, then it is going to be very difficult to enforce any regulation,” he said.

Indian social media has become flooded with deepfake content targeting politicians and parties. The country ranked sixth for vulnerability to deepfakes, according to The State of Deepfakes report released by Italian tech firm DeepTrace Technologies in 2019.

On Instagram, AI-generated videos showing Modi singing popular songs in several regional languages have surged in popularity recently.

But the deepfake content being created for political parties is mostly not created in-house, it’s outsourced to private consulting firms instead.

Divyendra Singh Jadoun, who runs a “synthetic media” start-up called The Indian Deepfaker, said it used to take his team 15 days to craft a 60-second video, but it’s now possible for anybody to create a deepfake in less than three minutes using a single image.

“It’s not limited to people with knowledge of coding. It’s now openly available. So I am worried that it can happen on a large scale during the general elections,” he said.

Jadoun, who told This Week in Asia he was working with a number of different politicians ahead of the coming polls, said many of India’s political parties were integrating AI into their electoral campaigns.

Ever wish you could hear some new Beatles songs? With AI, you can

It was his company who cloned Ashok Gehlot’s voice for the personalised WhatsApp messages the politician sent out ahead of Rajasthan’s assembly elections in November.

As generative AI continues to improve, the technology available to detect deepfakes is struggling to catch up, Jadoun said. “All the companies like Intel that are using deepfake detector tools can only detect old deepfakes of poor quality. However, the quality of deepfakes is increasing day by day.”

As an example, he cited an Instagram advert he came across featuring a faked interview with cricketer Virat Kohli, which the platform said didn’t violate its policies when Jadoun reported it.

“These platforms are not able to detect high-quality deepfakes. So we cannot stop it technically,” he said.

02:18

From K-pop to salesgirls: AI goes mainstream in South Korea

From K-pop to salesgirls: AI goes mainstream in South Korea

Hafiz Malik, a professor of electrical and computer engineering at the University of Michigan-Dearborn in the US, agreed that deepfake technology was evolving rapidly and would affect elections globally.

This was a particular concern in countries like India where WhatsApp is widely used and trusted, he said.

“Unlike platforms such as Facebook and Twitter, which have mechanisms like deepfake detection and alert generation to satisfy regulators, WhatsApp lacks such checks and balances,” Malik said.

Microsoft has likewise pledged to assign digital watermarks to content, providing information on whether AI was used in its creation, to assist campaigns and election watchdogs in dealing with cybersecurity threats.

Meta, the company that owns Facebook, Instagram, WhatsApp and others, has said it will require political adverts to disclose if AI was used in their creation, but such disclosures will not be needed for other unpaid posts.

But regulation alone won’t be enough, Malik said – stressing the need for continuous technological development and updates to effectively address the problem.

“You can develop a tool for the sake of satisfying the regulator, but it is bound to fail if it is not up to date. The only way to regulate deepfakes is to develop more advanced technology,” he said.

“At the end of the day, it is a technology, and technology can be used or abused.”

S.Y. Quraishi, a former chief election commissioner of India, told This Week in Asia that the only mechanism available to the country’s electoral authorities to stop deepfakes was educating voters about the issue – adding that rumours during election time had been a long-standing problem.

“I’ve witnessed how rumours during voting can lead to riots and booth capturing,” he said, referring to a form of electoral fraud whereby supporters, or their hired muscle, gain physical control of a polling booth to ensure their candidate wins through threats and intimidation.

Deepfake turns Ukrainian YouTuber Russian to sell goods online in China

“With deepfakes, spreading fake news is as easy as pressing a button, greatly increasing the risk of misinformation.”

Britain’s cybersecurity agency warned in November of the growing threat AI poses to elections, while the US Federal Election Commission has said it is looking at ways to regulate AI-generated deepfakes in political adverts.

“Deepfakes are a global problem. There is no law in the world that can regulate it,” Quraishi said.