How disinformation was used to try inflame India Pakistan tensions

Key Concepts

• Deepfakes: Synthetically generated media, often videos, that depict individuals saying or doing things they never did.
• Disinformation: False or inaccurate information that is deliberately spread to deceive.
• Media Literacy: The ability to access, analyze, evaluate, and create media in a variety of forms.
• AI Amplification: The use of Artificial Intelligence to rapidly and widely disseminate information, including disinformation.
• Polarization: The division of a society into sharply contrasting groups.
• Authoritarian Rule: A form of government characterized by strong central power and limited political freedoms.

Deepfakes and Regional Tensions: The Pakistan-India Case Study

This segment of the transcript details a stark warning about the escalating threat of disinformation, specifically focusing on the circulation of deepfake videos designed to inflame regional tensions between Pakistan and India.

The Deepfake Incident

• Nature of the Disinformation: Deepfake videos are being widely shared on social media, making entirely false claims about rising tensions between Pakistan and India.
• Specific Example: A deepfake video was created by twisting words from an interview with Alima Khanum, the sister of former Pakistani Prime Minister Imran Khan.
• Fabricated Claims: The deepfake falsely attributes to Alima Khanum an accusation that the Chief of Pakistan's Army, Asim Munir, desires war with India due to his "Islamic radicalization and conservatism."
• Original Context: In the actual interview, Alima Khanum was discussing her brother's imprisonment and only mentioned Field Marshal Munir in the context of blaming him for Imran Khan's detention. She made no reference to a conflict with India.
• Attribution of the Deepfake: The transcript implies the clips were shared by politicians in Pakistan.

Spread and Impact of the Deepfake

• Viral Circulation: The deepfake clip went viral on social media and was picked up by mainstream media outlets in the subcontinent.
• Mainstream Media Adoption: Press on the subcontinent treated the deepfake as legitimate news, with headlines reflecting the fabricated claims.
• Platform for Spread: The transcript notes that the clip was shared and reposted thousands of times on X (formerly Twitter) in both India and Pakistan.
• Lack of Fact-Checking: Alarmingly, not a single post on X had a community note fact-checking the claims.
• Shaping Narratives: The deepfake quickly shaped mainstream narratives, with several media outlets reporting on it without verification.
• Fragile Media Ecosystem: This highlights the polarized and fragile media ecosystem in the subcontinent, where disinformation can easily infiltrate news cycles.
• Dangerous Implications: The spread of such disinformation is particularly dangerous given that India and Pakistan are nuclear powers. The transcript states that this can have "dangerous and detrimental impact on not just society but policy."

Contributing Factors to Vulnerability

• High Internet Penetration, Low Digital Literacy: South Asia has extremely high internet penetration but very low media and digital literacy. This combination makes the region incredibly vulnerable to manipulated videos.
• Rapid Dissemination: A clip that appears authentic can travel through millions of phones before the possibility of it being fake is even considered.
• Generation and Verification Lag: The deepfake appears to have been generated quickly after the interview and picked up rapidly without proper verification.
• Nationalistic Lens: The deepfake is instantly picked up across the border and interpreted through a nationalistic lens, amplified by partisan media and online networks.
• Feedback Loop: This creates a feedback loop where fabricated content contributes to outrage, deepens polarization and division, and pressures governments to respond to events that never occurred.

The Challenge of Reversing Disinformation

• Difficulty in Reversing Narratives: It is extremely hard to reverse a narrative once it has been established by deepfakes intended to influence public opinion.
• Limited Impact of Clarifications: Even with clarifications from the individual targeted by the deepfake, it is unlikely to reach and convince the public, as the fake content is already widespread.

Accountability and Solutions

• Two Levels of Accountability: Accountability is needed on two levels:
  1. AI Developers: They must take responsibility for building adequate safeguards to prevent malicious use of their technology.
  2. Social Media Platforms: They also need to implement safeguards to prevent the spread of political or malicious content.
• Amplification of Existing Propaganda: AI is not inventing new harms but is "massfully amplifying existing propaganda strategies."
• Real-World Consequences: In a region with a high level of mistrust and volatile relations, a single manipulated video can escalate perceptions of hostility and have real-world consequences extending beyond the digital sphere.

Political Context

• Authoritarian Tendencies: The disinformation emerges at a time when Pakistan is moving towards authoritarian rule, with the constitution being amended to grant unprecedented powers to the army chief.
• Existing Fault Lines: The deepfake feeds directly into existing fault lines in the subcontinent, characterized by decades of mistrust, recurring military standoffs, and polarized public opinion.

Expert Testimony

• Rakir Pameid Nike (Executive Director, Center for the Study of Organized Hate):
  • Emphasizes the difficulty in distinguishing deepfakes initially, noting "small glitches in the facial expressions and the audio" as indicators upon closer inspection.
  • Highlights the "razor-thin margins of trust" between India and Pakistan, making them particularly susceptible to such disinformation.
  • States that AI "massfully amplifies existing propaganda strategies."
  • Warns that in regions with low digital literacy, this amplification becomes dangerous, allowing what once required coordinated effort to be generated and circulated at scale in minutes.

Conclusion

The transcript underscores the critical danger posed by deepfakes and disinformation, particularly in geopolitically sensitive regions like South Asia. The incident involving Alima Khanum serves as a potent example of how manipulated content can rapidly spread, shape public opinion, and exacerbate existing tensions between nuclear-armed neighbors. The lack of digital literacy, coupled with the amplification capabilities of AI and social media, creates a fertile ground for malicious actors. Addressing this threat requires a multi-pronged approach involving accountability from AI developers and social media platforms, alongside a concerted effort to improve media and digital literacy among the public. The potential for real-world consequences, including political instability and conflict, necessitates urgent attention to these issues.