The growing impact of deepfake health scams | 7.30

Key Concepts:

• AI-generated deepfake scams
• Unproven supplements (Glyco Balance, Keto Flow Plus ACV)
• Misinformation regarding diabetes treatment (Metformin)
• Exploitation of trusted figures (Dr. Norman Swan, Professor Jonathan Shaw)
• Social media platforms' (Meta/Facebook) role in scam propagation
• ACCC's (Australian Competition and Consumer Commission) regulatory powers
• Copyright infringement vs. harm caused by health misinformation
• Industrialized scam operations
• Counter-fraud measures and their limitations
• Erosion of trust in healthcare information

1. Main Topics and Key Points:

• Deepfake Scams: AI-generated videos featuring doctors and celebrities are used to promote unproven supplements, misleading Australians into abandoning prescribed medications.
• Exploitation of Trust: Scammers are using the likeness of trusted medical professionals like Dr. Norman Swan and Professor Jonathan Shaw to endorse products like Glyco Balance and Keto Flow Plus ACV.
• Misinformation about Diabetes: The scams often target individuals with diabetes, urging them to replace Metformin, a safe and routinely prescribed medication, with unproven supplements.
• Financial Impact: Victims are not only losing money on ineffective supplements but also risking their health by discontinuing prescribed treatments. One example is Janice Merrill who was charged $339 instead of $244.
• Ease of Deepfake Creation: AI experts at the University of New South Wales demonstrate how easily deepfakes can be created using readily available technology, requiring minimal resources (laptop, internet, software). Voice cloning can be achieved with just 3-5 seconds of audio.
• Industrialized Scam Operations: These scams are often run by sophisticated operations, potentially backed by organized crime, that are highly resourced and quick to adapt to counter-fraud measures.
• Meta's Role and Response: Meta (Facebook, Instagram, WhatsApp) acknowledges the problem and claims to remove a large number of fake accounts (over a billion in Q4 2024). However, the sheer scale means that a significant number of scams still slip through.
• ACCC's Regulatory Efforts: New legislation empowers the ACCC to impose substantial fines on platforms that fail to remove scam content. Meta is expected to be one of the early designated sectors.
• Ineffective Reporting Systems: Victims and organizations find Meta's reporting system poorly responsive, with copyright complaints being more effective than health-related complaints in getting content removed.
• Multi-System Abuse: Scammers exploit multiple systems, including fake bank accounts, warehouses, websites, and instant payment systems, making it difficult to counter them effectively.
• Decline in Reported Losses: The ACCC reports a 25% decline in reported financial losses to scams in 2024, but losses still amount to $2 billion, indicating the ongoing scale of the problem.
• Erosion of Trust: These scams erode trust in healthcare information, making people skeptical of legitimate advice from medical professionals.

2. Important Examples, Case Studies, or Real-World Applications Discussed:

• David Bell: A diabetic who read a fake article supposedly written by Dr. Norman Swan and stopped taking Metformin in favor of Glyco Balance.
• Professor Jonathan Shaw: A leading diabetes researcher whose image and voice were used in deepfake videos promoting unproven supplements.
• Janice Merrill: A victim who ordered Glyco Balance after seeing a deepfake ad featuring Dr. Shaw and was overcharged and enrolled in a recurring order without her consent.
• University of New South Wales AI Demonstration: A demonstration of how easily deepfakes can be created using AI technology, cloning voice and appearance to generate fraudulent content.

3. Step-by-Step Processes, Methodologies, or Frameworks Explained:

• Deepfake Creation Process (UNSW Demonstration):
  1. Voice Cloning: Sample 3-5 seconds of audio to clone the target's voice.
  2. Appearance Cloning: Scan video frame by frame to map the target's face from different angles.
  3. Text Insertion: Add the desired text for the fake persona to say.
• Meta's Counter-Fraud Process:
  1. Proactive removal of fake accounts (99.7% removed before detection in Q4 2024).
  2. Responding to user reports of scam content.
  3. Adjusting tactics to counter scammers' evolving methods.

4. Key Arguments or Perspectives Presented, with Their Supporting Evidence:

• Scams are becoming increasingly convincing due to advancements in AI technology. Supporting evidence: The deepfake videos are difficult to distinguish from genuine content, as demonstrated by David Bell's experience.
• Social media platforms are not doing enough to combat these scams. Supporting evidence: The difficulty in getting scam content removed from Meta, even with multiple complaints. Copyright complaints are more effective than health-related complaints.
• These scams erode trust in healthcare information. Supporting evidence: Professor Shaw's concern that people will become skeptical of legitimate health advice due to the prevalence of misinformation.
• Scammers are quick to adapt and exploit multiple systems. Supporting evidence: Nathaniel Glasher's statement about adversarial tactics and the use of fake bank accounts, warehouses, websites, and payment systems.

5. Notable Quotes or Significant Statements with Proper Attribution:

• Dr. Norman Swan (in deepfake): "We've developed a product that actually works, it restores your metabolism."
• David Bell: "It was so convincing everything about that promotion was so convincing I stopped taking uh these pills these Gardom pills."
• Professor Jonathan Shaw (in deepfake): "Yeah I did call those who still treat type 2 diabetes with metformin in 2024 idiots."
• Janice Merrill: "Well I felt really shattered it was it was horrible cuz I thought this was really real."
• Nathaniel Glasher (Meta): "We're talking about sophisticated operations working at industrial scale often backed by criminal organizations by organized crime that are highly resourced very adversarial and are looking to target people wherever they can find them."
• Anthony Alban (ACCC): "...there will be financial penalties to that sector if they do not fulfill uh or take reasonable steps to fulfill their obligations under that law."
• Professor Jonathan Shaw: "what what happens as a result of this and other similar things is mistrust so all sorts of useful things that I and other people might say about health will not be trusted"

6. Technical Terms, Concepts, or Specialized Vocabulary with Brief Explanations:

• Deepfake: AI-generated video or audio that convincingly imitates a real person.
• Metformin: A safe and commonly prescribed medication for type 2 diabetes.
• Glyco Balance, Keto Flow Plus ACV: Examples of unproven supplements being promoted through deepfake scams.
• AI (Artificial Intelligence): Computer systems able to perform tasks that normally require human intelligence.
• ACCC (Australian Competition and Consumer Commission): The Australian government agency responsible for enforcing competition and consumer law.
• Meta: The parent company of Facebook, Instagram, and WhatsApp.

7. Logical Connections Between Different Sections and Ideas:

The video logically connects the problem of deepfake scams to the ease of creating them, the role of social media platforms in their propagation, the regulatory efforts to combat them, and the resulting erosion of trust in healthcare information. The case studies illustrate the real-world impact of these scams on individuals.

8. Any Data, Research Findings, or Statistics Mentioned:

• Meta removed over a billion fake accounts in Q4 2024, with 99.7% removed proactively.
• The ACCC reported a 25% decline in reported financial losses to scams in 2024, but losses still amounted to $2 billion.

9. Clear Section Headings for Different Topics:

(Covered in the structure above)

10. A Brief Synthesis/Conclusion of the Main Takeaways:

Deepfake scams are a growing threat, exploiting trust and misinformation to deceive individuals into purchasing unproven supplements and abandoning prescribed medications. The ease of creating these scams, coupled with the challenges in regulating social media platforms, makes it difficult to combat them effectively. The resulting erosion of trust in healthcare information poses a significant risk to public health. A multi-faceted approach involving technological solutions, regulatory enforcement, and increased public awareness is needed to address this problem.