Why a US Counterterrorism Expert Left to Build a $14M AI Safety Startup | Cinder, Glen Wise

Key Concepts

• AI and its dual nature: AI's ability to placate and agree with users, but also its use for malicious purposes like spam, phishing, and non-consensual intimate imagery.
• Trust and Safety Operations: The necessity of systems to combat online abuse to allow for continued innovation.
• Cinder: A central platform for trust and safety operations, acting as an "internet club bouncer" to enforce platform rules.
• Threat Intelligence: The practice of gathering and analyzing information about potential threats to proactively address them.
• Adversarial Mindset: Thinking like an attacker to anticipate and defend against malicious activities.
• Platform Building Challenges: The complexity and extensive requirements of building a comprehensive platform from scratch.
• Customer Problem Focus: The importance of deeply understanding and addressing the core customer problem before building solutions.
• Decision Spectrum Framework: A methodology for categorizing and addressing trust and safety issues based on their complexity.
• Burnout in Trust and Safety: The emotional toll and challenges faced by professionals in the field due to the constant exposure to online harm.
• Mission-Driven Motivation: The critical role of a strong mission and purpose in driving innovation and resilience in the trust and safety space.

Cinder: Building a Safer Internet

The Open Pandora's Box of Technology and the Need for Trust and Safety

The transcript highlights that technology, particularly Artificial Intelligence (AI), has opened a "Pandora's box." Children are growing up interacting with AI systems that are designed to be agreeable and placating. However, this same technology is being exploited for "horrible things," ranging from AI-generated spam and "AI slop" to highly targeted phishing campaigns and the creation of non-consensual intimate imagery. The speaker emphasizes that there is a critical need for systems to prevent these harmful activities to ensure that innovation can continue. Without such interventions, the inevitable outcome will be stringent regulation that could stifle innovation and slow societal progress. The speaker's goal, through Cinder, is to be at the forefront of stopping these harms and innovating in this space, thereby enabling others to innovate rapidly.

Introducing Cinder: The Internet's Club Bouncer

Glenn Wise, co-founder and CEO of Cinder, describes the company as a "central platform for trust and safety operations." He uses the analogy of a "club bouncer for the internet," explaining that Cinder empowers customers to define and enforce rules on their platforms regarding what content and user behavior is permissible. Cinder's system is used to enforce these rules across all users. The company has successfully raised $14 million from investors, including Y Combinator and Accel, with a clear mission to make the internet a safer place.

The Genesis of a Mission: From 9/11 to Counterterrorism

Wise's personal journey and motivation are rooted in his childhood experience of 9/11 in New Jersey. This event profoundly impacted him and his community, instilling a desire to be part of a "greater mission." This led him to pursue opportunities in government agencies like the NSA, CIA, and FBI, focusing on issues related to preventing such events from recurring. His career path then led him to work on counterterrorism issues in government, which subsequently transitioned to a similar role at Facebook on a threat intelligence team.

Facebook's Threat Intelligence and the Adversarial Mindset

Facebook began investing heavily in threat intelligence before the prevalence of platform harm. Wise's experience on a "red team" at Facebook is described as one of the most enjoyable aspects of his career, as it involved actively trying to "break things." He instills this "adversarial mindset" in his current team, emphasizing the need to acknowledge and think like those who are actively trying to circumvent security measures.

The Birth of Cinder: Identifying a Market Need

Wise observed numerous companies struggling to manage and understand the abuse occurring on their platforms. This realization, coupled with his long-standing contemplation of the problem, led him to consider founding a company. Initially, his hesitation stemmed from a perceived lack of experience. His initial thought was to join a smaller company and help build an internal platform. However, upon reaching out to various companies, he realized that the need was widespread, making a centralized solution more logical and efficient. This business rationale ultimately led to the decision to found Cinder.

Cinder's Mission: Enabling Innovation Through Safety

Wise reiterates the core mission: to stop harmful activities so that innovation can flourish. Cinder aims to be the entity that addresses these harms and innovates in this domain, allowing other companies to deploy and build new technologies rapidly without the burden of dealing with the "underbelly of the internet" and malicious actors.

The Y Combinator Journey and Building a Team

Wise recounts his application to Y Combinator, submitted shortly before the deadline. He received a call from Michael Seibel, who was enthusiastic about the idea but advised Wise to find a co-founder. Wise identified former colleagues from Facebook whom he respected and believed would be instrumental. Despite the initial challenge of convincing them to leave Facebook, he successfully assembled a strong team within three months, a moment he describes as "special."

Scale of Harm and Evolving Threats

Cinder has processed over 100,000 reports from its customers this year alone, including those from the National Center for Missing and Exploited Children. This figure, while substantial, is acknowledged as only "scratching the surface" of the pervasive online harm. A significant concern is the increasing sophistication of abuse, with harms that were once only executable by nation-states now being carried out by teenagers with laptops. This necessitates the development of societal systems to combat abuse before it escalates into a "total hellscape."

The Challenges of Platform Development and a Pivotal Learning Experience

Building a platform from the ground up, as Cinder initially attempted, is described as "really challenging." It requires integrating numerous signals, ensuring high configurability and customization, and accommodating unique customer data and workflows. The initial focus was on building a "threat intel platform for trust and safety," driven by the pain points experienced in previous roles. However, when this platform was presented to potential customers, the feedback was that while "interesting," it wasn't what they truly needed. Instead, they required a solution for their "overall operations." This feedback, particularly the experience of investing significant time in building a threat intel platform only to be told customers wanted something else, was a "huge learning" for the team. It underscored the critical importance of focusing on the "core customer problem" and deeply understanding it before building.

Empathy and Problem-Solving: Cinder's Approach

Wise's career focus has consistently been on preventing exploitation and abuse. Empathizing with the customer has been paramount. Even without a fully built product, listening to customers, understanding their challenges, offering suggestions, and then delivering a product that solves a real problem has been key to Cinder's development.

Addressing Non-Consensual Intimate Imagery (NCI) and the Decision Spectrum

A prominent harm Cinder addresses is Non-Consensual Intimate Imagery (NCI). The transcript notes that online harms are becoming increasingly complex. Cinder employs a framework called the "decision spectrum" to categorize and address these issues. On one end of the spectrum are simple decisions, such as identifying nudity in an image or fraud in a chat message. On the other end are complex investigations involving nation-states or sophisticated adversaries attempting to conceal their actions.

Combating Burnout and Mission-Driven Resilience

The sheer volume of harmful content makes prioritization difficult, as "everything is a harm." This constant exposure leads to significant burnout within the industry, with professionals feeling like they are fighting an "uphill battle." Despite these challenges, Cinder's team is deeply motivated by their mission to create a world where the internet and innovation are safe for everyone. The founding team's direct experience in building these systems and experiencing harm firsthand provides them with the necessary insight and drive. The work is described as "difficult," "complex," and "nuanced," requiring a strong sense of purpose. Hearing stories of successful interventions and prevented abuse serves as a powerful motivator for the team to continue their work. The speaker concludes by acknowledging the ease with which one can become "bogged down in the harm" but emphasizes the existence of a path forward and the necessity of someone to combat these issues.