Knowledge Graphs in Litigation Agents — Tom Smoker, WhyHow

Key Concepts

• Graph RAG: Using knowledge graphs for retrieval-augmented generation, particularly in the legal domain.
• Multi-Agent Systems: Breaking down complex workflows into specific, testable steps, often controlled by a graph.
• Litigation Agents: Agents designed to support class action and mass tort lawsuits.
• Legal Discovery: The process of gathering information relevant to a lawsuit, often involving large volumes of documents.
• Case Research: Identifying potential lawsuits by monitoring online complaints and other sources of information.
• Schema: A structured framework for organizing and understanding data within a graph.
• Data-Driven Decision Making: Using data and analytics to inform decisions, particularly in legal contexts.

Graphs in the Legal Industry: Turning Documents into Knowledge

The speaker discusses the application of graph technology, specifically graph RAG, within the legal industry, focusing on how their company, WHA.ai, uses graphs and multi-agent systems to find cases for lawyers, particularly in the realm of class action and mass tort lawsuits.

WHA.ai's Business Model

WHA.ai finds potential class action and mass tort cases before other entities by:

• Web Scraping: Scraping the entire web for relevant information.
• Lead Qualification: Filtering the scraped data to identify potential leads.
• Agent Systems: Using a multi-agent system to qualify leads.
• Graph Storage: Storing the qualified information in a graph database.
• Lawyer Collaboration: Providing these leads to lawyers who specialize in these types of cases.

The speaker uses the example of a pharmaceutical product causing harm to many people. WHA.ai helps collect these individuals and supports law firms in collectively suing the pharmaceutical company. This process involves creating a schema that includes individuals, products, ingredients, concentrations, and ID numbers, forming a large, schematized dataset with valuable insights for domain experts.

Defining Graphs and Their Value

The speaker defines graphs as relations, emphasizing the visual and backend elements. The key benefits of using graphs are:

• Connectivity Visualization: Seeing what is connected to something else.
• Explicit Relationships: Clearly defining the relationships between entities.
• Mass Analytics: Performing large-scale analytics on these connections.

These relations can be node-to-node or multi-hop, allowing for varied and distributed representations.

Challenges with Graphs

While graphs offer numerous advantages, the speaker highlights some challenges:

• Inconsistent Understanding: Different people may interpret the same graph differently.
• Data Representation: Varying ways of using, storing, and talking about data within graphs.
• Lack of Standardization: Absence of agreed-upon standards, especially as graphs become more popular.

Multi-Agent Systems in Legal Applications

The speaker defines multi-agent systems as a way to break down complex workflows into specific steps that can be tested and controlled. In WHA.ai's application:

• Workflow Decomposition: Breaking down complex legal workflows into specific steps.
• IO Testing: Rigorously testing each step in the workflow.
• State Control: Managing the state of each step, often using a graph.

The speaker emphasizes the importance of accuracy in legal applications, noting that lawyers require precise and correct information. Probabilistic large language models (LLMs) alone are insufficient due to their potential for errors. Therefore, a structured schema and control mechanisms are necessary.

Issues with Multi-Agent Systems

The speaker also addresses potential issues with multi-agent systems:

• Variable Importance: Some parts of the workflow are more critical than others.
• Agent Reliability: The quality of an agent depends on the prompt used, and bad prompts can lead to bad agents.
• Accuracy Concerns: Achieving 95% accuracy for a single agent is challenging, and chaining multiple agents together can significantly reduce overall accuracy.

The speaker highlights the problem of decision-making under uncertainty when building these systems. WHA.ai addresses these issues by:

• Guardrailing: Implementing measures to ensure accuracy and prevent errors.
• Episodic Memory: Capturing and pruning information state over time.
• Graphical State Management: Using graphs to structure, extend, prune, and query state.

The speaker notes that even with 95% accuracy for each agent, a sequence of five agents can result in only 77% overall accuracy.

Legal Industry Needs: Accuracy and Creativity

The speaker describes the legal industry as requiring both accuracy and creativity. Lawyers need to be detail-oriented and creative in applying those details to a case.

• Accuracy: Everything needs to be perfect and written in the correct way.
• Creativity: Creative arguments are essential for successful litigation.

The speaker uses the example of Netflix and Blockbuster to illustrate creative legal arguments. The precedent of Blockbuster being sued for keeping too many details about DVD rentals was used to argue that Netflix was also capturing too much user data.

Legal Discovery Challenges

Legal discovery involves reviewing large volumes of documents, such as emails, to find relevant information. This process is often manual and time-consuming. Graphs can help by:

• Information Extraction: Extracting and structuring information from documents.
• Data Reduction: Reducing the amount of information that needs to be reviewed.
• Augmentation: Augmenting the information from discovery with external data.
• Visualization: Providing a visual representation of the data to aid in decision-making.

The speaker reiterates the pharmaceutical example, explaining how graphs can help identify individuals affected by a problematic ingredient concentration.

Case Research and Web Scraping

The speaker discusses how WHA.ai uses web scraping to find potential cases by monitoring online complaints.

• Web Scraping: Scraping the entire web for relevant information.
• Lead Qualification: Filtering the scraped data to identify potential leads based on specific schemas.
• Personalized Reports: Generating reports tailored to the specific needs of each lawyer.

The speaker emphasizes that there is no "perfect case," but rather cases that are specific and personalized to the needs of individual lawyers. LLMs, multi-agent systems, and structured information are valuable in this process.

Case Study: Car Fires

The speaker presents a case study involving cars catching fire. WHA.ai tracks complaints on government websites, forums, and subreddits to identify potential lawsuits.

• Complaint Tracking: Monitoring online complaints about car fires.
• Density Analysis: Analyzing the density of complaints based on the number and velocity of complaints.
• Early Lead Detection: Identifying potential lawsuits significantly earlier than traditional methods.

WHA.ai can find these leads within 15 minutes, allowing law firms to take on the lawsuit much earlier.

Future Directions

The speaker concludes by discussing the future directions of WHA.ai and the role of technology in the legal industry.

• Early Lawsuit Detection: Finding lawsuits earlier to compensate harm.
• Iterative Schema Building: Continuously improving the schema based on feedback and new information.
• GenAI Integration: Using LLMs to pipe together ML filtered systems.

The speaker emphasizes that GenAI is not a replacement for traditional machine learning but rather a complementary technology. LLMs have enabled WHA.ai to provide end-to-end value by connecting different parts of the workflow. The ability to iteratively build and prune the graph is a key factor in the company's success.

Conclusion

The speaker effectively demonstrates how graph technology and multi-agent systems can be applied to the legal industry to improve efficiency, accuracy, and decision-making. WHA.ai's approach involves scraping the web, structuring data in graphs, using multi-agent systems to qualify leads, and generating personalized reports for lawyers. The speaker highlights the challenges and benefits of using these technologies, emphasizing the importance of accuracy, creativity, and iterative schema building. The case study of car fires illustrates the potential of this approach to identify lawsuits early and compensate harm.