Surfacing Semantic Orthogonality Across Model Safety Benchmarks: A Multi-Dimensional Analysis

AI Safety Benchmarks: A Deep Dive into Semantic Coverage and Clustering

Key Concepts:

• AI Safety Benchmarks: Question and answer datasets used to measure Large Language Models (LLMs) in terms of potential harms.
• Hype vs. Reality: The contrast between exaggerated claims about AI capabilities and the actual limitations and challenges faced in development and deployment.
• Semantic Space: A high-dimensional space where text prompts are represented as vectors, with proximity indicating semantic similarity.
• Clustering: Grouping similar prompts together based on their semantic embeddings to identify categories of harm.
• Silhouette Score: A metric used to evaluate the quality of clustering, measuring how well each prompt fits within its assigned cluster.
• Overindexing: When a specific benchmark disproportionately focuses on a particular category of harm compared to others.
• Embedding Models: Algorithms that convert text into numerical vectors (embeddings) representing their semantic meaning.
• Dimensionality Reduction: Techniques used to reduce the number of dimensions in the embedding vectors, making clustering more efficient.
• Distance Metrics: Functions used to measure the distance between embedding vectors, determining how similar or dissimilar prompts are.

Introduction and Context

The talk addresses the critical need for robust AI safety benchmarks, acknowledging the gap between the hype surrounding AI and the reality of its current capabilities and potential harms. The speaker emphasizes that while AI is here to stay, preventing harms and accurately measuring AI safety remain essential. The presentation focuses on a paper analyzing existing AI safety benchmarks to understand their semantic coverage and identify potential gaps.

Methodology: Clustering and Analysis of AI Safety Benchmarks

The core of the paper involves a methodology for clustering and analyzing AI safety benchmarks to identify harm categories and assess the coverage of different benchmarks. The process involves several key steps:

1. Data Aggregation: Appending multiple open-source AI safety benchmarks into a single dataset.
2. Data Cleaning: Removing duplicates and outliers based on prompt length using both IQR and Z-score methods. The Z-score method was chosen due to the right-skewed distribution of prompt lengths.
3. Embedding: Converting text prompts into high-dimensional vectors using embedding models like MiniLM and MPNet.
   • MiniLM: Chosen for its scalability, high-quality embeddings, and efficient memory usage.
   • MPNet: Explored for its superior contextual and sequential encoding, albeit with higher memory requirements.
4. Dimensionality Reduction: Reducing the dimensionality of the embedding vectors using techniques like t-SNE and UMAP.
   • t-SNE: Preserves local structure but struggles with global relationships.
   • UMAP: Preserves both local and global structure while scaling efficiently. Hyperparameters like perplexity (for t-SNE) and n_neighbors and min_dist (for UMAP) were optimized based on past research.
5. Clustering: Applying k-means clustering to group similar prompts together based on their reduced-dimensionality embeddings. C-means clustering was used.
6. Cluster Optimization: Iteratively refining the clustering process by varying embedding models, dimensionality reduction techniques, distance metrics, and hyperparameters. The goal was to maximize the silhouette score, a measure of cluster separation and cohesion.
7. Centroid Analysis: Identifying the prompts closest to the centroid of each cluster and using LLMs to infer the underlying harm category.
8. Evaluation: Evaluating the clustering results using silhouette score and performance time. Bird score was initially considered but discarded due to consistently high scores across clusters.

Results: Harm Categories and Benchmark Coverage

The analysis revealed six primary harm categories across the analyzed benchmarks:

• Controlled Substances
• Suicide and Self-Harm
• Guns and Illegal Weapons
• Criminal Planning and Confessions
• Hate (including Identity Hate)
• PII and Privacy

The clustered results showed that different benchmarks overindex on specific harm categories, indicating variations in their focus and coverage. For example, the hate and identity hate category was found to be very focused, suggesting a need for broader exploration.

Sample Size Presumptions

The sample size calculation was based on the following assumptions:

• Maximum clusters: 15 (based on existing taxonomies with a 10% increase)
• Significance level: 0.15 (adjusted for 15 clusters)
• Effect size: 0.5 (based on past research indicating the importance of a substantial effect size)

This resulted in a minimum required sample size of 1,635 per benchmark and a total sample size of 8,175 across all benchmarks.

Variance and Optimization

The paper details the various combinations of embedding models, dimensionality reduction techniques, and distance metrics explored during the optimization process. The top eight combinations, ranked by silhouette score, are presented, highlighting the overlap in confidence intervals and the importance of considering efficiency (processing time) alongside cluster quality.

Insights and Bias

The analysis revealed potential biases in the benchmarks, such as the underrepresentation of anthropomorphism and other psychological harms. The distribution of prompt lengths also varied across benchmarks, suggesting further biases.

Limitations and Future Research

The study acknowledges several limitations:

• Methodological limitations, such as the potential for increased sample size.
• Information loss during dimensionality reduction.
• Inherent biases in the embedding models.
• Limited generalizability due to the small number of benchmarks analyzed and the exclusion of private benchmarks.
• Potential biases in the research team's technical background and Western perspectives.

Future research directions include:

• Developing harm benchmarks for more cultural contexts.
• Exploring prompt-response relationships.
• Applying the methodology to domain-specific datasets.

Conclusions

The paper concludes with four key takeaways:

1. Six primary harm categories were identified with varying coverage and breadth across benchmarks.
2. Semantic coverage gaps exist across recent benchmarks and will evolve over time.
3. An optimal clustering configuration framework was developed for this specific use case and can be scaled for other benchmarks or LLM applications.
4. Plotting semantic space provides a transparent evaluation approach that offers more actionable insights than traditional metrics like ROUGE and BLEU.