Wiki Founder: AI bots scraping Wikipedia are costing us a lot of money

Key Concepts

• AI Bot Scraping
• Wikipedia
• Donors
• Enterprise Product
• Legal Action
• AI Crawler Bots
• Cloudflare

Cost of AI Bot Scraping on Wikipedia

The primary concern raised is that AI bots are extensively scraping Wikipedia, which incurs significant costs for the organization. These costs are borne by donors who contribute to Wikipedia with the intention of supporting its mission, not to subsidize the operations of AI companies like Sam Altman's.

Proposed Solution: Enterprise Product

The Wikimedia Foundation believes that AI companies should utilize their enterprise product. This product is designed to provide a structured and potentially paid access method for large-scale data consumption, thereby compensating Wikipedia for the resources used.

Discussion on Legal Action

While legal action against AI companies for scraping Wikipedia has not been pursued "yet," the possibility is acknowledged. The current stance is described as "too friendly," suggesting a preference for collaborative solutions before resorting to litigation.

Existing Partnerships and Future Considerations

• Google: Google is highlighted as a "great customer" of Wikipedia's enterprise product, indicating a precedent for paid access.
• Other AI Companies: Discussions are ongoing with other AI entities regarding their use of Wikipedia's data and the potential for them to adopt the enterprise solution.
• Cloudflare: The potential use of services like Cloudflare to block or manage AI crawler bots is mentioned as a consideration, though the speaker is not directly involved in those specific technical decisions.

Argument for Fair Compensation

The core argument is that it is "really not fair" for AI companies to freely consume Wikipedia's data, which is costly to maintain and is funded by donations. The expectation is that these companies should "pay" for the resources they utilize, as their current practices are diverting donor funds from their intended purpose.

Potential for Blocking AI Crawlers

The possibility of implementing measures to "block the AI crawler bots" is something Wikipedia "would probably consider." This indicates a willingness to take more assertive steps if a mutually agreeable solution, such as the enterprise product, is not adopted.

Synthesis/Conclusion

The transcript outlines a significant challenge faced by Wikipedia: the substantial cost incurred by AI bots scraping its content. The organization advocates for AI companies to use its enterprise product as a means of fair compensation, arguing that donor funds should not be used to subsidize AI operations. While legal action is not currently on the table, the possibility exists, and measures to block AI crawlers are being considered as a potential recourse if a collaborative solution isn't reached. The existing relationship with Google as an enterprise customer serves as a model for future engagements with other AI entities.