The biggest names in AI are backing a new way to opt out of scraping
OpenAI, Google, Microsoft and Anthropic say they will honor a single shared robots.txt policy token, __usage_ai-training, that lets a website opt out of AI model training across all of them in one line. The signal comes from an IETF Internet-Draft proposed by Anthropic's Kevin Kelley; Anthropic has already begun implementing it, OpenAI plans to follow in the coming months, and Google and Microsoft aim for 2027. It matters because publishers currently must set separate rules for each company, though the token is voluntary and does not remove content already collected.
SOURCES
- Hacker News (AI)2026-10-31 04:49