What changed
OpenAI announced that it plans to honor a common signal in robots.txt that website operators can use to communicate preferences about foundational generative AI training. The company says it aims to implement the signal in the coming months and will continue to respect its existing crawler controls. The work is connected to the Internet Engineering Task Force's AI Preferences discussions and is intended to reduce the need for publishers to maintain separate rules for every provider.