May 3, 2024

Programmatic

In a world where nearly everyone is always online, there is no offline.

Half of Top News Sites Blocked OpenAI’s Crawlers in 2023

At the end of 2023, nearly one-half (48%) of the top news websites, based on reach, across 10 countries blocked OpenAI’s crawlers, while nearly one-quarter (24%) blocked Google’s AI crawler, according to a study by Reuters Institute. Reuters Institute analyzed the robots.txt of the 15 online news sources with the widest reach, including titles like…
Adweek Feed