There are a lot of interesting stats in this Cloudflare blog post about AI bots. I still worry that we might over-correct when blocking LLM training. For example, CCBot is mentioned, but Common Crawl goes back 15+ years and has a variety of non-AI uses.

Manton Reece @manton