Jeremy Keith follows up on fighting AI bots, quoting a couple things I’ve said. He closes with:
There is nothing inevitable about any technology. The actions we take today are what determine our future. So let’s take steps now to prevent our web being turned into a dark, dark forest.
I agree with these statements in isolation. Maybe what we disagree on is whether AI is inherently destructive to the web, so all AI bots should be stopped, or whether we can more narrowly minimize AI slop from spreading.
Even without AI, Google referrers to blogs have also been going down, with Nilay Patel arguing that we are heading to Google Zero. In other words, Google is already taking more from the web than they are giving back.
The solution to that is Google alternatives that get us back to the style of old-school search engines: “10 blue links”, with a focus on real blogs and news sites, weeding out content farms and other spam shenanigans. We have spammers creating accounts in Micro.blog every day, trying to pollute the open web. It’s depressing. I want to create more tools that highlight human-generated content, like the audio narration we added.
Jeremy didn’t quote one of my responses about trying to insert text into posts to confuse bots, so I’ll add it here for completeness. I replied with:
I think it’s a bad precedent. It’s already hard enough for legitimate crawling because of tricks that paywalls use, or JavaScript that gets in the way. Mucking up text and images is bound to create problems for non-AI tools too. There’s gotta be a better way to address this.
I viewed source to see how Jeremy is handling this on his blog. His technique doesn’t appear to be causing any problems with Micro.blog’s bookmarking, which saves a copy of the text in a blog post for reading later, because the prompt injection is outside the <article> and h-entry for the post. But it’s not hard to imagine a well-behaved, non-AI bot getting tripped up by this.
I don’t think technological determinism is an appropriate summary of my thoughts. There are a bunch of questions to resolve around generative AI, for sure, including rogue bots, but there’s a lot of potential good too.
