Werewolf
banner
werewolfuk87.bsky.social
Werewolf
@werewolfuk87.bsky.social
He/Him. CisHet Aromantic & Greysexual man from the UK. 🇬🇧 🏳️‍🌈🏳️‍⚧️

Blogger 📝, Martial Artist 🥋, Whovian and Trekkie 🖖. I'm also learning Japanese 🇯🇵.

https://talkingtoavoidcalledtheinternet.wordpress.com/
I have also added some helpful bots that you can optionally allow, as well as the ‘Crawl-delay’ command for every disallowed bot with a duration of 24 hours, or 86400 seconds.

In theory this should mean that banned bots won’t try to access your site more than once per day.
December 26, 2025 at 1:52 PM
It’s quite telling to see which bots get blocked by the big sites like Facebook or Wikipedia, for example. In any case, this list was compiled from the robots.txt files of major websites and from other such lists.
December 26, 2025 at 1:52 PM
I have collated this list by looking at the robots.txt of major websites, which anyone can do by going to the homepage of any website and adding ‘/robots.txt’ to the end. For example:

bsky.app/robots.txt
bsky.app
December 26, 2025 at 1:52 PM
The big problem is that the robots.txt is a voluntary standard, meaning bots can ignore it. Thankfully a lot of bots still do respect robot.txt files, and it is better to have a robots.txt instead of nothing.

www.rfc-editor.org/rfc/rfc9309....
RFC 9309: Robots Exclusion Protocol
This document specifies and extends the "Robots Exclusion Protocol" method originally defined by Martijn Koster in 1994 for service owners to control how content served by their se...
www.rfc-editor.org
December 26, 2025 at 1:52 PM
I'd hazard the guess that the advert is a YouTube video which got hit by a copyright strike for the music they tried to use.

Still, running an advert for a soundbar without any sound is peak irony.
December 21, 2025 at 11:21 AM
Well at least 2 people knew what to get me for my birthday.

@jayforeman.bsky.social @markcooperjones.bsky.social

I also realise that by tagging them both, it implies Mark and Jay are the two people who bought them for me.
December 12, 2025 at 4:55 PM