cm0002@lemmy.world to Technology@lemmy.worldEnglish · 10 months agoOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comexternal-linkmessage-square20linkfedilinkarrow-up1182arrow-down12
arrow-up1180arrow-down1external-linkOpen Source devs say AI crawlers dominate traffic, forcing blocks on entire countriesarstechnica.comcm0002@lemmy.world to Technology@lemmy.worldEnglish · 10 months agomessage-square20linkfedilink
minus-squaremesa@lemmy.worldlinkfedilinkEnglisharrow-up38·10 months agoYep same thing. I have some small servers and was getting hammered by openai ip controlled ai crawlers not respecting robots.txt. had to block all their IP addresses and create an AI black hole in order to stop them ddos ing my tiny site(s).
minus-squaretempest@lemmy.calinkfedilinkEnglisharrow-up24·10 months agoDid you not pay your protection money to CloudFlare?
minus-squareCosmic Cleric@lemmy.worldlinkfedilinkEnglisharrow-up17arrow-down4·10 months agoMan, this current age of AI really sucks. This comment is licensed under CC BY-NC-SA 4.0
minus-squareburghler@sh.itjust.workslinkfedilinkEnglisharrow-up11arrow-down1·10 months agoYou licensed your comment? You just saw first hand that AI crawlers don’t care about legal barriers.
minus-squareWhyJiffie@sh.itjust.workslinkfedilinkEnglisharrow-up4·10 months agoI suppose the goal isa kind of poisoning the well, in case at any time in the future it becomes enforceable somehow
minus-squareCosmic Cleric@lemmy.worldlinkfedilinkEnglisharrow-up2arrow-down1·10 months agoSome info about that … https://lemmy.world/post/26711096/15639879 This comment is licensed under CC BY-NC-SA 4.0
Yep same thing. I have some small servers and was getting hammered by openai ip controlled ai crawlers not respecting robots.txt. had to block all their IP addresses and create an AI black hole in order to stop them ddos ing my tiny site(s).
Did you not pay your protection money to CloudFlare?
Man, this current age of AI really sucks.
This comment is licensed under CC BY-NC-SA 4.0
You licensed your comment? You just saw first hand that AI crawlers don’t care about legal barriers.
I suppose the goal isa kind of poisoning the well, in case at any time in the future it becomes enforceable somehow
Some info about that …
https://lemmy.world/post/26711096/15639879
This comment is licensed under CC BY-NC-SA 4.0