Luu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 1 year agoTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comexternal-linkmessage-square86linkfedilinkarrow-up155arrow-down10
arrow-up155arrow-down1external-linkTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comLuu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 1 year agomessage-square86linkfedilink
minus-squareMojave@lemmy.worldlinkfedilinkEnglisharrow-up0·1 year agoThey obfuscate their traffic by randomizing user agents, so it’s either add a global rate limit, or let them ass fuck you
minus-squareWhyJiffie@sh.itjust.workslinkfedilinkEnglisharrow-up0·1 year agothe article told all source IPs can be tracked back to bytedance. Wouldn’t it be possible to block them? maybe even blocking all IPs of a specific ASN
They obfuscate their traffic by randomizing user agents, so it’s either add a global rate limit, or let them ass fuck you
the article told all source IPs can be tracked back to bytedance. Wouldn’t it be possible to block them? maybe even blocking all IPs of a specific ASN