Hacker News new | ask | show | jobs
by salomonk_mur 74 days ago
I'm surprised at the effectiveness of simple PoW to stop practically all activity.

I'll implement Anubis at low difficulty for all my projects and leave a decent llms.txt referenced in my sitemap and robots.txt so LLMs can still get relevant data for my site while.keeping bad bots out. I'm getting thousands of requests from China that have really increased costs, glad it seems the fix is rather easy.
2 comments
gruez 74 days ago
>I'm surprised at the effectiveness of simple PoW to stop practically all activity.

It's even dumber than that, because by default anubis whitelists the curl user agent.

    curl -H "User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36" "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/?id=v7.0-rc5&id2=v7.0-rc4&dt=2"
    <!doctype html><html lang="en"><head><title>Making sure you&#39;re not a bot!</title><link rel="stylesheet"
vs

    curl "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/?id=v7.0-rc5&id2=v7.0-rc4&dt=2"
    <!DOCTYPE html>
    <html lang='en'>
    <head>
    <title>kernel/git/torvalds/linux.git - Linux kernel source tree</title>
link
functionmouse 74 days ago
shhhh don't tell the bots !
link
marginalia_nu 74 days ago
Anubis' white lists and block rules are configurable though. The defaults are a bit silly.
link
xena 74 days ago
The default is to allow non-Mozilla user agents so that existing (good) automation continues to work and so that people stopped threatening to burn my house down. Lovely people in the privacy community.
link
otterley 74 days ago
Sorry xe :(
link
wolvoleo 74 days ago
It's definitely more than enough to stop me as a human wanting to visit the site, so yeah.

In that case a better solution would be to take the site down altogether.

link
xboxnolifes 74 days ago
Take down the site entirely because a couple humans get into a fit about it?
link
jlarocco 74 days ago
The site's down entirely anyway. The silly "proof of work" finishes only to tell me the site is down.

What a waste of time.

link
wolvoleo 74 days ago
I'm just saying, making visitors wait at least a minute while making their device turn red hot is going to stop 99,9% of your visitors. So at that point what's the point in trying to serve the content?
link