[Leftium]

You can hide data in random static (for a novel captcha). Very obvious to most humans (if it becomes popular, the bots could probably be designed solve it; just requires diffing the images):

- https://fx.leftium.com/static

- https://www.youtube.com/shorts/tPfzNzy24to

---

I noticed most of the traffic to my hobby sites was bot traffic (probing for common vulnerabilities). So I added a robots.txt for well-behaved bots and rules for the bad ones. It works pretty well.

robots.txt: https://github.com/Leftium/robots-txt

Sample cloudflare rules:

    (
      http.request.uri.path contains "/wp-login"
      or http.request.uri.path contains "/wp-admin"
      or http.request.uri.path contains "/wp-content"
      or http.request.uri.path contains "/wp-includes"
      or http.request.uri.path contains "/xmlrpc.php"
      or http.request.uri.path contains "/wordpress/"
      or http.request.uri.path contains "/wp-json/"
      or http.request.uri.path contains "/wp-config"
      or http.request.uri.path contains "/wp/"
      or http.request.uri.path.extension eq "php"
      or http.request.uri.path.extension eq "aspx"
      or http.request.uri.path.extension eq "asp"
      or http.request.uri.path contains ".env"
      or http.request.uri.path contains "sftp-config"
      or http.request.uri.path contains ".vscode/"
      or http.request.uri.path contains "credentials"
      or http.request.uri.path contains "stripe"
      or http.request.uri.path contains "secrets"
      or http.request.uri.path contains "config.json"
      or http.request.uri.path contains "config.yml"
      or http.request.uri.path contains "config.yaml"
      or http.request.uri.path contains "config.py"
      or http.request.uri.path contains "config.inc"
      or http.request.uri.path contains "config.rb"
      or http.request.uri.path contains "settings.json"
      or http.request.uri.path contains "settings.yml"
      or http.request.uri.path contains "settings.py"
      or http.request.uri.path contains "application.yml"
      or http.request.uri.path contains "application.properties"
      or http.request.uri.path contains "parameters.yml"
      or http.request.uri.path contains "appsettings.json"
      or http.request.uri.path contains "docker-compose"
      or http.request.uri.path contains "/.git/"
      or http.request.uri.path contains "/.circleci/"
      or http.request.uri.path contains "/.aws/"
      or http.request.uri.path contains "/.bitbucket/"
      or http.request.uri.path contains "/.travis.yml"
      or http.request.uri.path contains "serverless.yml"
      or http.request.uri.path contains "debug.log"
      or http.request.uri.path contains "error.log"
      or http.request.uri.path contains "/storage/logs/"
      or http.request.uri.path contains "_profiler"
      or http.request.uri.path contains "phpinfo"
      or http.request.uri.path contains "/debug/"
      or http.request.uri.path contains "/horizon/"
      or http.request.uri.path contains "/manage/"
      or http.request.uri.path contains "fileman"
      or http.request.uri.path contains "assetmanager"
    )

[kassner]

In a similar tone, I made Caddy module that temporarily blocks an IP address once it tries to access specific URLs: https://github.com/kassner/caddy-trapdoor It’s great to catch those bursts of requests, nowadays I see a single IP probing hundreds of paths in a couple of seconds. This catches majority of them for me.

[prologic]

I'm actually almost tempted to build a reverse proxy that detects this type of "scanning" behaviour and just drops it to the floor.