[IncreasePosts]

Regarding point two, OP should connect to a VPN in Japan or somewhere he very isn't, use incognito mode, and see if the same content is served. I've seen hacked sites that are set up to serve normal content to where the attacker thinks the owner of the site lives, but serve phishing content or malware or whatever to everywhere else.

A 301 fits that bill because then the owners browser even when traveling will serve the good content

[preinheimer]

Our service testlocal.ly can grab screenshots for you from different countries really quickly if you want a free check.

[Pikamander2]

Oh hey, I've used your site before. Thanks for setting it up!

One quick point of feedback: The "Learn more about our features and pricing" button appears to be broken, at least on Chrome Android.

The click gets intercepted by the registration form somehow, like by some type of overly-broad selector targeting "form button" or similar.

Instead of being taken to the pricing page, it takes me to the next step of the form, which I don't want to fill out before seeing the pricing.

[Scoundreller]

Can you get Google Safe Search to do that? I feel like my reports fall on deaf ears because SMS spammer's URLs would only serve 'bad' pages to $MyCountry (and nowadays do it behind a captcha, fuck you hcaptcha).

[jasongill]

I have seen attacks where directly visiting the site doesn't show anything out of the ordinary, but visits coming from Google (referer) show different content. Have also seen ones where only User-Agent: Googlebot would see the modified version of the site.

(I doubt that is the case in OP's situation, but I have seen both of those methods of "hiding" multiple times now)

[bluesix]

Yes, this is how most Wordpress malware works - they inject/publish ad or keyword spam content on the site if the user agent is googlebot. Regular users don't get the ads. It's partially why most people never realise their site has been hacked.

[miramba]

Scams on every possible level - the internet has become so depressing.

[AutistiCoder]

Doesn't Google have countermeasures against this?

[nneonneo]

Or, try a mobile user-agent. I've seen loads of phishing pages that will only serve their malicious payloads to phones - this is especially common with the scams that are sent via SMS.

[TrueDuality]

Yeah this is a good call-out. If the site is being used for drive-by or targeted malware there are other checks that may be happening alongside the redirect such as user agent, country of origin (like you mentioned), plugins installed, OS, or even time of day.

If they detect something that matches what they want, they may throw some intermediate 301's to pages that attempt to infect the user with something still ultimately redirecting to the "normal" page.

[SlightlyLeftPad]

Just a note 301s are super sticky and browsers cache them even across incognito modes. Your best bet is to use a new browser after reconnecting to avoid false results.

[avoutic]

On Chromium-based browsers, if you open the Developer Tools (F12 or Inspect in right click) and you go to the Network tab, you can click 'Disable Cache'.

In my experience, this solves the sticky 301 issue and you should have no issues with cached 301s anymore.

Works perfect for these kind of investigations or if you made a mistake during site development.

[SlightlyLeftPad]

Of course, there are ways to clear it but that’s never something you could expect a non-technical user to do.

[nneonneo]

Really? That seems like a fantastic way to fingerprint people. I would be a bit surprised if that was the case...

(Fingerprint usage: have https://myfingerprint.example.com 301 to https://myfingerprint.example.com/unique_id_3b136c1cb, then embed https://myfingerprint.example.com in an iframe and see which request is made.)

[kqr]

I'm not GP but a decade ago when I started out as a web developer I made the mistake of using 301s in production and at the time we never figured out how to get the browser to re-learn the responses for those pages without drastic measures.

I still never use 301s for that reason. Things may have changed, but I dare not try!

[geocar]

> I still never use 301s for that reason. Things may have changed, but I dare not try!

I use 301 for http:->https: redirects because (a) I doubt we're going back, (b) it prevents some cleartext leaks (like the Host header), and (c) it is slightly cheaper.

> we never figured out how to get the browser to re-learn the responses for those pages without drastic measures.

If you control the target URL it is easy, just redirect back. Seriously: The browser won't loop, it'll just fetch the content again and now not seeing a 301 will forget that nonsense ever happened. This is why 301 is usually a fine default for same-site redirects, or if the redirect target is encoded in the URL (such as in tracking URLs).

The big no-no is don't 301 to a URL you can't control unless you have the appropriate Cache-Control headers on the redirect.

[throwawaymobule]

Isn't there a https upgrade header specifically for this kind of thing?

[swiftcoder]

> If you control the target URL it is easy, just redirect back. Seriously: The browser won't loop

Just uh... don't do this if you have a CDN infront of your site. We had an incident where Cloudfront cached the 301's in both directions

[SlightlyLeftPad]

Interesting use case actually. I had never thought of this. I wonder if it’s used in the wild

[jezek2]

You can disable caching in Firefox's developer tools, this covers such cached redirects. Very useful combined with a persistent log of network activity to avoid clears after redirects.

[saalweachter]

Try curling the urls with a referrer of Google.

There's a related site compromise where a hacked webserver behaves normally except, when the referrer is google.com, it adds a JavaScript redirect to the end of any page.

You go to example.com, everything looks normal. You click a link to example.com, you end up on a page selling herbal dick pills. Site owner yells at Google thinking it's their fault. Googlebot never gets served the redirect.

You should be able to do the same thing with 301 redirects.