Hacker News new | ask | show | jobs
by hrjet 3430 days ago
From a privacy point of view, it would be awesome if you stabilise the NoJS version of your search page: https://duckduckgo.com/html/

* The above landing page doesn't work (typing a query and pressing enter returns me back to the same page).

* Even if I manually enter a search URL like this https://duckduckgo.com/html/?q=test it doesn't fully work (page navigation and requerying is broken).

2 comments

Interesting, it works fine for me. What browser are you using?
Chromium with uMatrix installed. (exact versions irrelevant, because it hasn't been working for me since atleast 2 years)
Like MaxLeiter, this page works normally for me, so I thought I would do a bit of testing. I know this is a plain HTML version of DDG and no AJAX is involved, but bear with me, I want to illustrate a point.

From the browser console, the origin is that of the web page you're currently viewing, so you're free from CORS concerns.

So, from my browser console in Chrome 56, with duckduckgo.com/html/ open in the browser tab, I should be able to do this without any trouble:

    fetch(window.location, {
      method: 'POST', 
      body: 'q=testddg'
    })
    .then(result => {
      result.text()
      .then(text => { 
        document.documentElement.innerHTML = 
          new DOMParser()
            .parseFromString(text, 'text/html')
              .documentElement.innerHTML
      })
    })

But, surprise surprise, it doesn't work. There's no change in the document, where I'd obviously expect to see the search results page loaded, because the response was the home page -- as it is in your case, when you try to search for stuff.

So I tried to make a POST request without a body instead, appending the query parameter to the URL as one would with a GET request:

    fetch(window.location + '?q=testddg', {
      method: 'POST'
    })
    .then(result => {
      result.text()
      .then(text => { 
        document.documentElement.innerHTML = 
          new DOMParser()
            .parseFromString(text, 'text/html')
              .documentElement.innerHTML
      })
    })

This worked perfectly, all the search results were loaded. Works as a plain GET request too. So what we seem to be able to surmise is that posting a request body causes the search to be rejected...

But that's absurd! The native submit behaviour of a form with method 'POST' necessarily sends a POST request with a body, the body can be of multiple types, specified by the enctype attribute, default being x-www-form-urlencoded, which mimics a GET request.

In my browser's dev tools, I checked the request in the networking panel. The fetch requests were all there, but they differed from the native requests in an important way: the body was called 'Form data' in native requests, where in fetches it appeared as 'Request payload'.

The latter is obviously not going to be identified by DDG or any website as form data to parse, so I tried fetching with the correct content-type header explicitly set:

    fetch(window.location, {
      method: 'POST', 
      headers: {
        'Content-Type': 'application/x-www-form-urlencoded'
      },
      body: 'q=testddg'
    })
    .then(result => {
      result.text()
      .then(text => { 
        document.documentElement.innerHTML = 
          new DOMParser()
            .parseFromString(text, 'text/html')
              .documentElement.innerHTML
        })
    })
Boom, worked perfectly, just as we'd expect.

So what was the point of this? We haven't really learned much, because handling form submissions via AJAX always requires such considerations, and that's irrelevant here anyway because the web page in question does not use AJAX, it's a plain HTML version of the site!

What we have determined is that there's most probably nothing wrong with their web page and the problem is more likely to be originating from your side.

I get the feeling that your browser is, for some reason, not applying the correct default enctype for the DDG/html form or uMatrix is interrupting the request and re-sending the request as a generic POST without the explicit content-type.

I've been unable to reproduce the bug on my machine. Do you use any other browser extensions? Does the site work in other browsers you may have?

Nope, there are no other browser extensions. Just Chromium with uMatrix.

Here's the complete request/response data in HAR format: https://gist.github.com/hrj/f538ed3e67636c94934e221efe67b323

(Note that the UA string is being spoofed by uMatrix, but I don't think that's relevant. I tried disabling the UA spoofing with the same result.)