[ajanuary]

I think you're reading it incorrectly.

You can have an empty segment in the path. The BNF for a segment is:

    segment       = *pchar

Which according to RFC2234 section 3.6 means zero or more repetitions.

[gpvos]

But then the server may still decide that an empty segment is so meaningless that it will refuse it.

In fact, it would not be a smart move to just treat double slashes the same as single ones, because of relative URLs: a ".." segment only removes one slash, so the hierarchy levels would get messed up. thttpd is doing the smart thing here.

As one of my teachers at university would say: the empty segment is also a segment.

[lmm]

The server can of course interpret the path as it wants, but it should allow an application running under the server to give 'foo//bar' a meaning if that application wants to, IMO.

[gpvos]

True. I was writing about the case when the URL simply mapped to a file system location. Applications should be able to apply their own interpretation.

[stevekemp]

Agreed.

(The problem in my case was just stupid spiders that were crawling my sites.)

[ajanuary]

Yes, it's a valid URI but //robots.txt is different resource to /robots.txt. It seems thttpd is probay doing the right thing.

[zimpenfish]

The difference between `path-abempty` and `path-absolute` is bloody confusing but I think you're right.