Can someone point to a good resource for best practises in protocol design? Am currently faced with this problem and would like to avoid having to reinvent the wheel...
There is a lot of "philosophy" about protocol design, which seems mostly sound. Although I wonder what happened to the protocol mentioned (BXXP or BEEP). I think HTTP was "good enough" as a universal application protocol?
This one also talks about HTTP being wildly successful beyond its original design parameters:
Gosh, i don't know of anything off the top of my head.
The only real tip i have is to byte length encode variable length requests, because scanning strings sucks.
having a frame for messages for messages is nice, stick your fixed with stuff up front, and create fixed with length indicators. It sucks that you waste a little space with 0's, but avoiding the scan is pretty wonderful.
GET /foo HTTP/1.1
GET HTTP 01.10 0004/foo
Some things need to happen in order, you need to be authenticated before being granted access to resources, but that doesn't necessarily require the client knowing. the state at each step,
Traditionally you'd have something like
HELO jfoutz hunter2
OK
REQEUEST 1
OK <bytes for resource 1>
REQUEST 2
OK <bytes for resource 2>
but really, the server knows if you're authenticated
so i could send
HELO jfoutz hunter2
REQUEST 1
REQUEST 1
and then get back either
OK
OK <bytes for resource 1>
OK <bytes for resource 2>
or
NO
NO
NO
or, whatever. generally it's just going to be a bunch of request/response pairs. sometimes you can't make later requests without actually looking at a response. You don't know what css or images to request until the html is parsed, for example. Usually, you can request a bunch of stuff at once, and it'll work out however it works out.
The only real tip i have is to byte length encode variable length requests, because scanning strings sucks.
This is really, really good advice to use whenever possible: it means clients can determine apriori how much data they need to read, and perhaps decide whether the length is valid and/or allocate sufficient memory. One of my annoyances with the traditional "almost-text-based" protocols like HTTP, FTP, SMTP, etc. is that parsing them is not trivial and often you need to keep reading until you hit the delimiter or reach an internal limit. In contrast, "read a length, then read length bytes" makes for simple and efficient implementation.
There are certainly cases when the amount of data cannot be determined ahead of time, and in those cases I'd suggest chunked-length-prefixing; delimiters are really a method of last resort.
There is a lot of "philosophy" about protocol design, which seems mostly sound. Although I wonder what happened to the protocol mentioned (BXXP or BEEP). I think HTTP was "good enough" as a universal application protocol?
This one also talks about HTTP being wildly successful beyond its original design parameters:
https://tools.ietf.org/html/rfc5218