Hacker News new | ask | show | jobs
by tlb 3209 days ago
I recently switched from zeromq to straight libuv sockets with jsonl (\n-separated json) payloads. Because I'm working inside a Node process, combining zmq's threading model with Node's threading model was a pain. Now, there's a single IO thread which is the same as the Javascript engine thread, and I can use uv_work to run CPU-intensive tasks on multiple cores.
1 comments

Do you not allow \n inside your JSON or encode your JSON as base64? If not you might have problems with disambiguating frame ends from line breaks inside frames.

A common way for framing is to prepend each frame with it's encoded length. That's easier, faster and less error-prone than searching for ASCII delimeters.

If you do the JSON serialization yourself, there's no reason for newlines to be in the JSON. (Newlines within strings are encoded as \n.)
I'm generating the JSON, either with custom C++ marshaling routines or with JSON.stringify which doesn't include newlines unless you give it extra arguments. I believe that any valid JSON can be converted to a single line by changing any '\n' bytes to ' '. Literal '\n' bytes are not allowed inside strings, and outside strings any whitespace is equivalent.
a newline is encoded as the two characters "\n" in json and would not be confused with a literal \n (aka \x0a) character