Hacker News new | ask | show | jobs
by boricj 29 days ago
It's fine as a serialization/deserialization primitive for on-disk files, as long as the NULL character is invalid.

String tables in most object file formats work like that, a concatenated series of ASCIIZ strings. One byte of overhead (NUL), requires only an offset into one to address a string and you can share strings with common suffixes. It's a very compact layout.
1 comments
diffuse_l 29 days ago
Nothing prevents you from using a shared pool of strings that don't have null terminator. It can even be more efficient, since you don't have the null byte to handle at string end. Depending on the maximum string length you want to support, it doesn't even have to take more space.
link
boricj 29 days ago
How do you represent that pool of strings on-disk?

If we concatenate the raw strings together without the null terminator, either all string references will require a length on top of the offset (25% size penalty for a Elf32_Sym), or we'll need a separate descriptor table that stores string offsets and lengths to index into.

If we prepend strings with a length (let's say LEB128), we'll be at best tied with null-terminated strings because we'd have a byte for the length vs. a byte for the terminator. At worst, we'll have a longer string table because we'd need more than one byte to encode a long string length and we would lose the ability to share string suffixes.

Out of all the jank from a.out and COFF that was eliminated with ELF, that representation for the string table was kept (in fact, the only change was mandating a null byte at the beginning to have the offset 0 indicate a null string). It works fine since the 1970s and doesn't cause undue problems, as nothing prevents a parser to spit out std::string_view instead of const char* for the application code.

link
diffuse_l 25 days ago
For short strings (probably most of them) - use a byte for the length (at the string/symbol definition site, alongside the offset) (adds 1 byte * symbols, use high bit if necessary to add bytes for longer strings). You need the offset into the table anyway. It isn't strictly better, but it isn't strictly worse, and it gives you the option to reuse sub-strings.
link