Hacker News new | ask | show | jobs
by wyldfire 3325 days ago
> The include_* macros are great for “packaging”. Being able to compile small assets directly into the binary and eschewing run time file loading is fantastic for a small game.

For C/C++/etc devs looking for something similar, BFD objcopy supports an "-I binary". It will emit an object file with _binary_objfile_start, _binary_objfile_end and _binary_objfile_size symbols.

But I have got to say that making it a language feature means that Rust is truly a batteries included language.

4 comments

Agreed. I code in C# and there's facility for bundling assets into the binary but they're handled at the project file level and not the C# file level, and this means the keys to access this binary content is not available as a compile-time constant. You're still just passing strings around and hoping they match.

Having first-class language support for resource files looks fantastic.

> and this means the keys to access this binary content is not available as a compile-time constant

Isn't that exactly what resource files give you or am I confusing something? Anything you add in resource files have are accessible as a static variable.

It looks like you're right (for C# at least): https://msdn.microsoft.com/en-us/library/7k989cfy(v=vs.90).a...
I wonder how XNA's asset pipelines worked. I also wonder if you could "pack" a binary into an assembly by writing a tiny bit of IL around it.
Yeah, that's the workflow I was mentioning, cooking the file into a resource by referencing it as resource in the csproj, but then it's keyed by a string of the file-path within the project file, which isn't nearly as nice as having it bound to a variable in C#.
There is also an option to use assembler or inline asm. I found quite a nice utility that uses inline asm [0]. It's widely portable and I think that I will use it instead of my naive asm/shell combo that doesn't work with mingw asm.

The problem with objcopy outside of unconvenient usage and naming is that naive objcopy will result in your binary having executable stack [1]. You can change a symbol name, but that's also unconvenient.

Check resulting binary with:

  $ readelf -lW the_binary | grep GNU_STACK
    GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RWE  0x8
  $
Notice: RWE instead RW.

Also: https://wiki.gentoo.org/wiki/Hardened/GNU_stack_quickstart#H...

[0] https://github.com/graphitemaster/incbin

[1] https://news.ycombinator.com/item?id=10816322#10818085

EDIT:

My shell script - bin2o.sh:

  #!/bin/sh
  set -e
  
  filename="$1"
  name=$(echo "$1" | sed "s/[^A-Za-z0-9]/_/g")
  obj="$2"
  
  echo \
  "	.section .rodata
  	.global ${name}
  	.type ${name}, @object
  	.global ${name}_size
  ${name}:
  	.incbin \"${filename}\"
  1:
  ${name}_size:
  	.int 1b - ${name}
  	.section .note.GNU-stack,\"\",%progbits
  " | gcc -x assembler -c - -o "$obj"
To embed binary assets into any language, write a program that emits the binary as a string containing hex codes.

E.g. bytes [0, 12, 99] would become "\x00\x0C\x63".

Almost every language has facilities to treat a string as a set of bytes, so this works basically everywhere. It's nice because you don't need any special language-level support for it.

While I do agree that it is possible to do something like that, I think we can all agree that the following (available in Rust) is soooooo much nicer:

    let stuff = include_bytes!("my.file");
While include_bytes is nice, xxd -i in a makefile is pretty workable solution in the scale of things.
Fair point, however I believe the OP made the game work for many different OSes, and I don't think xxd or Makefiles are the best when you have to deal with Windows (whereas Rust's include_bytes! provides a platform-independent solution).

I had never heard of xxd until your comment though, so thank you very much!

Somehow I'm reminded of the old adage "UNIX is IDE for C"
Now I have to rewrite my arcade game emulator in rust just so I can use that to load rom images...
include_bytes! is for compile-time inclusion of files though, not for runtime.
Yeah but when it only runs 15 games with a max size of 32K... But yeah it makes it non-distributable because it would then include the images.
C++ string literals usually have fairly low maximum length (C++ standard suggests 64K in [implimits/2.16]). Initialized arrays can be longer in practice but they also have a limit (the same standard suggests 16K but even that will give you 128Kb if you initialize uint64). Objcopy, as suggested elsewhere in the comments has more comfortable limits.
That has also been a feature of Windows for quite some time, you have your IIRC .exe icon and various string resources packaged in the .exe itself. It's pretty useful on the whole.