[JoshTriplett]

> The binary is also 53M

That's a debug binary, and the vast majority of that is debug symbols. A release build of this project is 4.3M, an order of magnitude smaller.

Also, compiling out the default features of the git2 crate eliminates several dependencies and reduces it further to 3.6M.

https://github.com/bgreenwell/lstr/pull/5

https://github.com/rust-lang/git2-rs/pull/1168

Stripping the binary further improves it to 2.9M, and some further optimizations get it to 2.2M without any compromise to performance. (You can get it smaller by optimizing for size, but I wouldn't recommend that unless you really do value size more than performance.)

[esafak]

No offense, but 4.3MB is huge for what it does. Most shells take less space than that! Where's all the bloat coming from?

[koito17]

> Most shells take less space than that!

Most shells dynamically link to a runtime your OS provides "for free". The 4.3 MiB binary in question is bundling the Rust runtime and its dependencies.

For reference, a statically-compiled C++ "Hello, World" is 2.2 MiB after stripping.

  % cat hello.nix
  {
    pkgs ? import <nixpkgs> { crossSystem = "aarch64-linux"; }
  }:
  
  pkgs.stdenv.mkDerivation {
    name = "hello-static";
    src = pkgs.writeText "hello.cpp" ''
      #include <iostream>
      int main() {
        std::cout << "Hello, World!" << std::endl;
        return 0;
      }
    '';
    dontUnpack = true;
    buildInputs = [ pkgs.glibc.static ];
    buildPhase = "$CXX -std=c++17 -static -o hello $src";
    installPhase = "mkdir -p $out/bin; cp hello $out/bin/";
  }
  
  % nix-build hello.nix
  ...
  
  % wc -c result/bin/hello
  2224640 result/bin/hello

[esafak]

2.2MiB for "Hello, World"? I must be getting old...

The executable takes 33KB in C, 75KB in nim.

[koito17]

By switching to e.g. musl, you can go down to a single megabyte ;)

But in all seriousness, my example is quite cherrypicked, since nobody will actually statically link glibc. And even if they did, one can make use of link-time optimization to remove lots of patches of unused code. Note that this is the same strategy one would employ to debloat their Rust binaries. (Use LTO, don't aggressively inline code, etc.)

[3836293648]

Just a `puts("Hello world!")` with -Os statically linked to musl is 22k

[AdieuToLogic]

Just for fun, I wondered how small a canonical hello world program could be in macOS running an ARM processor. Below is based on what I found here[0] with minor command-line switch alterations to account for a newer OS version.

ARM64 assembly program (hw.s):

  //
  // Assembler program to print "Hello World!"
  // to stdout.
  //
  // X0-X2 - parameters to linux function services
  // X16 - linux function number
  //
  .global _start             // Provide program starting address to linker
  .align 2

  // Setup the parameters to print hello world
  // and then call Linux to do it.

  _start: mov X0, #1     // 1 = StdOut
          adr X1, helloworld // string to print
          mov X2, #13     // length of our string
          mov X16, #4     // MacOS write system call
          svc 0     // Call linux to output the string

  // Setup the parameters to exit the program
  // and then call Linux to do it.

          mov     X0, #0      // Use 0 return code
          mov     X16, #1     // Service command code 1 terminates this program
          svc     0           // Call MacOS to terminate the program

  helloworld:      .ascii  "Hello World!\n"

Assembling and linking commands:

  as -o hw.o hw.s &&
  ld -macos_version_min 14.0.0 -o hw hw.o -lSystem -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.2.sdk -e _start -arch arm64

Resulting file sizes:

  -rwxr-xr-x  1 <uid>  <gid>    16K Jun 18 21:23 hw
  -rw-r--r--  1 <uid>  <gid>   440B Jun 18 21:23 hw.o
  -rw-r--r--  1 <uid>  <gid>   862B Jun 18 21:21 hw.s

0 - https://smist08.wordpress.com/2021/01/08/apple-m1-assembly-l...

[creatonez]

> The executable takes 33KB in C, 75KB in nim.

Did you statically link Glibc...? Or is this with a non-GNU libc?

Either way, it probably is true that this 2.2MiB number would be smaller on, say, Debian 5. And much smaller on PDP Unix.

[3836293648]

We just have large standard libraries now

[surajrmal]

lto will remove most of it.

[wahern]

> Most shells dynamically link to a runtime your OS provides "for free"

Rust binaries also dynamically link to and rely on this runtime.

[mtndew4brkfst]

That's not intrinsically or pervasively true, although it's not uncommon.

[creatonez]

Yes. The main areas where Rust code uses dynamic linking by default are Glibc and OpenSSL (through the popular `native-tls` crate). Most things outside of that will be statically linked by default. There is room to improve the situation by making more C wrapper libraries support either method.

For Rust code talking to Rust libraries (such as `std`), it's a totally different challenge, similar to what you face in C++. You can compile your pure Rust app in a way to divide it up into DLLs. The problem is that the polymorphism in Rust means these DLLs must all be built together. Calling a polymorphic function means code has to be generated for it. The only way around this is for your Rust library to speak the C ABI, which bloats code as building your C-style API surface will resolve all the polymorphized functions/structs, but at least gets you swappable dynamic linking.

[3836293648]

The only way to avoid it is to be on linux with no_std, use musl statically or be on embedded. You cannot (or at least are supposed to not be able to) link to glibc statically and on every other OS you can only call syscalls via the system libraries. (Well, technically you can on most systems, it's just not stable across updates. OpenBSD will actively block it though)

[wahern]

Unless the majority of Rust builds on Linux statically link musl libc or use no_std, then it's pervasively true. And it's true on most non-Linux targets, including the BSDs and macOS. It's the same situation with Go.

[fuzztester]

why did you embed the c++ code in the .nix file?

just to have everything in one file? how to show how to do it with nix?

because it seem simpler to have a separate C++ file, and a simple shell script or makefile to compile it.

e.g. although I could figure out roughly what the .nix file does, many more people would know plain unix shell than nix.

and where is $out defined in the .nix file?

[AnthOlei]

The nix file is besides the point - it gives you a totally hermetic build environment. Not OP, but it’s the only way I know how to get gcc to use a static glibc. All you should pay attention to is that it’s using a static glibc.

$out is a magic variable in nix that means the output of the derivation - the directory that nix moves to its final destination

[JoshTriplett]

> Not OP, but it’s the only way I know how to get gcc to use a static glibc.

    /tmp$ gcc -O3 test.c -o test
    /tmp$ ldd test
     linux-vdso.so.1 (0x00007f3d9fbfe000)
     libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3d9f9e8000)
     /lib64/ld-linux-x86-64.so.2 (0x00007f3d9fc00000)
    /tmp$ gcc -static -O3 test.c -o test
    /tmp$ ldd test
     not a dynamic executable

[fuzztester]

>> Not OP, but it’s the only way I know how to get gcc to use a static glibc.

> /tmp$ gcc -static -O3 test.c -o test /tmp$ ldd test not a dynamic executable

yes, that last line above means it's a statically linked executable.

yes, i had a doubt about what the GP said, about their nix way being the only way to create a statically linked executable.

but I didn't remember all the details, because it's been a while since I worked with C in depth (moved to Java, Ruby, Python, etc.)(though I did a lot of that earlier, even in pre-Linux years), so I didn't say anything else. thanks, Josh Triplett for clarifying.

but one thing I do remember, is that static linking was the only option in the beginning, at least on Unix, and dynamic linking came only some time later.

when I started working on UNIX and C, there was no dynamic linking at all, IIRC.

https://en.m.wikipedia.org/wiki/Static_library

("dynamic linking" topic in above page links to the below page in Wikipedia: )

https://en.m.wikipedia.org/wiki/Dynamic_linker

[3836293648]

I thought glibc had some hacks in it to prevent it from working fully when statically linked? Is this just a myth or outdated or only affects C/C++ or what?

[fuzztester]

thanks.

[fuzztester]

and by the way, ignorant mindlessly downvoting dudes who don't even bother to check if a comment is right or not, can shove it up, and take a flying leap into Lake Titicaca. they'll meet a lot of their brothers there, like giant frogs.

from a Google search:

>Overview Lake Titicaca, straddling the border between Peru and Bolivia in the Andes Mountains, is one of South America's largest lakes and the world’s highest navigable body of water. Said to be the birthplace of the Incas, it’s home to numerous ruins. Its waters are famously still and brightly reflective. Around it is Titicaca National Reserve, sheltering rare aquatic wildlife such as giant frogs.

:)

[o11c]

For reference, some statically-linked shells on my system:

  2288K   /bin/bash-static (per manual, "too big and too slow")
  1936K   /bin/busybox-static (including tools not just the shell)
  192K    /usr/lib/klibc/bin/mksh
  2456K   zsh-static

For comparison, some dynamically-linked binaries (some old)

  804K    ./bin/bash-3.2
  888K    ./bin/bash-4.0
  908K    ./bin/bash-4.1
  956K    ./bin/bash-4.2
  1016K   ./bin/bash-4.3
  1092K   ./bin/bash-4.4
  1176K   ./bin/bash-5.0
  1208K   ./bin/bash-5.1
  1236K   /bin/bash (5.2)
  124K    /bin/dash
  1448K   /bin/ksh93 (fattest when excluding libc!)
  292K    /bin/mksh
  144K    /bin/posh
  424K    /bin/yash
  848K    /bin/zsh

(The reason I don't have static binaries handy is because they no longer run on modern systems. As long as you aren't using shitty libraries, dynamic binaries are more portable and reliable, contrary to internet "wisdom".)

[JoshTriplett]

Among the features it has: an interactive terminal GUI, threaded parallel directory walking, and git repository support. In around a thousand lines of code, total, including tests, half of which is the GUI.

[oguz-ismail]

*TUI. Not GUI

[have-a-break]

I feel like that's just the result of having a native package manager making natural bloat and a compiler which hasn't had decades of work.