kernel+boot+basicpackages 500MB, with C it could be 16MB.
Golang is designed to be static all-in-one packages for each application(e.g. microservices), when you run a lot of them in any given system, it becomes really large together.
I don't mind to use one or two go binaries in my embedded board, when I need a few of them, I am concerned about the storage size(possibly memory size as well).
The only place on the readme where I could find those figures you came up with was the partition table and obviously a partition of 500MB doesn’t mean the applications are going to be 500MB in size. In fact it’s quite the opposite; it pretty much guarantees the applications are going to be less than 500MB given you are going to want some headroom to work with.
To be fair, this is designed for Raspberry Pi, and nowadays it's almost impossible to find a micro SD that's less than 16 GB and costs significantly less than a 16 GB one, I don't think a couple hundred MBs are a concern (even if it hurts my soul too)
What's the max/typical read bandwidth from an SD card on a Pi? Even if the storage space is there, you might still get perceptible latency improvements with order-of-magnitude smaller binaries.
On a Raspberry Pi 3 with a random SD card I found laying around:
pi@octopi:~ $ sudo hdparm -tT /dev/mmcblk0
/dev/mmcblk0:
Timing cached reads: 1366 MB in 2.00 seconds = 683.30 MB/sec
Timing buffered disk reads: 62 MB in 3.02 seconds = 20.53 MB/sec
So it's going to take a couple seconds to page this thing into memory. (The memory speed isn't great either.) It's not so slow that I'd bother to write application software in C just to make startup 5 seconds faster, though.
Meanwhile on a Threadripper 3970x with SSD (but virtualized):
$ sudo hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 20200 MB in 2.00 seconds = 10109.10 MB/sec
Timing buffered disk reads: 5296 MB in 3.00 seconds = 1763.28 MB/sec
If that’s an issue just cache the file in RAM. 500MB is still less than the system memory on a Pi (and as I’ve pointed elsewhere, the OP was reading the partition table not the application file sizes).
It is basically just a normal computer and I feel you’re optimising the wrong thing here too given we’ve already established the OP was wrong when they threw around what they believed to be the applications file sizes.
The Raspberry 3 Model A+ only has 512MB of RAM, so whilst the above might be 'technically correct', it doesn't seem like that would work in practice, at least on this one (the other types of Model 3 (and the 4s) do have 1GB RAM or more.
Am assuming that the reference to Raspberry Pi 3 and 4 might have been made without considering the above Model since otherwise there would be no need to exclude the Zero models, which also have 512MB of RAM.
I know how much memory the Pis support, I’ve owned every RPi since the very first one. Model As are lower specs than their B counterparts (eg the 3 B+ has 1GB of RAM) but that’s all moot because you and your siblings are still missing one important detail that I’ve stressed several times already and at this point I feel I shouldn’t have to making this point again, but…
Your application is not going to be 500MB!!
The OP completely misunderstood this entire premise of this project and every comment that has followed has been just as misinformed.
It’s almost comical just how badly you all have misunderstood this project yet still proceeded to lecture everyone about how unsuitable it and Go is. So let me explain one last time:
500MB is just a partition for you to put YOUR own application on. That application might only be 5 (five) megabytes or it could be 50MB but the partition needs to be large enough to offer some breathing room (eg logs, external assets if required, etc) and frankly for a non-embedded mini-PC, a 500MB partition isn’t a ridiculously high suggestion.
Clearly none of you guys bitching about the unsuitability of Go for the Pi have never actually used Go on a Pi or even bothered to read this GitHub readme. I’ve done both and frankly you’re all being ridiculous. Go works fine on the Pi (even the very first model) and no Go executable is going to be half a gigabyte(!!!) - I know Go binaries are large but stop for a moment to thing about the actual file size you are all talking about for a moment. Even Electon doesn’t inflate that big in most cases.
Honestly, it’s ridiculous the way you’ve all carried on. Did nobody stop to think for a moment before posting?
It appears the authors assume the user will choose an SD card as the rootfs. Another alternative is to run the RPi with a tmpfs mounted directory as the rootfs and use the SD card as tertiary storage. This way I do not have to worry as much about the quality and condition of the SD card. To conserve RAM, I need the bootloader, kernel and the "always-installed" userland to be relatively small. With NetBSD, I can usually fit this all in under 20MB, leaving the rest of RAM as free tmpfs space to use however I wish. If I need to use some large Go binary for some specific task, I install it after boot in the free space. Then I remove it when the task is completed. Another alternative is to put these larger programs in a compressed filesystem such as cloop or squashfs, stored as a file on the SD card. NetBSD allows me to mount cloop2 filesystems read-only from tertiary storage. https://man.netbsd.org/vndcompress.1
upx will take time/resource to unzip, which could be slow in some embedded systems.
a helloworld go binary is about 8MB, a C is about 20KB(shared libraries), go binary size adds up really fast due to the fact most do not use shared libraries even when there are many go binaries in the system.
UPX also prevents re-using pages when multiple instances of the executable are loaded in RAM which is bad on low end systems.
The best solution would be to use BTRFS/ZFS with transparent compression (can use fast algorithms) or even SquashFS if you can deploy the entire app image mounted in a directory.
You're taking a few F/OSS components (in this case the Linux kernel, the golang toolchain and some custom software) and you're assemblying them into a bootable system.
That's almost quite literally the definition of "gnu/linux distribution".
I think you might be misunderstanding what GNU/Linux means.
GNU/Linux, or GNU plus Linux as I’ve recently taken to calling it, means combining a GNU userland (GNU libc, GNU coreutils, GNU compiler collection) and the Linux kernel. The Linux kernel itself is not a GNU project. If you want to use a GNU kernel then you need Hurd, hence GNU/Hurd.
When you replace the whole userland with components that are not written by the GNU project, then it no longer is GNU/Linux. As such this is not a GNU/Linux distribution, and neither is Alpine for example since by default it doesn’t use a GNU userland.
the size of the image concerns me:
kernel+boot 100MB, with C it could be 4MB.
kernel+boot+basicpackages 500MB, with C it could be 16MB.
Golang is designed to be static all-in-one packages for each application(e.g. microservices), when you run a lot of them in any given system, it becomes really large together.
I don't mind to use one or two go binaries in my embedded board, when I need a few of them, I am concerned about the storage size(possibly memory size as well).