Hi everyone, resin.io founder here, happy to answer any and all questions. We just released resinOS at ELCE a few hours ago (Embedded Linux Coference, Europe), happy to see it made HN!
Hi! Discovered resin.io already some time ago and think Docker containers on embedded devices is a very interesting approach.
Just skimmed a little bit through the resin-os docs and was left with the following questions:
- What's your plans for audio and video? The base system doesn't seem to provide any audio or video manager. Should those (e.g. wayland) also be ran inside a container, which could make access from other containers difficult? Or should they if required be included through the Yocto image build process? Or is A/V currently out of scope for you and you focus on pure networked devices?
- Haven't found any description on how prebuilt docker images can be packaged and flashed during the installation process or later on? I only found the rdt push description which seems to push the application sources to the device and builds the image there. This might be interesting for development but would not be anything that I want at all for production. E.g. I don't want to give customers my app sources, I need company git repository access (credentials) during image building, etc.
- What's your reasoning about building all those build and development tools in coffeescript for node? From a pure tool user perspective I know that it's a turn-off for lots of users that it requires node6 on the PC (it's not LTS, other stuff might require other versions, what is that node-thing anyway, ...). And from a developer perspective I would rather not prefer an exotic programming without static typing and a rather volatile ecosystem for a reliable long-term system. I don't want to say here that it isn't possible to create solid project with it and yours might work great - it's just not the first option that comes to my mind.
If you run a privileged container, you can add any sort of audio and video manager in the container. We have users running X11 and Electron inside their containers, for instance.
Preloaded containers is something we have built out for resin.io, and will be extending to resinOS standalone as well. The current release is just the development version. Production version with preloading should follow.
We use node.js for the development toolkit, not for the OS components. There will always be programming language disagreements, but we find we are productive in node.js and it works for us. Coffeescript is admittedly getting a bit long in the tooth and we should switch to ES6 or TypeScript, but I personaly see that as a minor change, though a good one.
I think I see where you fit in - deeply embedded systems with real-time constraints tend to use a certified hypervisor solution to separate runtimes. Traditional IT solutions can use Linux KVM or a full Docker implementation. ResinOS fits in the middle.
Familiarity with Docker will allow makers to get multiple runtimes up and running on higher-end (but small) SoCs...but for what type of applications?
There are a lot of companies looking at the intersection of containers and embedded. Some call it "Fog Computing" others "Edge Computing", others still "Industrial Internet" and so forth. We have a number of use cases on the resin.io website of people using this technology in production today, but some of the larger ones we're not allowed to talk about :(. ResinOS does include a full Docker engine however. It's just that a lot more is needed to do containers right on embedded Linux devices.
There's a lot of devices that need to run unsupervised, but still have (relatively) beefy hardware and commodity software. Digital signage stations are one example, some customers want a (relatively modern) Webkit/Blink based browser based system so they can deploy React/Angular/etc based applications on it instead of needing to hire people who know Qt Embedded or similar. The overhead doesn't really matter for such applications: The backlight for a 40+ inches large touchscreen will always consume more power than the other components combined, so you can put in an actively cooled 20W dual/quadcore APU just fine, and having a full OS allows running other services on it (SSH/VPN solutions for remote management, e.g.).
(Disclaimer, we've been doing exactly that since 2011. I've been wanting to move to a containerized approach for years, but our customers are happy enough with the current solution that they don't want to fund it. Alas.)
The hypriot guys are doing great work, and we share a common mission in spreading containers to embedded development. ResinOS is built with production deployments as a first goal, and so has a read-only rootFS, supports atomic host OS updates, is ported (and portable) to a lot of device types (via Yocto) and in general has many more "embedded" characteristics than Debian, on which hypriotOS is based. I do however want to reiterate that we know the Hypriot guys decently well and appreciate the work they're doing to make Docker accessible to embedded hackers.
Excellent response and yes, HypriotOS is based on Raspbian, which is a Debian derivative. Glad to hear you're using Yocto as well. I'll certainly have to check this out.
Do you have anyone using Kubernetes on ResinOS by chance? I've got a small fleet of small hardware and had started looking at running k8s on HypriotOS, but this looks a bit more custom built.
Haven't really tried to go that route. There are some fundamental limitations to Kubernetes and Docker Swarm due to their dependence on etcd, which behaves very badly when the devices are not on the same LAN. Depending on your use case you may want to have a look at resin.io as a management/orchestration service of sorts (pardon the plug), though our focus is purely on embedded/IoT scenarios, so it may or may not apply to your use case.
Nah I also do quite a bit of work with microcontrollers such as the esp8266 (32 bit 80Mhz RISC) and the just released esp32. It's relevant and I'll check them out.
We're generally very active in the Yocto community, the resin.io team already maintains the raspberrypi, CHIP, and Samsung Artik BSPs. Not sure what their interaction has been with the vitrualisation layer and don't want to misrepresent them, but I do know Andrei and the rest of the team are generally good citizens and work hard to feed patches upstream as much as possible.
I have used Yocto for building a new of embedded Linux systems in production. I understand that Yocto is used for the resin hostOS images and Docker is used for the images run as containers.
Is it possible or has any thought been put into making it possible to use Yocto for building the containerized images that are deployed to resin (probably in the form of docker images)? Although this is less convenient than using off-the-shelf docker images and customizing with your own docker files, I believe many of the benefits of Yocto in terms of building a custom embedded linux system might also be recognized within the context of resin.
Yes! We've actually created extremely small base images with Yocto for our own agent at resin.io, and it's a really cool approach, though it is very complex to set up and handle, and has approximately none of the elegance of a Dockerfile. Also, resinOS images are available as containers for some very specific use cases, so that's another instance where we've done something like that. Perhaps we should write up more about the approach.
Awesome, I would love to see hear more about the approach you guys used for building base images with Yocto. I'm willing to give up a little elegance for control and reproducibility.
I just added the BeagleBoard X15 to the board support list.
Do you anticipate supporting loading the other embedded processors? The BeagleBoard X15 has 2 - Cortex M4s and 2 DSPs but I think a simple case would be the single core dsp on the Beaglebone.
You mean that you tried the image and it worked on the X15? That's fantastic! We hadn't tried the board before.
What does support of the other processors entail? Is it a kernel modification? Are you sure the dsp on the Beagle doesn't work? Say hi on the gitter channel ( https://gitter.im/resin-os/chat ) and the resinOS team will be there tomorrow morning (european time) to talk more.
resinOS is designed for embedded devices, already supports many of them running several architectures (all the way down to ARMv5). While I have heard that rancherOS can run somewhat on a raspberry pi, that's a long way from being able to properly function in an embedded environment (read-only rootFS, reliable networking and DNS stack, support for atomic host updates, etc). There's nothing we'd like more than to have someone make the OS for us, but at this point we're pretty sure we'll have to do it ourselves. The variance in the embedded world is simply too large for a cloud OS to handle it without major rearchitecting.
Transactional updates are pretty much standard for embedded devices that care about updates. Android Brillo, ChromeOS, Android, Snappy, and many many others employ similar strategies.
ResinOS does indeed have a smaller footprint, a broader set of supported architectures (Snappy only supports ARMv7 and above), and uses Docker instead of LXD+snaps.
Just skimmed a little bit through the resin-os docs and was left with the following questions:
- What's your plans for audio and video? The base system doesn't seem to provide any audio or video manager. Should those (e.g. wayland) also be ran inside a container, which could make access from other containers difficult? Or should they if required be included through the Yocto image build process? Or is A/V currently out of scope for you and you focus on pure networked devices?
- Haven't found any description on how prebuilt docker images can be packaged and flashed during the installation process or later on? I only found the rdt push description which seems to push the application sources to the device and builds the image there. This might be interesting for development but would not be anything that I want at all for production. E.g. I don't want to give customers my app sources, I need company git repository access (credentials) during image building, etc.
- What's your reasoning about building all those build and development tools in coffeescript for node? From a pure tool user perspective I know that it's a turn-off for lots of users that it requires node6 on the PC (it's not LTS, other stuff might require other versions, what is that node-thing anyway, ...). And from a developer perspective I would rather not prefer an exotic programming without static typing and a rather volatile ecosystem for a reliable long-term system. I don't want to say here that it isn't possible to create solid project with it and yours might work great - it's just not the first option that comes to my mind.