I know that co-location means “customer owned hardware”, but in this case, I think I’d way rather rent data center owned RPis and just pay them money rather than sending in hardware, having to cycle out hardware if/when it fails, etc.
It also means the colo is running whatever random power supply I send them, which seems like something they’d want to avoid and means that there’s all the inefficiency of 12 supplies per U rather than one beefy +5.1V supply (with battery backing) feeding the Pis via the GPIO pins.
I was hoping they'd have a DC power supply per rack, but their FAQ makes it clear that this is not the case. Bit of a missed opportunity there. Handling heterogeneous power supplies sounds like a nightmare.
What a bummer. Something like an 80PLUS Platinum ATX PC PSU could do around 40 amps on the 5V rails. Redundant server PSUs seem like an obvious choice here.
if you're using amazon.com/dp/B077R5BVVR/ removing just the outer enclosure and removing that fan should be safe, but it might not be the loudest fan...
otherwise the quietest PC is the one in a different room, but you probably already know that
For USB power? I don't think there should be any way for Pis plugged into the same freestanding hub to talk to each other, but if so you could prevent that with power-only cables.
So docker is involved but you get a pi all to yourself. Does that make it so much easier to manage that they can pay for the pi themselves and still have much lower costs?
It looks like they have a custom built rack for the Pi, so supplying a custom 5v rail probably wouldn’t be hard at all and would almost certainly be cheaper than large deployments of PoE switches.
Thanks for your comment about the power supplies. Not every power supply will work. a) As stated in the FAQs and again when successfully booked: " Note that the power supply should have at least 3.0A / 5.1V power. It must be CE certified and support DC power." b) The DCs are located in Germany: 230 V - DC.
I have also read all the other comments on this subject: Yes, currently each Pi runs with its own power supply.
Currently, our team is developing a dedicated power supply for all Pi slots within the rack. Then it will also be possible for users to turn the power on and off via the colocation dashboard. However, the development will still take a little time. The reason for this is that some components are not immediately available and especially not in large quantities. See analogously the availability of Pis.
It seems like either feeding 5v via custom power supply on GPIO or requiring PoE HATs (though those are slightly less efficient...) would be a better scalable option.
One massive thing that seems to be missing here (unless I've missed it) is any kind of remote ability to manage the server, eg at a minimum remote power cycling, if the Pi locks up. It would also be nice to get remote console but that would require even more effort and potentially slight customization on the Pis' boot config (to enable UART).
Agree, a central supply with a 3A MOSFET for control and a PTC for basic protection would give a lot more functionality and reduce customer downtime and smart-hands touches.
At that point what you're describing has little resemblance to a hosted RPI though, the complexity justifies just making something custom that's better suited to the task than shoe horning Raspberry Pi hardware into a rack.
Sorry but that’s bullshit.. they’re describing a basic power switch (even simpler than a PoE hat) which is nothing compared to the design complexity of an MCU board with memory, peripherals and chip level power conversion.
If it doesn't have to be a Pi and just needs to be ARM, then AWS, GCP and Oracle all offer ARM instances and the ability to have custom images. I'm pretty sure they all have options that cost less than .05 USD an hour for on demand, and I know that GCP and AWS have tons of free credit available.
I like the idea in theory, but I can’t entirely agree with the “Green” designation. Putting 12 Raspberry Pis, 12 USB SSDs, 12 switch ports, and cabling and power supplies for all of the above adds up quickly.
From a pure compute-per-watt perspective using typical cloud workloads, I’d still expect a run of the mill shared cloud server to be more efficient. It would also allow for more burst overhead for individual workloads.
This is an interesting option for people who need a specific Raspberry Pi hosted somewhere.
> This is an interesting option for people who need a specific Raspberry Pi hosted somewhere.
What is that use case though? The page says that they only host regular Pis and optionally a USB SSD. So they can't do anything that a regular cloud server can't do - no custom hats, etc.
I have a Pi 4 home server, and the biggest issue right now is that my home upload is a bit weak for remote video streaming. So this product could interest me, in theory - saves me from having to migrate all my data & configuration to a cloud server. But I would rather pay Hetzner a very similar amount of money to get a VPS that's about as powerful as a Pi (probably more) and still have the physical Pi here at home as a fallback.
Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient?
Edit: This is in response to the "fleet of Pi's" question, obviously a Mac Mini is not going to be cheaper to rent than a single Pi! The aforementioned VPS route is the better way to go for that case.
Scaleway will give you an 8 core 16GB RAM 256GB SSD M1 Mac Mini for €0.1/h. It may not sound like much of an increase from core count but it is ~10x faster for multicore which means it probably comes out on top for perf/€, perf/Watt, and total perf compared to a rack of Pi 4's for any such distributed ARM use case.
For pure traditional cloud a Graviton2 instance on AWS is probably more green, albeit probably less cost efficient to the user.
To be clear this isn't an alternative to hosting a single Pi it was in response to the distributed case:
> Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient?
Assuming the task really requires ARM, is perfectly scalable among multiple systems, doesn't require more than 10mbps between the nodes, and doesn't require dedicated control/scheduling nodes (i.e. best case for the Pi's) a fleet in multiples of 10 Pi's per would be $59.90 month each plus the up front cost of the Pi's, power adapters, SSDs, and shipping. And even if you wrote off the up front hardware as on hand it would still be significantly less green to run.
"Green", now removed from the title, but still on the landingpage refers to the power used by our data center. Our data centers operate inside wind & solar parks, therefor we only use green energy. If there is no wind or sun we directly purchase green energy. Of course every DC is connected to the public grid for continuous operation.
Raspberry Pis may be "green" in that they are cheap, but power efficient they aren't. They have barely any power management support, making their idle power usage higher than even some x86 chips.
It's not hard for x86 laptops to idle below 4W (see Surface https://www.notebookcheck.net/Microsoft-Surface-Go-Pentium-6... , even Pro ones with screen on they idle well below 10W). With 10W just for the SoC you get into desktop or gaming laptop territory.
I have a full x86 system that idles at 1.7W _at the wall_ https://news.ycombinator.com/item?id=26639929 . This is an off-the-shelf ASUS PN40 mini-desktop, running an N4000, and includes 8GB RAM, a SATA SSD, and Gigabit ethernet, all running and accepting requests.
> I have a full x86 system that idles at 1.7W _at the wall_
A single stick of DDR1/2/3 ram uses more than that alone.
SATA SSDs are typically 0.5-1W idling.
The mobo chipset, figure at least a few watts
That doesn't even account for CPU idle power consumption.
Whatever you're using for power monitoring isn't accurate. Probably doesn't fully integrate, instead using sampling, and the sampling rate isn't high enough to show true power consumption from your PC's switching PSU.
It's DDR3L, not DDR3. Even the active power use of the stick is not going to be > 0.5W. And RAPL/ACPI can show the CPU/chipset (respectively) consumption, and it averages less than 1W. The official ASUS measurement is at ~3.7W https://csr.asus.com/english/file/ErP_PN40.pdf , and that is with Windows, 8 GB stick and 2 SSDs (personally I couldn't get it to idle at less than 6-7W with Windows).
Again: there are laptops that idle at these values, and this is even worse than a CPU laptop -- it's an Atom. Think S0i3... which keeps the RAM (in SR), chipset, and a shitton of things awake and yet consumes way less than 1W.
> Whatever you're using for power monitoring isn't accurate.
Sure (as in: one can never disagree with that), but the point is: it's still at least half the consumption of the Raspberry Pi 4, and lower than the consumption of a Raspberry Pi 3 where I disabled almost everything. For which apparently the measurements match to the 2 digits what I can find online, which I can compare since it's much easier to use the same software-hardware combo.
This is not ARM vs x86. It's just the RPI that is crap regarding power management, since by design _they are not even trying_. They just wanted to be cheap. There are many (ARM and other) CPUs that idle even lower than that. If they wanted to be efficient, they would beat the many year old Atom in an instant. Probably already do in (peak) performance/watt.
While I completely understand the allure of running on your own hardware, if you just want a cheap server to host a personal page or similar, you cannot beat Scaleway's Stardust VM instances. For less than 2 EUR a month you get 1 vCPU, 1 GB of RAM, 1 IPv4+IPv6 address, 10GB of storage and unlimited traffic. They claim up-to 100Mbps bandwidth but I regularly get much more than that. This sounds like a commercial but I'm just really happy with the service.
The lottery approach pissed me off so bad, I decided to leave Scaleway altogether. They also aren’t upfront about their contingents. I wanted to try out an M1 instance and before registering and putting my credit card info in, it seemed like it’d be all fine, I just have to put in cc info. I did, they told me there are no instances available. The fair way would be to be upfront about it in my opinion.
It isn't a lottery as much as an availability constraint.
First you have a limit of 1 stardust per datacenter per customer. Second, they only spin up a fixed number of new stardust servers per day.
It is a loss leader just like in any other business. They are losing money on one instance to get you in the door and you realize their other services are awesome.
I think local hardware makes sense for LAN-only sites - e.g. a company wiki, a media center or a file storage with web interface.
For anything that is supposed to be visible on the internet, I'd always use a hosted server - if nothing else, because I really don't want to open an ingress into my personal home network, even if my ISP permitted that.
For use-cases were you have to handle certain incoming requests even though your setup is mostly LAN-only otherwise (webhooks, ACME, adding some dashboard you can access from your phone...), services like PageKite[1] sound promising.
That pagekite looks handy! I’ve recently got gigabit fiber at home but my routers ipv6 firewall is crap. This seems easier than just making a custom vpn to a cloud instance.
I would recommend Racknerd as well. Not affiliated with them except a happy customer. I pay $36/yr for my 2 GB memory, 2 vCPU, 50 GB SSD VPS that I run Nextcloud on. I also have a $16/yr VPS with 1.5 GB memory and 30 GB SSD for K3s
"Instead of using a public IP the Pi is accessed by combining a public hostname with dedicated TCP ports. The hostname points to one of the ExaMesh gateways and is assigned to the colocation along with the available TCP ports in the booking process."
So maybe useful for an extra node for redundency, but maybe not as useful as having an actual address. Perhaps an extra encrypted Syncthing node or something
Thanks for mentioning it. Yes, there are quite a few use cases as these are already implemented by customers: Backup, Sync, Nodes (e.g. BTC Full Nodes), Relay, ...
I'm gonna ask the dumb/obvious question: Why would I want this?
It's certainly not for the compute. Isn't the point of a Raspberry Pi controlling periphery on the edge? But that's not possible here?
???
It's not even needing ARM cores, as those are now cheaply offered by all the cloud computing companies.
Is it just for some cheap fun? But if I'm going to host something on cheap amateur grade hardware, why would I not also just use my home connection? Is this for the experience and education?
... I really don't see what it's good for (explanations welcome).
For many years I was part of a bandwidth cooperative. We had a cabinet and a fat pipe and a bunch of sysadmins who wanted a place to keep their stuff. Early on it was all 1U or 2U systems. But later there was enough demand for Mac Minis that we dedicated a shelf to them.
It didn't make much sense from a professional syadmin's perspective. But for a Mac user who already had their little project on a Mini and wanted to get it off their home bandwidth, it made sense to them in that it was one simple, incremental change. I imagine the market here is similar.
Anything benefiting from a static IP address, such as running your own VPN, mail server, Bitcoin node, TOR node... the latter of which got me banned by my bank's security team because I was marked as "suspicious traffic" (wasn't even an exit node) - preventing me from using online banking. Talking to support proved fruitless, however the ban was lifted as soon as I changed my IP address.
Why a Pi though? You're obviously not making use of any of that expensive IO other than the eth...why not just offer a "Pi-compatible " custom board* that's actually designed in a sensible way for this use-case? Would be substantially cheaper and more energy efficient.
*Or really just shared hosting w/ containers running Raspbian on standard server hardware with a nice onboarding workflow for migrating from a real Pi would likely be sufficient for most people's use-cases—if you're not using peripherals I imagine you don't have any need for the real time OS features?
I have used numerous Pi-alternatives and the compatible distros are always a mess, not to mention finding support for why some package/GPIO isn't working is hit-or-miss. Try getting developers of a $10 board to patch bugs.
I have even spent more and used e.g. Asus Pi-compatible boards and it was always a dumpster fire - the last one I picked up was advertised as supporting 4K output, but getting it to actually work was beyond my capability, and when it did work, was topping out at 10 FPS. Writing to support ended up being a waste of time, all I ever got was vague answers as to why it should work without any actual solutions.
"Substantially cheaper and more energy efficient" - 9W max is already quite low[0] and cheaper? Pi devices are already pretty cheap, and the money goes to a good cause as opposed to cheap knock-offs where the money goes where? Also the cheaper you go, the less support you will probably receive. As noted by the link, StackExchange has a dedicated site just for Raspberry Pi questions. Good luck getting even close to the same level of information about any compatible boards, which probably cut corners by using sketchy hardware which may or may not be be patched in the future in their forked distro.
I fix this by running a script on a cron job that updates dns records based on my current ip, using cloudfront to only allow known ips through my ufw rules. It doesn't work for 100% uptime, but I've never had an instance where visiting my domain failed.
It may not work if you're running a tor node, depending on how cloudfront deals with them, but it does work for mostly-reliable dns resolution on a non-static, residential isp connection.
Not negating the project, just offering an alternative for people that want a static ip without renting vps/metal and without the isp static upcharge.
I have a script that updates the dns record on a free tier plan with Cloudflare, and since traffic passes through them I can open my residential network to only ports 80 and 443 from their ip4/6 addresses.
Next to the other arguments, the colocation is pretty cheap. In Germany, you can calculate ~20ct per Wattmonth for electricity, so ~1€ of this would go to electricity alone. Hosting at home also tends to come without static IP and non-symmetrical, somewhat unstable connections (speaking from painful experience).
For this service you pay ~6€ per month (assuming 50€ for the Pi and two years of runtime, no SSD) for a rather powerful VM. Just as a comparison, at Linode, you get 1 shared CPU and 1G of RAM for roughly the same price, compared to 4 core and 4-8 gigs with the Pi. Storage is even more expensive, so if you attach a large SSD, the calculation becomes even better (but the 10Mbit might become a bottleneck quickly).
Security perhaps. VPS is no longer as secure with the rowhammer and cache exploitation vulnerabilities. And if you only need a tiny system, a raspberry pi is pretty ideal
An RPI4 w/ SSD for $6 is probably the best compute for the buck right now in colo prices. This basically looks like a BYO hardware setup where they can maximize economics due to the RPI form factor being consistent.
I’d like to see the same thing but with a Mac mini.
Thank you, as far as we could find out, this is really unique.
GPIO: Currently there are no plans for this yet. It is not possible at the moment, we point this out during checkout and on the colocation administration page. However, we think nothing is impossible. If you have a use case, feel free to write me at colocation@examesh.de ;)
I think I'd rather have a decently specced KVM VM on a x86-64 hypervisor somewhere, I can run mainline debian on, for $6/mo than a raspberry pi. For that money if you look you can get something with 2 pseudo cores, 2 gigs of ram, and probably 40GB of storage.
At least I can have more confidence that the storage won't spontaneously fail, and network throughput greater than 10 Mbps.
This seems like a cool idea and all and it's certainly cheap for hobby projects. But I wonder how viable it really is as a business model. Doing the math on person-hour costs if just one pi requires 15 minutes of support/human attention from a person at the ISP, once, you're losing money on that customer forever.
This is neat but from a scaling perspective it doesn’t make sense. A single server grade Xeon chip can expose the same compute power as a cluster of these devices, with better performance across the board (memory access, peripherals, etc)
The use case is greenwashing and separating people from their money because everyone thinks Pis are just the bee's knees.
The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400...so in theory a 5600x is twice as energy efficient as a Pi4 if fully loaded. Sure this doesn't account for system fans and the motherboard, but they don't use that much compared to the CPU.
The whole point of virtualization is that most systems are idle a lot of the time. At datacenter scale virtualization, you can dramatically over-provision and shut down/sleep unnecessary nodes, firing them up when you need to. You can get near 100% utilization on your hardware, making the very most of every watt that doesn't go to actually computing.
Here they're going to have a zillion Pi4's, most of them sitting idle, but still using a couple watts. They're not even bothering to use any sort of shared power to improve PSU efficiency. They're not even bothering to use Pi4 compute modules.
Now, the interesting bit is that now there's the Pi Zero Wireless 2. It has nearly the compute power of the 3B+, but the highest energy efficiency per watt of any Pi board so far...
> The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400...
That's more energy efficient, sure. But it sets a lower boundary on the power draw much higher than a normal Pi. A Pi plus a single external HDD draws 12W at the socket, according to my measurements. A PC CPU draws 4-5x that, just by itself. The other components on the motherboard need power too, even if you use integrated graphics or no graphics at all.
A PC only becomes more power-efficient if your load can't fit in three or more Pis. For plenty of uses, more than two Pis are an overkill.
That's exactly why you virtualize, so you don't just have a bunch of small Pis being mostly idle but less but more powerful machines running a lot of virtual istances that on their own are also mostly idle but you can put enough on to put enough load on the higher-powered machine to be worth it.
In a data center they should have enough instances that virtualizing makes sense from the perspective of power efficiency.
Virtualisation means you can get noisy (or nosy: see Rowhammer) neighbors. VMs are fine, but there are valid reasons to prefer bare metal for hosting critical applications.
"12 pis in 1U" where 1U is defined as the height of a pi on its side plus shelf, rather than the definition of 1U in every other 19" rackmount data center
Somewhat ironically, I'd guess that putting a server inside a wind turbine makes it less likely that you are utilising green energy. The power and comms connections to that location are there primarily to monitor the turbine, and they want that to work all the time, and especially when the blades aren't turning. So you don't go powering it with the wind farm itself.
Installing the server anywhere else means there is a chance that its power is being generated by that wind turbine!
One could power the equipment off the turbine when it's operating, and off an alternative supply otherwise.
It seems a little silly to worry about where the specific electrons came from to power the equipment, though. If powering that equipment enables a wind turbine to produce more power than it would have without that equipment, then it seems like the existence of that equipment is "green" whether or not its power came from dirtier sources.
Wind turbines generate 690v three phase. It isn't stepped down until it gets to a substation near the consumer.
The power needed onsite (lights, control systems, energy to start the blades spinning, etc) usually comes from a natural gas generator or a direct feed from a fossil fuel plant if one is nearby. Due to circular dependencies, you can't power them off the energy they generate.
> Wind turbines generate 690v three phase. It isn't stepped down until it gets to a substation near the consumer.
Large utility-scale wind turbines do not have power leaving them at 690V. It's many kilovolts. If they put out 690V, just for the sake of example a 2MW turbine would be generating ~3,000A.
> The power needed onsite (lights, control systems, energy to start the blades spinning, etc)
Self-motoring is only necessary on stall-regulated turbines. Most large utility-scale turbines are pitch-regulated.
Here's video of a turbine starting up, showing its control panel with power in/out indicated:
> usually comes from a natural gas generator or a direct feed from a fossil fuel plant if one is nearby
That is not how electrical grids work.
If a wind generator is not generating power and goes online, the power for its control systems comes from the grid, which encompasses all the electrical generation devices attached to said grid. Power for control systems could be coming from other wind turbines, solar, hydro, stored energy, nuclear, whatever.
> Due to circular dependencies, you can't power them off the energy they generate.
If the generators are connected to the grid and generating power, all the control systems, lighting, whatever - is connected to the same grid. If it's not generating power, all those systems are still connected to the grid.
I don't know anything about the domain, but I'm pretty sure it wouldn't be a big deal to tap off a few kilowatts or so using a secondary winding. Something similar to the alternator on a car perhaps. It doesn't need to be the same circuit and efficiency as the main power generation path.
What do you mean circular dependencies? You mean that you need an independent source of power for all the infrastructure to turn the turbines on when they're not currently generating? Sure, that makes sense. It seems like there's a ton of ways to implement that. Grid power, generators, batteries and solar panels, etc.
I totally understand all of the drawbacks here, I agree that it’s hard to think of an actual use case, and all that aside, there’s something aesthetically pleasing here in an “I’d read about this in a William Gibson novel” kind of way. “My compute fleet is distributed across a field of windmills in Europe” just _sounds_ cool.
A question to ExaMesh, in case someone from there happens to be reading: could you publically offer a guarantee and promise that not a single byte will be read off of (or written to) customers' SD cards or SSD drives before being installed, or after being uninstalled from your racks?
Not ExaMesh, but there are a variety of ways to encrypt the card and remotely unlock it.
You can use dropbear to ssh into the Pi and provide the key during boot.
You can have the Pi connect to a remote system to retrieve the key.
Some methods are obviously not perfect, but it'll definitely make it more complicated than just "copy the card." Remote key retrieval would let you audit when the system was booted and so on.
There's a more complex purpose-built open source software package specifically for handling remote disk unlocks but I'm blanking on the name and my google searches aren't turning it up. I vaguely recall it had fairly high levels of paranoia in its design.
I personally vouch for our team, but think about it: your Pi is connected to the Internet. You are responsible for the security of the Pi (system security, updates, encryption ...). You send us the hardware in the mail. How many hands does the package go through? Of course, no one gets into the data center without access authorization, etc. pp.
Thanks for the assurance. The concern with "mail order co-lo" is of course that the customers themselves don't travel to the DC to install and power up the server.
I can’t imagine having any hardware colocated without proper out-of-band KVM access. Who is going to drive out to the wind turbine and flash a new disk image to my Pi?
10Mbps is also excruciatingly slow. I was ready to see a 100Mbps cap.
I think they'll need to iterate a bit to find product market fit. The 10mbit bandwidth limit, calling it "Decentralized", no public IP downsides are off-putting even at this price.
I bet everyone who has a raspberry pi had this idea. Throwing a raspberry pi with a solar panel and a sim card to a random place. It could be for backup, vpn or to access some private network. But having it be a rackmounted VM in a fixed location doesn't sound that fun to me.
Hey this is sort of the mirror opposite of my startup (we try to bring the internet to your home-pi, rather than ship your home-pi to a datacenter!). Neat tho! I'm not entirely sure it's that power efficient versus a carved up hypervisor tho...
The main gotchas: 1) no public IP of your own but instead you get a few dedicated ports NATd to you on a shared IP, 2) bandwidth is fixed at 10/10 Mbit/s.
Pi co-lo for just four bucks a month sounded great until the fine print was revealed...
They were a revision of the first gen Raspberry Pi. A bit more stable and seemed to not corrupt the root filesystem after a few hours, which the initial one seemed to invariably do for me.
They're not offering it any more, no.
I moved to self-hosting after that. Even got a static IP address for it.
I find static IP a bit overpriced when you can easily build your own dynamic IP solver with a few instances with static IPs that you need anyway to get low latency around the globe, also most home IPSes block ports 25 and 53 so if you want mail and dns you still need additional boxes.
I'm using google (bad I know but they are the best currently) as my main 3 continent provider (my own HTTP with JSON db, SMTP and DNS implemented from scratch) and I have redundancy in asia on AWS and in central US with IONOS, that way I have the cheapest double redundant cloud you can get with the most customers in low latency reach!
There were more than a handful of free Raspberry Pi colocations. However, since there is real manual work and infrastructure behind it, there are no such offers anymore, as far as we could check.
If it's BYO power brick you could also colo a second computer inside the power brick, using the rpi as a "modem" for it. Put a tamper resistant device inside that case.
Most domain extensions are already registered. And since we are from Germany, the term "Piccolo" sound very similar to "Picolo" but means something very different: https://de.wikipedia.org/wiki/Piccolo_(Sekt)
However, we would have definitely pursued an overlap/commonality of the word with, say, beer!
It also means the colo is running whatever random power supply I send them, which seems like something they’d want to avoid and means that there’s all the inefficiency of 12 supplies per U rather than one beefy +5.1V supply (with battery backing) feeding the Pis via the GPIO pins.